Compare commits

...
251 Commits
Author SHA1 Message Date
Daniel Salazar a8a78736bb feat: kv list ordering toggle + addressable path improvements (#3843)
* fix: put-1787

* feat: kv list reverse
2026-09-09 17:24:03 -07:00
Juan Fernando Castro 9b465aba9f Merge pull request #3837 from HeyPuter/juancastro/team-count-active-seats
feat: add TeamStore.countActiveSeats for the billable seat count
2026-09-09 19:33:55 -04:00
Daniel Salazar 0edee23dd9 Update CODEOWNERS (#3842) 2026-09-09 15:55:19 -07:00
Daniel Salazar 99f3037110 Update CODEOWNERS (#3841) 2026-09-09 15:53:36 -07:00
Juan Castro 7ab48c590e feat: add TeamStore.countActiveSeats for the billable seat count
Per-account billing needs "how many accounts is this team getting the
benefit of", and neither existing count answers it. `countSeats` includes
suspended seats, because it bounds the seat cap and a suspended seat still
holds its username. `countPayers` counts the owner. Billing off `countSeats`
overcharges for suspended accounts, contradicting what the admin console
tells the owner: "0 suspended accounts cost nothing per account."

So this is a third, separate count rather than a change to either — the cap
must keep counting suspended seats.

An unactivated seat is deliberately still counted: it holds a temporary
password, not a suspension, and it is paid for. That is why the filter is
narrower than `listDirectory`'s, which also excludes
`requires_password_change`.

Falsified: dropping the suspended clause fails exactly one test
("drops a suspended seat from the active count but not from the cap",
expected 3 to be 2) and leaves the other 34 green.
2026-09-09 18:13:04 -04:00
404oopsandClaude Fable 5.1 6146ab9dd3 feat(ai): refresh txt2vid catalogs, retire Sora, default to Veo 3.1 Lite (#3828)
OpenAI shuts the Sora Videos API down on 2026-09-24 and sora-2 was the
default txt2vid model, so the default moves to Veo 3.1 Lite on Gemini
and the OpenAI video provider goes. While there, the video catalogs are
brought in line with what each vendor serves today, the request options
are unified across providers, and the txt2vid docs are rewritten.

- driver: default provider gemini-video-generation with
  veo-3.1-lite-generate-preview; a request under the generic `ai-video`
  driver name lands on the default instead of the first-registered
  provider; `WIDTHxHEIGHT` sizes map onto tier catalogs by the shorter
  side and fill width/height
- openai video provider, the `openai-video-generation` alias, its
  config template and migration entries, and Together's openai/sora-2*
  rows removed
- gemini: Veo 3.1 Fast rates 10/12/30 cents per second for
  720p/1080p/4K, Veo 3.1 Lite accepts reference images, URL image
  inputs are fetched server-side through the SSRF-guarded fetch
- together: drop nine models retired upstream, add eighteen from the
  live listing; per-second models are estimated from the catalog rate,
  clamped to remaining credit and billed at the cost Together reports
  on the job; tier-sized models take resolution/ratio;
  input_reference/last_frame map onto keyframes; generate_audio is
  forwarded
- byteplus: Seedance 2.5 (dreamina-seedance-2-5-260628) with per-model
  reference-image caps
- util/imageInput: string-level image helpers shared by the image and
  video drivers; ai-image/inputImage re-exports them unchanged
- puter.js types: provider and generate_audio options; docs: txt2vid
  page rewritten with per-provider model tables, unified options and
  four new playground examples

Known follow-up: Veo returns a key-protected Google file URL, so the
default clip cannot be played directly by a browser until the provider
fetches it server-side.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-09 14:56:17 -07:00
404oopsandClaude Fable 5.1 c8f4906af4 fix(ai): keep per-provider failures when every chat route fails (#3836)
When the chat fallback chain is exhausted, the driver already records
each attempt (model, provider, status, code, message, timeout) in the
error's `fields.attempts`, but the alarm keyed on the classified message
alone and the alarm client printed the error at inspect depth 2, so the
log and Slack line read `internal_error:All providers failed` with
nothing about which providers failed or why. Deduped repeats printed
only a count.

- The HTTP alarm gate now attaches an HttpError's `fields` to the alarm
  under a single `details` key. One key can't shadow the gate's own
  request fields, and a repeat from another thrower on a shared id
  replaces it instead of merging into it.
- The chat driver logs one warn line per exhausted chain with the
  completion id, the resolved route, the classified code and the
  attempts as JSON, so every occurrence is greppable by trace even when
  the alarm dedupes it. Chains marked `noAlarm` don't log.
- Docs: `fields` reaches both the client and the alarm, so it has to be
  safe to show the caller.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-09 14:56:11 -07:00
Reynaldi Chernando 435629afc5 Add gpt 2.5 image (#3839) 2026-09-09 14:29:41 -07:00
Juan Fernando Castro e132f765d6 Merge pull request #3745 from HeyPuter/juancastro/put-1736-team-directory
🏗️ PUT-1736: a team directory apps can read, once the team opens it
2026-09-09 16:09:03 -04:00
Juan Castro 4e45fdc172 feat: a team directory apps can read, once the team opens it
Members can already enumerate each other: `/teams/:uid/members` needs a user
actor and nothing more. The only thing this adds is admitting an app actor to
the same names, so an app can offer colleagues without the member driving it.

That is the whole risk, so it is off until the team owner turns it on.
`group.directory_enabled` defaults to 0, and a team that has not opted in
answers 404 rather than 403 -- whether a team has this on is not something an
app should be able to probe for either.

Three things bound what an app sees. The membership tested is always the
person's, never the app's, so an app installed by a member of one team can
never read another's. The page carries username and uuid and nothing else --
no email, activation state, usage or role. And suspended accounts and ones
that never took up their credential are left out, since offering someone who
cannot sign in is noise and their existence is not this list's to disclose.

Activation is the forced-change flag clearing, not the password existing: a
provisioned seat holds its temporary password from birth, so testing
`password IS NOT NULL` would have leaked exactly the accounts meant to be
excluded. A test covers that distinction.

Turning the directory on or off writes an audit row, because it changes who
can read the member list and that is not something a team should be able to
alter silently. Setting it to the value it already has records nothing.

The toggle lives in TabTeams, and turning it on asks for confirmation while
turning it off does not -- one grants access, the other only takes it away.

Closes PUT-1736.
2026-09-09 14:51:05 -04:00
Juan Fernando Castro f6b519874e Merge pull request #3742 from HeyPuter/juancastro/put-1740-1762-workspace-gui
🏗️ PUT-1740 + PUT-1762: the workspace surfaces in the GUI
2026-09-09 14:50:56 -04:00
Reynaldi ChernandoandCopilot Autofix powered by AI 59be4ad237 more recipes (#3819)
* more recipes

* addition

* final

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* styling fix

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-10 01:07:26 +07:00
Juan Castro 9121e66c3f feat: the team surfaces in the GUI
Two places a team becomes visible to a person rather than an API: the
Dashboard tab that administers it, and the share dialog that shares with it.

Squashed because they are one change to a reviewer. Both are the first GUI
consumers of `puter.teams`, both add strings to the same `en.js`, and both had
to answer the same question -- what a team looks like to someone who has
never seen one. Splitting them would mean reviewing that answer twice and
resolving the same translation conflict twice.

TabTeams (PUT-1740)

  Registered in `builtinTabs` after `TabFiles`, as a plain object with
  `html()` and `init($el_window)` like every other tab. The member table
  carries the operations the API allows and nothing it does not: provision,
  resend an activation, disable, restore, delete.

  Deleting asks for confirmation naming the account, because the API deletes
  on a single call -- PUT-1732 places the reversible step at disable, not in
  front of the request, so the client is where a confirmation belongs.

  The wording says plainly that disabling stops the per-account charge and
  keeps the storage one, and that deleting is the only thing that ends it.
  This is the one place a person decides between the two, so leaving them to
  infer the difference costs them money.

  Rendering and the billing arithmetic are pure functions in
  `teamsConsole.js`, with tests beside them; the tab file is the DOM.

Share dialog (PUT-1762)

  `UIWindowShare` gains the teams the caller owns as recipients. A
  team is one entry in the list, not its members expanded -- sharing
  reaches every member, and the per-person wording would understate what the
  grant does.

  `shareWorkspaces.js` holds the resolution and the deduplication, so the
  dialog does not gain a second source of truth about who a recipient is.

Closes PUT-1740 and PUT-1762.
2026-09-09 11:51:44 -04:00
Juan Castro afb5b0d82a feat: add the puter.teams SDK module
Team administration reaches the backend through the `/teams` routes
rather than a driver interface, because the gates it needs are route-level:
a user actor, a verified account, and a dual-window rate limit.

The module follows `apps/` and `perms/` in layout -- one file per method,
a thin `index.js`, a JSDoc-only `types.js`, and the `METHODS` rebinding so a
destructured method keeps its `this`. It does not follow `perms/lib/req.js`:
those endpoints resolve `{ error: true }` for backward compatibility, and
nothing here has callers to keep compatible, so this throws `PuterJSError`
with the backend's own code.

Every method takes a team `uid`. A handle is a mutable label that
deleting the team releases, so a stored handle can later resolve to a
different team.

The list methods offer the three forms `puter.apps.list()` does -- an array
by default, the page envelope under `cursor`/`includeTotal`, an async
iterator under `stream`. They refuse `offset`: these routes are keyset-only
and would otherwise return page one however far you asked to skip.

The surface covers the routes that exist today. Usage totals and member-email
correction have no backend route yet and are deliberately absent rather than
shipped as methods that 404. `deleteMember()` is here because the route it
needs lands in the commit below this one.
2026-09-09 11:51:44 -04:00
Juan Castro a60a4a98b4 feat: delete a team seat for good, once it is disabled
A disabled account persists indefinitely. Removing it is an explicit request,
never a timer, and it is refused on a live account with
`account_must_be_disabled_first` — which puts a reversible step in front of the
only irreversible operation in the feature.

The audit row is written before `cascadeDelete` runs. The FKs are ON DELETE SET
NULL and the `_keep` columns carry the identifiers, so the record of what was
done survives the account it names.

No second billing emit here: `cascadeDelete` already captures the seat and
fires `team.account.deleted` through UserAccountService, and emitting again
would close the storage charge twice. Disabling closed the per-account charge;
this closes the storage one, and it is the only thing that does.

There is no restore window, and none was wanted: the reversible step already
exists earlier at disable, a disabled account costs only the bytes it holds so
nothing pressures a hasty delete, and a restore promise means retaining data
the team explicitly asked to be rid of.

Published in rate-limits-and-quotas.md alongside the team-deletion note,
since the two are easy to confuse and only one of them frees a seat.

Closes PUT-1732.
2026-09-09 11:51:44 -04:00
Juan Castro 81d15f412b feat: tell a team member what was done to their account
Two halves of the same question -- what did the team do to me, and how do
I find out. One is pull, the other push.

The activity view (PUT-1746) was already user-scoped and already 404'd a
non-member; what was missing is the load-bearing half. A member is told a reset
happened, but a reset only matters alongside who signed in afterwards, so
`SessionStore.listSignIns` merges their own sign-ins into the same stream,
newest first, with a per-stream cursor. Without that row the audit says a
credential was issued and never says whether it was used.

The notices (PUT-1733) cover the two things a member cannot discover for
themselves: their account being disabled, and their team being closed.
Deliberately one each -- closing a team disables every account in it, so
sending both would tell one person twice about one event. Both say plainly that
nothing was deleted; the accounts persist, suspended, holding their files.

Squashed because they are one change to a reviewer: same audience, same
purpose, seven files, overlapping only in TeamService. Neither touches shared
platform code.
2026-09-09 11:51:44 -04:00
Juan Castro f5b2365e83 feat: enforce the forced password change on team seats
`user.requires_password_change` shipped with the team columns but nothing
enforced it and nothing ever cleared it, so a provisioned seat kept its
administrator-issued password indefinitely and `reissueCredential`'s
"already activated" 409 was unreachable.

Adds the fourth clause to `assertVerifiedAccount`, the only place a
verification gate may live -- WebDAV builds its own actor and calls that
function directly, so a second implementation would bypass it the way the
phone and card gates once were bypassed.

A gate that refuses everything also refuses the endpoint that clears it,
so `/user-protected/change-password` opts out with `allowUnconfirmed`.
That widens the route: an account pending email, phone or card
verification can now change its password, which it could not before. The
caller is authenticated and proves the current password, so this is
benign, but it is a behaviour change to a shared route.

Also here, because the gate is worthless without them:

- change-password and the recovery-token path clear the flag, and record
  an `activate` entry when the account is a seat.
- Reset takes a live account back with a fresh credential, capped at 20
  per day and audited as `reset_member_password` with no credential in
  the row. Re-issue is audited the same way; it stays closed once a seat
  has chosen its own password.
- An issued credential expires after 24h (new `temp_password_expires_at`
  column, three dialects) and login refuses it after that, so an unused
  reset dies instead of becoming a standing credential.
- 2FA is untouched by a reset, so a reset alone is not takeover.
2026-09-09 11:51:44 -04:00
Juan Fernando Castro f3943f6a55 Merge pull request #3729 from HeyPuter/juancastro/put-1726-put-1727-put-1729-team-share-recipient
🏗️ PUT-1726 + PUT-1727 + PUT-1729: share with a workspace
2026-09-09 10:41:21 -04:00
Juan Castro 7a02fa7741 feat: tell every team member when something is shared with them
`notifyShared` keyed every map off `ResolvedShare.holderId`. A team share
has no individual holder -- the grant is one row against the group and
`holder_user_id` is NULL -- so it was skipped everywhere and the call returned
having told nobody. Nothing errored: the share was created, resolved and listed
correctly. The failure mode was silence.

`#recipientsOf` now expands a team share to its live members at
announcement time, so the grant stays one row and only the telling fans out.
`#announce` and `#emailHolder` are both inside the per-recipient loop, so
members get the in-app notification and the email digest.

Bounded by `NOTIFY_FANOUT_CAP`, which logs when it bites rather than truncating
silently -- a shortened fan-out otherwise reads as "everyone knows". The
existing per-recipient budgets absorb the volume from there.

Wording is unchanged: "shared N items with you", not the team name.

A member who joins later resolves the grant through the scan but is not
retroactively told; announcements describe a moment, not a state.
2026-09-09 10:22:56 -04:00
Juan Castro da68074401 feat: share with a team
Covers PUT-1726, PUT-1727 and PUT-1729. Together because a recipient without
the listing work produces a share that is created correctly, resolves
correctly, and never appears in "Shared with me" — a state nobody would ship.

The recipient. `ShareRecipient` gains `team` (uid) and `teamHandle`, resolved
before the email and username branches and never falling through to them.
Passing both is an error rather than a precedence rule, so a call site always
shows which was chosen: handles are released on soft delete and can be
reclaimed by an unrelated team, and a scripted share to a handle would
silently retarget. There is no bare-string spelling, since that would change
how existing strings are interpreted.

ACLService.setUserGroup mirrors setUserUser: same read-modify-write, same
one-mode-per-node rule, under a node lock keyed on the group.

One grant against the team, not one per member, so membership changes
apply without touching the grant and a share spends one unit of the daily
quota however many members there are. A member who joins afterwards gets
access, which is asserted.

The index row carries `holder_group_id` and leaves `holder_user_id` NULL, so
`0077`'s group index constrains it rather than the user-holder one.

Listing. `listByHolder` and `countByHolder` union the caller's teams into
the same keyset page — `ORDER BY id` still holds — and `#grantEvidence` gains
group grants as a third source. Without that third source the share is
filtered out of every listing as dead: nothing errors, the share simply is not
there, which is the one place a user would look for it.

Unsharing revokes the grant as well as deleting the index row. Deleting the
row alone would hide the share while leaving every member holding real access.

Closes PUT-1726, PUT-1727 and PUT-1729.
2026-09-09 10:22:56 -04:00
Juan Castro 9f615cc632 feat: add the two-team fixture and group permission grants
Two tickets, together because the fixture has no behaviour of its own — the
grant tests are its first real use, and "a grant to team A must not
resolve for B's members" is exactly a two-team assertion.

PUT-1719. Every query resolving team-scoped data has to take the
team as a parameter rather than infer it. One that forgets returns the
right rows against a database holding a single team, so a one-team
fixture does not give weaker coverage — it gives false confidence.

The fixture provides two teams, each with its own master and two
activated seats, plus a user in neither. Each team having its own master
is also what the shipped `max_teams_per_user: 1` requires, so it runs
against the real cap rather than lifting it the way the existing team suites
have to. Seats are activated before use: provisioning leaves
`requires_email_confirmation` set and `requireVerified` rejects them outright,
so an unactivated seat cannot call anything and every authorization assertion
built on one would be vacuous.

PUT-1725. The write half of group grants; the read half already worked, since
`#scanUserGroup` joins `jct_user_group` itself and resolves for whoever is a
member at scan time.

  PermissionStore   resolveGroupId, listGroupMemberUuids,
                    upsertUserGroupPerm, deleteUserGroupPerm,
                    auditUserGroupPerm
  PermissionService grantUserGroupPermission, revokeUserGroupPermission

Both rewrite the permission first, so `fs:/path:mode` collapses to
`fs:<uuid>:mode` and a revoke names the same string the grant wrote; both gate
on `canManagePermission`; both audit into the existing
`audit_user_to_group_permissions`.

The delete is scoped to the issuer, so one issuer's revoke cannot drop
another's identical grant — the PK is (user_id, group_id, permission) and
user_id is the issuer.

Cache invalidation goes through `bumpCacheGenerations` with every member uuid
at once. That already announces to peer regions in a single event, so the
fan-out costs one broadcast rather than one per member.

There is no flat-KV equivalent for group grants, so unlike the user-to-user
path there is no second write to keep consistent and no second invalidation
surface.

Closes PUT-1719 and PUT-1725.
2026-09-09 10:22:56 -04:00
Daniel Salazar 8a70fa2e5c fix: CODEOWNERS + perm dialogs outside puter dashboard/desktop (#3833) 2026-09-08 22:56:03 -07:00
Felix-Ayush 9bb5bffd01 fix(puter-js): account for base64 padding in estimateDataUrlSize (#3805)
Math.ceil(len * 3 / 4) over-counts by 1-2 bytes on padded payloads, so thumbnails at the size limit were wrongly rejected. Subtract padding like ai dataUriByteLength.
2026-09-08 22:33:33 -07:00
Felix-Ayush d7877359d4 fix(puter-js): return key_undefined for missing kv keys (#3803)
get/del/incr/decr/add/expire/expireAt only checked size, so undefined keys threw TypeError. Add assertKeyPresent first, matching set/update/remove.
2026-09-08 22:32:53 -07:00
Felix-Ayush cdc524f504 fix(puter-js): preserve zero amount in kv incr/decr (#3802)
parseCounterArgs used a falsy check, so incr(key, 0) sent amount 1. Only default when amount is undefined/null.
2026-09-08 22:30:02 -07:00
dependabot[bot] 8b872545dc chore(deps): bump docker/setup-buildx-action from 3 to 4 (#3823)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:27:06 -07:00
dependabot[bot] 52f9fa57a6 chore(deps): bump docker/metadata-action from 5 to 6 (#3822)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:26:49 -07:00
dependabot[bot] eb28012897 chore(deps): bump peter-evans/dockerhub-description from 4 to 5 (#3826)
Bumps [peter-evans/dockerhub-description](https://github.com/peter-evans/dockerhub-description) from 4 to 5.
- [Release notes](https://github.com/peter-evans/dockerhub-description/releases)
- [Commits](https://github.com/peter-evans/dockerhub-description/compare/v4...v5)

---
updated-dependencies:
- dependency-name: peter-evans/dockerhub-description
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:26:27 -07:00
Daniel Salazar e5f322f68c fix: misc event hardening + perm fixes (#3831) 2026-09-08 19:16:21 -07:00
Neal Shah ab3bba5525 add file query param (#3829) 2026-09-08 21:39:19 -04:00
Juan Fernando Castro 9f3c658975 Merge pull request #3830 from HeyPuter/juancastro/phase3-land-on-main
Teams phase 3: land the remaining three PRs on main (#3733, #3724, #3725)
2026-09-08 18:27:11 -04:00
Juan Castro bd19fd18ec feat: cap teams per user and seats per team
A seat is a real Puter account: it takes a name from the global username pool
and gets a home directory. Nothing charges for one — that is prod's job — so
until it does, the only bound on creation is the request rate limit, which
bounds the rate and not the total.

  max_teams_per_user   default 1
  max_seats_per_team   default 50

Ordering is the substance of both checks. The team cap is tested before
the handle, so a capped user is told they are capped rather than that the name
they picked was unusable. The seat cap is tested before any account state
exists, so a refused provision does not burn a global username.

Soft-deleted teams do not count toward the owner's cap, so deleting frees
the slot — which does mean create, provision, delete, repeat still consumes
usernames over time, bounded by the daily rate limit. The caps raise the cost
and make the cycle audited; they do not close it.

Lowering the seat limit blocks new provisioning and disables nobody.

Both limits are published in rate-limits-and-quotas.md, and both keys are
documented in config.template.jsonc and config.default.json.

Closes PUT-1758.
2026-09-08 16:59:29 -04:00
Juan Castro 41a05b68fd feat: announce credit-state transitions so billing can act on them
A seat that exhausts its allowance is refused and cannot fix it itself — only
the team owner can add capacity. Something has to tell the owner, and that
something lives in prod, so OSS's job is to say when it happened.

`MeteringService` emits `metering.credit-state` from `rememberRemainingCredits`,
the single point where the verdict is computed, at 90% of the allowance and
again on exhaustion.

Emitted on a transition, never per request. A blocked account keeps trying and
every retry recomputes the verdict, so without that a retry loop would announce
hundreds of times. The state is tracked per uuid in process, under the same FIFO
bound as the credit cache, and dropped in `#dropCachedCredits` — so added
capacity re-arms the alert for the rest of the month.

The dedup that turns this into exactly one notice per member per month belongs
with whoever sends the mail, and needs a cross-region marker rather than a
per-cluster one.

Part of PUT-1750; the notification half is prod's.
2026-09-08 16:59:29 -04:00
Juan Castro 0c6045ed90 feat: emit team billing events for the payment integration
The charge lives outside this repo, the way the marketplace extension cancels
Stripe subscriptions off `user.delete`. This is the trigger, and it is the
whole of what OSS owes billing.

  team.account.created    a seat exists and can be used
  team.account.disabled   it stopped, and still holds its bytes
  team.account.enabled    it resumed
  team.account.deleted    it is gone
  team.deleted            the team is gone; its accounts are not

Each carries the team uid, the affected account, and the owner's
`stripe_customer_id`. That column ships in the mysql and postgres schemas but
not sqlite, so the read is guarded and degrades to null, as `cascadeDelete`
already does.

Deleting a team emits one `team.account.disabled` per seat plus the
team event, rather than one bulk event: the accounts persist, suspended,
holding their files and their usernames. Deleting a team is not a way to
stop paying for the accounts in it.

`team.account.deleted` is captured before the row is deleted.
`jct_user_group.user_id` is ON DELETE CASCADE, so by the time a listener on
`user.delete` runs, nothing can say which team paid for the account.

`getOrgSeat` deliberately admits soft-deleted teams: their accounts still
exist, so the charge is still running.

Closes PUT-1712.
2026-09-08 16:59:28 -04:00
Juan Fernando Castro 00afea1285 feat: a seat is created, never adopted (#3720)
`addMember` refuses to turn an account that already has a password into a
team seat. No service path did this — `provisionAccount` always creates —
but the store permitted it, and the design rules out existing accounts joining
a team.

Provisioning passes the guard because it admits the account before setting its
temporary password.

There is no bypass parameter. Both HTTP suites now provision a real seat and
authenticate as it, using the same token-minting the harness uses for
`POST /login`. That surfaced something worth knowing: an unactivated seat
cannot call the API at all. Provisioning leaves `requires_email_confirmation`
set and `requireVerified` rejects it, so the suites activate the seat first —
which is the state a member is actually in when making requests.

`listMembers` and `getMembership` also return `u.uuid`, which the billing
events need in order to name the account without a second lookup.
2026-09-08 16:45:16 -04:00
dependabot[bot] 84368f8d02 chore(deps): bump otpauth from 9.5.1 to 9.5.2 (#3820)
Bumps [otpauth](https://github.com/hectorm/otpauth) from 9.5.1 to 9.5.2.
- [Release notes](https://github.com/hectorm/otpauth/releases)
- [Commits](https://github.com/hectorm/otpauth/compare/v9.5.1...v9.5.2)

---
updated-dependencies:
- dependency-name: otpauth
  dependency-version: 9.5.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 13:26:41 -04:00
Daniel Salazar 86050f131b fix: hardening events for shared kv (#3827) 2026-09-08 09:04:50 -07:00
dependabot[bot] e50f2911b6 chore(deps): bump actions/upload-artifact from 4 to 7 (#3824)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:58:37 -07:00
dependabot[bot] 414c121ae4 chore(deps): bump actions/setup-node from 4 to 7 (#3825)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 7.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:58:24 -07:00
Toshit ChawdaandClaude Opus 5 1d6d0da950 feat(puter.js): transferables in AppConnection.postMessage (#3776)
`postMessage(message, transfer)` (or `{ transfer }`) moves ArrayBuffers,
MessagePorts, streams and the rest to the target app instead of copying
them. The second argument is optional, so existing callers are unchanged.

The transfer list rides in the message body as well as the real transfer
list: structured clone's memory map keeps those objects identical to the
ones inside `contents`, which is how the desktop picks them out mid-relay
and keeps transferring them onward rather than leaving copies behind. Both
`messageToApp` paths carry it — the direct iframe relay and the connection
path that `launchApp()` between apps actually uses.

The desktop forwards the list as-is and lets postMessage judge it. A list
that arrived through the SDK was already validated by the browser on the
first hop, and validating again here would mean an allowlist of
transferable types that silently downgrades an unrecognised one to a copy.
A bad list only reaches us from an app that hand-wrote the envelope; that
throws without detaching anything, and is caught so it cannot escape
`ipc_listener` as an unhandled rejection.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-09-08 08:57:52 -07:00
dependabot[bot] 72889d72c8 chore(deps): bump undici from 7.29.0 to 7.29.1 (#3821)
Bumps [undici](https://github.com/nodejs/undici) from 7.29.0 to 7.29.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.29.0...v7.29.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.29.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:52:50 -07:00
Nariman Jelveh 8a6daee9ab feat: let extensions customize recommended apps 2026-09-07 16:44:41 -07:00
404oops e980b131a0 Merge pull request #3815 from HeyPuter/filipkujundzic/ai-vision-canonical-media-parts 2026-09-08 00:54:41 +02:00
404oopsandClaude Fable 5.1 1cb5472c4c fix(ai): canonicalize image parts once and translate them per provider
The SDK shorthand sends `{ image_url: { url } }` with no type. Claude
forwarded it untouched (Anthropic: "content.1.type: Field required"),
the OpenAI/Azure Responses providers sent a Chat Completions part the
Responses API rejects, and Mistral emitted snake_case `image_url` where
its SDK validates `imageUrl`. All three only appeared to work because
the fallback loop re-served them through OpenRouter.

- utils/mediaParts: canonicalise every inbound media part (untyped
  shorthand, bare string URLs, Responses input_image, Anthropic image
  blocks, Gemini inline_data) to the Chat Completions form in the
  driver, so providers translate from one shape
- Claude → Anthropic image blocks (url / base64 sources), copy-on-write
- Responses processor → input_image with a string URL and detail, and
  copies messages instead of mutating the caller's objects
- Mistral → camelCase imageUrl; the test that asserted the old shape
  is corrected
- driver: 400 up front when the catalog says the model has no image or
  video input; resolve puter_path parts per attempt for providers
  without their own upload path (Claude keeps the Files API)
- move the Moonshot http→data-URL inliner to utils/inlineImages and
  apply it to Grok-via-Azure and Gemini, whose upstreams cannot fetch
  http image URLs

Fixes #3409

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-07 15:23:40 +02:00
Daniel Salazar 927317bc4e fix: events hardening (#3814) 2026-09-06 22:44:07 -07:00
Daniel Salazar a441e7f748 fix: harden events (#3813)
* fix(events): rename carries from, one retry on deploy-timeout, forward-path counters

- FSService.rename passes the pre-rename path so a folder subscription
  sees op move with from, like a real move does
- deploy-timeout from the dispatcher gets one retry with the deployed
  header and no second upload (ALREADY_DEPLOYED_MISS_REASONS)
- OTel counters events.forward.sent/received and events.single.attempt
- cross-app KV subscribe error hints at the three-segment parse
- docs: lease is 60 s, kv prefix example is fully qualified, move covers
  rename

* feat(events): forward session subscriptions across regions, fast bumps, worker session cleanup

- session (onLocal) subscriptions now receive writes committed in other
  regions: a transition-maintained remote-watch index (ev:sc / ev:rw),
  watch/event forward items, replay through dispatchForwarded against
  session rows only; events.forwardSession=false is the kill switch
- subscription and presence generation bumps also ride the addressed
  forward channel (kind bump) so a peer sees a new durable row within
  a queue window; the webhook fan stays as backstop
- workers.destroy revokes every holder's events:handlers session; app
  deletion reaps the app's rows, backlog and handlers and revokes the
  sessions; an hourly sweep revokes sessions whose app is gone
- docs: cross-region latency, per-region caps footnote, session
  lifecycle
2026-09-06 16:44:07 -07:00
dependabot[bot] 2784feb47d chore(deps): bump fast-uri from 3.1.5 to 3.1.7 (#3810)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.5 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.5...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-06 16:43:02 -07:00
Daniel Salazar 2f70cd1a14 feat(email): sendTransactional alias and streaming file inputs (#3812)
- puter.email.sendTransactional is the new name; send stays as a
  deprecated alias with the same arguments and result
- fileInput: openFileInputStream / resolveFileInputEntry expose the
  ACL-checked FS read as a stream; loadFileInput wraps them
- EmailAttachment accepts a `path` the transport streams on its own
2026-09-06 16:41:02 -07:00
Felix-Ayush b1d3a7c191 fix(puter-js): require prompt in ai.txt2img (#3804)
txt2vid rejects missing prompts client-side with prompt_required, but txt2img sent the request to the backend. Add the same guard before the driver call.
2026-09-06 16:39:35 -07:00
Nariman Jelveh 092a9245ff Fix PDF thumbnail worker loading from CDN bundles 2026-09-06 14:31:42 -07:00
Nariman Jelveh e6cedde732 Add resilient PDF thumbnails to GUI uploads (#3809)
* Add resilient PDF thumbnails to GUI uploads

* Keep GUI image thumbnails working with SDKs lacking the callback context

The desktop loads puter.js from js.puter.com by default, and the SDK
there predates the thumbnail callback context, so passing the PDF
generator made every image upload lose its thumbnail until the SDK
deploys. Fall back to the SDK's bundled image generator whenever the
running SDK passes no usable context, and cover both paths in the unit
and browser tests.

* Ignore preparation failures that land after an upload is cancelled

Cancelling during preparation already rejects the upload and fires the
abort callback. If the step that was in flight then fails, such as a
dropped directory that cannot be read, the error callback also fired and
the GUI showed an upload error for an upload the user had just
cancelled. Skip error reporting once preparation has been aborted.

* Give each PDF thumbnail worker four seconds

The per-PDF budget covers downloading PDF.js as well as rendering, and
the first PDF of a session on a slower connection ran out of time
before its assets had even loaded. Four seconds fits that first load on
ordinary connections while staying under the five-second batch cap, so
one stuck PDF still leaves the rest of the batch a chance.
2026-09-06 13:51:02 -07:00
404oops 9b4d8f11d4 Merge pull request #3801 from HeyPuter/filipkujundzic/remove-retired-imagen-moonshot-models
fix(ai): remove retired Imagen 4 and legacy Moonshot models
2026-09-06 13:26:14 +02:00
dependabot[bot] bf4e810f9c chore(deps-dev): bump browserstack-local (#3808)
Bumps [browserstack-local](https://github.com/browserstack/browserstack-local-nodejs) from 1.5.5 to 1.5.9.
- [Release notes](https://github.com/browserstack/browserstack-local-nodejs/releases)
- [Commits](https://github.com/browserstack/browserstack-local-nodejs/compare/v1.5.5...v1.5.9)

---
updated-dependencies:
- dependency-name: browserstack-local
  dependency-version: 1.5.9
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:43:42 -07:00
dependabot[bot] 89a4947f68 chore(deps): bump webpack from 5.101.3 to 5.110.3 in /src/puter-js (#3800)
Bumps [webpack](https://github.com/webpack/webpack) from 5.101.3 to 5.110.3.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Changelog](https://github.com/webpack/webpack/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack/compare/v5.101.3...v5.110.3)

---
updated-dependencies:
- dependency-name: webpack
  dependency-version: 5.110.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:47 -07:00
dependabot[bot] 9ebe1092f1 chore(deps): bump dependabot/fetch-metadata from 2 to 3 (#3763)
Bumps [dependabot/fetch-metadata](https://github.com/dependabot/fetch-metadata) from 2 to 3.
- [Release notes](https://github.com/dependabot/fetch-metadata/releases)
- [Commits](https://github.com/dependabot/fetch-metadata/compare/v2...v3)

---
updated-dependencies:
- dependency-name: dependabot/fetch-metadata
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:22 -07:00
dependabot[bot] 1caf3e121f chore(deps): bump actions/checkout from 3 to 7 (#3764)
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:13 -07:00
dependabot[bot] 3620f7bd85 chore(deps): bump docker/build-push-action from 5 to 7 (#3765)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:39:57 -07:00
dependabot[bot] e48802252e chore(deps): bump actions/download-artifact from 4 to 8 (#3767)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:39:21 -07:00
Daniel Salazar da65b7f569 feat: the invoking backend deploys an events worker the dispatcher cannot find (#3807)
The dispatcher's rehydrate callback reaches whichever backend answers the
API's public hostname. A backend with the runtime flag on that is not behind
that hostname, or the only one in the fleet with it on, could never get its
scripts deployed that way — the callback answered "disabled". The invoking
backend already knows the app and script, so on a dispatcher miss it deploys
the set itself and retries once, telling the dispatcher to skip its callback
and negative cache. The callback stays the path for evicted scripts.
2026-09-05 14:40:44 -07:00
404oopsandClaude Fable 5.1 ded3d3695b fix(ai): remove retired Imagen 4 and legacy Moonshot models
Google shut down imagen-4.0-fast/standard/ultra in the Gemini API on
2026-08-17; live calls now return 404 "not found ... or is not supported
for predict". Drop the three entries, the generateImages code path that
only they used, and their tests; the integration test moves to
gemini-2.5-flash-image.

Moonshot's live /models listing and the Kimi pricing docs now carry only
kimi-k3, kimi-k2.7-code(-highspeed) and kimi-k2.6. Drop kimi-k2.5 and the
whole moonshot-v1-* family and repoint the unit and integration tests at
current models. kimi-k2.5 still resolves through OpenRouter.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-05 22:22:16 +02:00
dependabot[bot] e451d507ad chore(deps): bump lodash from 4.17.21 to 4.18.1 in /src/puter-js (#3797)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.21 to 4.18.1.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.21...4.18.1)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.18.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 12:21:36 -07:00
dependabot[bot] 7099cadc6f chore(deps): bump shell-quote from 1.8.1 to 1.10.0 in /src/puter-js (#3774)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.8.1 to 1.10.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.8.1...v1.10.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.10.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 12:19:35 -07:00
dependabot[bot] 73cb60f5ae chore(deps): bump docker/login-action from 3 to 4 (#3762)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:58:57 -07:00
Daniel Salazar e478087260 fix: keep matrix legs of test/coverage jobs always instantiated (#3799)
Skipping the whole matrix job via a job-level `if` collapses it to a
single check with the unevaluated name template ("test (\${{
matrix.artifact }})") instead of expanding to "test (base)"/"test (pr)",
since there's no matrix context to fill in when the job never runs. That
check never matches a required-status-check pinned to "test (pr)", so
PRs that skip backend/puterjs testing block forever waiting on a check
that will never be reported. Gate the steps instead so the matrix always
expands with stable names, doing no real work when irrelevant.
2026-09-04 23:58:12 -07:00
dependabot[bot] b6fb4f3cae chore(deps-dev): bump @eslint/js from 9.39.4 to 9.39.5 (#3791)
Bumps [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js) from 9.39.4 to 9.39.5.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/commits/v9.39.5/packages/js)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.39.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:56:03 -07:00
dependabot[bot] 61566431cc chore(deps): bump fs-extra from 11.3.5 to 11.3.6 (#3785)
Bumps [fs-extra](https://github.com/jprichardson/node-fs-extra) from 11.3.5 to 11.3.6.
- [Changelog](https://github.com/jprichardson/node-fs-extra/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jprichardson/node-fs-extra/compare/11.3.5...11.3.6)

---
updated-dependencies:
- dependency-name: fs-extra
  dependency-version: 11.3.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:55:31 -07:00
dependabot[bot] d3baf0ef7e chore(deps): bump js-yaml from 4.3.1 to 4.3.2 (#3792)
Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.3.1 to 4.3.2.
- [Changelog](https://github.com/nodeca/js-yaml/blob/4.3.2/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/4.3.1...4.3.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:54:54 -07:00
dependabot[bot] 4c3fe8ef7b chore(deps-dev): bump @vitest/coverage-v8 from 4.1.10 to 4.1.11 (#3793)
Bumps [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) from 4.1.10 to 4.1.11.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.11/packages/coverage-v8)

---
updated-dependencies:
- dependency-name: "@vitest/coverage-v8"
  dependency-version: 4.1.11
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:53:49 -07:00
dependabot[bot] c164b5724f chore(deps): bump nodemailer from 9.0.3 to 9.0.6 (#3795)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.0.3 to 9.0.6.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.0.3...v9.0.6)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 9.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:52:48 -07:00
Daniel Salazar bc9cb2d7e7 fix: withdrawing background consent revokes the app's events session (#3777)
* fix: withdrawing background consent revokes the app's events session

A background handler runs as a worker session for the subscriber and app.
Revoking `events:background` or uninstalling the app suspended the
subscriptions but left that session valid, so a token a handler had copied
out kept working until the user found the row in the sessions list. The
revocation settle now revokes the session too; the next consented delivery
mints a fresh one.

* fix: cleanup docs
2026-09-04 21:10:27 -07:00
dependabot[bot] 5669a2d960 chore(deps): bump fastify from 5.10.0 to 5.12.3 (#3749)
Bumps [fastify](https://github.com/fastify/fastify) from 5.10.0 to 5.12.3.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/commits)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 21:01:11 -07:00
dependabot[bot] 50aea0cc64 chore(deps-dev): bump vite from 8.1.3 to 8.1.5 (#3790)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.1.3 to 8.1.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.1.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.1.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:36 +00:00
dependabot[bot] c67f1c3653 chore(deps): bump uuid from 14.0.1 to 14.0.2 (#3789)
Bumps [uuid](https://github.com/uuidjs/uuid) from 14.0.1 to 14.0.2.
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/uuidjs/uuid/compare/v14.0.1...v14.0.2)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 14.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:13 +00:00
dependabot[bot] 78f44c8f8c chore(deps-dev): bump vitest from 4.1.10 to 4.1.11 (#3788)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 4.1.10 to 4.1.11.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.11/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-version: 4.1.11
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:03 +00:00
dependabot[bot] 45ee0ef102 chore(deps-dev): bump @types/node from 24.13.2 to 24.13.3 (#3787)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.2 to 24.13.3.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:52:45 +00:00
dependabot[bot] c4da663533 chore(deps): bump sharp from 0.35.3 to 0.35.4 (#3786)
Bumps [sharp](https://github.com/lovell/sharp) from 0.35.3 to 0.35.4.
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.35.3...v0.35.4)

---
updated-dependencies:
- dependency-name: sharp
  dependency-version: 0.35.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:52:31 +00:00
dependabot[bot] 69b5901278 chore(deps): bump mysql2 from 3.22.5 to 3.22.6 (#3784)
Bumps [mysql2](https://github.com/sidorares/node-mysql2) from 3.22.5 to 3.22.6.
- [Release notes](https://github.com/sidorares/node-mysql2/releases)
- [Changelog](https://github.com/sidorares/node-mysql2/blob/master/Changelog.md)
- [Commits](https://github.com/sidorares/node-mysql2/compare/v3.22.5...v3.22.6)

---
updated-dependencies:
- dependency-name: mysql2
  dependency-version: 3.22.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:54 +00:00
dependabot[bot] 99fb4c2dad chore(deps-dev): bump prettier from 3.9.4 to 3.9.6 (#3783)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.4 to 3.9.6.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.4...3.9.6)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:41 +00:00
dependabot[bot] 25b6a80cb5 chore(deps): bump libphonenumber-js from 1.13.6 to 1.13.12 (#3782)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.6 to 1.13.12.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/commits/v1.13.12)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:34 +00:00
dependabot[bot] b334961f75 chore(deps): bump @esbuild/linux-x64 from 0.25.11 to 0.25.12 (#3781)
Bumps [@esbuild/linux-x64](https://github.com/evanw/esbuild) from 0.25.11 to 0.25.12.
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG-2025.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.11...v0.25.12)

---
updated-dependencies:
- dependency-name: "@esbuild/linux-x64"
  dependency-version: 0.25.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:10 +00:00
dependabot[bot] aa95a76253 chore(deps): bump @prelude.so/js-sdk from 0.12.0 to 0.12.1 (#3780)
Bumps [@prelude.so/js-sdk](https://github.com/prelude-so/web/tree/HEAD/packages/js-sdk) from 0.12.0 to 0.12.1.
- [Changelog](https://github.com/prelude-so/web/blob/main/packages/js-sdk/CHANGELOG.md)
- [Commits](https://github.com/prelude-so/web/commits/HEAD/packages/js-sdk)

---
updated-dependencies:
- dependency-name: "@prelude.so/js-sdk"
  dependency-version: 0.12.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:50:51 +00:00
dependabot[bot] 85bf595d2d chore(deps): bump highlight.js from 11.11.1 to 11.11.2 (#3779)
Bumps [highlight.js](https://github.com/highlightjs/highlight.js) from 11.11.1 to 11.11.2.
- [Release notes](https://github.com/highlightjs/highlight.js/releases)
- [Changelog](https://github.com/highlightjs/highlight.js/blob/main/CHANGES.md)
- [Commits](https://github.com/highlightjs/highlight.js/compare/11.11.1...11.11.2)

---
updated-dependencies:
- dependency-name: highlight.js
  dependency-version: 11.11.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:50:38 +00:00
Daniel Salazar 2f15e40a77 ci: add CODEOWNERS, restrict Dependabot to patch bumps, make test workflows always-run (#3778)
- CODEOWNERS: default owners @Salazareo @ProgrammerIn-wonderland @jelveh
  @jfcastro92, plus @reynaldichernando for src/docs.
- dependabot.yml: ignore semver-minor/major npm updates, patch only.
- backend-tests.yaml / puterjs-tests.yaml: trigger unconditionally and gate
  the real work behind a dorny/paths-filter job instead of a workflow-level
  path filter, so the checks always report (skipped when irrelevant) and
  can be marked required. Also pins matrix job names to the artifact label
  instead of the branch ref, which previously made the check name change
  per PR and unusable as a required check.
2026-09-04 20:49:14 -07:00
Daniel Salazar b784b51cf3 fix: harden the events stack for flag-on (#3752)
* fix: harden events dispatch, single delivery and KV share handles

Dispatch: a filtered subscription used the anchor path stored at subscribe
time, so renaming or moving the anchor folder silently ended its deliveries;
dispatch now resolves the anchor's live path from the event's own ancestor
chain. A move out of a watched folder now reaches that folder's subscribers,
with `from` only for rows that watched the source side. Gap markers are
authorized like deliveries and coalesced per subscription and subject instead
of fanning per lost event. Session subscriptions: the per-socket cap decides
on the write, not before it; an orphaned watched-set token heals on refresh;
durable rows keep their watch window when a session subscribe touches the
same keys. `self` is false when the acting user is unknown.

Single delivery: a subscription in backoff or suspended with a backlog pinned
the sweeper's head and starved everyone behind it — the sweep now defers it.
Only a settled handler run bills a delivery. A socket-only account row no
longer wedges after two attempts nobody received. The lease is twice the
handler timeout; remote candidates have their own attempt counter; the region
depth reconcile runs once a minute region-wide with a bounded scan.

KV share handles: a grantee no longer sees the owner's namespace and absolute
prefix on the subscribe answer or listing, nor in the delivery token; revoking
a wider handle retires the handles it covers; minting the same handle twice
returns the existing one, after the delegation check; a row whose event
cannot be re-based onto its handle is dropped rather than delivered raw.

* fix: presence survives replication, long sessions and region churn

One presence item per (user, app) with per-region map fields lost a region
whenever two regions joined inside the replication window, and nothing ever
put it back. Presence is now one item per (user, app, region): each region
writes only its own, a leave or repair retires it conditionally on its own
write stamp, and a read is a prefix query. Items carry a 48 h ttl refreshed by
a claim-gated write off the existing socket renew path, at most once per
12 h, so a tab that stays connected keeps its region in the row. A region
that answered "no socket" or completed a leave releases a shared pin, so a
reconnect on another node rejoins and a flapping client cannot force a
replicated write per cycle. Cached rows expire after a minute; unaddressable
region names are filtered and pruned; relayed acks settle under a bounded
concurrency; the forward queue is bounded in bytes as well as items.

* feat: indexes for the event_subscriptions hot queries

Handler publish, remove and listing, and the hourly expiry and suspension
sweeps, all scanned `event_subscriptions`. Adds (app_uid, handler_name),
(expires_at) and (suspended_at, id), guarded on every engine. Existing
migrations: the postgres widens are now guarded so a boot does not take an
exclusive lock for a no-op, the kv_share_handles grantee FK gets an index,
the sqlite notification rebuild is transactional and idempotent.

* fix: notification writes go through the registry

The driver's `create` bypassed the type registry, producing uncatalogued
rows with no size bound; it now requires a registered type, caps the payload,
and answers 400 rather than 500 for a bad one. `mark_acknowledged` emits the
ack other tabs listen for, and only when a row was actually changed.

* fix: the handler scanner, unsubscribe, and the in-tab handler environment

The free-variable scanner skipped arrows inside a declaration's initializer,
so `const ids = event.items.map(x => x.id)` was refused, and treated a name
after a comma in a nested initializer as bound, so a real free variable slipped
through to fail on first delivery. `unsubscribe()` now drops the durable
routing entry so the events socket can close. A broadcast handler running in
the tab gets `user` and `fetch` like the worker gives it. `single` without
a handler name is refused before the round trip.

* docs: events limits, error codes and the background-workers section

Retention is deployment-configured rather than a fixed 14 days, and the
template no longer ships it armed. Documents `events_terminal`, the two
per-event gap reasons, the subject length and listing caps, the `from` field
on moves, and the handle-relative anchor. The sessions manager hides the
background-workers section when the server has none to show.

* feat: a background handler acts as the app does for its user

A handler's `user` was a five-minute access token scoped to the subscription's
`list` grant, which could stat the changed file but not read it, and could
not reach the app's KV or AppData — so an app told that a file was written
could do nothing with it. It now runs with the same authority the app has for
that user in a tab: an app-under-user worker session, one row per (user, app)
named `events:handlers`, visible and revocable in the sessions list. The
`events:background` consent is what authorizes running it unattended, and is
re-checked before every mint.

The wider token exposed two things: puter.js opens a filesystem socket the
moment it has a token, which would have parked the isolate in the app's own
delivery room and steered deliveries at it; the events client now opts out of
sockets (and the per-open bookkeeping) before construction, and is memoized
per token in the isolate. And four filesystem operations assumed a socket
exists; they no longer do.
2026-09-04 17:32:57 -07:00
Daniel Salazar f30baa2a1c feat: the per-app events worker runtime (#3697)
* feat: bake published handlers into a generated events worker

* feat: deploy and address the per-app events worker behind a flag

* test: single delivery end to end through a real local worker

* feat: events workers run their own runtime, in their own namespace

An events worker was being deployed as an ordinary worker: default dispatch
namespace, a `subdomains` row, the router preamble, and an app-scoped worker
token baked in. The public dispatcher resolves any script in that namespace
straight off the hostname, so the worker answered at `<name>.puter.work`, and
the only thing in front of it was an unguessable name plus a check that a
`puter-auth` header was present — which the router never validates. Anyone who
learned the hostname could run an app's handlers with a body of their choosing,
in an isolate holding the owner's token as `me`.

Instead:

- Handlers run on their own runtime (`src/worker/src/events-runtime.js`), which
  provides no `router` and no `me`, owns the single invoke route, and hands a
  handler only `{ event, ctx, user, fetch, ack }`. `user` is built from the
  invocation's delivery token, so a handler acts as the subscriber whose
  delivery it is and nothing wider. The preamble build emits one bundle per
  runtime; the shared half of the template is now included by both.
- The deploy target carries the runtime to prepend, the source to deploy, and
  whether to mint a worker token at all, so an events worker deploys into the
  `events` dispatch namespace from generated source with no token binding, no
  `subdomains` row, and no claim on the owner's worker quota or worker list.
- An invocation carries a key derived from the deployment secret and the script
  name, bound as a secret and checked in constant time inside the isolate,
  which reads it once and drops it before handler code runs.
- Scripts are named after the handler set they contain, so publishing writes
  rows and deploys nothing: a set is deployed the first time a delivery needs
  it, and a changed set is a new script rather than an overwrite of a running
  one. Publish responses keep the shape they had before the runtime existed.
- Invocations reach a worker only through the events dispatcher, which has no
  zone route and requires the internal secret; the backend's own deploy path is
  the rehydrate route the dispatcher calls on a namespace miss. Locally there is
  no dispatcher, so the controller hands the service an in-process transport
  that deploys on miss itself.

The SDK stops allowlisting `puter` as a handler global — a handler that reaches
for an ambient SDK is now refused at publish time, naming `user` instead, rather
than passing the scan and failing on its first delivery.

Requires `events.workerNamespace`, `events.dispatcherUrl` and
`events.internalSecret`; without them nothing is addressable and background
deliveries stay retriable, as they did with the runtime off.

* fix: a handler's delivery token gets through the read routes

An events handler acts as the subscriber through the access token its
invocation carried, but every FS read route refused scoped access tokens
outright, so `user.fs.stat(event.path)` — the design's own example — answered
403 inside the worker. The read-side routes now admit them; the ACL each
handler already runs intersects the token's grant with its issuer's, which is
the check that keeps a token to what it was minted for. The end-to-end suite
asserts the stat from inside the isolate.

* fix: shorthand-method handlers publish as functions

`{ ingest({ event }) { … } }` stringifies without the `function` keyword, so
its source is not an expression and the events worker baked it as a broken
stub — every delivery a retriable 500 until the subscription suspended, with
nothing at publish time to say why. The SDK now gives a shorthand method the
keyword before hashing and sending; getters, setters and computed names are
left for the server-side check to refuse.

* feat: an app's events worker is listable and destroyable

An app with published handlers has an events worker, and hosted deployments
bill it monthly per app, so its owner needs to see it and be able to take it
down. The core announces the lifecycle on the bus — `events.worker.create`
when an app's first handler is published, `events.worker.destroy` when its last
one goes — with the owner as the actor, so pricing can plug in from outside.
`GET /events/workers` lists the caller's workers (paginated, with the script
each set deploys as) and `POST /events/workers/destroy` removes every handler
of an app under the same owner scoping as the handler routes, suspending the
subscriptions bound to them. `puter.events.workers.list/destroy` in the SDK,
a docs page, and a 5 MB cap on an app's combined handler source
(`events_worker_too_large`) so a set that publishes can always deploy.

* fix: harden the events worker runtime for production

- A 4xx is terminal only when it carries the handled marker the runtime (and
  the dispatcher) stamp on every answer that came from a script; an unmarked
  4xx — an edge 404 for a wrong dispatcher hostname, a WAF page — stays
  retriable and is logged, once per script per minute, with the runtime's
  reason header.
- Script names are scoped to this backend's exposed API origin, so two
  backends sharing a namespace never resolve one script with the wrong
  endpoint binding or key. Shape unchanged.
- Each handler is validated in the exact context it is emitted into and the
  whole generated file is compiled once; a source that would break the script
  marks every handler broken instead of deploying a SyntaxError.
- Locally, events scripts live under their own registry key: the public local
  worker host cannot reach them and an ordinary worker cannot take their name.
- A suspended or deleted app owner stops invocations; deploys are throttled
  per app per hour; in-flight deploys are keyed by app and script; the
  upstream deploy call times out; the generated source is size-capped with a
  margin over the publish cap; boot fails when the runtime is on but its
  preamble is not built. Byte-length secret compare, appUid shape check,
  dispatcher URL prefix preserved, wider connection pool.

* feat: background workers are listed in the sessions manager

A user paying for an app's events worker needs somewhere to see it and take it
down. The sessions manager gets a section listing the apps that run event
handlers in the background, with a Destroy action that removes their published
handlers.
2026-09-04 17:32:57 -07:00
dependabot[bot] dfab3d50cd chore(deps-dev): bump browserslist from 4.28.5 to 4.28.9 (#3759)
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.28.5 to 4.28.9.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.28.5...4.28.9)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-version: 4.28.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:20:29 +00:00
dependabot[bot] 9daaec4ec4 chore(deps): bump qs from 6.15.3 to 6.16.0 (#3757)
Bumps [qs](https://github.com/ljharb/qs) from 6.15.3 to 6.16.0.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.15.3...v6.16.0)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.16.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 16:17:36 -07:00
dependabot[bot] b7effe1e01 chore(deps): bump fast-uri from 3.1.0 to 3.1.7 in /src/puter-js (#3756)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.0 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.0...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 16:17:08 -07:00
Daniel Salazar 2a68e7aafb ci: add Dependabot config and patch auto-merge workflow (#3760)
Weekly npm/docker/github-actions updates plus vulnerability PRs;
patch-level Dependabot PRs auto-merge, minor/major stay manual.
2026-09-04 16:16:04 -07:00
Daniel Salazar 3355190a17 feat: add astra to models (#3755) 2026-09-04 15:56:53 -07:00
Nariman Jelveh 6ebbc6ae0c feat(gui): let users reposition and zoom a new profile picture (#3754)
* feat(gui): let users reposition and zoom a new profile picture

Picking a photo in the dashboard's Account tab used to stretch the whole
image into a 150x150 square, so anything that was not already square came
out distorted and off-center. The pick now opens an adjust step: a
dashboard-style modal (centered card on desktop, bottom sheet on phones)
where the user drags to reposition and zooms with the slider, pinch, or
wheel before saving. The saved result is the same 150x150 PNG as before.

Geometry lives in profilePictureCrop.js with unit tests; the modal owns the
DOM and pointer handling.

* fix(gui): stop double-encoding the crop modal's hint and frame label

i18n() already HTML-encodes its output, so wrapping it in html_encode()
again turned any apostrophe or ampersand in a translation into a literal
"&apos;" / "&amp;" on screen. Also puts the new profile_picture_* keys
in alphabetical order.

* fix(gui): let the crop frame take focus on click so arrow keys work after a drag

pointerdown's preventDefault() also cancels the click-to-focus that a
mousedown would have done, so after dragging the photo the arrow keys and
+/- went to the dialog container and did nothing. The frame now focuses
itself on pointerdown. Focus that arrives by pointer draws no ring; the
first key press lifts that so keyboard users still see where they are.

Also ignores secondary mouse buttons and treats a lost pointer capture as
a release so a pointer can't stay stuck in the gesture map.

* fix(gui): return focus to the avatar when the crop modal closes

The modal remembered document.activeElement to restore focus later, but
at that moment focus sits inside the file picker, which closes right
after -- so on Save, Cancel or Escape focus fell to <body>. The Account
tab now names its avatar as the place focus returns to, and the avatar
becomes a real button (role, tabindex, label, Enter/Space) so it can
hold that focus and be reached from the keyboard at all.

* fix(gui): announce the crop zoom as a magnification, not a 0-100 slider value

Screen readers read the range input's raw value, which maps to nothing a
user can picture. aria-valuetext now carries the zoom factor (1.0x-4.0x)
and follows every zoom source: slider, buttons, keys, wheel, pinch.
2026-09-04 14:38:55 -07:00
404oopsandClaude Opus 5 7248951410 fix(ai): make chat fallback reach streamed Claude calls and rank Azure explicitly (#3743)
* fix(ai): make chat fallback reach streamed Claude calls and rank Azure explicitly

- ClaudeProvider opens the upstream stream and awaits its connection
  before returning the populator, so an overloaded or rate-limited route
  throws from complete() and reaches the driver's fallback loop instead
  of surfacing as an error frame on a 200
- the OpenAI-compatible chat and completions routes only pin a provider
  when the caller sent one, so they get the same preferred healthy route
  puter.js callers do and unhealthy-route skipping applies to their
  first attempt
- Azure is ranked ahead of the vendors it fronts by an explicit tier in
  modelRouting rather than a price tie plus registration order
- drop Together's synthetic always-failing model-fallback-test-1 entry
- test that a 4xx leaves a route in rotation while a 503 marks it

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai): surface swallowed Claude stream errors, keep compat-route defaults

Review follow-ups on the fallback work.

- The pre-created event iterator only receives an error if a reader is
  already waiting on it, so a failure landing between the connect and the
  populator's first pull ended the stream cleanly — truncated content
  billed and reported as a success. Rethrow when the stream is errored.
- A refused stream deleted its Anthropic uploads but left the caller's
  message parts pointing at those file ids, so the fallback route was
  handed handles it cannot resolve. processPuterPathUploads now returns a
  restore() that both failure paths call.
- The OpenAI-compat routes keep pinning OpenAI when the caller sends no
  model at all, so the default model stays put instead of moving to
  Azure's.
- Say why /openai/v1/responses and /anthropic/v1/messages stay pinned:
  each translates one provider's native shape by hand.
- PREFERRED_PROVIDERS is unexported and its doc now states the rank is
  unconditional; the duplicated hidden-model list is one constant.

Co-Authored-By: Claude Opus 5 <[email protected]>

* fix(ai): undo the puter_path rewrite by field instead of snapshotting the part

Copying the content part kept whatever the caller sent on it — a large
inline `source` or `text` alongside `puter_path` — reachable until the
request ended, where overwriting the field used to make it garbage right
away. The only fields this function writes are `type`/`source` on success
and `type`/`text` on failure, and the fallback uploader keys off
`puter_path` alone, so restore undoes those three by name.

Co-Authored-By: Claude Opus 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-04 11:10:05 -07:00
Reynaldi Chernando 2966cee6b0 explicit mention of kv and fs being user scope and workers for backend (#3751) 2026-09-05 01:02:22 +07:00
Reynaldi Chernando d8735d950e fix kv set value type (#3750) 2026-09-05 00:30:17 +07:00
Reynaldi Chernando 48abf73d1a kv add array example (#3748) 2026-09-05 00:22:46 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 11c5a7ce52 dynamic workers docs (#3747)
* dynamic workers docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-04 22:57:11 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI b6876c50bf PUT-1477 Add fs commands to cli (#3746)
* Add fs commands to cli

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-04 21:31:33 +07:00
Daniel Salazar 6d7be8bd6a docs: kv key layout is the ACL boundary (PUT-1689) (#3693) 2026-09-03 19:43:28 -07:00
Daniel Salazar b451d05d10 feat: app-minted kv share handles (PUT-1688) (#3692) 2026-09-03 19:43:27 -07:00
Daniel Salazar f139588f89 feat: revocation settles kv handles (PUT-1687) (#3691) 2026-09-03 19:43:27 -07:00
Daniel Salazar ac5446f877 feat: cross-user KV share grants and handles (PUT-1686) (#3690) 2026-09-03 19:43:27 -07:00
Daniel Salazar adde6c3a29 feat: notification GUI rides the events surface (PUT-1685) (#3689)
Widen the notif: match filter and fetch scope for a session's own
generic developer/app-user subscribe: today it pins ref to the
session's own uuid, so a row naming an app (handler-suspension
notices, app-bound worker deploys) never matches live and never
replays on reconnect, even though the audience predicate already
grants the holder every such row it owns. The predicate is the
authority and already reruns per row/page after the match, so
widening the filter to it (account is unaffected — it never names an
app) adds no exposure.
2026-09-03 15:39:22 -07:00
Daniel Salazar fe5f81e52e feat: scoped app access to notifications (PUT-1684) (#3688)
An actor holding an app reads the `app-user` rows naming that app, plus its
`developer` rows when the holder owns it. `account` rows reach no app, and a
slice an actor may not see comes back empty rather than refused. The audience
predicate becomes the enforced read path in the same change that lifts the
blanket app-actor 403, layered behind an audience/app_uid SQL scope; two-segment
`notif:` subjects expand server-side from the actor's own app, so an app can
never name another app's uid.

No feature flag: `audience` defaults to 'account', so every pre-registry row is
default-denied to app actors and the backfill can only narrow.
2026-09-03 15:39:22 -07:00
Daniel Salazar 796683a133 feat: missed-event fetch and notification fold-in (PUT-1681) (#3687) 2026-09-03 15:39:22 -07:00
Daniel Salazar eb8f497e9f feat: presence and cross-region event forwarding (PUT-1679) (#3686)
* feat: presence and cross-region event forwarding (PUT-1679)

* fix: fan cache bumps to sibling nodes and stop the forward shed cascading (PUT-1679)

`outer.events.generationBumped` and `outer.events.presenceBumped` rode
`outer.*`, which the broadcast service only webhooks to peer regions;
only `outer.pubsub.*` also fans over Redis to a region's other nodes.
Both caches are per-process maps with no expiry, so a bump landing on
one node left its siblings stale until that user's next transition.
Renamed onto `outer.pubsub.events.*`; the listeners already accept the
`from_outside` copy the Redis re-emit carries.

`PeerForwardQueue.push` called `onOverflow` synchronously and the
handler pushed markers straight back, each of which re-tripped the
bound and shed the next item: one item over a 5000 bound recursed ~2200
deep, threw a RangeError, and turned ~2200 queued deliveries into gap
markers. It also re-summed `bytes` over the whole queue per drop. The
handler now returns its markers and the queue appends them past the
bound check, sheds deliveries before markers, keeps one pending marker
per (peer, subscription), and subtracts bytes per dropped item.
2026-09-03 15:39:21 -07:00
Daniel Salazar 32a838d2e2 fix: one ** per match pattern and one * per segment (#3736)
`compileMatch` turned every `*` into an unbounded `[^/]*`, so a subject
like `fs:~/x/*a*a*a*a*a*a*a*a*a*a*z` tested against a 24-char filename
with no `z` — both named by the same subscriber — cost the regex engine
C(34,10) splits before failing: 1.0 s per event at ten stars, 3.7 s at
eleven, on the fs-write dispatch path, inside the 256-character cap.

With one `*` per segment the delimiters pin each star and a wrong split
dies in one step; the single `**` is the only choice point left, so the
worst allowed shape is O(depth × length): 0.4 ms at depth 60 of 200-char
segments. Every documented pattern (`*.png`, `**/build.log`, `**/*.png`,
`dir/**`, `report-?.csv`) stays valid; `*a*`, `a**b` and `**/x/**` are
refused with `invalid_subject_pattern`. The rule is stated where the
syntax is introduced, on the limits page, and in onLocal's error table.
2026-09-03 15:08:49 -07:00
404oops 1b777cd760 Merge pull request #3730 from HeyPuter/filipkujundzic/put-1620-video-generation-timeouts-return-500-and-page-on-call
fix: stop video generation timeouts and failures from paging (PUT-1620)
2026-09-03 22:54:58 +02:00
Daniel Salazar 799fc4ac3c feat: send app icons as a subdomain URL plus an API fallback (#3734)
App payloads carried only the /app-icon endpoint URL, which 302s to the
icons hosting subdomain. Networks that mangle that redirect render no icon
at all, and every icon load pays a round trip for the hop.

Ship the direct subdomain URL as `iconCdnUrl` alongside it (taskbar items,
installedApps, recent/recommended launch apps, suggested apps), and have the
GUI load that first with the endpoint URL as a one-shot retry - desktop
taskbar, start menu, dashboard app grid and recents. Only rows whose `icon`
column is already an http(s) URL get one: a data: column means the resize
pipeline has not written anything to the subdomain yet.

Also folds the four copies of the generated-size list into one exported
APP_ICON_SIZES.
2026-09-03 13:32:55 -07:00
Juan Fernando Castro d855315db8 Merge pull request #3714 from HeyPuter/juancastro/put-1708-26-teamcontroller-and-audit-writes
🏗️ PUT-1708 + PUT-1709 + PUT-1743: TeamController, teams_enabled, and the isolation suite
2026-09-03 16:21:02 -04:00
Juan Castro 9bc2cd741c feat: add TeamController, the teams_enabled flag, and the isolation suite
Covers PUT-1708, PUT-1709 and PUT-1743.

Twelve routes, every one setting requireUserActor -- that option is what
installs requireAuthGate, requireVerifiedAccount and requireNonAccessTokenGate,
because server.ts derives `needsAuth` from the route options. Reads need it as
much as writes: without an auth option a route gets no suspension check and
admits access tokens, so a just-disabled member could still read the roster and
a scoped third-party token could read the audit log.

Authority is checked before anything observable. Validating the body first made
POST /members answer 400 before 403, and resolving :username first turned the
member routes into a global username-existence oracle.

Provisioning applies the same username and email rules as signup rather than
its own -- USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and
validator.isEmail, now exported from AuthController. Without them a workspace
could mint accounts signup would refuse, claim unregistered reserved names, and
mail arbitrary unvalidated addresses.

Handle problems are 400 or 409 rather than a bare Error, which the server turns
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable drops sessions through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key; without that a
disabled member kept authenticating from cache for the session TTL, which is
exactly the "takes effect on the next request, not after a cache TTL" property
disable is supposed to have. Revoking also preserves last_ip/last_user_agent,
which the member-facing audit view reads.

Audit writes live in TeamService at the point of each action rather than in the
route, so a caller reaching the service directly cannot skip them, and the SQL
lives in TeamStore. Audit reads map internal user ids to usernames, and remain
readable by the owner after the workspace is soft-deleted -- otherwise the
delete_team entry was written and immediately unreachable.

teams_enabled gates route registration through an optional isEnabled() the
server honours, so with it off the paths do not exist rather than existing and
refusing. It does not gate DDL.

TeamIsolation.http.test.ts asserts the negative the feature rests on: the
workspace manages accounts and cannot read them, including through a
full-access token and after the member is disabled. It asserts outcomes rather
than the absence of an implicator.
2026-09-03 15:57:23 -04:00
Juan Castro 673f7fe75f feat: provision org accounts from a workspace
Covers PUT-1705. The master account supplies { username, email }; the account
is created with no password, gets the default filesystem tree, joins with
org_owned = 1, and receives a one-shot activation link.

Activation reuses password recovery rather than new token machinery: the same
pass_recovery_token, the same one-hour purpose-scoped JWT, the same
/action/set-new-password link. No team_activation table, no new token type,
and no unauthenticated endpoint on the team surface. Activation state needs no
column either -- an unactivated account is one with no password.

Applies the same username and email rules as signup rather than its own:
USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and validator.isEmail,
now exported from AuthController. Without them a workspace could mint accounts
signup would refuse -- the username becomes the /username home-directory
segment -- claim unregistered reserved names, and send activation mail to
arbitrary unvalidated addresses at the route's daily limit.

Usernames come from Puter's global pool, so a taken one is refused with free
alternatives rather than silently modified: a suffixed name would appear in
every share dialog that person ever sees, and they never agreed to it. The
check runs before any write, so a rejected provision leaves no orphaned user
row -- asserted by a test on the workspace's member count.

The new account carries requires_email_confirmation, since the address came
from the administrator rather than its holder.

Adds a team_account_activation email template stating what the workspace can
and cannot do -- including that it can reset the password, which the design
requires be said rather than only claiming files are private.

free_storage stamping and the billing event are phase 3.
2026-09-03 15:57:23 -04:00
Juan Castro da2b42d7fe feat: add TeamService workspace lifecycle and disable/re-enable
Covers PUT-1704 and PUT-1707: creating a workspace, admitting the master
account, and the whole of offboarding.

`createWorkspace` admits the creator with org_owned = 0, which is what makes
the master pay for itself and stay an invalid target of every member route.
`checkOwnerInvariant` asserts the rule no dialect can express -- the owner is
a member with org_owned = 0 and the only such member -- and a test breaks it
deliberately, since the schema cannot refuse a second one.

Three authority checks: 404 to a stranger so the endpoint is not an existence
oracle, 403 to a member who is not the master, and the master refused as a
target of member routes.

Handle problems surface as 400 (unusable) or 409 (taken), including the
unique-index race. `TeamStore` throws a bare Error, which the server would turn
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable writes `user.suspended` as well as suspended_at and suspended_reason.
PUT-1707 named only the latter two, but those are siblings added by 0061 and
0063 -- `userProtected` rejects on `if (user.suspended)` and reads neither.
Setting only the timestamp and reason would have recorded a disable that never
took effect, and disable is the whole of offboarding here.

Sessions are dropped through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key with its double-delete
pattern; without that a disabled member keeps authenticating from cache for the
session TTL, which is exactly the "next request, not after a cache TTL"
property disable is supposed to have. Revoking also preserves last_ip and
last_user_agent, which the member-facing audit view reads.

Files are untouched and re-enable restores the account.

Adds team_not_found, not_the_master_account and not_an_org_account to the
HttpError legacy codes, which the controller also needs.

Billing events, invalidateActorSubscription, audit rows and the GUI push are
deliberately not here -- they belong to phase 3 and PUT-1708.
2026-09-03 15:57:23 -04:00
Juan Castro 8b4e259421 feat: add team membership reads and writes to TeamStore
Membership management for workspaces: addMember, removeMember,
getMembership, isMember, listMembers and listTeamsForUser. The permission
scan is untouched -- readUserGroupPerms already joins jct_user_group and
resolves group grants; this is the management side.

Resolves the ticket's "do not leave two writers" by splitting domains and
enforcing the split in SQL rather than by convention. Every existing caller
of GroupStore targets a seeded system group -- ADMIN_GROUP_UID,
default_user_group, default_temp_group -- never a team, so the two stores
were already disjoint in practice. GroupStore.addUsers/removeUsers now carry
`AND kind IS NULL`, making a team uid a no-op there, which costs no extra
query because it folds into the existing subquery and matches how addUsers
already treats an unknown username. TeamStore's writes select group_id from
a kind-filtered subquery, so neither store can reach the other's rows.

org_owned is written here but never accepted from a request; TeamService
sets it at provisioning and workspace creation only.

listMembers is keyset-paginated on id per doc/pagination.md, using the
shared cursor and limit helpers and fetching one row past the limit to
decide whether a cursor is warranted.

Passes 1/0 for org_owned rather than db.booleanValue, which yields a real
boolean on postgres and is rejected by the smallint column there -- sqlite
accepted it silently.
2026-09-03 15:57:16 -04:00
Juan Castro 37cb11df8c feat: add TeamStore for workspace CRUD
`GroupStore` has only addUsers/removeUsers; nothing creates, reads back or
lists a group at runtime. `TeamStore` is that missing half, scoped to rows
with `kind = 'team'`.

A workspace is addressed by `uid`, which `group` has carried as NOT NULL
UNIQUE since 0015. `handle` is a mutable display label with no addressing
role, so a rename invalidates nothing and a stale reference can never
resolve to a different workspace.

Soft delete releases the handle and keeps `name`. Nothing points at a
handle, so the name returns to the pool instead of being reserved forever
by a global unique index that cannot exclude dead rows -- mysql has no
partial indexes, so that exclusion was never available.

Handles validate to ^[a-z0-9]+(-[a-z0-9]+)*$, 3-64 chars, against a
reserved list. The charset is deliberately narrower than the column so the
engines' collations cannot disagree: mysql's utf8mb4_unicode_ci also folds
accents and eszett, which sqlite's NOCASE and postgres's lower() do not.

Every read filters `kind = 'team' AND deleted_at IS NULL`, which is what
makes the seeded admin/system groups unreachable rather than merely absent.
Handle lookups compare lower(handle) on postgres, where the index is on
that expression rather than the column.
2026-09-03 15:20:07 -04:00
Juan Fernando Castro ad70e8763b Merge pull request #3708 from HeyPuter/juancastro/put-1701-13-add-audit_team_membership-and-shareholder_group_id
🏗️ PUT-1701: Add audit_team_membership and share.holder_group_id
2026-09-03 15:17:34 -04:00
Juan Castro 64db3fa050 feat: add audit_team_membership and share.holder_group_id
Insert-only record of what a workspace administrator did to an account,
shaped like \`audit_user_to_group_permissions\` after 0019: nullable FK
beside a NOT NULL \`_keep\` column. The FKs are ON DELETE SET NULL, never
CASCADE, so hard-deleting an account cannot erase the record of the
resets performed on it.

Two indexes rather than one. The member's own view is the only place a
reset becomes visible to the account it was performed on, so
(user_id_keep, id) is a read path, not an optimisation.

\`share.holder_group_id\` mirrors \`holder_user_id\` from 0067. The existing
unique index does not constrain team shares at all -- it leads with
\`holder_user_id\`, which is NULL on every team share, and NULLs are
distinct -- so the group-scoped unique index is what prevents duplicates.

Drops the \`role\` column from the specified DDL: it contradicted the
settled single-administrator model.
2026-09-03 15:07:53 -04:00
Juan Fernando Castro 43f62cf7d0 Merge pull request #3705 from HeyPuter/juancastro/put-1700-12-deduplicate-jct_user_group-before-adding-the-unique-pair
🏗️  PUT-1700: Deduplicate jct_user_group and make membership writes unique
2026-09-03 15:07:49 -04:00
Juan Castro dc7b456afa fix: run the membership dedup once instead of on every boot
Review follow-ups on the dedup migration.

mysql and postgres track no per-file applied state and re-execute every
migration on each boot, so the unguarded DELETE self-joined the whole table at
every process start, forever. Both now sit behind the same index-existence
check that guards the ALTER, which also stops a rolling deploy deleting on one
instance while another adds the index.

The dedup test was a false green. `targetVersion: 67` never applies 0071 -- the
loop breaks on `threshold + 1 >= targetVersion` but stamps the target anyway --
so the fixture asserted the current schema version on a database missing a
migration. It now replays the real 0072 file against a fully migrated database,
and a second test pins the off-by-one so nobody builds a fixture on it again.
2026-09-03 14:33:46 -04:00
Juan Castro c1c1588939 fix: deduplicate jct_user_group and make membership writes unique
\`jct_user_group\` had no unique constraint on (user_id, group_id) and
\`GroupStore.addUsers\` had no conflict clause, so re-adding a member
inserted a second row. \`readUserGroupPerms\` joins the junction table on
group_id alone, so each duplicate returned another copy of every group
permission the user holds.

Deduplicate keeping the lowest id, add the unique pair index, and make
\`addUsers\` ignore conflicts via the existing \`insertIgnoreInto\` helpers --
without that last part the index turns a re-add into a raised error, which
five call sites would log as a failed signup step.

mysql cannot delete from a table it reads in a subquery (error 1093), so
it uses a self-join with the same lowest-id-wins semantics.
2026-09-03 14:32:30 -04:00
Juan Fernando Castro abb426a717 Merge pull request #3704 from HeyPuter/juancastro/put-1699-11-add-team-columns-to-group-and-jct_user_group-three
🏗️ PUT-1699: Add team columns to group and jct_user_group
2026-09-03 14:26:58 -04:00
Juan Castro eae25459f4 fix: make handle lookups case-insensitive and pin the reset flag
Review follow-ups on the team columns.

NOCASE moves onto the `handle` column itself, not just the index. Index-only
NOCASE makes uniqueness case-insensitive while leaving `WHERE handle = ?`
case-sensitive, so the same lookup would match on mysql (utf8mb4_unicode_ci)
and miss on sqlite. Postgres still needs lower(handle) at the call site.

`requires_password_change` becomes NOT NULL DEFAULT 0 on all three dialects,
matching the three sibling `requires_*` flags. Left nullable, any query written
as `= 0` would silently exclude every pre-existing user.
2026-09-03 12:10:48 -04:00
Juan Castro 2a5c089dbe feat: add team columns to group and jct_user_group
Schema foundations for teams. Ships dark — nothing reads these columns yet.

group:          kind, name, handle, plan_id, deleted_at
jct_user_group: org_owned
user:           requires_password_change

`kind = 'team'` marks a workspace; the seeded system groups keep it NULL so a
team query can never surface them. `org_owned` distinguishes accounts the
workspace created from the master account, deciding who pays rather than who
may read. `requires_password_change` is a fourth `requires_*` flag for
assertVerifiedAccount, needed by the phase 5 reset flow but shipped here to
avoid a second three-dialect migration.

`handle` uniqueness is case-insensitive, and the dialects disagree by default:
mysql gets it from utf8mb4_unicode_ci, sqlite needs COLLATE NOCASE (as
0055 does for usernames), and postgres indexes lower(handle). Without this the
same migration would accept `Design-Team` next to `design-team` on sqlite and
postgres while mysql rejected it. Postgres handle lookups must therefore
compare lower(handle) to use the index.

idx_group_owner is sqlite-only: mysql and postgres already index that column.
idx_jct_user_group_group is composite, unlike the existing single-column keys.
2026-09-03 12:10:40 -04:00
404oops 55fa5cadcb Merge pull request #3732 from HeyPuter/gemini-38-flash
gemini 3.8 flash direct integration
2026-09-03 17:35:22 +02:00
Reynaldi Chernando 644f59f9dd gemini 3.8 flash direct integration 2026-09-03 22:18:02 +07:00
404oopsandClaude Fable 5.1 8b01077d44 docs: describe the real txt2vid error body and per-provider wait windows
The Errors section claimed a `{ message, code }` body and a single
ten-minute wait. The body also carries `error`, `errorCode`, `provider`,
`upstreamCode` and, for pre-job rejections, `upstreamStatus`, and the
Sora provider still waits five minutes and folds refusals into
`upstream_failed`. Document the fields and the exceptions.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-03 16:57:36 +02:00
404oopsandClaude Fable 5.1 627a5b2d5e fix: stop video generation timeouts and failures from paging (PUT-1620)
A video job that outlived its poll window, an SDK request that timed out,
or a Veo operation that finished with an error all reached the HTTP error
handler as plain Errors. Each became an unhandled 500 with critical
severity and paged on-call for what is the provider's pace or the
provider's fault.

Video providers now share one poll loop that gives up with a 504
`upstream_timeout`, treats a transient poll failure (timeout, dropped
connection, 408/429/5xx) as a missed poll rather than a failed job, and
stops polling with a 400 `client_aborted` when the caller disconnects, so
nothing is metered for a clip nobody will receive. The driver controller
exposes the disconnect as an `abortSignal` on the request context. The
window is ten minutes for every provider; Together and BytePlus move up
from five.

Failed jobs are classified: content-filter refusals become a 400
`bad_request` with `errorCode: moderation_flagged`, rejected parameters a
400 `upstream_bad_request`, and anything else a 502 `upstream_failed`,
each carrying the provider's own code. Veo's filtered output keeps
`disallowed_value` and gains the same `errorCode`. The sanitizer and
content-filter pattern move from the Replicate provider into a shared
util so image and video agree.

Status-less SDK connection timeouts are translated to a 504
`upstream_timeout` at the driver boundary, and the chat driver records
them per attempt so an all-timeout chain is a 504 and a mixed chain is
`upstream_failed` instead of an `internal_error` 500. The Together chat
client gets the same ten-minute request timeout as the other providers.

The OpenAI video provider is left alone beyond an import path: its API is
scheduled to shut down on 2026-09-24.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-03 16:47:16 +02:00
Daniel Salazar 72011fa6ed feat: events metering and quotas (PUT-1683) (#3685)
* feat: events metering and quotas (PUT-1683)

* fix: drop the standing subscription charge and price single deliveries at 100 µ¢

An idle durable row costs nothing worth billing; the plan quotas bound how
many an account holds. Removing the daily line also removes the global
day-claim, the whole-table scan and the timer it rode on.

* test: stop asserting on documentation pages

The limits test read rate-limits-and-quotas.md and grepped it for numbers,
so every rewording of the page failed the backend suite. Docs are kept in
step by the PR and checked in review; AGENTS.md now says so.
2026-09-03 01:39:07 -07:00
Daniel Salazar 41b837b398 feat: background delivery seam, retries, and consent (PUT-1682) (#3684) 2026-09-03 01:39:06 -07:00
Daniel Salazar 04c00385b9 feat: event handlers, context, and suspension machine (PUT-1680) (#3683) 2026-09-03 01:39:06 -07:00
Daniel Salazar 9626ab9c71 feat: KV change events and KV subjects (PUT-1678) (#3682) 2026-09-03 01:39:06 -07:00
Reynaldi Chernando 3de6eeb474 init docs recipes (#3728) 2026-09-03 10:04:01 +07:00
Neal Shah af27115f68 Attempt 1: Fix PUT-1625 (#3727) 2026-09-02 21:36:04 -04:00
Daniel Salazar d8ebfc8023 feat: delivery re-check cache, revocation and anchor settle (PUT-1677) (#3681)
* feat: delivery re-check cache, revocation and anchor settle (PUT-1677)

* fix: authorize re-anchors, settle each row once, purge revoked backlog (PUT-1677)

- A path-form row whose anchor is deleted only climbs to an ancestor its
  holder may still watch under the mode it subscribed with; otherwise it ends
  with `anchor_deleted`. It used to land on any surviving ancestor (a guest's
  row on the owner's home), where the re-check denied every delivery but the
  row still held an anchor slot and a filter evaluation there.
- After a climb the new anchor is re-verified and the climb repeated if a
  recursive delete took that level too, instead of leaving the row on a dead uid.
- suspend() is one conditional write per row and reports which rows it was the
  one to suspend; concurrent settles of the same grant (an unshare revokes
  several strings) no longer each purge, forget and notify the same rows.
- One "subscriptions ended" notification per holder and app, carrying the count
  and subjects, instead of one per row.
- A revoke that removed nothing no longer announces; the sweeper purges (not
  defers) the backlog of a permission_revoked row; the reap purges pending
  entries with the row.
- The delivery auth cache indexes entries by subscription so forget() is not a
  scan of the whole cache.
2026-09-02 16:00:14 -07:00
Daniel Salazar 41d4850ab4 feat: pending event deliveries and delivery-class invariants (PUT-1676) (#3680)
* feat: pending event deliveries and delivery-class invariants (PUT-1676)

* fix: make pending delivery claims and drains atomic, keep the region under its ceiling (PUT-1676)

- claim() and the drain-time reindex run as Lua over the subscription's own
  {subId}-tagged keys. Two claimers can no longer both lease the head, and a
  drain that finds the queue empty deletes it in the same step it checks, so a
  concurrent enqueue is never wiped between the two.
- An append writes the entry and its queue position in one MULTI (same slot),
  with the index seeded before it and corrected after, so an entry is never
  visible without its position and never left out of the sweeper's index.
- Pipelines no longer mix slots (index/counter vs. per-subscription keys), so
  the store works on a multi-shard cluster, not only a single-shard one.
- Region shedding counts the marker it leaves behind; it used to stop one over
  the ceiling and convert a real event into a marker on every enqueue after.
- A claimed or suspended subscription moves to the back of the sweeper's index,
  so a delivery nobody settles cannot hold the head against every other backlog.
- `single` rows must carry a `worker` target: with sockets exhausted and no
  handler, an unacknowledged delivery would sit at the head forever.
- A gap marker for a row with no socket target is dropped rather than counted
  as a delivery of nothing.
- Backlog keys carry a 7-day TTL, refreshed by every claim, as a backstop for
  keys a purge/enqueue race left unindexed.
2026-09-02 16:00:14 -07:00
Daniel Salazar 0dfbceb047 feat: durable event subscriptions store, cache, and routes (PUT-1673) (#3679)
* feat: durable event subscriptions store, cache, and routes (PUT-1673)

* fix: durable subscription hardening (PUT-1673)

- Expired rows stop delivering at dispatch time and no longer count toward
  the per-account cap, instead of waiting for the sweep.
- The expiry sweep runs hourly with a jittered first pass shortly after boot;
  a 24 h interval never fired on a fleet that redeploys more often than that.
- Only a durable generation bump marks peer regions cold. A session
  subscribe/unsubscribe in one region used to force a primary read in every
  other region on its next dispatch.
- `subject`/`anchor_path` widen to varchar(4096) to match `fsentries.path`,
  and subjects longer than that are refused with `invalid_subject` rather than
  failing the insert on MySQL/Postgres.
- The dispatch and durable integration suites wait for the specific delivery
  they expect and assert only within their own folder; the old any-delivery
  `settle()` let a late event from a previous test satisfy or pollute the
  next one under CI load.
2026-09-02 16:00:13 -07:00
Daniel Salazar cab5253923 feat: puter.events SDK module (PUT-1675) (#3678)
* feat: puter.events SDK module (PUT-1675)

* fix: end event subscriptions on a server-side disconnect and retry budget refusals (PUT-1675)

- A disconnect socket.io will not retry (`io server disconnect`) now fails
  every subscription with `events_connection_failed` and drops the socket, so
  the next onLocal() starts a fresh connection instead of reusing a dead one.
- The connection closes when the last subscription lapses on re-subscribe.
- A re-subscribe refused with `too_many_requests` is retried after 10 s
  rather than ended for good.
- The API suite no longer passes when it cannot connect; every runtime it
  runs on (node, browser, workerd) carries the socket.
2026-09-02 16:00:12 -07:00
404oopsandClaude Fable 5.1 d89dc347a2 fix: keep the credit-bounded max_tokens on the context-length retry (PUT-1628) (#3718)
OpenRouter and Together reject a request whose prompt plus max_tokens
overflows the model's context window. Both providers retried by deleting
max_tokens, which threw away the output cap the credit gate had sized to
the caller's remaining balance and let the retry run to the model's full
output limit with no second gate and no new hold.

The retry now goes through a shared helper that sizes a new cap from the
window and input count the rejection reports, falling back to the model's
declared context and a doubled prompt estimate, and never exceeds the cap
the gate set. When no window can be determined or no output fits, the
original rejection is rethrown instead of retrying uncapped. The rejected
params are copied rather than mutated, so the first attempt's record is
not rewritten after the fact.

Also corrects the estimator's own comment, which described the mean of
two approximations as a deliberate halving.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-02 13:28:36 -07:00
Daniel Salazar 66dc0a6396 feat: subscribe ACL and app scoping for event subscriptions (PUT-1672) (#3677) 2026-09-02 13:08:23 -07:00
Daniel Salazar fb8e2976dc fix: don't redirect puter.com (#3723) 2026-09-02 11:57:30 -07:00
404oopsandClaude Fable 5.1 fd7b517448 fix: classify Replicate prediction failures instead of 500ing (PUT-1608) (#3719)
A Replicate prediction that ran and ended `failed` reaches the provider as a
plain Error with no HTTP status, so the driver-boundary translator could not
classify it and it surfaced as an unhandled 500, a critical alarm, and an
on-call page. Most of these are the model's content filter refusing the
user's prompt.

Wrap the run call and classify the failure: content-filter refusals become
a 400 with `errorCode: moderation_flagged` (the code chat refusals already
use); anything else becomes a 502 `upstream_failed`, which the alarm gate
skips. Status-bearing SDK errors pass through untouched so the boundary
translator keeps handling them. Upstream messages are stripped of markup and
bounded so an HTML error page can no longer ride into a response body or an
alarm signature.

Documents the codes callers can now act on in the txt2img reference.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-02 11:51:08 -07:00
Neal Shah 262f1dc5c5 allow concept of "home regions" (#3699)
* allow concept of "home regions"

* remove extraneous config value not applicable to repo
2026-09-02 14:11:55 -04:00
Daniel Salazar 7ec674d33b feat: app sockets join their own events room (PUT-1671) (#3676) 2026-09-02 09:01:22 -07:00
Daniel Salazar b07d2e109f feat: session event subscriptions and dispatch hot path (PUT-1666) (#3675) 2026-09-02 09:01:22 -07:00
Daniel Salazar 92e1b40361 feat: events subject registry, parser, anchors, filters (PUT-1660) (#3674) 2026-09-02 09:01:21 -07:00
Daniel Salazar d0ee19a2c2 chore: remove dead notification surfaces (PUT-1669) (#3673) 2026-09-01 22:04:33 -07:00
Daniel Salazar 82076ea767 feat: notification retention sweep (PUT-1668) (#3672) 2026-09-01 22:04:32 -07:00
Daniel Salazar 5f37817a5c feat: notification type registry and audience rules (PUT-1665) (#3671) 2026-09-01 22:04:32 -07:00
Daniel Salazar 148b930182 feat: notification scope columns and backfill (PUT-1659) (#3670) 2026-09-01 22:04:31 -07:00
Daniel Salazar 0385072e06 feat: better verification step handling (#3717) 2026-09-01 21:17:19 -07:00
Daniel Salazar 1736d23ee7 Ds/put 1674 (#3715)
* feat: app-scoped share listing and revoke surface (PUT-1670)

* fix: address review on app-scoped share surface

- Scope the uid-addressed revoke to the named row: only that row's
  issuer's grant is withdrawn, and only that one invite cancelled —
  an app or owner addressing one row no longer takes another issuer's
  grant on the same (item, recipient) pair with it.
- Delete a pending row directly on uid-addressed revoke, so an invite
  whose address registered but never claimed can still be withdrawn.
- Read the legacy `issuerAppUid` data key in the SQL app filter and
  grouping, alongside the unified `issuedByApp`.
- Refuse malformed `appUid` input (duplicated param, empty string)
  instead of silently listing everything, and refuse app-listing
  cursors that decode but name no appUid.
- Derive the acting app from `effectiveApp` alone, per the actor
  contract; drop the second derivation site.
- Pin the attribution semantics with tests: one row records one
  issuance, so re-sharing the same pair re-attributes it to whoever
  issued last, in both directions.
- Soften the uniform-404 docblocks to what the gates actually answer.

* feat: readable grant audit trail (PUT-1674)

* fix: cover the apps summary's no-app-group first page (PUT-1670)

listOutboundApps sorts the no-app group first via an empty-string
sentinel. Add a regression test pinning that a first page (no cursor)
actually returns it, and that the cursor it hands back resumes past it
into the app-keyed groups rather than skipping or repeating.
2026-09-01 14:07:15 -07:00
Daniel Salazar 66a975f659 feat: global outbound share listing (PUT-1664) (#3694)
* feat: global outbound share listing (PUT-1664)

* fix: address review on outbound share listing

- Check share-row liveness per (holder, entry, issuer) so a grant
  withdrawn outside unshare doesn't stay listed while another issuer
  still reaches the same holder; batch the permission reads across the
  whole page instead of per holder.
- Retire a revoked issuer's unclaimed invites in the revoke cascade,
  and hide invites whose issuer lost their authority at read time.
- Unify the pending/active app-attribution key on `issuedByApp` and
  dual-read the legacy `issuerAppUid` spelling.
- Add the missing share issuer index (sqlite, postgres) and correct
  the listOutbound plan comment.
- Refuse cursors that decode but name no id instead of silently
  restarting from page one.
- Consolidate the five hand-built ResolvedShare literals and the two
  listing endpoints' parse/shape code.
- Ship the SDK surface: puter.fs.listSharedByMe() with docs, types,
  suite coverage, and the rate-limit page entry.
2026-09-01 13:52:37 -07:00
404oopsandClaude Fable 5.1 6478a8f47d Add Claude Fable 5.1 (#3711)
Register claude-fable-5-1 in the Claude catalog and gate it in the
provider the same way as Fable 5: no sampling params, effort via
output_config, adaptive thinking with summarized display. The bare
claude-fable / claude-fable-latest aliases move to 5.1, matching how
the Opus aliases moved when Opus 5 landed.

Fable 5.1 keeps Fable 5's $10/$50 per MTok, 1M context and 128K output,
but bills cache reads at 0.025x input instead of the 0.1x every other
Claude model uses, so the catalog row carries its own rate and a test
pins it.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-01 15:34:49 -04:00
Nariman Jelveh fdd86ceb03 Stop server-rendered landings flashing for signed-in users (#3707)
The shell renders its anonymous markup — the marketing homepage, an
`/app/<name>` landing — off the session cookie alone, and that cookie is
set with no maxAge, so a browser drops it on quit while the GUI's
localStorage token lives on. A returning user is served the anonymous
page and the GUI only tears it down once `whoami` answers, a network
round-trip after first paint. That teardown is the flash.

Gate it before the paint instead. The shell now emits, as the first thing
in <head>, a rule hiding `.hide-if-logged-in` under an <html> class that
an inline script adds iff `auth_token_v2` is in localStorage. The rule is
already in the cascade when the markup is parsed, so a browser holding a
token never paints it at all.

`initgui` settles the guess the token represents: `whoami` confirming the
session removes the nodes outright (replacing the old `#appLanding`
removal), and no session — none stored, or one `whoami` rejected — drops
the class so the markup comes back. The gate carries its own 12s failsafe
so a bundle that never boots can't strand a blank page.

SEO is unaffected: the HTML is byte-identical for every client, nothing
branches on user-agent, and a crawler has no stored token so it never
adds the class. Unreadable storage fails open the same way.

Anonymous markup opts in with `class="hide-if-logged-in"`, which
`home.html` already carried.
2026-09-01 09:19:00 -07:00
Daniel Salazar 7584dffc4b fix: metering for ai (#3703) 2026-09-01 09:11:04 -07:00
jelveh b400e06473 Remove login landing fade-out overlay
Drop the delayed fade-out when a logged-in user reaches the GUI and remove the landing overlay immediately instead. This avoids the overlay sitting above the email/phone/card verification gates because of its z-index.
2026-08-31 20:13:58 -07:00
jelveh 22c3316109 Hide app landing after session restore
Remove the /app landing overlay once `whoami` confirms the user is authenticated, even if the server rendered the page as anonymous because the session cookie was missing. This prevents the overlay from covering the email, phone, and card verification gates during localStorage-based session restores.
2026-08-31 20:07:47 -07:00
404oops a27852de0a Merge pull request #3655 from HeyPuter/FK/normalize-openai-format
Comprehensive Normalization.
2026-08-31 22:24:34 +02:00
jelveh f80a0d72b3 fix: dashboard 'Add Existing Account' dialog buried on phones/tablets
On device-phone/device-tablet, style.css pins every .window to
z-index 9999999 !important — including the fullpage dashboard window.
The Add Existing Account login dialog opens with backdrop: true, and
the .window-backdrop wrapper is not a .window, so it kept its small
inline z-index and rendered entirely behind the dashboard.

Pass stay_on_top: true like every other backdropped dashboard dialog,
and make the Forgot-password child dialog inherit the flag so it does
not open buried under the now stay-on-top login window's backdrop.

Verified with Playwright on iPhone 13 emulation and a desktop
viewport: login and recover-password dialogs stack above the
dashboard in both.
2026-08-30 21:45:43 -07:00
Nariman JelvehandClaude Fable 5 3e8b2735bb Cap /app/<name> meta descriptions at 150 characters (#3698)
The app shell fed the row's full description into <meta name=
"description">, og:description and twitter:description. Long
descriptions now collapse whitespace and cut at a word boundary with an
ellipsis, never exceeding 150 characters.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-30 20:17:04 -07:00
Daniel Salazar 5065bcbf0a fix: document allowedAppIdsGate's real non-app-gate behavior (PUT-1667) (#3669)
The JSDoc claimed non-app actors are rejected; they are passed through, and
the check reads `actor.app` rather than `effectiveApp`, so an app in the
token chain is invisible to it. Callers depend on the pass-through — routes
combining `adminOnly` + `allowedAppIds` are reached with an admin's browser
session, and the dev-account surface is called from the desktop's own
session — so the docs are corrected instead of the behavior, on the gate and
on the `allowedAppIds` route option. Tests cover the gate per actor shape
(user session, worker, full-access token, app-issued access token,
app-under-user allowed/disallowed) and the admin composition.
2026-08-30 12:55:39 -07:00
404oopsandClaude Fable 5 7e1cdefd98 feat(ai): backfill provider catalogs from live /models listings
A live audit of every keyed provider's /models endpoint against the hardcoded
catalogs found no stale entries but large gaps. This backfills them under four
rules: nothing vendor-deprecated, nothing without a price confirmed on the
vendor's official pricing page (each entry's source was recorded during
review), nothing absent from the live /models listing, and nothing that fails
a live routing probe.

Added: 46 Alibaba entries (qwen3/3.5/3.7/3.8 families, VL/omni/MT lines, and
Model Studio's hosted GLM/DeepSeek/Kimi third-party models) plus 9 dated
aliases; OpenAI chat-latest and gpt-4o-2024-11-20 plus 16 snapshot aliases;
Gemini gemma-4-31b-it and gemma-4-26b-a4b-it (vendor-documented free tier)
plus rolling -latest aliases; Mistral-hosted zai-glm-5-2 and a
mistral-medium-3.5 alias; deepseek-v4-flash-vision-exp; glm-5.3-flash.

Culled by the rules: 15 vendor-deprecated OpenAI entries (the 3.5/4/4-turbo
legacy line, gpt-4o-2024-05-13, o1-pro, four chat-latest predecessors, the
5.x codex line — deprecations page, most shut down 2026-10-23) and dated
aliases onto the deprecated o1/o3-mini/o4-mini; qwen3-vl-flash-2025-10-15
(live routing probe returned upstream 400 twice). Tiered Alibaba prices are
encoded at the base tier and busy-hour rates where time-of-day priced, noted
in comments.

Every surviving addition was verified end-to-end through a local deployment:
58/58 answered a live prompt, including all 46 Alibaba entries and every
spot-checked alias.

Not changed, flagged for maintainers: pre-existing o1, o3-mini and o4-mini
entries are now vendor-deprecated (shutdown 2026-10-23); the pre-existing
deepseek-v4-flash/-pro prices no longer match DeepSeek's current pricing
page; gemma-4 emits its own <thought> markup inline in content.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-30 20:50:46 +02:00
404oopsandClaude Fable 5 4199a2506d feat(ai): forward Mistral prompt_mode so magistral can emit separable thinking
Live-probing magistral-small-latest showed the model inlines its reasoning as
answer prose in a flat string — no ThinkChunk content, no markers, nothing a
client can separate. Mistral's chunked thinking shape is requested via
`prompt_mode: 'reasoning'`, which the provider previously dropped on the
floor: there was no way to even ask for it.

`custom.prompt_mode` now forwards to the SDK's `promptMode`, following the
BytePlus custom-params precedent. Opt-in rather than a default because the
API rejects the mode where the account/model lacks it ('Reasoning prompt
mode is not enabled for this model', code 3051) — verified end-to-end: the
3051 travels back through the stack, which also proves the parameter is
delivered. The moment Mistral enables the mode, the ThinkChunk content flows
into the existing splitter and comes out as `message.reasoning` and
`reasoning` stream chunks with no further changes.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-30 20:50:46 +02:00
404oopsandClaude Fable 5 4aa66c7c9b feat(ai): make puter.ai.normalize tri-state — unset means the release-date policy
The SDK-wide flag now mirrors the per-call option's tri-state. Left unset
(the default), the release-date policy applies: models released on or after
2026-09-01 are normalized to the OpenAI shape, older models keep their
vendor-native shape, and nothing rides the wire — the server's policy
resolution decides. Setting `true` force-normalizes every chat() call
regardless of release date; `false` disables normalization for every call;
either explicit value is sent on each call that doesn't set its own.
A per-call `normalize` overrides the flag in both directions, and assigning
`undefined` restores the policy.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-30 20:50:15 +02:00
Daniel Salazar afe46a17a0 fix: alaem cascade (#3668) 2026-08-29 03:41:23 -07:00
Juan Fernando Castro 81d700d146 fix: publishing a directory takes ownership, not write (PUT-1619) (#3654)
Creating a hosted subdomain gated `root_dir` on `write`, and hosting serves
everything under that directory with the ACL deliberately bypassed. So a
recipient of a `write` share could point a `*.puter.site` subdomain at the
owner's folder and make the subtree world-readable — continuously, covering
files the owner added later, with the row under the recipient's account where
nothing the owner can list would show it. `update` had the same gate for a
changed `root_dir`.

`#checkPublishAccess` now decides both: the actor's own tree still takes
`write`, anyone else's takes `manage` — "Can edit & share", the level that
delegates the decision.

Keyed on who owns the entry rather than asking for `manage` outright, which is
what the ticket proposed. `manage`'s is-owner implicator declines to answer for
app actors, so a flat `manage` would refuse every app publishing a directory
its user handed it, with no way for the app to obtain the grant. The write
check still runs first — it is what masks a directory the caller cannot see as
a 404 — and `manage` satisfies every lower mode, so the order costs a
manage-holder nothing.

The GUI's Publish As Website item reuses the own-it-or-`manage` answer it
already computes for sharing, so it is not offered where this would refuse.
Docs state the rule on `hosting.create()` and in `share()`'s level list.

Regression tests fail without the driver change: a write-share recipient is
refused on create and on repointing an existing subdomain, while `manage` and
the actor's own directory are accepted.
2026-08-29 01:46:14 -07:00
LEANDER ROSHAN L d5e9c14b0d fix: update sharing indicator #3649 (#3665) 2026-08-29 01:43:40 -07:00
Juan Fernando Castro c17613669c fix: open read-only shared files instead of forwarding write_url=undefined (#3667)
Opening a file you hold read-but-not-write access on (a read-only share)
failed silently. For such a file the backend correctly omits write_url from
the /open_item signature, but launchApp appended it to the app iframe URL
unconditionally. URLSearchParams coerces undefined to the string \"undefined\",
so the app received puter.item.write_url=\"undefined\" — truthy, so the editor
believed the file was writable, and an invalid URL, so it broke on open. Only
read-only shares hit this; files you can write carry a real write_url.

Extract the puter.item.* param building into append_signed_item_params and
guard the write_url append so it is only added when present. launchApp.js is
too coupled to UIWindow/jQuery/window globals to unit-test directly, so the
pure helper carries the logic and its own test, matching the helpers/ pattern.

Regression test pins both directions: a read-only signature omits write_url
entirely (no \"undefined\"), and a writable signature still forwards it.
2026-08-29 01:42:46 -07:00
Juan Fernando Castro cdfd38bd4b fix: stop an embedded fs: in a path from escalating to full FS access (#3666)
The fs-path-to-uid permission rewriter split permission strings on the raw
\`fs:\` substring instead of parsing on component boundaries. A path can
itself contain \`fs:\` — a home dir named \`…fs\`, or \`fs\` in the mode
position — and the raw split mistook that for the mode delimiter.

Crafting \`fs:/<victim>fs:junk:read\` made the split do two things at once:
it dropped the \`junk:read\` mode, and it consumed the trailing \`fs\`, turning
the harmless-looking nonexistent path shown in the consent dialog into the
victim's real home path. The rewriter then stored a bare \`fs:<home_uuid>\`,
which subsumes every mode via parent-permission matching — full
read+write+delete from a request the user saw as \`junk:read\`.

Parse with PermissionUtil.split() in both matches() and rewrite(), matching
every other permission parser, and locate the \`fs\` component by position
(after an optional \`manage\` prefix). The path component is now exactly what
sits between \`fs\` and the next unescaped colon, and all trailing components
are preserved as the mode — so an embedded \`fs:\` is data, never a delimiter.
The crafted input now addresses the nonexistent \`/<victim>fs\` and 404s.

Regression tests pin both halves: the exact PoC must 404, and an \`fs:\` in
the mode position must survive the rewrite instead of collapsing to a bare
permission. Both fail on the old rewriter and pass on the fix.
2026-08-29 01:41:40 -07:00
Juan Fernando Castro 693e86d763 Merge pull request #3657 from HeyPuter/juancastro/put-1616-shareblocks-is-missing-requireuseractor-any-authorized-app
🔧 PUT-1616: Close /share/blocks to app tokens
2026-08-28 16:46:04 -04:00
Daniel Salazar 6d10ef2821 fix: stop a URL parameter putting puter.js into app mode (#3660)
`env = 'app'` was decided by the presence of a `puter.app_instance_id` query
parameter and nothing else, so a crafted link put any page that loads the SDK
into app mode — and app mode is what makes the URL's `puter.api_origin`
authoritative for every credentialed call.

App mode now also requires the document to be framed. The GUI only ever
launches an app into an iframe, so this costs a real app nothing while a
top-level document carrying the parameters is treated as the third-party site
it is. It is not an attestation that the framing document is the GUI — a
cross-origin ancestor's identity is not readable — so the token paths carry the
rest:

- The `web` boot branch adopted a stored token without consulting the origin it
  was bound to, which is what completed the fixation: one link plants a token
  bound to an attacker's origin, and every later visit adopted it. It now
  applies the same binding rule the app branch does, and drops a token that
  fails it rather than leaving it to be re-read.
- `signIn()` had no env guard, and in app mode delivered a real token to
  whatever `puter.api_origin` the launching URL named. Apps get their token
  from the session that launched them, so it now rejects there with
  `not_available_in_app`. Nothing internal reaches it in app mode:
  `authenticateWithPuter` and both implicit-auth call sites already gate on
  `env === 'web'`.
- The cross-origin-isolated branch polled `${this.APIOrigin}/login/wait` and
  adopted whatever came back. Pinned to `defaultAPIOrigin`, the same way the
  popup and its message handler already pin `defaultGUIOrigin`.

Backward compatibility: no signature, response field or existing error code
changes. The only behaviour a caller can observe is the new `signIn()`
rejection, which replaces a call that could not have worked correctly.

Covers the SDK side of the parameter PUT-1395 and PUT-1427 closed on the GUI.
2026-08-28 11:16:10 -07:00
Daniel Salazar 9fab4742c9 fix: complete the malformed-input guards, including the two paths still live (#3663)
Review of the previous change found three of its claims unmet.

The image-generation crash it reported fixed is still reachable. The assert was
scattered across three helpers, and Gemini and OpenAI call `isHttpUrl` directly
on `input_images` without going through any of them — so two of seven providers
still 500 on a non-string. `isHttpUrl` now refuses a non-string itself, and the
shape is settled once in `ImageGenerationDriver.generate`, where the driver call
arrives, rather than per helper. That also covers `input_images` that isn't an
array, which produced a different crash per provider.

The sixth case in the ticket, previously unlocated, is
`Messages.js` reading `tool_call.function.name` with no guard — reachable with
`{"messages":[{"role":"assistant","tool_calls":[{"id":"x"}]}]}`. Guarded, along
with the same shape in `make_claude_tools`: a TypeError there carries no status,
so the retry loop reads it as a provider failure and marks the route unhealthy
for every caller.

`#hardExpiryFromExpiresIn` returning null for a bad type moved the failure past
the session INSERT, leaving an orphaned non-expiring row and still answering
500. Reverted; the controller guard is the fix, now covering fractions,
negatives and unparseable durations rather than only wrong types.

Also: the batch write handlers check that the body is an array but not what is
in it, so a null element 500s the same way; `#requireObjectBody` accepted an
array despite its name; `handleCreateAccessToken` destructured a body that may
be absent; and two AGPL notices had been rewrapped with a Markdown link.
2026-08-28 11:14:30 -07:00
Daniel Salazar f3a46a9be4 fix: stop sockets outliving the session that authenticated them (#3658)
A socket was checked once at handshake and never again. Nothing in the backend
disconnected one, so logout-everywhere, password reset, session revoke and
suspension all left every connection streaming legacy FS entries, upload paths
and notification bodies — up to 400 per account — on a credential that had
already been revoked.

Three gaps, three fixes:

- The handshake skipped the suspension and pending-verification checks every
  authenticated HTTP route gets. `decideSocketAuth` now applies both.
- `revokeCascade` reports which rows it revoked, AuthService announces that as
  `auth.sessions.revoked`, and SocketService drops the account's room. The
  whole room goes, not just the revoked session: narrowing it would need
  `fetchSockets`, which the adapter builds on `serverCount()` — and that calls
  node-redis's `send_command`, which ioredis does not implement. A connection
  whose session survived reconnects on its own and re-authenticates.
- A bulk suspension writes `user.suspended` without touching `sessions`, so no
  revoke fires. A five-minute sweep re-verifies each live socket's token and
  drops the ones that no longer authenticate. De-duplicated by token, since a
  browser's tabs share one.
2026-08-28 11:12:27 -07:00
Daniel Salazar e460e9034c fix: answer 400 instead of 500 on four malformed-input paths (#3662)
Each of these read a field off caller input that wasn't the shape the code
assumed, threw a TypeError, and was served as a 500 with a critical page.

- POST /fs/write and /startWrite: a request whose body never parsed left
  `req.body` undefined, and the first read of `fileMetadata` threw.
- POST /drivers/call, image generation: `input_image` / `input_images` entries
  are documented as strings but nothing checked, so a number or an object
  reached `.startsWith`. Type confusion on caller input, so reachable on
  demand rather than by accident.
- POST /auth/create-access-token: `expiresIn` went to the expiry parser
  unvalidated, where anything but a string or a number has no `.trim`.
- POST /login/wait: destructuring `session` out of an absent body threw.
  Same class as the POST /login body ticket; that one covers /login itself.
2026-08-28 10:22:51 -07:00
Daniel Salazar 5c8defb940 fix: bound failed WebDAV Basic-auth attempts per account and per address (#3659)
The only thing in front of the bcrypt compare on the DAV host was the 600/min
request ceiling, keyed on a fingerprint that rotates with client-controlled
headers. /login guards the same credential with a captcha and two much tighter
buckets; DAV had neither, which left password and TOTP guessing viable from a
host that answers any origin.

The request ceiling can't double as a credential ceiling — a working DAV client
resends its credentials on every request — so the new buckets count only
verifications that failed: 10 per account and 50 per address per 15 minutes,
sized like /login's. They're read before the compare, so an exhausted bucket
costs no bcrypt round, and successful requests never draw them down. `-token`
attempts are held by the address bucket alone; bucketing them per account would
let bad tokens lock out good ones.

Reading a bucket without spending from it is new, hence `peekRateLimit` and the
matching `peek` on all three backends.

Also documents the DAV limits, which were undisclosed.
2026-08-28 10:21:34 -07:00
Daniel Salazar 88b851a423 fix: derive the signup client IP from req.ip instead of x-forwarded-for (#3661)
Both signup paths read the x-forwarded-for header directly for the
puter.signup.validate event, the puter.signup.success event and the
signup_ip_forwarded column. Behind an appending proxy a client can prefix that
header with anything it likes, which gives every per-IP abuse signal a fresh
bucket per request.

req.ip is the value `trust proxy` resolves, so it is the one the server can
stand behind; server.ts already uses it for the ip.validate gate for exactly
this reason. All three now share one derivation, so a per-IP counter is written
and read under the same key. The raw chain is still recorded, but only as
audit_metadata.ip_fwd, where nothing keys on it.

Rows written before this hold whatever the proxy appended, so per-IP velocity
over the trailing window is undercounted until they age out.
2026-08-28 10:19:24 -07:00
Juan Castro 94d798ccce fix: close /share/blocks to app tokens
All three block routes set only `subdomain`, `requireVerified` and a rate
limit, and the controller's actor check accepts anything carrying a user, so an
app-under-user token passed. Unlike `share` and `revoke`, nothing else bounded
them: an app the user authorized once could read exactly whom they had blocked,
then clear the list — blanket switch and per-username — and reopen the channel.

A block list is a personal safety control, not something an app can otherwise
reach, so it falls outside the "an app can share what it can already reach"
carve-out the other share routes rely on. `requireUserActor` on all three, and
deliberately without `allowFullAccessToken`: this is security management, which
stays closed to every access token, personal ones included.

The only caller is the desktop's Blocked Senders window, which sends the user's
own session token.

Tests fail without the gate: an app token minted through
/auth/get-user-app-token is refused on all three routes while the user's own
session still succeeds, and a metadata check pins the gate on every block route
so a fourth cannot quietly ship without it.
2026-08-28 12:32:17 -04:00
404oopsandClaude Fable 5 07998c32ed test(ai): point the provider-level replay test at the real driver fallback test
The comment claimed the fallback loop cannot be driven from a test — false
since ChatCompletionDriver.test.ts gained a two-attempt fallback test that
drives the actual loop. Say where that test lives instead.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 04d12d9524 fix(ai): route Mistral streamed thinking to the reasoning channel, unconditionally
Third triple-check round. Ten findings were put through independent skeptics
first; six did not survive — pre-existing on main, inert, or resting on a false
premise — and are not acted on here.

Mistral streamed thinking now goes to `reasoning` on every path. The previous
commit gated the split, which made this the only place in the repo where
chain-of-thought reached the visible text channel, and made Mistral the only
provider whose streamed chunk *types* depend on a response-format flag. Every
other reasoning path routes thinking to `reasoning` unconditionally —
ClaudeProvider's thinking_delta, the DeepSeek/OpenRouter rename, and this
branch's own Responses summary-delta handler. Removing the gate restores that
uniformity and makes the documented promise that streaming is unaffected by
normalization true again; the two opposing tests collapse into one that runs the
same fixture with and without `normalize` and asserts identical event streams.

Docs stop claiming older models are unchanged. Four reasoning fields were made
consistent across all models, ungated, and one of them removes a field: on
non-streaming responses `message.reasoning_content` is now `message.reasoning`.
chat.md gains a table naming all four so a caller reading `reasoning_content`
learns why it disappeared, instead of reading that nothing changed for them.

Adds the driver-level fallback test. Writing it surfaced that the invariant it
was meant to assert is false and always was: the driver rewrites string
`content` into text blocks in place on the caller's own messages
(`normalize_single_message`, pre-existing) before any provider runs. The test
now asserts what is true and load-bearing — both attempts receive the same array
reference, and the reasoning artifacts survive attempt 1 so attempt 2 can still
replay them.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oops 875aac5db0 Revert "docs: record AI-normalization review rules in CLAUDE.md"
This reverts commit f220276724fdc1a16ae56f9c95e41773fe4231ed.
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 216e8d68ed fix(ai): scope the normalized flag, gate the Mistral stream split, correct four doc claims
Acts on a second triple-check audit.

`normalized` means one thing. The driver stamped it on both the OpenAI-shape
path and the legacy `response.normalize` path, which converts toward Anthropic
blocks — so a caller could get `normalized: true` alongside array content, and
the flag told them nothing they could branch on. The legacy branch no longer
sets it. That branch is reachable only by a direct driver call with
`response.normalize` and no `normalize`; the four wire routes pin
`normalize: false`, which skips it.

Mistral streaming, split by concern. Flattening a reasoning model's chunked
`delta.content` to a string is a correctness floor and stays ungated — the
shared handler passes the value straight to `addText`, so an array reaches the
caller as stringified objects. Splitting the thinking text out into a
`reasoning` delta is the dialect change and now sits behind the policy gate
like the non-streaming remap. On the native path the thinking text is kept
inline rather than dropped.

Mistral `finishReason` is deleted only once its value carried over. The delete
ran unconditionally, so a non-string `finishReason` with no `finish_reason`
left the choice with no finish reason at all.

Four doc claims corrected against the code paths they cover: `content` is
string-or-null on normalized responses (tool-only turns carry no text, and the
`// always a string` example comment was wrong); `reasoning_details` is not
scoped to normalized responses, since Responses models emit it either way; and
the release-date rule depends on the serving provider's own dates — OpenRouter
derives them from its live API, so models newly listed there from 2026-09-01
normalize by default.

Adds the test the copy-on-write fix was actually for: one messages array sent
through two sequential calls, asserting the caller's array is untouched and
both attempts carried the thinking signature. That is the fallback hazard; the
harness wires one provider per model, so the fallback loop itself cannot be
driven from a provider test.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 c1b420e480 fix(ai): gate the Mistral remap, unify the reasoning join, stop mutating caller messages
Acts on a triple-check audit of this branch.

Gate the Mistral dialect remap. The camelCase→snake_case rewrite and the
chunked-content flattening were firing for every Mistral call regardless of
`normalize` or the cutoff, deleting `finishReason` and `message.toolCalls` out
from under any caller reading them. Both now sit behind the policy resolution
the driver already used, extracted as `shouldPresentAsOpenAI` so the provider
and the driver cannot drift. The streaming chunk-array split stays ungated:
handing an array to `addText` is a plain bug, and streamed chunks are
provider-uniform by design. The conformance matrix now passes `normalize: true`,
which is the contract it was always testing.

Unify the reasoning join. Three code paths produced two separators while one
doc sentence described them all: the coercer joined thinking segments with '',
the Responses handler and Mistral with '\n\n'. The coercer now matches, and
chatresponse.md's claim is true for every path it covers. Text blocks still
join with '' — Anthropic splits prose mid-sentence across them.

finish_reason is an open set. chat.md's normalize bullet and the SDK
ChatMessage typedef still declared a closed four-value set, contradicting the
documented pass-through of unmapped vendor reasons and the coercer that
implements it.

Stop mutating caller messages. Both reasoning-replay input paths deleted
output-only fields from the caller's own message objects, which the driver
reuses across fallback attempts. Both strip a copy now; tests pass a frozen
message through each.

Drop three dead things the type cleanup left: the no-op ChatProvider
checkModeration stub (no subclasses, no callers — its removal restores a
pre-existing baselined TS2420), the redundant second normalizeReasoningContent
call in BytePlus and ZAI, and the coercer's bare-string branch that no provider
reaches. A bare string now passes through by reference instead of being
coerced.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 fbb9038651 revert: restore the 31 typecheck-baseline entries this branch removed
Commit 6b4f5b01f dropped 31 entries (49 error instances) from
tools/typecheck-baseline.json as a side effect of fixing the AI-code type
errors they suppressed. The file is a ledger the maintainer owns, and shrinking
it inside a feature PR ships a CI-gate change nobody asked for.

Restoring is free: tools/typecheck.mjs fails only on *regressions*
(count > baseline), so stale entries are non-fatal — the gate prints "49
baselined error(s) fixed. Run npm run typecheck:update to lock that in." and
exits 0. Nothing else in the repo reads the file, vitest included, and
tsconfig.build.json's noCheck:true means emit is unaffected. The errors stay
fixed in the source either way; only the ledger's own bookkeeping is deferred
to whenever the maintainer chooses to regenerate it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 74640d4e83 fix(ai): flatten Mistral reasoning models' chunked content
`magistral-*` returns `message.content` as a ContentChunk[] rather than a
string, with the thinking text nested one level deeper inside `thinking`
chunks. The camelCase remap did not touch it, so a non-streamed magistral
response reached the caller as an array with no `reasoning` — the one case
left where a provider did not produce the equalized shape this branch
promises. Streaming had the matching bug: the chunk array was handed to
addText, which would have stringified it into the text stream.

Both paths now split chunked content into a string `content` plus a
`reasoning` string, joining multiple thinking chunks with a blank line as the
Responses handler and the Anthropic coercer do. The streaming fix rides the
existing Mistral-only `chunk_but_like_actually` hook, so no new deviation is
introduced.

The conformance matrix had no Mistral reasoning fixture, which is why it
missed this; it now has one carrying chunked content, verified to fail
without the flattening.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 3b8de6a90b docs: record AI-normalization review rules in CLAUDE.md
Rules from the post-normalization review: don't silently change what existing
models return, verify doc claims against every code path they cover, record
self-disclosed defects in the PR draft, finish with a fresh build + suites, and
check git stash before concluding an edit was lost. Plus the standing
typecheck-baseline no-go.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 a6830f341a fix(ai): round-trip reasoning artifacts and pass unmapped stop reasons through
Closes the reasoning gaps left open by the normalization work.

Reasoning replay. The coercer dropped Anthropic thinking-block signatures and
the Responses handler dropped reasoning item ids/encrypted_content, so a
normalized reasoning turn could not be replayed — Anthropic rejects an
extended-thinking tool-use continuation whose thinking blocks lost their
signature. Both now ride `message.reasoning_details` verbatim, and both input
paths accept them back: ClaudeProvider splices the blocks ahead of the content
(Anthropic requires them to lead), and the Responses input processor expands
them into standalone `reasoning` items. Output-only fields a replayed message
carries (`reasoning`, `refusal`, `normalized`) are stripped on both paths,
since neither upstream accepts them. The docs caveat recommending
`normalize: false` for agentic Claude loops is gone; it is no longer true.

Unmapped stop reasons. chatresponse.md promised a vendor `finish_reason` with
no OpenAI analog "passes through unchanged" — true for the Mistral remap, false
for the Anthropic coercer, which discarded it. Anthropic's `pause_turn` means
"continue this turn", so flattening it to `stop` destroyed the signal. The
coercer now passes unmapped values through verbatim, matching both the doc and
the Mistral path, and the docs gain the full Anthropic stop-reason table.

Reasoning summaries. Multi-part summaries joined with '' instead of a blank
line, and the streaming Responses path emitted no reasoning at all;
`response.reasoning_summary_text.delta` now feeds the same `reasoning` stream
channel the chat-completions handler uses.

Types. `text?: string & { verbosity?: ... }` was an uninhabitable intersection
(providers read `text?.verbosity` as an object), and the verbosity enum was
`'concise' | 'detailed'` where OpenAI accepts `'low' | 'medium' | 'high'`.
Adds `reasoning`, `reasoning_details`, and `refusal` to the SDK ChatMessage
typedef.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Fable 5 548d26f884 test(ai): cover Hoonify in the provider-consistency matrix
Adds the upstream Hoonify provider (feat #3499, merged after this
branch was cut) to the cross-provider output conformance suite. It
speaks the OpenAI chat-completions dialect and conforms out of the box.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-28 01:08:48 +02:00
404oops 9b67a2b1dd Add AI response normalization controls
Introduces a new `normalize` option for chat completions, plus release-date based default normalization (post-2026-09-01) to coerce provider-native outputs into a consistent OpenAI-style shape. Adds shared normalization utilities, extensive driver/provider consistency tests, and controller safeguards that pin provider-native output where route-specific translators are used. Also wires the option through puter.js (`chat` options and `puter.ai.normalize` default), updates AI/chat response docs and examples, and resolves related TypeScript typing issues reflected in the typecheck baseline.
2026-08-28 01:08:47 +02:00
Juan Fernando Castro f952b5006b Merge pull request #3652 from HeyPuter/juancastro/put-1594-permission-requests-should-auto-return-true-and-resolve
🔧 PUT-1594: Settle a permission request from what is already granted
2026-08-27 19:02:15 -04:00
Nariman Jelveh 56b4ef7f2b Leave worker notifications out of the dashboard entirely 2026-08-27 15:59:25 -07:00
Nariman Jelveh 1d3963539f Keep worker notifications out of the dashboard toasts 2026-08-27 15:47:57 -07:00
Juan Castro c4be7fabac fix: settle a permission request from what is already granted
`puter.perms.request()` already pooled a permission read and prompted only
for what was missing. The raw `puter.ui.requestPermission()` did not, so
every caller still on it re-asked the user on each launch — including
`perms.requestAppData()`, whose own docs promise the opposite, and the
driver-denial retry.

- puter.js: `ui.requestPermission()` reads what is held before prompting and
  resolves true when the whole request is covered. Only in env=app and
  env=web, the environments that raise a prompt; elsewhere the method still
  answers false without asking anyone. A check that cannot be made — no
  token, an unreadable request shape, a failed read, or one that outlasts its
  timeout — falls through to the prompt rather than standing in for an
  answer. Public signature unchanged.

- GUI: the request-permission popup asks the same question as the app, using
  the user-app token its own exchange already mints, and skips the dialog
  when the access is held. This is the one case the SDK cannot settle for
  itself: a signed-out site holds no token to check with. An origin the
  browser does not vouch for never reaches the check, since the exchange
  fails first.

Both checks are time-boxed, because each one stands in front of something
that is waiting: the popup's gates the dialog, so a stalled read would leave
the prompt unshown and the opener pending, and the SDK's spends the browser's
transient activation, which a slow read would cost the popup.

Note that driver, service and feature scopes are implicitly granted to every
app (backend/data/hardcoded-permissions.js), so requests for those now
settle silently — the dialog was asking about access the app already had.
Consent scopes (email, fs, apps, subdomains, app-data, app-root-dir) are
unaffected and still prompt until granted.

Fixes a bug this method already had on the way past: `pollDecision` read an
undeclared `permission`, so every attempt threw a ReferenceError into its
network-failure catch and the COOP-severed-opener recovery burned its full
five-minute timeout before answering false. It polls `requested` now, and
requires the whole list.

Tests: the e2e suite drove its dialogs with an implicitly-held driver
permission, so the fixture now asks for a driver nothing implies, fresh per
page load, which also removes the cross-test grant carry-over the old
revokes worked around. The reconciliation tests ask for the held scope plus
an unheld one, since a fully-held request no longer reaches a dialog. Adds a
backend contract test for check-permissions under an app-under-user actor,
which is what the two new client paths rest on.
2026-08-27 16:35:06 -04:00
Nariman Jelveh 167e281cc8 Keep read notifications in the dashboard panel
The panel listed only what the user hadn't dismissed, so acting on a
notification made it vanish. It now shows the 30 most recent regardless
of state, with read/unread as a visual flag: unread rows carry a dot and
a check to mark them read; reading one (check, click-through, toast
close, mark-all, or an ack from another tab) leaves it listed but
quieter. The badge, tab title, and header count follow the unread count.
2026-08-27 13:31:39 -07:00
Nariman Jelveh c6ecc0c786 Soften dashboard notifications empty-state title 2026-08-27 13:14:05 -07:00
Nariman Jelveh f38dec07ea Remove empty-state hint from dashboard notifications 2026-08-27 12:59:01 -07:00
Nariman Jelveh 7332993ff3 Show share notifications in the dashboard (#3653)
* Show share notifications in the dashboard

The dashboard opens its own socket but never listened for notif.* events,
so a file shared with a user on the dashboard was announced to nobody.

Add a notification center to the dashboard: a bell in the sidebar with the
unread count (a dot when collapsed, and on the mobile hamburger), a panel
anchored to it — a bottom sheet on phones — listing what the server still
holds as unacknowledged, and toasts for what arrives live. Clicking a share
lands on Files › Shared with the item selected; dismissing, and "Mark all as
read", acknowledge on the server so other tabs clear too. A toast timing out
is not a dismissal: the entry stays unread in the panel.

Arrivals fold in by uid (a regrouped share rewrites its row and toast in
place), the panel refreshes on open, reconnect and tab focus so shares the
backend folds in silently past the sender's budget still surface, and a
burst on connect is capped at three toasts plus a summary.

Backend: notif.unreads carries created_at so delivered-on-connect items can
be dated, and listings break same-second ties by id so order is stable.

The desktop's mark-ack calls move to a shared helper; UINotification gains an
optional auto-hide timeout that pauses on hover/focus.

* fix: look up notification glyphs and icons by own key only

A notification whose `source` or `icon` was a prototype key such as
`constructor` made the glyph lookup return a function, and the toast
icon builder then threw inside the socket handler — which dropped the
whole burst of toasts it was part of. The list rendered the same entry
as "[object Object]".

* fix: show dashboard toasts above open app windows

App windows opened from the dashboard are stay-on-top and stack in the
99999999+ band, so the toast container at 10000000 sat underneath them:
any notification arriving while an app was open — the usual state of a
dashboard session — was drawn behind the app and never seen.

* fix: reveal the dashboard when a toast is clicked over an open app

Clicking a share toast switches the dashboard to Shared, and the "N new
notifications" toast opens the panel — both inside the dashboard window.
With an app maximized over it, all of that happened out of sight and the
click appeared to do nothing. Minimize the covering app windows first,
the way the minimize controls do, landing the URL on the dashboard's own
route, and act once the history pop has settled so the tab switch is not
traversed over.

* fix: keep keyboard focus in the notifications panel across re-renders

Every change to the list rebuilds its rows, which dropped focus to the
document body: dismissing an entry with Enter threw the keyboard out of
the dialog (the row it had just been moved to was replaced), and so did
any arrival while an entry was focused. Put focus back on the same entry
after a render, or on the one that took a dismissed entry's place, and
let Shift+Tab from the panel itself wrap to the last control instead of
leaving the dialog.

* fix: keep the notifications panel inside short viewports

The anchored panel is bottom-aligned with the bell and grows upward, but
its height was capped only by the viewport (560px or 100vh - 24px), not
by the room above the bell — on a short window the header ran off the
top of the page. Cap it to that room, so the list scrolls instead.

* fix: make notifications with nowhere to go inert in the panel

An entry with no target (a worker deploy result) showed a default cursor
yet, when clicked, was dismissed and took the panel with it — nothing
opened, the entry was gone, and the list had closed. Render such entries
as text with the ✕ as their only action; only share entries remain
buttons. The ✕ now names its notification for screen readers, since on
an inert entry it is the only focusable control.

* fix: stop a listing in flight from resurrecting dismissed notifications

The list is refreshed on open, on reconnect, and when the tab comes back
into view; a dismissal landing while that listing is in flight was undone
when it resolved, since the server's snapshot predates the ack. When the
listing also resolved after the server's own ack event, nothing removed
the entry again and it stayed until the next refresh. Leave out of the
reconciliation whatever was acknowledged — here or in another tab —
since the listing was requested.
2026-08-27 12:49:20 -07:00
Daniel Salazar 909949c68d fix: failed email alarms (#3651) 2026-08-26 23:22:14 -07:00
Juan Fernando Castro 5a6a1020a1 Merge pull request #3644 from HeyPuter/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat
🔧 PUT-1585: Share flag in readdir shared users in stat
2026-08-26 18:51:30 -04:00
Nariman Jelveh 7472002cf3 feat: add Hoonify AI chat provider (#3499) 2026-08-26 15:39:29 -07:00
Juan Fernando Castro 7bd6a2d89b Merge pull request #3646 from HeyPuter/juancastro/put-1590-sharing-shared-files-are-not-different-than-regular-files
🔧 PUT-1590: Mark shared items in the file listings
2026-08-26 17:55:19 -04:00
Juan Castro 3f7dac63b9 Merge remote-tracking branch 'origin/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat' into juancastro/put-1590-sharing-shared-files-are-not-different-than-regular-files
# Conflicts:
#	src/gui/src/UI/Dashboard/UIShareModal.js
#	src/gui/src/UI/UIWindowShare.js
2026-08-26 17:06:25 -04:00
Juan Fernando Castro 3d28b2bb99 Merge pull request #3647 from HeyPuter/juancastro/put-1586-1599-share-dialog-correctness
🐛 PUT-1586 + PUT-1599: Share dialog says what it can grant, and what it did
2026-08-26 16:58:36 -04:00
Juan Castro 63688f81b4 Merge branch 'main' into juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat 2026-08-26 16:55:51 -04:00
Juan Fernando Castro 14ac6542e3 Merge pull request #3645 from HeyPuter/juancastro/put-1589-shared-file-fs-events-seem-to-be-lost
🐛 PUT-1589: Shared file fs events seem to be lost
2026-08-26 16:54:09 -04:00
Juan Castro 5739f4e59f fix: stop pushing file metadata to revoked recipients (PUT-1611)
The realtime fan-out resolved its audience straight from the `share`
index, which has no live-grant check. `/auth/revoke-user-user` deletes
the permission and leaves the index row, so a revoked recipient's socket
kept receiving name, size, masked path and mtime for every write and
move under the folder, with no expiry.

The service already solves this elsewhere — `#reachingHolders` returns
exactly the holder/entry pairs whose grant still stands, and
`listSharedWithMe` was moved onto it for the same reason. The realtime
path never got the same treatment; it does now.

Free on the unshared path: with no share rows reaching the entry there
are no holders to check, so the write path every user takes is unchanged.
Pinned by a test that counts permission reads.
2026-08-26 16:35:38 -04:00
Juan Castro cae51ce67a fix: trashing a top-level share is a removal, not a move
A grant on the entry itself is keyed on uuid, so it follows the entry
into the owner's Trash. Both ends of the move then resolved, and the
recipient was told the shared item had moved — to the GUID name Trash
gave it. Their own copy got renamed to a GUID and stayed on screen.

`shared-with-me` has always omitted trashed entries, so the listing and
the event disagreed; only the event was wrong. Trashing now reports
item.removed at the path the recipient knew, which is also what the
desktop's data-path selector needs to find the row. Restoring out of
Trash reports item.added.

A move that leaves the recipient's masked address unchanged now stays
quiet — a share masks its own root, so the owner shuffling it around
their tree is invisible to the recipient and the event carried nothing.
2026-08-26 16:25:02 -04:00
Juan Castro 8e622189ca fix: tell every recipient when a shared file disappears
Both ways a file can vanish were silent for anyone holding the folder
above it, so a third party's window kept showing a file that was gone
and 404'd on click.

A delete built its audience from the permission rows it removed, and a
file inside a shared folder has no grant of its own — only the folder
does — so the audience was empty. A move resolved its audience from the
entry's new path, and the GUI's Delete is a move to the owner's Trash,
where no recipient has a share.

Resolve the audience from where the entry was rather than from the
grants that went with it:

- Deletes also fan out to holders reaching the entry through an
  ancestor, coalesced by parent so a subtree stays a couple of queries.
  A holder covered by both passes is told once.
- Moves resolve both ends. Reaching both is item.moved, only the
  destination item.added, only the origin item.removed.

Recipients are named by the path they knew, masked through their own
share rather than the owner's tree.
2026-08-26 14:54:06 -04:00
Daniel Salazar c1c830fc32 docs: add puter site config docs (#3648) 2026-08-26 11:11:25 -07:00
Juan Castro cf00ebb9d4 Merge branch 'main' into juancastro/put-1589-shared-file-fs-events-seem-to-be-lost 2026-08-26 13:33:24 -04:00
Juan Fernando Castro b05ee63cb2 Merge pull request #3641 from HeyPuter/juancastro/put-1568-rename-srcguisrchelpers-files-to-camelcase
🧹 PUT-1568: Rename GUI helper files to camelCase
2026-08-26 13:06:58 -04:00
jelveh 40667bc811 Open Document Picture-in-Picture windows on behalf of apps
Browsers only allow documentPictureInPicture.requestWindow() from a
top-level document, and an app lives in an iframe, so an app calling it
gets NotAllowedError ("only allowed from a top-level browsing context").
The `document-picture-in-picture` token in the iframe's `allow` list does
nothing — it is not a policy feature the browser knows. Video PiP
(video.requestPictureInPicture) already works inside apps.

The GUI is the top-level document, so a new PictureInPictureService opens
the window for the app and fills it with an iframe of a page the app names,
which must come from the app's own origin (checked against the message's
origin, now carried on the IPC caller context). One window per app
instance; it closes with the app's window, and the app hears about a close
it didn't ask for. The window's opener is the GUI, so the page inside it
can reach its app's frame through parent.opener.frames and share objects
directly — a MediaStream included, which postMessage cannot carry (tracks
are not transferable between windows in Chromium).

puter.js gains puter.ui.requestPictureInPicture({ url, width, height,
onClose }) and puter.ui.exitPictureInPicture(), with docs.
2026-08-26 09:10:02 -07:00
Juan Castro 4b3d1112d1 Say when a share changed nothing
Sharing a file with someone who already had it answered "Shared with
X", the same as a first share, so the dialog claimed to have done
something it had not.

The service already knew — it computes isNew to decide whether to
notify the recipient — but the flag stopped at the controller. It now
travels on share results only; a listing describes standing access and
says nothing about it.

Which of the three things happened is settled client-side, since the
mode each recipient holds is already on screen: created access, raised
or lowered it, or changed nothing. That keeps the previous mode off the
wire, and an older backend that omits isNew still reads as a share,
which is what these dialogs said before.
2026-08-26 11:57:05 -04:00
Juan Castro 70edef561f Stop offering an access level a delegate cannot grant
A recipient given "can edit & share" could not pass that level on: the
dialog offered it, the server refused it, and the refusal was a bare
403 Forbidden that reads as a bug. Handing out manage needs authority
over manage, which only the owner has — the refusal is right, the dead
end and the silence were not.

The dropdown now withholds it from anyone who does not own the item; a
row already set to it keeps it, so opening the dialog cannot downgrade
the owner's own grant, and a mixed selection follows its strictest item.

The server says why, and only to someone who can already share the item
— a stranger still gets the ACL's own safe error, which does not admit
the node exists.

Verified against a running server: a delegate grants read and write as
before, and manage now answers cannot_delegate_manage with a sentence
naming the owner as the one who can.
2026-08-26 11:48:54 -04:00
jelveh e430ade7bd Reach Shared and Trash from the dashboard Files tab on phones
Below 480px the Files tab hides its directories sidebar, which was the only
entry point to Shared (a query, not a directory) and Trash (filtered out of
the Home listing). Home now carries a row for each; CSS shows them only at
the widths where the sidebar is hidden, using the complementary breakpoint so
the two never both show or both disappear.

The rows borrow the item markup for layout but aren't `.item`, so sorted
insert, selection restore, share-link selection and socket updates ignore
them. The footer count and keyboard select-all exclude them explicitly, and
the placeholder-removal sweeps leave them in place. They navigate on tap and
offer the same menu as their sidebar entry via the ⋯ button or long-press;
the Trash icon tracks empty/full alongside the sidebar's.

Also guard the `window.user.directories` lookup in renderDirectory: it is
undefined for some sessions, and since `puter://shared` doesn't look like a
path it always reached that branch, threw outside the try, and left the
spinner up with navigation stuck — from the desktop sidebar as well.
2026-08-26 08:18:13 -07:00
Juan Castro 519b132355 Inset the shared marker onto the icon glyph
It sat on the icon box's corner, which is 5px outside the artwork on
every side, so the dot read as clipped — half of it hanging over empty
padding with the icon's own drop-shadow falling across it. Nudged in on
both axes: 7px on the desktop, 2px on the dashboard rows.
2026-08-26 11:16:49 -04:00
Juan Castro b08dc70654 Move the shared marker onto the icon's corner and shrink it
Review feedback: too big, and in the wrong place. It sat in the badge
cluster, which the dashboard pins to the row's top-left corner rather
than to the icon — and at 12px it dominated a 24px row icon.

Now a dot on the icon itself, lower-right: 9px on the desktop's 45px
icons, 7px on the dashboard's 24px rows. Anchoring to .item-icon rather
than to the badge cluster is what keeps it on the corner at both sizes.

Dropping the people glyph with it — unreadable at either size, and
colour was the signal the ticket asked for.
2026-08-26 11:07:48 -04:00
Juan Castro a1d027e4a1 Mark shared items in the file listings
A shared file looked exactly like a regular one. The data to tell them
apart arrived with the readdir/stat share flag; nothing rendered it.

Adds the badge to both listings — UIItem (desktop, explorer windows,
file dialogs) and the dashboard's Files rows — fed from is_shared, and
keeps it in step with the share dialogs: both funnel every grant, mode
change and revoke through one render, so the badge follows without
waiting for a re-listing.

Inherited access is deliberately not badged. It is a state of the folder
that was shared, so marking every file inside would repeat one fact on
hundreds of items; the backend flag is direct-only for the same reason.

The icon (owner-shared.svg) and the strings (item_shared_by_you, in 40
locales) were already in the tree, unused — only the wiring was missing.
The blue ring is doing the work: list view shrinks badges to 8px, where
a glyph is illegible and the white circle the sibling badges use
disappears into the row.
2026-08-26 10:56:10 -04:00
jelveh defa37c106 Fix dashboard files tab mobile scrolling
Replace the Files tab's viewport-height sizing with flex-based layout sizing through the dashboard content chain. This keeps the internal file list within the visible content area on mobile browsers, preventing the bottom rows from being clipped when browser toolbars are shown.
2026-08-26 07:42:44 -07:00
Juan Castro 1514ab3cb5 Ignore fs events replayed by replication
Raised in review: could an event from another node re-trigger the
fan-out? Not today — broadcast carries outer.* and pubsub.* only, so
fs.* never crosses a node boundary, and the emitted outer.gui.* is
consumed on the peer by SocketService while ShareService listens to
fs.* alone, so nothing re-enters.

That safety is a property of what broadcast happens to replicate, which
is not this service's to rely on. The handlers now skip anything tagged
from_outside: the node that did the write has already told the audience,
and a second fan-out would only duplicate it.
2026-08-26 09:22:49 -04:00
roar_zx ca25915a38 docs: link Dutch and Odia README translations (#3635) 2026-08-26 00:59:16 -07:00
Juan Castro ee2b5adcf9 Tell share recipients about creates and renames
A recipient's client keeps its cache fresh from fs events pushed over
their socket, and ShareService fans those out to holders — but only for
write, move and delete. A new entry emits fs.create.<flavor>, not
fs.write.file, and an in-place rename emits fs.rename; neither had a
listener, so a recipient watching a shared folder never learned that a
file appeared in it or was renamed. Part of why: those keys and
outer.gui.item.renamed were missing from the typed event map, so a
listener for them did not compile.

Delivering the event is only half of it. Paths were masked against the
entry itself, so item.added named a parent no cached listing was keyed
on, and the payload carried no dirpath, which is how the desktop finds
the container to render into — the event would have arrived and changed
nothing. Paths are now masked at the share the holder reached the entry
through, which is the address their own reads returned, and from_path on
a move and old_path on a rename travel the same way (dropped when the
move started outside the share, self-masked when the share is on the
entry itself, where the root already carries the new path).

Creates fire per entry, so an upload would have cost one share lookup
per file; they are coalesced by parent folder the way subtree deletes
already are. Measured on a 25-file burst into one folder: 25 lookups
before, 1 after. A holder with a share on both a folder and something
inside it is told once, by the nearer of the two.
2026-08-25 19:38:38 -04:00
Juan Castro 2ef5d00d9b Type is_shared and say who the flag counts
stat() and readdir() return FSItemRead, so the is_shared the docs lead
with typechecks for TypeScript consumers rather than erroring on FSItem.

The docs said "you have shared", but the query has no issuer predicate:
a manage delegate's re-share sets the owner's flag too, which is the
useful answer and matches getShares().
2026-08-25 18:15:15 -04:00
Juan Castro bac3dde945 Forget cached entries when sharing changes
Nothing invalidated the SDK entry cache on share or unshare — the socket
handlers only fire on item mutations — so is_shared, which now rides in
the cached entry, stayed stale for every consistency: 'eventual' read.
That includes the GUI's own listing refresh, which defaults to it, so a
badge would not have appeared until an unrelated write flushed the cache.
2026-08-25 18:15:05 -04:00
Juan Castro 1600dc38a5 Retire the share index row with the grant it records
revoke-user-user withdraws a grant without touching the share index, so
the row outlived the access — invisible until now, because listSharesOf
filters against live grants, but the new flag reads the index and would
report a file as shared to nobody, permanently.

Drop the row where the grant goes. The alternative, filtering liveness on
the read side, is the per-entry work the flag exists to avoid.
2026-08-25 18:14:52 -04:00
Juan Castro 0c8dd8b6e4 Document the share flag and returnShares 2026-08-25 17:45:15 -04:00
Juan Castro 3217533594 Give the puter.js API suites rate-limit headroom
Around 500 tests run through one seeded free-tier account, so the
per-tier windows were throttling the suite rather than anything it
tests — adding a single readdir was enough to trip fs:readdir-burst.
Resolve the seeded users to the unlimited policy, alongside the
subscription gate the harness already turns off for the same reason.
Test config only; no published limit changes.
2026-08-25 17:44:56 -04:00
Juan Castro 0591f13cc2 Expose is_shared and returnShares in puter.js
Carry the flag into the v1 entry shape and add returnShares to stat(),
whose shares are mapped into the same Share objects getShares() returns.
returnShares joins the dedupe key and bypasses the entry cache in both
directions — a result carrying share data is never written to it, so a
later plain stat cannot serve other people's names from cache.
2026-08-25 17:44:50 -04:00
Juan Castro ee2f14576b Report sharing in stat and readdir
isShared on /fs/stat and /fs/readdir, is_shared on their legacy
counterparts, and return_shares now fills the shares array the legacy
stat has been stubbing with []. Null for entries the caller does not own,
so a share recipient is never told who else can reach the owner's files.
The share-to-wire mapper moves out of ShareController so both controllers
publish one shape.
2026-08-25 17:44:41 -04:00
Juan Castro 5115fdd48c Answer which entries are shared in one indexed query
Directory listings need a shared/not-shared flag per entry. Ask the share
index once per batch, chunked, rather than a query per entry, and answer
only for entries the caller owns so listing a foreign directory never
queries at all. Direct shares only — inherited access stays with
listSharesOf, which already reports it.
2026-08-25 17:44:25 -04:00
jelvehandClaude Fable 5 e006b19773 puter.peer: room names, relayed guest grants, servers that come back
A peer server could only be reached by the invite code the signaller
minted for it, which is unknowable ahead of time and dies with the
server's socket. Any app that wanted a link to share in advance or reuse
had to run a registry of its own that maps a stable code to whatever
invite is live — and keep it live, since the signaller socket can drop
without the library saying a word (`onclose` was an empty function),
leaving a host serving an invite nobody can dial.

`serve({ name })` serves under a room name of the caller's choosing;
`connect(name)` dials it. A name is held by whoever serves it and free
once they stop; serving a held name from another identity rejects with
`name_in_use`, from the same identity it takes over. A connection to a
room nobody serves fails with `no_host`, so a lobby has a definite answer
to poll on instead of guessing between "not yet" and "gone".

`serve({ guestGrant })` and `server.setGuestGrant()` leave a guest grant
with the signaller, which hands it to every anonymous guest in the connect
reply; the connection redeems it before making its offer, so a guest with
no session gets relays without the app publishing the grant anywhere.

A server whose signaller socket drops now re-registers on its own with
backoff — under the same name, or a fresh code announced by the new
`reconnect` event — and pings the socket to keep idle proxies from cutting
it. It fires `close` (`replaced`, `name_in_use`) when it has to stand
down for good. Existing connections are peer-to-peer and never affected.

Signaller refusals reach the connection's `error` event as an Error with
a `code` next to the message. Keepalive replies and stray frames no longer
throw in the message handlers.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-25 13:24:45 -07:00
Juan Castro 07e73efa40 Merge remote-tracking branch 'origin/main' into juancastro/put-1568-rename-srcguisrchelpers-files-to-camelcase 2026-08-25 15:10:29 -04:00
404oops 2a45a10854 Merge pull request #3630 from HeyPuter/FK/dedupe
refactor: dedupe model catalog aliases and stop mutating shared catalogs
2026-08-25 18:07:27 +02:00
Nariman Jelveh e052aec7fb Carry a share link through the OIDC round trip (#3642)
A share email lands on `/?shared=…`, and its recipient is signed out or
in a temporary session more often than not. Password login reloads the
same URL, so the link came back with it — but an OIDC login leaves the
origin and returns to a URL the backend builds from a whitelist of bare
paths. `return_to` was a pathname only, and `/` wasn't on the list, so
the recipient came back signed in on the Dashboard's Apps tab with
nothing to say what had just been shared with them.

`shared` now travels with the path. The GUI sends it along with the
return path; the backend accepts it as the one parameter a `return_to`
may carry, and only values shaped like the masked share path the mail was
built from — everything else rejects the whole value, the path is still
matched as a raw string so a protocol-relative one can't smuggle an
origin through, and the query is rebuilt rather than reflected. The items
ride onto the error page too, so a failed attempt retries on the link
instead of losing it.

The root joins the whitelist for this, but only when it names something:
on its own it is where the flow already lands.
2026-08-25 08:42:47 -07:00
Daniel Salazar cf4d743854 fix: testing for peer (#3643) 2026-08-25 08:31:45 -07:00
Daniel Salazar 3c866c645b feat: guest turn for peer (#3640) 2026-08-25 07:51:02 -07:00
Juan Castro 4a5d689af7 refactor: rename GUI helper files to camelCase
src/gui/src/helpers was mostly snake_case, disagreeing with itself and
with the camelCase file convention in AGENTS.md. Rename the 45 offenders
with git mv and repoint every import specifier at the new paths.

Files only — exported symbols are unchanged. applyItemAddedToContainers.js
still exports apply_item_added_to_containers and no call site is edited
beyond its import line; those identifiers cross module boundaries and some
hang off window, so renaming them is a separate, riskier change.
2026-08-25 10:49:59 -04:00
jelveh 6f1548e26a Move Share above Download in file context menus 2026-08-25 07:28:53 -07:00
Daniel Salazar 764ce2f8e2 feat: rep score middleware gateway (#3639) 2026-08-25 07:11:41 -07:00
Daniel Salazar 717c098503 fix: metering buffer improvements for manual editing (#3633) 2026-08-24 18:36:43 -07:00
Nariman Jelveh 2c9a5c4f7e Land share-email links in the Dashboard's Shared view (#3634)
* Land share-email links in the Dashboard's Shared view

A share email's links opened the item on the desktop, and "Open Puter"
went to the bare origin — the recipient arrived on Home with nothing to
say what had just been shared. Every link now lands in the Dashboard's
Files tab, on Shared, with the shared items selected.

Each named file still links to itself. "Open Puter" carries every item
in the mail (?shared=a&shared=b), so the whole batch lands highlighted;
the digest records now keep each item's masked path for that. The
button's href is rendered raw like the item links, so `=` and `&` read
the same in the html and text parts, and the unsubscribe link is built
from a separate origin value.

On the GUI side `/?shared=` boots the dashboard rather than the desktop;
`/desktop?shared=` keeps opening the item in place. The Files tab
selects the rows by uid, which survives a rename. A share only reaches a
real account, so a share link never mints a temp user and an existing
temp session is asked to sign in first, as the desktop already did. The
address-bar cleanup both shells need moves into a helper.

* Keep the share email's button link within what mail clients tolerate

The "Open Puter" link names every item in the mail, capped at twenty. But
a single item's parameter is ~150 characters once the owner, uuid and an
encoded name are in it, so twenty of them run to several kilobytes — past
the ~2000 characters where older mail clients cut a URL off or stop making
it clickable — and the count alone couldn't do what its comment promised
for the mail's primary button.

Add a length budget beside the count: items go in, in digest order, only
while the whole link stays under it, so the button always works and the
first items are the ones highlighted. A single item always fits.

* Keep a share link's own item however long its name runs

`shareDeepLink` now builds through `sharedViewLink`, whose length budget
applied to the first item too: a parameter that alone overran it was
dropped, and the link came out as a bare `?shared=`. A name of a few
hundred characters — the GUI allows five hundred, and encoding triples
every non-ASCII one — was enough, so that item's own link in the mail
landed on Shared with nothing picked out, where it used to open the item.

The first item now always goes in; the budget only decides how many more
join it. One long link is still the item the mail is about, and it is no
worse than a bare origin for the clients that truncate it.
2026-08-24 17:01:55 -07:00
jelveh 34af8d18d0 Share a whole selection from the Dashboard's Files tab
Sharing was one item at a time: the only way to give someone four files
was to open the dialog four times. A multi-selection now offers Share in
its context menu and in the mobile selection bar, and UIShareModal takes
a list of items.

With several items the access list folds into one row per person rather
than repeating per file, so a grant, a mode change or a revoke is one
decision about a person. A row says what it can't otherwise show: how
much of the selection the person reaches ("On 2 of 4 items", with an
"Add to all" that extends them), a mode select that rests on a
placeholder when their grants disagree rather than presenting one item's
mode as the batch's, and inherited grants left uncontrollable where they
belong. Changing a mode touches only the items the person already holds
— nothing here widens access without saying so.

Requests are chunked to the documented 50-items-per-request cap, and a
selection past it skips the per-item listing instead of firing one
request per file on every refresh.

The eligibility rule the Share entry already used moves into a shared
can_share() helper, since the multi-select menu needs the same answer
for every row.
2026-08-24 14:08:37 -07:00
Juan Fernando Castro 0527bcd98e Merge pull request #3632 from HeyPuter/juancastro/put-1560-file-sharing-deeplink-in-emailnotifications
🔧 PUT-1560: Deep-link the shared item from email and notifications
2026-08-24 12:30:07 -04:00
Juan Castro ea530e604c Merge branch 'main' into juancastro/put-1560-file-sharing-deeplink-in-emailnotifications 2026-08-24 12:21:21 -04:00
Juan Fernando Castro 97d137b94b Merge pull request #3623 from HeyPuter/juancastro/put-1564-perms-docs-drop-the-app-root-dir-raw-permission-string-wire
🛠️ PUT-1564: Perms docs — drop the app-root-dir raw string, wire the playground examples
2026-08-24 10:59:56 -04:00
jelveh 1ceffbe291 Add sudoku to recommended apps
This change adds Sudoku to the curated recommended apps list so it appears alongside the other bundled games in the app recommendations flow.
2026-08-23 21:59:52 -07:00
Daniel Salazar 684d6752f9 fix: provide fallback for signup verification (#3631) 2026-08-23 21:49:35 -07:00
jelveh 76cea02513 Treat .md like .txt; remove markus suggestion
Merge the .md extension case into the plain-text branch so it suggests ['editor', 'code'] instead of ['markus', 'editor', 'code']. Update the related test to use 'viewer'/'png' as the built-in guard example since 'markus'/'md' no longer applies.
2026-08-23 21:17:28 -07:00
404oopsandClaude Fable 5 308b78e96c refactor: dedupe model catalog aliases and stop mutating shared catalogs
- Add modelLookupNames() and route every provider's list() through it so
  repeated or self-referential aliases can't double-list a model.
- Stop the chat/image/video drivers' #buildModelMap from writing through
  to the module-level catalog constants (ids/aliases accumulated across
  instantiations); bucket copies preserve the models() wire shape.
- Remove self-aliases and exact duplicates from the claude, deepseek,
  groq, mistral, and openai catalogs; fold the duplicated
  gemini-3.7-flash entry — this corrects the live cached-token price
  from 8 to 7.5 and the knowledge date to 2026-03 (the previously
  effective first entry carried stale values).
- ClaudeProvider: derive default max_tokens from the resolved entry
  instead of a second name-based lookup that capped dated ids at 4096.
- Add cross-provider catalog invariants (modelCatalogs.test.ts) with a
  disk-scan registration guard, plus regression tests for catalog
  non-mutation, alias-dedup routing, wire shape, and max_tokens ceilings.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-24 03:42:47 +02:00
nevrkla 2d787c3fdf feat(i18n): add Czech GUI localization (#3617)
* feat(i18n): add Czech GUI localization

* feat(i18n): add Czech GUI localization

* fix(i18n): polish Czech GUI translation
2026-08-23 17:22:44 -07:00
Juan Castro 202c43f46e feat(share): deep-link the shared item from email and notifications
A share email named the file but had nowhere to go: the only link was "Open
Puter", and finding what someone shared meant hunting for it under Shared. Each
named file in the digest now links to itself, and a notification covering a
single item points at that item.

The link carries one parameter, the masked path a recipient is already given:

    https://puter.com/?shared=%2Falice%2F<uuid>%2Freport.txt

Its second segment is the uuid, so there is no second copy of it to disagree
with the path, and the GUI can still find the entry when a rename has left the
name segment stale - it stats the path, then falls back to the uuid.

Built from the owner, uuid and name rather than from `ResolvedShare.path`.
That path is masked for whoever made the request, and the issuer owns the
entry, so it comes back as the owner's *real* path - mailing it would tell the
recipient which folders the owner keeps things in, which is the one thing
masking exists to prevent. A test asserts the real path never reaches the mail.

`digestLines` now returns `lead`/`items`/`trail` beside `what`, so the template
can put an anchor around each name while Handlebars keeps escaping the names
themselves; the URL is machine-built from the configured origin and one encoded
path, so it stays literal. Concatenating the parts reproduces `what` exactly,
which a test pins - the linked and sentence forms must not describe different
shares.

Notifications carry the masked path rather than a URL: the recipient is already
in the GUI, which opens the item in place instead of reloading. Only a
single-item notification gets a target; folding into a group drops it rather
than picking one of five.

In the GUI, `?shared=` joins `?download=` and `?app=` as a param that keeps the
desktop booting at `/`, and the handler reuses the `/@user` public-folder flow -
extracted to `open_path_target`, which carried a TODO asking for exactly this -
so a file opens in its associated app and a folder in an explorer window. The
param is stripped from the address bar first, so a reload lands on the desktop
rather than opening the item twice.

Invites are deliberately not linked: there is no account to route to yet, and
the invite's own call to action is to create one.

Rolling-deploy safe: a digest entry queued before this has `names` and no
`items`, and still flushes - without links. New entries write both, so a node
on the previous build can flush them too.
2026-08-21 16:08:39 -04:00
Juan Castro e7be457a56 docs: redirect the retired perms pages, add the missing playground examples
Both were left as follow-ups on the previous commit.

Thirteen `puter.perms` method pages went when the surface collapsed onto
`request(resource, details)`, and `requestAppData` moved to `/Perms/appData`.
All fourteen had shipped, so an external link or a bookmark landed on nothing.
The build already generates meta-refresh redirects from a map; they are entries
in it now. Pages that only ever existed on the branch that removed them are
deliberately absent — nobody can hold a link to a URL that was never published.

Six documented APIs annotated a code block with a playground id but had no
example file behind it, so the build warned on every one and the Try-it link
resolved to nothing: `fs.share`, `fs.unshare`, `fs.listShared`, `fs.getShares`,
`ai.txt2speech` with Speechify, and `ui.showFeedbackDialog`. The files are the
documented blocks themselves, extracted rather than rewritten, so the example
and the page it appears on cannot drift. Each is indexed in `examples.js` beside
its siblings.

The build now runs clean: 12 warnings to none.
2026-08-21 11:26:32 -04:00
Juan Castro 9bf49b7574 docs(perms): drop the app-root-dir raw string, wire the perms playground
`app-root-dir:` was listed among the raw permission strings, but it reads as
nothing during a permission check - the rewriter yields the sentinel outside a
grant, so `check()` reports it ungranted even once it is and `request()` prompts
every call. Both contradict what the same page promises. The `'appRootDir'`
resource is the supported path and asks the server instead; the list now says so
rather than implying the string behaves like its neighbours.

The playground listed no perms examples at all: six files existed but nothing
referenced them in examples.js, so a freshly-reshaped API shipped with nothing
runnable. Adds a Perms group after UI, matching the sidebar's order, and two
examples for the features that had none - one prompt covering several resources,
and check() deciding whether to prompt.
2026-08-21 10:20:05 -04:00
821 changed files with 113557 additions and 7012 deletions

No files matched your search

+61
View File
@@ -0,0 +1,61 @@
# Default owner, including backend and SDK areas without a specialist below.
# The last matching rule takes precedence.
* @Salazareo
# GUI
/src/gui/ @jelveh @Salazareo
# Documentation, developer site, and CLI
/doc/ @reynaldichernando
/src/docs/ @reynaldichernando
/src/dev-center/ @reynaldichernando
/src/cli/ @reynaldichernando
# Backend: AI, hosting, workers, networking, and WebDAV
/src/backend/controllers/puterai/ @ProgrammerIn-wonderland
/src/backend/controllers/hosting/ @ProgrammerIn-wonderland
/src/backend/controllers/webdav/ @ProgrammerIn-wonderland
/src/backend/controllers/wisp/ @ProgrammerIn-wonderland
/src/backend/drivers/ai-*/ @ProgrammerIn-wonderland
/src/backend/drivers/util/ @ProgrammerIn-wonderland
/src/backend/drivers/workers/ @ProgrammerIn-wonderland
/src/backend/services/localworker/ @ProgrammerIn-wonderland
/src/backend/stores/subdomain/ @ProgrammerIn-wonderland
/extensions/workerSandbox* @ProgrammerIn-wonderland
/src/worker/ @ProgrammerIn-wonderland
/src/worker-types/ @ProgrammerIn-wonderland
/src/backend/controllers/peer/ @ProgrammerIn-wonderland
# Backend: sharing, permissions, and teams
/src/backend/controllers/share/ @jfcastro92
/src/backend/controllers/team/ @jfcastro92
/src/backend/services/share/ @jfcastro92
/src/backend/services/permission/ @jfcastro92
/src/backend/services/acl/ @jfcastro92
/src/backend/services/team/ @jfcastro92
/src/backend/services/apps/AppPermissionService* @jfcastro92
/src/backend/stores/share/ @jfcastro92
/src/backend/stores/permission/ @jfcastro92
/src/backend/stores/group/ @jfcastro92
/src/backend/stores/team/ @jfcastro92
# SDK: AI, hosting, workers, and networking
/src/puter-js/src/modules/ai/ @ProgrammerIn-wonderland
/src/puter-js/src/modules/hosting/ @ProgrammerIn-wonderland
/src/puter-js/src/modules/Workers* @ProgrammerIn-wonderland
/src/puter-js/src/modules/networking/ @ProgrammerIn-wonderland
/src/puter-js/src/lib/networkUtils* @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/ai.suite.ts @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/hosting.suite.ts @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/workers.suite.ts @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/net.suite.ts @ProgrammerIn-wonderland
/src/puter-js/src/modules/Peer* @ProgrammerIn-wonderland
# SDK: sharing and permissions
/src/puter-js/src/modules/perms/ @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/share* @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/unshare* @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/getShares* @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/listShared* @jfcastro92
/src/puter-js/tests/api/suites/perms.suite.ts @jfcastro92
/src/puter-js/tests/api/suites/sharing.suite.ts @jfcastro92
+20
View File
@@ -0,0 +1,20 @@
version: 2
updates:
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
@@ -17,10 +17,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
+54 -25
View File
@@ -1,28 +1,39 @@
name: Backend Tests
# Runs on backend and extension changes. Tests are co-located with the
# code (src/backend/**/*.test.ts, extensions/**/*.test.ts), so test-only
# changes match the same globs. puter.js SDK changes are covered by
# puterjs-tests.yaml instead.
# Always triggers so it can be a required PR check; the `changes` job below
# decides whether backend/extension paths actually changed (test-only changes
# match the same globs, since tests are co-located with the code), and every
# other job is skipped — not absent — when they didn't, which still satisfies
# a required check. puter.js SDK changes are covered by puterjs-tests.yaml.
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'src/backend/**'
- 'extensions/**'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
# The type-check job below reads these.
- 'tsconfig.json'
- 'tsconfig.build.json'
- '.github/workflows/backend-tests.yaml'
permissions:
contents: read
pull-requests: write
jobs:
changes:
runs-on: ubuntu-latest
outputs:
backend: ${{ steps.filter.outputs.backend }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
backend:
- 'src/backend/**'
- 'extensions/**'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'tsconfig.build.json'
- '.github/workflows/backend-tests.yaml'
# `tsconfig.build.json` builds with `noCheck: true`, so nothing else in CI
# runs the type checker and a missing export compiles to `undefined`,
# surfacing only when the call is finally reached at runtime. This diffs
@@ -35,13 +46,15 @@ jobs:
# Its own job rather than a step in `test`: that one runs a base/PR matrix for
# coverage comparison, and the check only needs the PR ref, once.
typecheck:
needs: changes
if: needs.changes.outputs.backend == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
@@ -52,7 +65,19 @@ jobs:
- name: Type check (new errors only)
run: npm run typecheck
# `needs.changes` gates the steps below, not this job, on purpose: a
# job-level `if` on a matrix job skips the whole job as one unexpanded
# check ("test (${{ matrix.artifact }})", literally, since there's no
# matrix context to fill it in) instead of the two real per-artifact
# checks a required rule pins to ("test (base)" / "test (pr)") — so the
# required check never gets reported and the PR blocks forever. Letting
# the matrix always expand keeps the check names stable either way.
test:
needs: changes
# Overrides the default matrix naming, which would otherwise bake the
# base/head branch name (matrix.ref) into the check name — making it a
# different string on every PR and impossible to pin as a required check.
name: test (${{ matrix.artifact }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
@@ -65,49 +90,53 @@ jobs:
steps:
- name: Checkout ${{ matrix.ref }}
uses: actions/checkout@v4
if: needs.changes.outputs.backend == 'true'
uses: actions/checkout@v7
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Set up Node.js
uses: actions/setup-node@v4
if: needs.changes.outputs.backend == 'true'
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
if: needs.changes.outputs.backend == 'true'
run: npm ci
- name: Run backend tests with coverage
if: needs.changes.outputs.backend == 'true'
run: npm run test:backend -- --coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
if: needs.changes.outputs.backend == 'true' && !cancelled()
uses: actions/upload-artifact@v7
with:
name: backend-coverage-${{ matrix.artifact }}
path: src/backend/coverage/
retention-days: 14
report-coverage:
needs: test
if: always()
needs: [changes, test]
if: always() && needs.changes.outputs.backend == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Download PR coverage
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: backend-coverage-pr
path: src/backend/coverage
- name: Download base coverage
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: backend-coverage-base
path: coverage-base
@@ -0,0 +1,25 @@
name: Dependabot auto-merge
on: pull_request
permissions:
contents: write
pull-requests: write
jobs:
automerge:
runs-on: ubuntu-latest
if: github.actor == 'dependabot[bot]'
steps:
- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v3
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
- name: Enable auto-merge for patch updates
if: steps.metadata.outputs.update-type == 'version-update:semver-patch'
run: gh pr merge --auto --squash "$PR_URL"
env:
PR_URL: ${{ github.event.pull_request.html_url }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+7 -7
View File
@@ -45,17 +45,17 @@ jobs:
fi
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -64,11 +64,11 @@ jobs:
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
@@ -82,7 +82,7 @@ jobs:
type=raw,value=main,enable=${{ github.event_name == 'push' || inputs.push_main }}
- name: Build and push Docker image
uses: docker/build-push-action@v5
uses: docker/build-push-action@v7
with:
platforms: linux/amd64,linux/arm64
context: .
@@ -95,7 +95,7 @@ jobs:
cache-to: type=gha,mode=max
- name: Sync README to Docker Hub
uses: peter-evans/dockerhub-description@v4
uses: peter-evans/dockerhub-description@v5
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
+2 -2
View File
@@ -11,10 +11,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
+2 -2
View File
@@ -9,7 +9,7 @@ jobs:
update-release-pr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v7
- name: Setup Git
run: |
git config user.name github-actions
@@ -20,7 +20,7 @@ jobs:
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
branch: release
- uses: actions/checkout@v4
- uses: actions/checkout@v7
if: steps.find-pr.outputs.number == ''
with:
ref: release
+61 -27
View File
@@ -1,28 +1,42 @@
name: Puter.js API Tests
# Runs on puter.js SDK changes — the suites live in src/puter-js/tests,
# so test changes are covered by the same glob — plus the pieces the
# runners depend on: the worker preamble, the in-memory test env in
# testUtil, and the backend vitest config the API-tests config extends.
# Backend and extension changes run backend-tests.yaml instead.
# Always triggers so it can be a required PR check; the `changes` job below
# decides whether puter.js-relevant paths actually changed (the suites live
# in src/puter-js/tests, so test-only changes are covered by the same glob,
# plus the pieces the runners depend on: the worker preamble, the in-memory
# test env in testUtil, and the backend vitest config the API-tests config
# extends), and every other job is skipped — not absent — when they didn't,
# which still satisfies a required check. Backend and extension changes run
# backend-tests.yaml instead.
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'src/puter-js/**'
- 'src/worker/**'
- 'src/backend/testUtil.ts'
- 'src/backend/vitest.config.ts'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/puterjs-tests.yaml'
permissions:
contents: read
pull-requests: write
jobs:
changes:
runs-on: ubuntu-latest
outputs:
puterjs: ${{ steps.filter.outputs.puterjs }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
puterjs:
- 'src/puter-js/**'
- 'src/worker/**'
- 'src/backend/testUtil.ts'
- 'src/backend/vitest.config.ts'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/puterjs-tests.yaml'
# The JSDoc in src/puter-js/src is the source of truth for the SDK's public
# types. The declarations shipped to npm are generated from it at build time
# and never committed, so what needs guarding is the JSDoc itself: this
@@ -32,14 +46,16 @@ jobs:
# Its own job rather than a step in `test`: that one builds bundles and
# installs a browser, and this needs neither.
types:
needs: changes
if: needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
@@ -51,14 +67,16 @@ jobs:
run: npm run check:puterjs:types
test:
needs: changes
if: needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
@@ -83,7 +101,18 @@ jobs:
# SDK coverage on both refs, reported as a PR comment — same shape as
# backend-tests.yaml. This run uses the istanbul-instrumented bundle
# (the `test` job above keeps exercising the production build).
# `needs.changes` gates the steps below, not this job, on purpose: a
# job-level `if` on a matrix job skips the whole job as one unexpanded
# check ("coverage (${{ matrix.artifact }})", literally, since there's no
# matrix context to fill it in) instead of two real per-artifact checks —
# confusing to read, and unusable if this ever gets pinned as a required
# check. Letting the matrix always expand keeps the check names stable.
coverage:
needs: changes
# Overrides the default matrix naming, which would otherwise bake the
# base/head branch name (matrix.ref) into the check name — making it a
# different string on every PR and impossible to pin as a required check.
name: coverage (${{ matrix.artifact }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
@@ -96,34 +125,39 @@ jobs:
steps:
- name: Checkout ${{ matrix.ref }}
uses: actions/checkout@v4
if: needs.changes.outputs.puterjs == 'true'
uses: actions/checkout@v7
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Set up Node.js
uses: actions/setup-node@v4
if: needs.changes.outputs.puterjs == 'true'
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
if: needs.changes.outputs.puterjs == 'true'
run: npm ci
- name: Install Playwright chromium
if: needs.changes.outputs.puterjs == 'true'
run: npx playwright install --with-deps chromium
# The base leg is best-effort — it only enriches the PR comment with
# a comparison, and the base ref may predate the coverage script.
- name: Run puter.js API tests with coverage
if: needs.changes.outputs.puterjs == 'true'
continue-on-error: ${{ matrix.artifact == 'base' }}
run: npm run test:puterjs:coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
if: needs.changes.outputs.puterjs == 'true' && !cancelled()
uses: actions/upload-artifact@v7
with:
name: puterjs-coverage-${{ matrix.artifact }}
path: src/puter-js/coverage/coverage-*.json
@@ -131,15 +165,15 @@ jobs:
if-no-files-found: ignore
report-coverage:
needs: coverage
if: always()
needs: [changes, coverage]
if: always() && needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Download PR coverage
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: puterjs-coverage-pr
path: src/puter-js/coverage
@@ -149,7 +183,7 @@ jobs:
- name: Download base coverage
id: base-coverage
continue-on-error: true
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: puterjs-coverage-base
path: coverage-base
+2 -2
View File
@@ -45,14 +45,14 @@ jobs:
fi
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Create alias tag
env:
+6 -5
View File
@@ -35,6 +35,8 @@ These apply everywhere — backend, puter.js, and GUI.
Keep comments light; prefer self-documenting code. Comment only when the _why_ is non-obvious or a usage detail would trip the next reader. Use `//` for single lines and `/** ... */` JSDoc when it genuinely needs more — if a comment runs long, it's probably too long. Don't restate the code and don't reference the current task, PR, or version — those rot. **No ticket references** (`PUT-1234`, `// fix for FOO-99`) in code, comments, or test names; describe the why in domain terms, not project-management terms. Use plain ASCII `-` in comment section dividers (`// -- Section --`), never box-drawing characters.
**Be concise — a comment is a sentence or two, not a paragraph block.** One or two lines for `//`, a few for JSDoc. Don't narrate the history ("used to swallow the failure, which meant…"), don't argue the design at length, and don't explain each parameter when the signature already does. State the constraint and stop. If the reasoning genuinely needs paragraphs, it belongs in a design doc, not above the function.
### Security & privacy
Before opening a PR, scan the diff for:
@@ -92,6 +94,7 @@ Follow the same layered structure inside an extension — unless it only needs a
- Vitest; test files sit next to the code they test (`*.test.ts` / `*.test.js`). Run with `npm run test:backend`.
- **Mock data, not methods.** Stub inputs (fixtures, fake rows, payloads), not the function under test or the layer beneath it — over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service).
- **Prefer the test server over mocking deps.** `setupPuterTestEnv()` in [src/backend/testUtil.ts](src/backend/testUtil.ts) boots a fully in-memory backend; hit a real database/client shape where reasonable — integration shapes catch what mocked unit tests miss.
- **Test code, not docs.** Never write a test that reads a file under `src/docs/` and asserts on its wording or numbers. Docs are kept in step by the PR (see the limits rule above) and checked in review; a test that greps a markdown page fails on every rewording and verifies nothing about behavior.
---
@@ -112,17 +115,15 @@ Declare a shape where it belongs, and reference it with an `import(...)` type fr
Use `@typedef {Object}` + `@property` for any shape whose fields need documenting — it is the only JSDoc form that carries a doc comment per field into the generated declaration. Keep the inline object-literal form for small internal shapes with nothing to say about each field, and prefer `unknown` over `*`:
```js
/** @typedef {{ key: string, value: unknown }} KVEntry */
/** @typedef {{ key: string; value: unknown }} KVEntry */
```
Public (exposed) methods must carry JSDoc types — parameters and return value — with one `@overload` block per accepted call form, since those overloads *are* the published signature. Unexposed/private helpers are typed at the contributor's discretion: annotate where it helps the next reader, and either way keep them clean. Members tagged `@internal` are stripped from the generated declarations, so use that tag rather than `@private` to keep something off the public surface.
Public (exposed) methods must carry JSDoc types — parameters and return value — with one `@overload` block per accepted call form, since those overloads _are_ the published signature. Unexposed/private helpers are typed at the contributor's discretion: annotate where it helps the next reader, and either way keep them clean. Members tagged `@internal` are stripped from the generated declarations, so use that tag rather than `@private` to keep something off the public surface.
Typing in JS files is encouraged: annotate with JSDoc `@type`/`@param`/`@returns` using the TypeScript type system, and define shared shapes with `@typedef`. API types must not be `unknown` or untyped `...args` — spell out the real parameter and return shapes; the only exception is values passed through transparently to an upstream layer that owns their type. For example:
```js
/**
* @typedef {{key:string, value: unknown}} KVEntry
*/
/** @typedef {{ key: string; value: unknown }} KVEntry */
/** @type {KVEntry[]} */
let entries = [];
+2
View File
@@ -113,7 +113,9 @@ This repository, including all its contents, sub-projects, modules, and componen
- [Korean / 한국어](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ko.md)
- [Malay / Bahasa Malaysia](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.my.md)
- [Malayalam / മലയാളം](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ml.md)
- [Dutch / Nederlands](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.nl.md)
- [Polish / Polski](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pl.md)
- [Odia / ଓଡ଼ିଆ](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.od.md)
- [Portuguese / Português](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pt.md)
- [Punjabi / ਪੰਜਾਬੀ](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pa.md)
- [Romanian / Română](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ro.md)
+3
View File
@@ -22,6 +22,9 @@
"default_temp_group": "b7220104-7905-4985-b996-649fdcdb3c8f",
"storage_capacity": 104857600,
"disable_user_signup": false,
"teams_enabled": false,
"max_teams_per_user": 1,
"max_seats_per_team": 50,
"strict_email_verification_required": false,
"gui_assets_root": "./src/gui",
"puterjs_root": "./src/puter-js/dist",
+29 -1
View File
@@ -273,6 +273,31 @@
// "emailBatchSeconds": 90
// },
// ── Teams / teams ──────────────────────────────────────────────
// Off by default, and off is the state every existing install stays in.
// The tables ship either way and sit inert; this gates whether `/teams`
// is registered at all, so with it off the paths do not exist rather than
// existing and refusing. It is also the backout: turning it off removes
// the feature without touching data.
"teams_enabled": false,
//
// Live teams one user may own. Soft-deleted ones do not count, so
// deleting yours frees the slot. Default 1.
"max_teams_per_user": 1,
//
// Seats one team may provision. Each seat is a real Puter account
// taking a name from the global username pool, so this is what bounds a
// team's blast radius until per-seat billing lands. Lowering it below
// a team's current seat count blocks new provisioning and disables
// nobody. Default 50.
"max_seats_per_team": 50,
// ── Notifications ───────────────────────────────────────────────────
// How long a notification is kept, in days from creation. Acknowledged or
// not, a row past this is swept. Off by default; any value above 0 arms
// the sweep.
// "notificationRetentionDays": 14,
// ── Alarms / alerting ───────────────────────────────────────────────
// Where system alarms go. Severity is the routing decision — each
// transport takes everything at or above its own `minSeverity`:
@@ -377,6 +402,10 @@
},
"zai": { "apiKey": "" },
"alibaba": { "apiKey": "" },
"hoonify": {
"apiKey": "",
"apiBaseUrl": "https://api.hoonify.ai/v1"
},
"together-ai": { "apiKey": "" },
// Local Ollama. `enabled: false` skips the auto-probe at startup
// (otherwise Puter logs ECONNREFUSED on every boot when no Ollama
@@ -398,7 +427,6 @@
"xai-image-generation": { "apiKey": "" },
// ─ Video generation ─
"openai-video-generation": { "apiKey": "" },
"together-video-generation": { "apiKey": "" },
"gemini-video-generation": { "apiKey": "" },
+13 -1
View File
@@ -78,6 +78,11 @@ this.clients.alarm.create(alarmId, message, fields, 'critical', {
The HTTP error handler uses it: its id is route + error signature, so a hot
loop of the same crash is one incident with N occurrences instead of N pages.
Dedup and Slack's repeat throttle mean a responder sees only the latest
occurrence, so the handler attaches the `HttpError`'s `fields` to the alarm as
`details` — put what a responder needs (which upstreams failed, and how)
there. Everything in `fields` is also returned to the client in the response
body, so it has to be safe to show the caller.
Reach for it anywhere else only when the id is that specific — otherwise a
per-request alarm can flood the pager.
@@ -121,7 +126,14 @@ config always has the last word.
### Repeat throttling
The chat transport won't repost the same alarm id within
Two levels, and they stack.
The client itself backs off before a repeat is reported at all: every
occurrence is reported up to a small burst, then one per interval. A fault
recurring hundreds of times a second still counts every occurrence, but it
does not write a log line or build an alert payload for each one.
Below that, the chat transport won't repost the same alarm id within
`repeatThrottleMs` (default 15 minutes). The first occurrence always posts,
and the next one that gets through reports how many piled up in between —
so a hot loop reads as one message with a count, not a wall of them.
+12
View File
@@ -96,6 +96,18 @@ extension.on('user.signup', (_key, data) => {
});
```
The `app.recommended` event lets extensions customize desktop recommendations.
Its `appNames` array starts as a fresh copy of the default ordered names on each
call. Listeners may mutate or replace it, including setting it to an empty array.
Async listeners are awaited before names are resolved to app summaries; names
that do not exist are skipped.
```ts
extension.on('app.recommended', (_key, data) => {
data.appNames = ['editor', 'camera'];
});
```
## Conventions
- **TypeScript preferred** in new code where feasible. Existing JS is fine; convert opportunistically when you're already touching a file.
+2 -1
View File
@@ -53,8 +53,9 @@ Both are supported ways to define an API. **Prefer a controller when you need fi
#### Middleware and gates
- **Controller routes** (and extension routes — same options) take [`RouteOptions`](../src/backend/core/http/types.ts): auth gates (`requireAuth`, `requireUserActor`, `noUserSession`, `adminOnly`, `allowedAppIds`, and the access-token controls), `subdomain` routing, per-route `rateLimit`, body parsers, and arbitrary extra `middleware`. The auth flavors are subtle and default-deny — read the JSDoc on each field before picking.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), `requireSubscription`, and `noUserSession`. The `/drivers/call` surface enforces them.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), `requireSubscription`, `requireReputation`, and `noUserSession`. The `/drivers/call` surface enforces them.
- **A surface only paying accounts should reach** declares `requireSubscription` — the route option on a controller endpoint, the per-method `@Driver({ requireSubscription })` block on a driver (`/drivers/call` is one shared route, so a driver's requirement can't live in route options). `true` accepts any plan that isn't free, so a plan registered by an extension counts without core naming it; an array of policy ids (`['business', 'pro']`) accepts only those; `false` is "no requirement", the same as leaving it out. An empty array is a boot error rather than a silent no-op — it reads as subscribers-only while admitting everyone. Off unless asked for, and answered from the metering service's cached per-actor subscription, so it costs a map lookup. Distinct from `requireCredits`: this asks which plan the account is on, not whether it has budget left. Deployments with no paid plans (self-hosted installs) turn the whole thing off with `meteringEnforcement.subscriptions: false`.
- **A surface only a trusted-enough account should reach** declares `requireReputation` — the route option on a controller endpoint, the per-method `@Driver({ requireReputation })` block on a driver. The value names a tier; what that tier takes is `reputationGate.tiers` in config, so the score a surface is worth is a deployment call and can be retuned without editing the surface. A tier the running config doesn't define is inert and everyone passes — an install that doesn't score its accounts must not start turning traffic away on a score it never computed — and `reputationGate.enabled: false` stops every declared gate at once. Opt-in, implies `requireAuth`, and denies with a bare 403 `reputation_required` that names neither the score nor the tier. Nothing in the tree declares one yet: the mechanism ships ahead of the enrollment.
- **An account must have verified a specific factor** — `requireVerified` (email, and only under `strict_email_verification_required`), `requirePhoneVerified`, `requireCardVerified`. These are opt-in and require the factor to have actually been verified, unlike the default-on gate that turns away accounts still carrying a pending verification the abuse harness asked for.
- **An endpoint that spends metered resources** on the caller's behalf — moving file content, making object-store requests, anything else the account is billed for — also declares `requireCredits: true`, which turns an account with nothing left of its budget away with a 402 before the handler runs. Endpoints that only describe or delete things deliberately don't: an account that has run out still has to be able to see what it has, clear it, and reach its billing pages. Drivers have no route options to declare this on, so they call `assertActorHasCredits` themselves ([src/backend/services/metering/enforcement.ts](../src/backend/services/metering/enforcement.ts)) — see `KVStoreDriver`, which does it once for every method.
+40 -1
View File
@@ -9,6 +9,7 @@ import {
vi,
} from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { extension } from '../src/backend/extensions.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleInstalledApps } from './installedApps.ts';
@@ -51,13 +52,14 @@ const seedUser = async () => {
});
};
const seedApp = async (ownerUserId: number) => {
const seedApp = async (ownerUserId: number, overrides = {}) => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.app.create(
{
name: `iaapp_${slug}`,
title: `Installed App ${slug}`,
index_url: `https://example.com/${slug}`,
...overrides,
},
{ ownerUserId },
);
@@ -145,6 +147,43 @@ describe('installedApps extension — handleInstalledApps', () => {
).toBe(false);
});
it('pairs iconUrl with the direct subdomain URL for a generated icon', async () => {
const user = await seedUser();
// The resize pipeline leaves an http(s) icon column behind, which is
// what says the sized PNG is on the icons subdomain.
const app = await seedApp(user.id as number, {
icon: 'https://api.puter.com/app-icon/x',
});
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
const { protocol, static_hosting_domain } = extension.config;
expect(list[0].iconCdnUrl).toBe(
`${protocol}://puter-app-icons.${static_hosting_domain}/${app!.uid}-256.png`,
);
});
it('reports no subdomain URL for an app with no generated icon', async () => {
const user = await seedUser();
const app = await seedApp(user.id as number);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list[0].iconCdnUrl).toBeNull();
});
it('flags apps with no owner_user_id as external', async () => {
const user = await seedUser();
const slug = Math.random().toString(36).slice(2, 8);
+6 -1
View File
@@ -2,7 +2,10 @@ import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import { extension } from '@heyputer/backend/src/extensions';
import { getAppIconUrl } from '@heyputer/backend/src/util/appIcon.js';
import {
getAppIconCdnUrl,
getAppIconUrl,
} from '@heyputer/backend/src/util/appIcon.js';
const clients = extension.import('client');
@@ -73,6 +76,8 @@ export const handleInstalledApps = async (
return {
...rest,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
// Direct subdomain URL for the client to try before iconUrl.
iconCdnUrl: getAppIconCdnUrl(app, extension.config),
external,
};
}),
+45
View File
@@ -9,6 +9,7 @@ import {
vi,
} from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { configContainer } from '../src/backend/exports.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleWhoami } from './whoami.ts';
@@ -118,6 +119,50 @@ describe('whoami extension — handleWhoami', () => {
expect(body).toHaveProperty('taskbar_items');
});
it('reports the SMS-to-card fallback only once a send has opened it', async () => {
const user = await server.stores.user.create({
username: `wuser_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'hashedpw',
email: `${Math.random().toString(36).slice(2, 8)}@example.com`,
requires_phone_verification: true,
} as never);
const prev = configContainer.phone_verification_card_fallback;
configContainer.phone_verification_card_fallback = {
enabled: true,
} as never;
try {
const before = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), before.res),
);
// Phone-gated, but the user has attempts left — no offer yet.
expect(
before.captured.body as Record<string, unknown>,
).toMatchObject({ card_fallback_available: false });
// Exhausting SMS attempts stamps this flag; whoami is then the only
// thing that can still tell a reloading GUI about the offer, since
// further sends are rejected by the route's own rate limit.
await server.stores.kv.set({
key: `card-fallback-open:${user.id}`,
value: true,
});
const after = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), after.res),
);
expect(
after.captured.body as Record<string, unknown>,
).toMatchObject({ card_fallback_available: true });
} finally {
configContainer.phone_verification_card_fallback = prev;
}
});
it('only forwards allow-listed feature flags', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
+33 -3
View File
@@ -1,5 +1,7 @@
import { Context } from '@heyputer/backend/src/core';
import { extension } from '@heyputer/backend/src/extensions';
import { isCardFallbackEligible } from '@heyputer/backend/src/util/cardFallback.js';
import { APP_ICON_SIZES } from '@heyputer/backend/src/util/appIcon.js';
import { getTaskbarItems } from '@heyputer/backend/src/util/taskbarItems.js';
import type { Request, Response } from 'express';
import TimeAgo from 'javascript-time-ago';
@@ -113,13 +115,12 @@ export const handleWhoami = async (
}
const oidcOnly = user.password === null;
const ALLOWED_ICON_SIZES = new Set([16, 32, 64, 128, 256, 512]);
const rawIconSize =
typeof req.query?.icon_size === 'string'
? Number(req.query.icon_size)
: undefined;
const iconSize =
rawIconSize !== undefined && ALLOWED_ICON_SIZES.has(rawIconSize)
rawIconSize !== undefined && APP_ICON_SIZES.includes(rawIconSize)
? rawIconSize
: undefined;
const noIcons = !iconSize;
@@ -155,6 +156,33 @@ export const handleWhoami = async (
// every app actor. Only the verification flag ships.
requires_phone_verification: user.requires_phone_verification,
requires_card_verification: user.requires_card_verification,
// The SMS-to-card escape hatch: true once this user is out of SMS send
// attempts and may verify a card instead. It has to ship from here
// because /send-confirm-phone can no longer say so — by the time the
// fallback opens, further sends are rejected by that route's own rate
// limit before any handler runs, so a page reload would otherwise lose
// an offer that stays valid for 24 hours. Only for user actors, and it
// costs no KV read unless the account is actually phone-gated.
card_fallback_available: isUser
? await isCardFallbackEligible(
extension.config,
user,
async (key) => (await stores.kv.get({ key })).res,
{
smsConfigured: () =>
Boolean(clients.prelude?.isConfigured()),
probeCardVerification: async () => {
const status = { enabled: null as boolean | null };
await clients.event?.emitAndWait(
'puter.card-verification.status',
status,
{},
);
return status.enabled;
},
},
)
: false,
desktop_bg_url: user.desktop_bg_url,
desktop_bg_color: user.desktop_bg_color,
desktop_bg_fit: user.desktop_bg_fit,
@@ -173,6 +201,7 @@ export const handleWhoami = async (
stores,
services,
apiBaseUrl: String(extension.config.api_base_url ?? ''),
config: extension.config,
},
{ iconSize, noIcons },
)
@@ -263,7 +292,8 @@ export const handleWhoami = async (
}
const subscription = details.subscription as
{ offering?: Record<string, unknown> } | undefined;
| { offering?: Record<string, unknown> }
| undefined;
if (subscription?.offering) {
delete subscription.offering.group;
delete subscription.offering.benefits;
+733 -395
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -76,7 +76,7 @@
"@aws-sdk/s3-request-presigner": "^3.1028.0",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.6",
"libphonenumber-js": "1.13.12",
"miniflare": "^4.20260617.1",
"open": "^10.1.0"
},
+54 -4
View File
@@ -375,18 +375,18 @@ describe('AlarmClient alarm registry', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 50; i++) {
for (let i = 0; i < 500; i++) {
client.create('hot', `occurrence ${i}`, { i });
}
const alarm = client.get('hot')!;
expect(alarm.count).toBe(50);
expect(alarm.count).toBe(500);
expect(alarm.occurrences).toHaveLength(20);
expect(alarm.timestamps).toHaveLength(20);
// The window kept is the most recent one, not the oldest.
expect(alarm.occurrences[19].message).toBe('occurrence 49');
expect(alarm.occurrences[19].message).toBe('occurrence 499');
// Trimming history must not rewind what the transports are told.
expect(seen[49]).toMatchObject({ repeatCount: 50, isRepeat: true });
expect(seen.at(-1)).toMatchObject({ repeatCount: 500, isRepeat: true });
});
it('names anonymous handlers by their registration order', () => {
@@ -519,3 +519,53 @@ describe('AlarmClient known-error rules', () => {
expect(chat[0].severity).toBe('info');
});
});
describe('AlarmClient repeat reporting', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('reports every occurrence up to the burst, then backs off', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 12; i++) client.create('flap', 'same fault');
expect(seen).toHaveLength(10);
expect(seen[9].repeatCount).toBe(10);
});
it('reports again once the interval is reached', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 500; i++) client.create('flap', 'same fault');
expect(seen).toHaveLength(11);
expect(seen[10].repeatCount).toBe(500);
});
it('keeps a fast-repeating fault out of the log after the burst', () => {
const client = makeClient();
capture(client);
const warn = vi.mocked(console.warn);
warn.mockClear();
for (let i = 0; i < 100; i++) client.create('flap', 'same fault');
// Occurrences 2-10 only; the first is an ACTIVE line on console.error.
expect(warn.mock.calls).toHaveLength(9);
});
it('skips the payload entirely when no transport accepts the severity', () => {
const client = makeClient();
const seen = capture(client, 'critical');
client.create('quiet', 'nobody is listening', {}, 'info');
expect(seen).toHaveLength(0);
});
});
+27
View File
@@ -69,6 +69,15 @@ const FALLBACK_SEVERITY: PagerSeverity = 'critical';
* and `count` keeps that.
*/
const OCCURRENCE_HISTORY_LIMIT = 20;
// A fast-recurring fault repeats far faster than it is useful to report, and
// every report allocates a log line and a payload. Report every occurrence up
// to the burst, then one per interval; `count` still tracks all of them.
const REPEAT_REPORT_BURST = 10;
const REPEAT_REPORT_INTERVAL = 500;
const shouldReportRepeat = (count: number): boolean =>
count <= REPEAT_REPORT_BURST || count % REPEAT_REPORT_INTERVAL === 0;
/** Keeps `info` alarms out of the paging system unless config says otherwise. */
const DEFAULT_PAGERDUTY_MIN_SEVERITY: PagerSeverity = 'warning';
/** Slack's ceiling once a pager exists: chat gets what doesn't page. */
@@ -456,6 +465,8 @@ export class AlarmClient extends PuterClient {
private handleRepeat(alarm: Alarm): void {
this.applyKnownErrors(alarm);
if (!shouldReportRepeat(alarm.count)) return;
console.warn(
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.count})`,
);
@@ -483,6 +494,18 @@ export class AlarmClient extends PuterClient {
);
}
private hasHandlerFor(severity: PagerSeverity): boolean {
for (const { minSeverity, maxSeverity } of this.alertHandlers) {
if (
meetsMinSeverity(severity, minSeverity) &&
withinMaxSeverity(severity, maxSeverity)
) {
return true;
}
}
return false;
}
private dispatchAlert(alarm: Alarm): void {
const resolved = this.resolveSeverity(alarm);
if (resolved === 'mute') {
@@ -494,6 +517,10 @@ export class AlarmClient extends PuterClient {
}
alarm.severity = resolved;
// Build nothing for a severity no transport accepts — the payload
// (cleaned fields plus the stack, twice) is the expensive part.
if (!this.hasHandlerFor(resolved)) return;
const fieldsClean = cleanFields(alarm.fields);
const repeatCount = alarm.count;
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 66;
const CURRENT_SCHEMA_VERSION = 79;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -81,6 +81,250 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
expect(await userVersionOf(client)).toBe(CURRENT_SCHEMA_VERSION);
});
it('stamps a version whose migration did not run (known off-by-one)', async () => {
// Existing behaviour, not an endorsement: the stamp overshoots the
// applied work by one entry, so 70 is reported without 0074 running.
const partial = await bootClient({ targetVersion: 70 });
try {
expect(await userVersionOf(partial)).toBe(70);
await expect(
partial.read(
"SELECT 1 FROM pragma_table_info('group') WHERE name = 'handle'",
),
).resolves.toEqual([]);
} finally {
partial.onServerShutdown();
}
});
it('applies the team columns and indexes from 0076', async () => {
const columnsOf = async (table: string) =>
(
(await client.read(
`SELECT name FROM pragma_table_info('${table}')`,
)) as { name: string }[]
).map((r) => r.name);
expect(await columnsOf('group')).toEqual(
expect.arrayContaining([
'kind',
'name',
'handle',
'deleted_at',
]),
);
expect(await columnsOf('jct_user_group')).toContain('org_owned');
expect(await columnsOf('user')).toContain('requires_password_change');
const indexes = (await client.read(
"SELECT name FROM sqlite_master WHERE type = 'index' AND name IN (?, ?, ?)",
['idx_group_handle', 'idx_group_owner', 'idx_jct_user_group_group'],
)) as { name: string }[];
expect(indexes.map((r) => r.name).sort()).toEqual([
'idx_group_handle',
'idx_group_owner',
'idx_jct_user_group_group',
]);
});
it('applies the event_subscriptions indexes from 0081', async () => {
const indexes = (await client.read(
"SELECT name FROM sqlite_master WHERE type = 'index' AND name IN (?, ?, ?)",
[
'idx_event_subscriptions_app_handler',
'idx_event_subscriptions_expires',
'idx_event_subscriptions_suspended',
],
)) as { name: string }[];
expect(indexes.map((r) => r.name).sort()).toEqual([
'idx_event_subscriptions_app_handler',
'idx_event_subscriptions_expires',
'idx_event_subscriptions_suspended',
]);
});
it('keeps `handle` unique but lets the seeded groups share a NULL one', async () => {
const seeded = (await client.read(
'SELECT COUNT(*) AS n FROM `group` WHERE `handle` IS NULL',
)) as { n: number }[];
expect(seeded[0].n).toBeGreaterThan(1);
await client.write(
'UPDATE `group` SET `handle` = ? WHERE `id` = (SELECT MIN(`id`) FROM `group`)',
['taken'],
);
await expect(
client.write(
'UPDATE `group` SET `handle` = ? WHERE `id` = (SELECT MAX(`id`) FROM `group`)',
['taken'],
),
).rejects.toThrow(/UNIQUE/iu);
});
it('treats `handle` case-insensitively, as usernames are treated', async () => {
await client.write(
'UPDATE `group` SET `handle` = ? WHERE `id` = (SELECT MIN(`id`) FROM `group`)',
['design-team'],
);
// Without NOCASE this differs per engine: mysql rejects, the others accept.
await expect(
client.write(
'UPDATE `group` SET `handle` = ? WHERE `id` = (SELECT MAX(`id`) FROM `group`)',
['Design-Team'],
),
).rejects.toThrow(/UNIQUE/iu);
});
it('matches a handle case-insensitively on lookup, not just on insert', async () => {
// Index-only NOCASE would leave `WHERE handle = ?` case-sensitive here while
// mysql's utf8mb4_unicode_ci column matched -- one query, two behaviours.
await client.write(
'UPDATE `group` SET `handle` = ? WHERE `id` = (SELECT MIN(`id`) FROM `group`)',
['design-team'],
);
await expect(
client.read('SELECT `handle` FROM `group` WHERE `handle` = ?', [
'Design-Team',
]),
).resolves.toEqual([{ handle: 'design-team' }]);
});
it('rejects a duplicate membership pair after 0078', async () => {
const [{ user_id, group_id }] = (await client.read(
'SELECT (SELECT MIN(`id`) FROM `user`) AS user_id, ' +
'(SELECT MIN(`id`) FROM `group`) AS group_id',
)) as { user_id: number; group_id: number }[];
const insert = () =>
client.write(
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
[user_id, group_id],
);
await insert();
// `GroupStore.addUsers` has no conflict clause, so before 0075 this
// second call silently doubled every group permission the user reads.
await expect(insert()).rejects.toThrow(/UNIQUE/iu);
});
it('applies the audit table and group-share columns from 0079', async () => {
const columns = (await client.read(
"SELECT name FROM pragma_table_info('audit_team_membership')",
)) as { name: string }[];
expect(columns.map((r) => r.name)).toEqual([
'id',
'group_id',
'group_id_keep',
'user_id',
'user_id_keep',
'actor_user_id',
'action',
'reason',
'created_at',
]);
const shareColumns = (await client.read(
"SELECT name FROM pragma_table_info('share')",
)) as { name: string }[];
expect(shareColumns.map((r) => r.name)).toContain('holder_group_id');
const indexes = (await client.read(
"SELECT name FROM sqlite_master WHERE type = 'index' " +
'AND (name LIKE ? OR name LIKE ?)',
['idx_audit_team_membership%', 'idx_share_holder_group%'],
)) as { name: string }[];
expect(indexes.map((r) => r.name).sort()).toEqual([
// The `_fk` three stop a user or group delete scanning the audit table.
'idx_audit_team_membership_actor_fk',
'idx_audit_team_membership_group',
'idx_audit_team_membership_group_fk',
'idx_audit_team_membership_user',
'idx_audit_team_membership_user_fk',
'idx_share_holder_group',
'idx_share_holder_group_entry_issuer',
]);
});
it('keeps an audit row after its group is deleted, blanking only the FK', async () => {
// 0043 leaves `PRAGMA foreign_keys = ON`, so this is real behaviour.
const [group] = (await client.read(
'SELECT MIN(`id`) AS id FROM `group`',
)) as { id: number }[];
const [user] = (await client.read(
'SELECT MIN(`id`) AS id FROM `user`',
)) as { id: number }[];
await client.write(
'INSERT INTO `audit_team_membership` ' +
'(`group_id`, `group_id_keep`, `user_id`, `user_id_keep`, ' +
'`actor_user_id`, `action`) VALUES (?, ?, ?, ?, ?, ?)',
[group.id, group.id, user.id, user.id, user.id, 'reset_member_password'],
);
await client.write('DELETE FROM `group` WHERE `id` = ?', [group.id]);
await expect(
client.read(
'SELECT `group_id`, `group_id_keep`, `action` FROM `audit_team_membership`',
),
).resolves.toEqual([
{
group_id: null,
group_id_keep: group.id,
action: 'reset_member_password',
},
]);
});
it('constrains team shares that the user-holder index cannot', async () => {
const [user] = (await client.read(
'SELECT MIN(`id`) AS id FROM `user`',
)) as { id: number }[];
const groups = (await client.read(
'SELECT `id` FROM `group` ORDER BY `id` LIMIT 2',
)) as { id: number }[];
await client.write(
'INSERT INTO `fsentries` (`uuid`, `name`, `user_id`, `modified`) ' +
'VALUES (?, ?, ?, ?)',
['11111111-1111-4111-8111-111111111111', 'shared.txt', user.id, 0],
);
const [entry] = (await client.read(
'SELECT `id` FROM `fsentries` WHERE `uuid` = ?',
['11111111-1111-4111-8111-111111111111'],
)) as { id: number }[];
const insertShare = (uid: string, groupId: number) =>
client.write(
'INSERT INTO `share` ' +
'(`uid`, `issuer_user_id`, `recipient_email`, `holder_user_id`, ' +
'`holder_group_id`, `fsentry_id`) VALUES (?, ?, ?, NULL, ?, ?)',
[uid, user.id, '[email protected]', groupId, entry.id],
);
await insertShare('share-team-a', groups[0].id);
// Different group, same file and issuer -- allowed.
await insertShare('share-team-b', groups[1].id);
// Both have `holder_user_id` NULL, so the user-holder index permitted them.
await expect(insertShare('share-team-c', groups[0].id)).rejects.toThrow(
/UNIQUE/iu,
);
});
it('matches a handle case-insensitively on lookup, not just on insert', async () => {
// Index-only NOCASE would leave `WHERE handle = ?` case-sensitive here while
// mysql's utf8mb4_unicode_ci column matched -- one query, two behaviours.
await client.write(
'UPDATE `group` SET `handle` = ? WHERE `id` = (SELECT MIN(`id`) FROM `group`)',
['design-team'],
);
await expect(
client.read('SELECT `handle` FROM `group` WHERE `handle` = ?', [
'Design-Team',
]),
).resolves.toEqual([{ handle: 'design-team' }]);
});
it('runs the javascript migrations, not just the .sql ones', async () => {
// The `system` user only exists because 0025 (a .dbmig.js file) ran
// inside the migration VM.
@@ -190,6 +434,190 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
});
});
// 0072 rebuilds `notification` for the scope tuple, and picks up the
// user_id index and the delete cascade mysql and postgres already had.
describe('0072 notification-scope', () => {
// Only the backfill re-runs: the rebuild half is once-only by
// construction, but mysql and postgres replay their whole file on
// every boot, so these statements have to be safe twice.
const BACKFILL = readFileSync(
new URL(
'./migrations/sqlite/0072_notification-scope.sql',
import.meta.url,
),
'utf8',
)
.split(/;\s*\n/)
.map((s) => s.trim())
.filter((s) => /(^|\n)UPDATE /.test(s));
const seedUser = async (): Promise<number> => {
const name = `notif-${Math.random().toString(36).slice(2, 10)}`;
await client.write(
'INSERT INTO `user` (`username`, `uuid`) VALUES (?, ?)',
[name, `${name}-uuid`],
);
const [row] = await client.read(
'SELECT `id` FROM `user` WHERE `username` = ?',
[name],
);
return Number(row.id);
};
const seedLegacy = async (
userId: number,
value: Record<string, unknown> | string,
): Promise<string> => {
const uid = `n-${Math.random().toString(36).slice(2, 12)}`;
await client.write(
'INSERT INTO `notification` (`uid`, `user_id`, `value`) VALUES (?, ?, ?)',
[
uid,
userId,
typeof value === 'string' ? value : JSON.stringify(value),
],
);
return uid;
};
const scopeOf = async (uid: string) => {
const [row] = await client.read(
'SELECT `type`, `audience`, `app_uid` FROM `notification` WHERE `uid` = ?',
[uid],
);
return row;
};
const runBackfill = async () => {
for (const stmt of BACKFILL) await client.write(stmt);
};
it('carries every backfill statement', () => {
expect(BACKFILL).toHaveLength(3);
});
it('classifies the markers legacy rows actually carried', async () => {
const userId = await seedUser();
const received = await seedLegacy(userId, {
source: 'sharing',
template: 'file-shared-with-you',
});
const claimed = await seedLegacy(userId, {
source: 'sharing',
template: 'file-shared-before-you-joined',
});
const deployed = await seedLegacy(userId, {
source: 'worker',
title: 'Successfully deployed https://x.puter.work',
template: 'user-requesting-share',
});
const failed = await seedLegacy(userId, {
source: 'worker',
title: 'Failed to deploy x! boom',
template: 'user-requesting-share',
});
await runBackfill();
expect(await scopeOf(received)).toEqual({
type: 'share.received',
audience: 'account',
app_uid: null,
});
expect(await scopeOf(claimed)).toEqual({
type: 'share.claimed',
audience: 'account',
app_uid: null,
});
// The app is unrecoverable — the payload only ever named a worker.
expect(await scopeOf(deployed)).toEqual({
type: 'app.worker.deployed',
audience: 'developer',
app_uid: null,
});
expect(await scopeOf(failed)).toEqual({
type: 'app.worker.deployFailed',
audience: 'developer',
app_uid: null,
});
});
it('leaves anything it does not recognise reading as legacy', async () => {
const userId = await seedUser();
const unknown = await seedLegacy(userId, {
source: 'sharing',
template: 'something-else',
});
const bare = await seedLegacy(userId, { title: 'hi' });
const notJson = await seedLegacy(userId, 'plain text');
await runBackfill();
for (const uid of [unknown, bare, notJson]) {
expect(await scopeOf(uid)).toEqual({
type: '',
audience: 'account',
app_uid: null,
});
}
});
it('does not reclassify on replay', async () => {
const userId = await seedUser();
const uid = await seedLegacy(userId, {
source: 'sharing',
template: 'file-shared-with-you',
});
await runBackfill();
// A row already classified as something else must survive a
// second pass untouched.
await client.write(
'UPDATE `notification` SET `audience` = ?, `app_uid` = ? WHERE `uid` = ?',
['app-user', 'app-1234', uid],
);
await runBackfill();
expect(await scopeOf(uid)).toEqual({
type: 'share.received',
audience: 'app-user',
app_uid: 'app-1234',
});
});
it('indexes user_id and the scope tuple', async () => {
const rows = await client.read('PRAGMA index_list(`notification`)');
const names = rows.map((r) => String(r.name));
expect(names).toContain('idx_notification_user_id');
expect(names).toContain('idx_notification_scope');
});
it('retires a deleted user rows', async () => {
const [{ foreign_keys: enforcing }] = await client.read(
'PRAGMA foreign_keys',
);
expect(enforcing).toBe(1);
const userId = await seedUser();
const uid = await seedLegacy(userId, { title: 'orphan-to-be' });
await client.write('DELETE FROM `user` WHERE `id` = ?', [userId]);
expect(
await client.read(
'SELECT `uid` FROM `notification` WHERE `uid` = ?',
[uid],
),
).toEqual([]);
});
});
it('indexes notification.created_at for the retention sweep', async () => {
const rows = await client.read('PRAGMA index_list(`notification`)');
expect(rows.map((r) => String(r.name))).toContain(
'idx_notification_created_at',
);
});
it('leaves an already-migrated database untouched on a second boot', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
const path = join(dir, 'nested', 'puter.sqlite');
@@ -223,6 +651,66 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
}
});
it('deduplicates pre-existing membership rows when 0078 applies', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-dedup-'));
const path = join(dir, 'puter.sqlite');
try {
// Stop at 67 so the duplicates exist the way a live database's do:
// written before the unique index, not after it.
const before = new SqliteDatabaseClient(
sqliteConfig({ inMemory: false, path, targetVersion: 67 }),
);
await before.onServerStart();
const [{ user_id, group_id }] = (await before.read(
'SELECT (SELECT MIN(`id`) FROM `user`) AS user_id, ' +
'(SELECT MIN(`id`) FROM `group`) AS group_id',
)) as { user_id: number; group_id: number }[];
for (const pair of [
[user_id, group_id],
[user_id, group_id],
[user_id, group_id],
[user_id, group_id + 1],
]) {
await before.write(
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
pair,
);
}
const seeded = (await before.read(
'SELECT `id`, `group_id` FROM `jct_user_group` ORDER BY `id`',
)) as { id: number; group_id: number }[];
expect(seeded).toHaveLength(4);
before.onServerShutdown();
const after = new SqliteDatabaseClient(
sqliteConfig({ inMemory: false, path }),
);
await after.onServerStart();
expect(await userVersionOf(after)).toBe(CURRENT_SCHEMA_VERSION);
// Three copies collapse to the lowest id; the distinct pair is left
// alone. Losing the higher ids costs nothing -- `addUsers` writes
// only the two id columns, so `extra` and `metadata` are NULL.
await expect(
after.read(
'SELECT `id`, `group_id` FROM `jct_user_group` ORDER BY `id`',
),
).resolves.toEqual([seeded[0], seeded[3]]);
await expect(
after.write(
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
[user_id, group_id],
),
).rejects.toThrow(/UNIQUE/iu);
after.onServerShutdown();
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('stops early at a configured target version', async () => {
const partial = await bootClient({ targetVersion: 5 });
try {
@@ -100,6 +100,19 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[63, ['0068_referral-code-unique.sql']],
[64, ['0069_user-block.sql']],
[65, ['0070_drop-orphaned-default-groups.sql']],
[66, ['0071_share_issuer_index.sql']],
[67, ['0072_notification-scope.sql']],
[68, ['0073_notification-created-at.sql']],
[69, ['0074_add_user_home.sql']],
[70, ['0075_event-subscriptions.sql']],
[71, ['0076_event-handlers.sql']],
[72, ['0077_teams.sql']],
[73, ['0078_jct-user-group-pair-unique.sql']],
[74, ['0079_team-audit-and-group-shares.sql']],
[75, ['0080_kv-share-handles.sql']],
[76, ['0081_event-subscriptions-indexes.sql']],
[77, ['0082_temp-password-expiry.sql']],
[78, ['0083_team-directory.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -1060,6 +1060,7 @@ CREATE TABLE IF NOT EXISTS `user` (
`signup_server` varchar(255) DEFAULT NULL,
`metadata` json DEFAULT (json_object()),
`reputation` smallint DEFAULT '100',
`home` varchar(64) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uid` (`uuid`),
UNIQUE KEY `username` (`username`),
@@ -1151,6 +1152,7 @@ CALL _puter_add_col('user', 'signup_user_agent', '`signup_user_agent` varchar(51
CALL _puter_add_col('user', 'signup_origin', '`signup_origin` varchar(255) DEFAULT NULL');
CALL _puter_add_col('user', 'signup_server', '`signup_server` varchar(255) DEFAULT NULL');
CALL _puter_add_col('user', 'metadata', '`metadata` json DEFAULT (json_object())');
CALL _puter_add_col('user', 'home', '`home` varchar(64) DEFAULT NULL');
CALL _puter_add_col('user', 'reputation', '`reputation` smallint DEFAULT ''100''');
--
@@ -0,0 +1,80 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Give `notification` a scope tuple: what the row is (`type`), the app it is
-- about (`app_uid`, NULL for platform), and who the recipient is in relation
-- to it (`audience`). Attribution lived inside the `value` JSON as two
-- free-form markers, `source` and `template`, that could disagree; these
-- columns replace them and are what a scoped list can be indexed on.
--
-- `app_uid` is char(40) to match `apps`.`uid`, which is `app-` plus a uuid.
--
-- Idempotent: columns go through `_puter_add_col` (mig_1), the index through
-- the guarded procedure. There is no per-file applied-state tracking, so every
-- statement here tolerates a re-run.
CALL _puter_add_col('notification', 'app_uid', '`app_uid` char(40) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
CALL _puter_add_col('notification', 'audience', '`audience` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT ''account''');
CALL _puter_add_col('notification', 'type', '`type` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT ''''');
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_scope_index()
BEGIN
-- Serves both the scoped list and the unacknowledged count.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_scope'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_scope`
(`user_id`, `audience`, `app_uid`, `acknowledged`);
END IF;
END //
DELIMITER ;
CALL _puter_add_notification_scope_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_index;
-- Backfill from the markers the payload actually carried. `type = ''` is the
-- not-yet-classified state, so re-running matches nothing the first pass
-- claimed, and a row matching no marker keeps the defaults (`''` / 'account')
-- and reads as legacy.
UPDATE `notification`
SET `type` = 'share.received', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.template')) = 'file-shared-with-you';
UPDATE `notification`
SET `type` = 'share.claimed', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.template')) = 'file-shared-before-you-joined';
-- Worker rows carried only `source: 'worker'` and a title, so the app they
-- belong to is unrecoverable and they stay unattributed. The title prefix is
-- the sole surviving signal of which way the deploy went.
UPDATE `notification`
SET `audience` = 'developer',
`app_uid` = NULL,
`type` = CASE
WHEN JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.title')) LIKE 'Successfully deployed %'
THEN 'app.worker.deployed'
ELSE 'app.worker.deployFailed'
END
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.source')) = 'worker';
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index `notification`.`created_at`. The retention sweep selects by age alone
-- and by nothing else, so no existing index on the table narrows it — every
-- pass would otherwise scan the whole table to find the oldest few hundred
-- rows.
--
-- Idempotent: the guarded procedure, as mig_26. There is no per-file
-- applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_add_notification_created_at_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_created_at_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_created_at'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_created_at`
(`created_at`);
END IF;
END //
DELIMITER ;
CALL _puter_add_notification_created_at_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_created_at_index;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Home region column. Mirrors SQLite migration 0074. Home region this
-- account's data belongs in — a node id, not a storage region. NULL for
-- accounts that predate the column, which fall back to `signup_server`.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations).
CALL _puter_add_col('user', 'home', '`home` VARCHAR(64) DEFAULT NULL');
@@ -0,0 +1,59 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Subscriptions that outlive the connection that made them. See
-- sqlite/0075_event-subscriptions.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_23. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `event_subscriptions` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`sub_id` varchar(80) NOT NULL,
`token` varchar(255) NOT NULL,
`owner_user_id` int unsigned NOT NULL,
`holder_user_id` int unsigned NOT NULL,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion. No foreign key: a row outlives the
-- app that made it, which is what keeps it revocable afterwards.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci
DEFAULT NULL,
`subject` varchar(4096) NOT NULL,
`anchor_uid` char(36) NOT NULL,
`anchor_path` varchar(4096) NOT NULL,
`match` varchar(1024) DEFAULT NULL,
`delivery` varchar(16) NOT NULL,
`ops` varchar(64) DEFAULT NULL,
`handler_name` varchar(128) DEFAULT NULL,
`targets` json NOT NULL,
`context` text,
`permission` varchar(32) NOT NULL,
`suspended_at` bigint DEFAULT NULL,
`suspended_reason` varchar(64) DEFAULT NULL,
`expires_at` bigint DEFAULT NULL,
`created_at` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_subscriptions_sub_id` (`sub_id`),
KEY `idx_event_subscriptions_token` (`token`),
KEY `idx_event_subscriptions_holder` (`holder_user_id`, `app_uid`),
KEY `idx_event_subscriptions_owner` (`owner_user_id`),
CONSTRAINT `fk_event_subscriptions_owner` FOREIGN KEY (`owner_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_event_subscriptions_holder` FOREIGN KEY (`holder_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,43 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions`.`anchor_uid` was sized for a filesystem node uuid.
-- A key-value subscription anchors on an app instead, and `apps`.`uid` is
-- `char(40)`, so the column has to hold one.
--
-- Idempotent: the guarded procedure, as mig_27. There is no per-file
-- applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_widen_event_subscriptions_anchor_uid;
DELIMITER //
CREATE PROCEDURE _puter_widen_event_subscriptions_anchor_uid()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'anchor_uid'
AND CHARACTER_MAXIMUM_LENGTH < 40
) THEN
ALTER TABLE `event_subscriptions`
MODIFY COLUMN `anchor_uid` varchar(40) NOT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_event_subscriptions_anchor_uid();
DROP PROCEDURE IF EXISTS _puter_widen_event_subscriptions_anchor_uid;
@@ -0,0 +1,37 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Named handlers an app deploys once. See sqlite/0076_event-handlers.sql for
-- the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_29. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `event_handlers` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`name` varchar(128) NOT NULL,
`source` mediumtext NOT NULL,
`source_hash` char(64) NOT NULL,
`created_at` bigint NOT NULL,
`updated_at` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_handlers_app_name` (`app_uid`, `name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,58 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Teams. See sqlite/0077_teams.sql for the column rationale.
-- No per-file applied-state tracking, so every statement must tolerate a re-run:
-- columns go through _puter_add_col, indexes through the guarded procedure below.
CALL _puter_add_col('group', 'kind', '`kind` varchar(20) DEFAULT NULL');
CALL _puter_add_col('group', 'name', '`name` varchar(255) DEFAULT NULL');
CALL _puter_add_col('group', 'handle', '`handle` varchar(64) DEFAULT NULL');
CALL _puter_add_col('group', 'deleted_at', '`deleted_at` timestamp NULL DEFAULT NULL');
CALL _puter_add_col('jct_user_group', 'org_owned', '`org_owned` tinyint(1) DEFAULT NULL');
CALL _puter_add_col('user', 'requires_password_change', '`requires_password_change` tinyint(1) NOT NULL DEFAULT ''0''');
DROP PROCEDURE IF EXISTS _puter_add_team_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_team_indexes()
BEGIN
-- Already case-insensitive: the table is utf8mb4_unicode_ci. NULLs stay distinct.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'group'
AND INDEX_NAME = 'idx_group_handle'
) THEN
ALTER TABLE `group` ADD UNIQUE INDEX `idx_group_handle` (`handle`);
END IF;
-- Composite, unlike the existing single-column `group_id` key.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'jct_user_group'
AND INDEX_NAME = 'idx_jct_user_group_group'
) THEN
ALTER TABLE `jct_user_group`
ADD INDEX `idx_jct_user_group_group` (`group_id`, `user_id`);
END IF;
END//
DELIMITER ;
CALL _puter_add_team_indexes();
DROP PROCEDURE IF EXISTS _puter_add_team_indexes;
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Deduplicate then constrain. See sqlite/0075 for why this is lossless.
-- Both steps sit behind the index check: mysql re-runs this file on every boot, and
-- guarding them together stops a rolling deploy deleting and indexing concurrently.
DROP PROCEDURE IF EXISTS _puter_add_membership_pair_index;
DELIMITER //
CREATE PROCEDURE _puter_add_membership_pair_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'jct_user_group'
AND INDEX_NAME = 'idx_jct_user_group_pair'
) THEN
-- Self-join because mysql refuses `NOT IN (SELECT ... FROM jct_user_group)` (1093).
DELETE dup FROM `jct_user_group` dup
JOIN `jct_user_group` keep
ON dup.`user_id` = keep.`user_id`
AND dup.`group_id` = keep.`group_id`
AND dup.`id` > keep.`id`;
ALTER TABLE `jct_user_group`
ADD UNIQUE INDEX `idx_jct_user_group_pair` (`user_id`, `group_id`);
END IF;
END//
DELIMITER ;
CALL _puter_add_membership_pair_index();
DROP PROCEDURE IF EXISTS _puter_add_membership_pair_index;
@@ -0,0 +1,84 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0076. Types follow mysql_mig_1; the guarded procedure follows mysql_mig_22.
CREATE TABLE IF NOT EXISTS `audit_team_membership` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`group_id` int unsigned DEFAULT NULL,
`group_id_keep` int unsigned NOT NULL,
`user_id` int unsigned DEFAULT NULL,
`user_id_keep` int unsigned NOT NULL,
`actor_user_id` int unsigned DEFAULT NULL,
`action` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`reason` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idx_audit_team_membership_group` (`group_id_keep`, `id`),
KEY `idx_audit_team_membership_user` (`user_id_keep`, `id`),
KEY `idx_audit_team_membership_group_fk` (`group_id`),
KEY `idx_audit_team_membership_user_fk` (`user_id`),
KEY `idx_audit_team_membership_actor_fk` (`actor_user_id`),
-- SET NULL, never CASCADE: deleting an account must not erase what was done to it.
CONSTRAINT `fk_audit_team_membership_group` FOREIGN KEY (`group_id`)
REFERENCES `group` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_audit_team_membership_user` FOREIGN KEY (`user_id`)
REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_audit_team_membership_actor` FOREIGN KEY (`actor_user_id`)
REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
CALL _puter_add_col('share', 'holder_group_id', '`holder_group_id` int unsigned DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_group_share_constraints;
DELIMITER //
CREATE PROCEDURE _puter_add_group_share_constraints()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_group'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_holder_group` (`holder_group_id`, `id`);
END IF;
-- Team shares leave `holder_user_id` NULL, so the existing unique index binds none.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_group_entry_issuer'
) THEN
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_holder_group_entry_issuer`
(`holder_group_id`, `fsentry_id`, `issuer_user_id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_holder_group_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_holder_group_fk`
FOREIGN KEY (`holder_group_id`) REFERENCES `group` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END//
DELIMITER ;
CALL _puter_add_group_share_constraints();
DROP PROCEDURE IF EXISTS _puter_add_group_share_constraints;
@@ -0,0 +1,68 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Opaque names for shared regions of a user's key-value namespace. See
-- sqlite/0080_kv-share-handles.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_29. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `kv_share_handles` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`handle` varchar(64) NOT NULL,
`owner_user_id` int unsigned NOT NULL,
`grantee_user_id` int unsigned NOT NULL,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion. No foreign key: a handle outlives
-- the app it was minted against, which is what keeps it revocable.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`key_prefix` varchar(1024) NOT NULL,
`permission` varchar(1024) NOT NULL,
`created_at` bigint NOT NULL,
`revoked_at` bigint DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_kv_share_handles_handle` (`handle`),
KEY `idx_kv_share_handles_owner` (`owner_user_id`, `id`),
CONSTRAINT `fk_kv_share_handles_owner` FOREIGN KEY (`owner_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_kv_share_handles_grantee` FOREIGN KEY (`grantee_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- A subscription made through a handle stores the grant string it was
-- authorized under rather than an access mode, and a grant string carries a
-- user uuid, an app uid and a key prefix. Guarded on the current length, as
-- mig_24: replaying a column change on a growing table rebuilds it every boot.
DROP PROCEDURE IF EXISTS _puter_widen_subscription_permission;
DELIMITER //
CREATE PROCEDURE _puter_widen_subscription_permission()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'permission'
AND CHARACTER_MAXIMUM_LENGTH < 1024
) THEN
ALTER TABLE `event_subscriptions` MODIFY `permission` varchar(1024) NOT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_subscription_permission();
DROP PROCEDURE IF EXISTS _puter_widen_subscription_permission;
@@ -0,0 +1,60 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions` grew three hot queries with no index behind them:
-- a handler's own rows, the expiry reaper and the suspended reaper. See
-- sqlite/0081_event-subscriptions-indexes.sql for the query shapes.
--
-- Idempotent: the guarded procedure, as mig_30/mig_34/mig_35. There is no
-- per-file applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_event_subscriptions_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND INDEX_NAME = 'idx_event_subscriptions_app_handler'
) THEN
ALTER TABLE `event_subscriptions`
ADD INDEX `idx_event_subscriptions_app_handler` (`app_uid`, `handler_name`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND INDEX_NAME = 'idx_event_subscriptions_expires'
) THEN
ALTER TABLE `event_subscriptions`
ADD INDEX `idx_event_subscriptions_expires` (`expires_at`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND INDEX_NAME = 'idx_event_subscriptions_suspended'
) THEN
ALTER TABLE `event_subscriptions`
ADD INDEX `idx_event_subscriptions_suspended` (`suspended_at`, `id`);
END IF;
END//
DELIMITER ;
CALL _puter_add_event_subscriptions_indexes();
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_indexes;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0082_temp-password-expiry.sql for the column rationale.
-- No per-file applied-state tracking, so the column goes through _puter_add_col.
CALL _puter_add_col('user', 'temp_password_expires_at', '`temp_password_expires_at` bigint DEFAULT NULL');
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0083_team-directory.sql for the column rationale.
-- No per-file applied-state tracking, so the column goes through _puter_add_col.
CALL _puter_add_col('group', 'directory_enabled', '`directory_enabled` tinyint(1) NOT NULL DEFAULT 0');
@@ -72,7 +72,8 @@ CREATE TABLE IF NOT EXISTS "user" (
signup_origin varchar(255),
signup_server varchar(255),
metadata jsonb DEFAULT '{}'::jsonb,
reputation smallint DEFAULT 100
reputation smallint DEFAULT 100,
home varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_user_email ON "user" (email);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- "Shared by me" pages by keyset: issuer equality, then `id` as the cursor.
-- The single-column issuer index (postgres_mig_1) serves the equality but not
-- the ordering; this composite lets the issued half of that listing range-scan
-- without a sort.
CREATE INDEX IF NOT EXISTS idx_share_issuer
ON share (issuer_user_id, id);
@@ -0,0 +1,55 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Scope tuple for `notification`. See mysql/mysql_mig_26.sql for the rationale.
-- `app_uid` is varchar(40) — the width of `apps`.`uid` — and varchar rather
-- than char for the same reason `uid` is here: blank padding is not what the
-- column means.
--
-- Idempotent via IF NOT EXISTS and the `type = ''` guard on the backfill.
ALTER TABLE notification ADD COLUMN IF NOT EXISTS app_uid varchar(40);
ALTER TABLE notification ADD COLUMN IF NOT EXISTS audience varchar(16) NOT NULL DEFAULT 'account';
ALTER TABLE notification ADD COLUMN IF NOT EXISTS type varchar(64) NOT NULL DEFAULT '';
-- Serves both the scoped list and the unacknowledged count.
CREATE INDEX IF NOT EXISTS idx_notification_scope
ON notification (user_id, audience, app_uid, acknowledged);
UPDATE notification
SET type = 'share.received', audience = 'account', app_uid = NULL
WHERE type = ''
AND value->>'template' = 'file-shared-with-you';
UPDATE notification
SET type = 'share.claimed', audience = 'account', app_uid = NULL
WHERE type = ''
AND value->>'template' = 'file-shared-before-you-joined';
-- Worker rows carried only `source: 'worker'` and a title, so the app they
-- belong to is unrecoverable and they stay unattributed. The title prefix is
-- the sole surviving signal of which way the deploy went.
UPDATE notification
SET audience = 'developer',
app_uid = NULL,
type = CASE
WHEN value->>'title' LIKE 'Successfully deployed %'
THEN 'app.worker.deployed'
ELSE 'app.worker.deployFailed'
END
WHERE type = ''
AND value->>'source' = 'worker';
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index `notification`.`created_at` for the retention sweep. See
-- mysql/mysql_mig_27.sql for the rationale.
CREATE INDEX IF NOT EXISTS idx_notification_created_at
ON notification (created_at);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Home region column. Mirrors SQLite migration 0074. Home region this
-- account's data belongs in — a node id, not a storage region. NULL for
-- accounts that predate the column, which fall back to `signup_server`.
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS home varchar(64);
@@ -0,0 +1,53 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Subscriptions that outlive the connection that made them. See
-- sqlite/0075_event-subscriptions.sql for the column rationale.
--
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS event_subscriptions (
id BIGSERIAL PRIMARY KEY,
sub_id VARCHAR(80) NOT NULL UNIQUE,
token VARCHAR(255) NOT NULL,
owner_user_id INTEGER NOT NULL
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
holder_user_id INTEGER NOT NULL
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_uid VARCHAR(40) DEFAULT NULL,
subject VARCHAR(4096) NOT NULL,
anchor_uid VARCHAR(36) NOT NULL,
anchor_path VARCHAR(4096) NOT NULL,
"match" VARCHAR(1024) DEFAULT NULL,
delivery VARCHAR(16) NOT NULL,
ops VARCHAR(64) DEFAULT NULL,
handler_name VARCHAR(128) DEFAULT NULL,
targets JSONB NOT NULL,
context TEXT DEFAULT NULL,
permission VARCHAR(32) NOT NULL,
suspended_at BIGINT DEFAULT NULL,
suspended_reason VARCHAR(64) DEFAULT NULL,
expires_at BIGINT DEFAULT NULL,
created_at BIGINT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_event_subscriptions_token
ON event_subscriptions (token);
CREATE INDEX IF NOT EXISTS idx_event_subscriptions_holder
ON event_subscriptions (holder_user_id, app_uid);
CREATE INDEX IF NOT EXISTS idx_event_subscriptions_owner
ON event_subscriptions (owner_user_id);
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions.anchor_uid` was sized for a filesystem node uuid. A
-- key-value subscription anchors on an app instead, and an app uid is 40
-- characters. See mysql/mysql_mig_30.sql.
--
-- Guarded: an unconditional ALTER COLUMN TYPE takes an ACCESS EXCLUSIVE lock
-- on every replay, even widening to the same type. Skipped once the column is
-- already wide enough.
DO $$
BEGIN
IF (
SELECT character_maximum_length FROM information_schema.columns
WHERE table_name = 'event_subscriptions' AND column_name = 'anchor_uid'
) < 40 THEN
ALTER TABLE event_subscriptions
ALTER COLUMN anchor_uid TYPE VARCHAR(40);
END IF;
END $$;
@@ -0,0 +1,34 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Named handlers an app deploys once. See sqlite/0076_event-handlers.sql for
-- the column rationale.
--
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS event_handlers (
id BIGSERIAL PRIMARY KEY,
app_uid VARCHAR(40) NOT NULL,
name VARCHAR(128) NOT NULL,
source TEXT NOT NULL,
source_hash VARCHAR(64) NOT NULL,
created_at BIGINT NOT NULL,
updated_at BIGINT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_event_handlers_app_name
ON event_handlers (app_uid, name);
@@ -0,0 +1,38 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Teams. See sqlite/0077_teams.sql for the column rationale.
-- Idempotent via IF NOT EXISTS; there is no per-file applied-state tracking.
ALTER TABLE "group" ADD COLUMN IF NOT EXISTS kind text;
ALTER TABLE "group" ADD COLUMN IF NOT EXISTS name text;
ALTER TABLE "group" ADD COLUMN IF NOT EXISTS handle text;
ALTER TABLE "group" ADD COLUMN IF NOT EXISTS deleted_at timestamp;
-- On lower(handle): text is case-sensitive here, unlike MySQL's utf8mb4_unicode_ci.
-- Handle lookups must therefore compare lower(handle) to hit this index.
CREATE UNIQUE INDEX IF NOT EXISTS idx_group_handle ON "group" (lower(handle));
-- No idx_group_owner here: idx_group_owner_user_id already covers it.
ALTER TABLE jct_user_group ADD COLUMN IF NOT EXISTS org_owned smallint;
-- Composite, unlike the existing single-column idx_jct_user_group_group_id.
CREATE INDEX IF NOT EXISTS idx_jct_user_group_group
ON jct_user_group (group_id, user_id);
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_password_change smallint NOT NULL DEFAULT 0;
@@ -0,0 +1,34 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Deduplicate then constrain. See sqlite/0075 for why this is lossless.
-- Guarded because postgres re-runs every file on each boot; `to_regclass` follows
-- search_path, so it resolves under a test schema too.
DO $mig15$
BEGIN
IF to_regclass('idx_jct_user_group_pair') IS NULL THEN
DELETE FROM jct_user_group
WHERE id NOT IN (
SELECT MIN(id) FROM jct_user_group GROUP BY user_id, group_id
);
END IF;
END
$mig15$;
CREATE UNIQUE INDEX IF NOT EXISTS idx_jct_user_group_pair
ON jct_user_group (user_id, group_id);
@@ -0,0 +1,57 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0076. Types follow `audit_user_to_group_permissions` in postgres_mig_1.
CREATE TABLE IF NOT EXISTS audit_team_membership (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-- SET NULL, never CASCADE: deleting an account must not erase what was done to it.
group_id integer REFERENCES "group" (id) ON DELETE SET NULL ON UPDATE CASCADE,
group_id_keep integer NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
actor_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
action varchar(255) NOT NULL,
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_team_membership_group
ON audit_team_membership (group_id_keep, id);
CREATE INDEX IF NOT EXISTS idx_audit_team_membership_user
ON audit_team_membership (user_id_keep, id);
-- SET NULL has to find the child rows; postgres does not index FK columns for you.
CREATE INDEX IF NOT EXISTS idx_audit_team_membership_group_fk
ON audit_team_membership (group_id);
CREATE INDEX IF NOT EXISTS idx_audit_team_membership_user_fk
ON audit_team_membership (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_team_membership_actor_fk
ON audit_team_membership (actor_user_id);
ALTER TABLE share ADD COLUMN IF NOT EXISTS holder_group_id integer
REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE;
CREATE INDEX IF NOT EXISTS idx_share_holder_group
ON share (holder_group_id, id);
-- Default NULLS DISTINCT, matching the other engines, so this binds only team shares.
CREATE UNIQUE INDEX IF NOT EXISTS idx_share_holder_group_entry_issuer
ON share (holder_group_id, fsentry_id, issuer_user_id);
@@ -0,0 +1,64 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Opaque names for shared regions of a user's key-value namespace. See
-- sqlite/0080_kv-share-handles.sql for the column rationale.
--
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS kv_share_handles (
id BIGSERIAL PRIMARY KEY,
handle VARCHAR(64) NOT NULL,
owner_user_id INTEGER NOT NULL
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
grantee_user_id INTEGER NOT NULL
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_uid VARCHAR(40) NOT NULL,
key_prefix VARCHAR(1024) NOT NULL,
permission VARCHAR(1024) NOT NULL,
created_at BIGINT NOT NULL,
revoked_at BIGINT DEFAULT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_kv_share_handles_handle
ON kv_share_handles (handle);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_owner
ON kv_share_handles (owner_user_id, id);
-- A cascading delete has to find the child rows, and postgres does not index
-- FK columns for you.
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_grantee_fk
ON kv_share_handles (grantee_user_id);
-- A subscription made through a handle stores the grant string it was
-- authorized under rather than an access mode, and a grant string carries a
-- user uuid, an app uid and a key prefix.
--
-- Guarded: an unconditional ALTER COLUMN TYPE takes an ACCESS EXCLUSIVE lock
-- on every replay, even widening to the same type. Skipped once the column is
-- already wide enough.
DO $$
BEGIN
IF (
SELECT character_maximum_length FROM information_schema.columns
WHERE table_name = 'event_subscriptions' AND column_name = 'permission'
) < 1024 THEN
ALTER TABLE event_subscriptions
ALTER COLUMN permission TYPE VARCHAR(1024);
END IF;
END $$;
@@ -0,0 +1,31 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions` grew three hot queries with no index behind them:
-- a handler's own rows, the expiry reaper and the suspended reaper. See
-- sqlite/0081_event-subscriptions-indexes.sql for the query shapes.
--
-- Idempotent via IF NOT EXISTS.
CREATE INDEX IF NOT EXISTS idx_event_subscriptions_app_handler
ON event_subscriptions (app_uid, handler_name);
CREATE INDEX IF NOT EXISTS idx_event_subscriptions_expires
ON event_subscriptions (expires_at);
CREATE INDEX IF NOT EXISTS idx_event_subscriptions_suspended
ON event_subscriptions (suspended_at, id);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0082_temp-password-expiry.sql for the column rationale.
-- Idempotent via IF NOT EXISTS; there is no per-file applied-state tracking.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS temp_password_expires_at bigint DEFAULT NULL;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0083_team-directory.sql for the column rationale.
-- Idempotent via IF NOT EXISTS; there is no per-file applied-state tracking.
ALTER TABLE "group" ADD COLUMN IF NOT EXISTS directory_enabled smallint NOT NULL DEFAULT 0;
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- "Shared by me" pages by keyset: issuer equality, then `id` as the cursor.
-- The issuer foreign key created no index here (unlike mysql, where the FK's
-- index implicitly ends in the primary key), so without this the issued half
-- of that listing walks the whole table.
CREATE INDEX IF NOT EXISTS `idx_share_issuer`
ON `share` (`issuer_user_id`, `id`);
@@ -0,0 +1,104 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Scope tuple for `notification`. See mysql/mysql_mig_26.sql for the rationale.
--
-- Done as the standard 12-step rebuild rather than three ALTERs, because the
-- same table is missing two things mysql and postgres have had since the v1
-- schema: an index on `user_id`, which every `listByUserId` scans without, and
-- the cascade that retires a deleted user's rows. A column add cannot
-- introduce a foreign key here.
--
-- Rows whose user is already gone are dropped rather than carried over: the
-- cascade this migration adds is what would have removed them.
--
-- Wrapped in one transaction: without it, a crash between the `DROP` and the
-- `RENAME` below would lose the table outright rather than leave it
-- unmigrated. `IF EXISTS`/`IF NOT EXISTS` guard a stale `notification_new`
-- from a previous interrupted attempt.
BEGIN;
DROP TABLE IF EXISTS `notification_new`;
CREATE TABLE `notification_new` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"user_id" INTEGER NOT NULL,
"uid" TEXT NOT NULL UNIQUE,
"value" JSON NOT NULL,
"acknowledged" INTEGER DEFAULT NULL,
"shown" INTEGER DEFAULT NULL,
"created_at" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
"app_uid" TEXT DEFAULT NULL,
"audience" TEXT NOT NULL DEFAULT 'account',
"type" TEXT NOT NULL DEFAULT '',
FOREIGN KEY("user_id") REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
INSERT INTO `notification_new` (
`id`, `user_id`, `uid`, `value`, `acknowledged`, `shown`, `created_at`
)
SELECT
`id`, `user_id`, `uid`, `value`, `acknowledged`, `shown`, `created_at`
FROM `notification`
WHERE `user_id` IN (SELECT `id` FROM `user`);
DROP TABLE IF EXISTS `notification`;
ALTER TABLE `notification_new` RENAME TO `notification`;
CREATE INDEX IF NOT EXISTS `idx_notification_user_id`
ON `notification` (`user_id`);
-- Serves both the scoped list and the unacknowledged count.
CREATE INDEX IF NOT EXISTS `idx_notification_scope`
ON `notification` (`user_id`, `audience`, `app_uid`, `acknowledged`);
-- Backfill from the markers the payload actually carried. `type = ''` is the
-- not-yet-classified state, so a re-run matches nothing the first pass
-- claimed, and a row matching no marker keeps the defaults (`''` / 'account')
-- and reads as legacy. `json_valid` guards rows whose `value` was stored as a
-- bare string.
UPDATE `notification`
SET `type` = 'share.received', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND json_valid(`value`)
AND json_extract(`value`, '$.template') = 'file-shared-with-you';
UPDATE `notification`
SET `type` = 'share.claimed', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND json_valid(`value`)
AND json_extract(`value`, '$.template') = 'file-shared-before-you-joined';
-- Worker rows carried only `source: 'worker'` and a title, so the app they
-- belong to is unrecoverable and they stay unattributed. The title prefix is
-- the sole surviving signal of which way the deploy went.
UPDATE `notification`
SET `audience` = 'developer',
`app_uid` = NULL,
`type` = CASE
WHEN json_extract(`value`, '$.title') LIKE 'Successfully deployed %'
THEN 'app.worker.deployed'
ELSE 'app.worker.deployFailed'
END
WHERE `type` = ''
AND json_valid(`value`)
AND json_extract(`value`, '$.source') = 'worker';
COMMIT;
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index `notification`.`created_at` for the retention sweep. See
-- mysql/mysql_mig_27.sql for the rationale. A plain index add, so no table
-- rebuild.
CREATE INDEX IF NOT EXISTS `idx_notification_created_at`
ON `notification` (`created_at`);
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Home region this account's data belongs in — a node id, not a storage
-- region. NULL for every account that predates the column, which falls back to
-- `signup_server`. What a name maps to is deployment config, so adding a
-- region needs no migration.
ALTER TABLE `user` ADD COLUMN `home` VARCHAR(64) DEFAULT NULL;
@@ -0,0 +1,90 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Subscriptions that outlive the connection that made them. Session
-- subscriptions never reach this table — they are Redis only — so every row
-- here is one somebody has to be able to find and revoke later.
--
-- - `sub_id` : `<appUid>#<uuid>`, or `user#<uuid>` for a row a user
-- session created. Unique table-wide; it is what
-- `unsubscribe` names.
-- - `owner_user_id` : owner of the anchor node. Dispatch only knows whose
-- resource changed, so this is the side a shared-folder
-- subscription has to be findable from, and the key a
-- region's cache is rebuilt under.
-- - `holder_user_id` : who subscribed — the delivery target, whose access is
-- re-checked, and who the quota counts against.
-- - `app_uid` : NULL for a row a user session created. No foreign
-- key: a row outlives the app that made it, which is
-- what keeps it listable and revocable afterwards.
-- - `anchor_*` : the resolved anchor. `match` is a glob relative to
-- `anchor_path`, so the path is not decoration — the
-- dispatch filter reads it on every event.
-- - `ops` : op filter; a comma-separated set, NULL for every op.
-- - `targets` : JSON array over socket|worker|push.
-- - `context` : plaintext JSON, hard-capped at 4 KB, read only on the
-- delivery path and never returned by `list`.
-- - `permission` : the ACL mode the subscribe check passed under,
-- re-checked per delivery.
-- - `suspended_*` : set when a subscription stops delivering without
-- being removed; the state machine that drives them
-- lands with the delivery classes that need it.
--
-- `created_at` and the two timestamps are unix seconds, matching
-- `app_feedback` and `user_block`.
CREATE TABLE IF NOT EXISTS `event_subscriptions` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"sub_id" TEXT NOT NULL,
"token" TEXT NOT NULL,
"owner_user_id" INTEGER NOT NULL
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
"holder_user_id" INTEGER NOT NULL
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
"app_uid" TEXT DEFAULT NULL,
"subject" TEXT NOT NULL,
"anchor_uid" TEXT NOT NULL,
"anchor_path" TEXT NOT NULL,
"match" TEXT DEFAULT NULL,
"delivery" TEXT NOT NULL,
"ops" TEXT DEFAULT NULL,
"handler_name" TEXT DEFAULT NULL,
"targets" TEXT NOT NULL,
"context" TEXT DEFAULT NULL,
"permission" TEXT NOT NULL,
"suspended_at" INTEGER DEFAULT NULL,
"suspended_reason" TEXT DEFAULT NULL,
"expires_at" INTEGER DEFAULT NULL,
"created_at" INTEGER NOT NULL -- unix seconds
);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_event_subscriptions_sub_id`
ON `event_subscriptions` (`sub_id`);
-- Dispatch reads the table only when a region's cache is cold; this is the
-- lookup it falls back to.
CREATE INDEX IF NOT EXISTS `idx_event_subscriptions_token`
ON `event_subscriptions` (`token`);
-- List, revoke and quota all key here — the index is the scope check rather
-- than a filter over a wider read.
CREATE INDEX IF NOT EXISTS `idx_event_subscriptions_holder`
ON `event_subscriptions` (`holder_user_id`, `app_uid`);
-- Rebuilding one region's cache for one owner, and the sweep that follows it.
CREATE INDEX IF NOT EXISTS `idx_event_subscriptions_owner`
ON `event_subscriptions` (`owner_user_id`);
@@ -0,0 +1,50 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Named handlers an app deploys once and its users' subscriptions bind to by
-- name. A handler is an addressable object with its own lifecycle: nothing
-- triggers by name, and a row here runs only when a subscription bound to it
-- has a delivery.
--
-- - `app_uid` : the namespace. Names are unique per app, and there is no
-- foreign key for the same reason `event_subscriptions`
-- has none — a row has to stay manageable after the app
-- row moves.
-- - `name` : the identity a subscription binds to, stable across
-- source changes.
-- - `source` : the serialized function. Read only by the delivery path
-- and never returned by `list`.
-- - `source_hash` : change detector and idempotency key. A publish carrying
-- the same hash is a no-op, and a subscription sending an
-- inline hash binds only when it matches this one.
--
-- `created_at` / `updated_at` are unix seconds, matching `event_subscriptions`.
CREATE TABLE IF NOT EXISTS `event_handlers` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"app_uid" TEXT NOT NULL,
"name" TEXT NOT NULL,
"source" TEXT NOT NULL,
"source_hash" TEXT NOT NULL,
"created_at" INTEGER NOT NULL, -- unix seconds
"updated_at" INTEGER NOT NULL
);
-- The name is the identity, and it is unique inside one app. This index is
-- also the lookup a subscribe binding check runs on.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_event_handlers_app_name`
ON `event_handlers` (`app_uid`, `name`);
@@ -0,0 +1,41 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- A `group` row with `kind = 'team'` owns accounts and pays for them; seeded
-- system groups keep `kind` NULL so a team query never returns them.
ALTER TABLE `group` ADD COLUMN `kind` TEXT DEFAULT NULL;
ALTER TABLE `group` ADD COLUMN `name` TEXT DEFAULT NULL;
ALTER TABLE `group` ADD COLUMN `handle` TEXT COLLATE NOCASE DEFAULT NULL;
ALTER TABLE `group` ADD COLUMN `deleted_at` TIMESTAMP DEFAULT NULL;
-- NOCASE on both column and index, so lookups match mysql's utf8mb4_unicode_ci.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_group_handle`
ON `group` (`handle` COLLATE NOCASE);
-- mysql and postgres already index this column; sqlite does not.
CREATE INDEX IF NOT EXISTS `idx_group_owner` ON `group` (`owner_user_id`);
-- 1 = team-created, 0 = the team owner. Decides who pays, not who may read.
ALTER TABLE `jct_user_group` ADD COLUMN `org_owned` INTEGER DEFAULT NULL;
-- Covers "list this team's members"; the unique pair index lands in 0072.
CREATE INDEX IF NOT EXISTS `idx_jct_user_group_group`
ON `jct_user_group` (`group_id`, `user_id`);
-- Fourth `requires_*` flag; assertVerifiedAccount will enforce it in phase 5.
ALTER TABLE `user` ADD COLUMN `requires_password_change` INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,30 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `GroupStore.addUsers` had no conflict clause, so a repeat call duplicated the pair.
-- `readUserGroupPerms` joins this table on group_id alone, so each duplicate reported
-- every group permission an extra time. The delete clears what accumulated.
-- Dropping the higher id is lossless: `addUsers` writes only the two id columns, and
-- nothing references `jct_user_group.id`.
DELETE FROM `jct_user_group`
WHERE `id` NOT IN (
SELECT MIN(`id`) FROM `jct_user_group` GROUP BY `user_id`, `group_id`
);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_jct_user_group_pair`
ON `jct_user_group` (`user_id`, `group_id`);
@@ -0,0 +1,66 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Insert-only record of what an admin did to an account, plus `share.holder_group_id`.
CREATE TABLE IF NOT EXISTS `audit_team_membership` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
-- Nullable FK beside a NOT NULL `_keep`, as `audit_user_to_group_permissions` since 0019.
"group_id" INTEGER DEFAULT NULL,
"group_id_keep" INTEGER NOT NULL,
"user_id" INTEGER DEFAULT NULL,
"user_id_keep" INTEGER NOT NULL,
"actor_user_id" INTEGER DEFAULT NULL,
"action" TEXT NOT NULL,
"reason" TEXT DEFAULT NULL,
"created_at" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- SET NULL, never CASCADE: deleting an account must not erase what was done to it.
FOREIGN KEY("group_id") REFERENCES "group" ("id") ON DELETE SET NULL ON UPDATE CASCADE,
FOREIGN KEY("user_id") REFERENCES "user" ("id") ON DELETE SET NULL ON UPDATE CASCADE,
FOREIGN KEY("actor_user_id") REFERENCES "user" ("id") ON DELETE SET NULL ON UPDATE CASCADE
);
-- The admin's view: one team, newest first.
CREATE INDEX IF NOT EXISTS `idx_audit_team_membership_group`
ON `audit_team_membership` (`group_id_keep`, `id`);
-- The member's view; the only place a reset becomes visible to its subject.
CREATE INDEX IF NOT EXISTS `idx_audit_team_membership_user`
ON `audit_team_membership` (`user_id_keep`, `id`);
-- Without these, SET NULL scans the audit table on every user or group delete.
CREATE INDEX IF NOT EXISTS `idx_audit_team_membership_group_fk`
ON `audit_team_membership` (`group_id`);
CREATE INDEX IF NOT EXISTS `idx_audit_team_membership_user_fk`
ON `audit_team_membership` (`user_id`);
CREATE INDEX IF NOT EXISTS `idx_audit_team_membership_actor_fk`
ON `audit_team_membership` (`actor_user_id`);
-- Mirrors `holder_user_id` from 0067: a `share` row is a listing entry, not the grant.
ALTER TABLE `share` ADD COLUMN `holder_group_id` INTEGER DEFAULT NULL
REFERENCES `group` (`id`) ON DELETE CASCADE ON UPDATE CASCADE;
CREATE INDEX IF NOT EXISTS `idx_share_holder_group`
ON `share` (`holder_group_id`, `id`);
-- Team shares leave `holder_user_id` NULL, and NULLs are distinct, so the existing
-- unique index binds none of them.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_share_holder_group_entry_issuer`
ON `share` (`holder_group_id`, `fsentry_id`, `issuer_user_id`);
@@ -0,0 +1,59 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- An opaque name for a region of one user's key-value namespace that somebody
-- else may watch. Key-value subjects carry no owner component, so without this
-- a shared key is unaddressable: the handle is what a filesystem node uid
-- already is, an owner-independent name for a shared root.
--
-- - `handle` : `kvh-<uuid>`, deliberately shaped unlike an app uid so
-- the subject parser can tell which slot it is in. It is
-- the only form the grantee ever sees, and it names
-- neither the owner's username nor their uuid.
-- - `permission` : the user-to-user grant this handle mirrors. Stored
-- rather than derived: it is the key revocation settling
-- matches subscriptions on, and deriving it would tie the
-- settle path to a lookup per row.
-- - `key_prefix` : the granted root, always ending on the key delimiter,
-- so `key_prefix + relative` is a whole key.
-- - `revoked_at` : set rather than deleted. A revoked handle stays visible
-- to its owner, which is the only record of what was
-- shared and when it stopped.
--
-- `created_at` / `revoked_at` are unix seconds, matching `event_subscriptions`.
CREATE TABLE IF NOT EXISTS `kv_share_handles` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"handle" TEXT NOT NULL,
"owner_user_id" INTEGER NOT NULL
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
"grantee_user_id" INTEGER NOT NULL
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
"app_uid" TEXT NOT NULL,
"key_prefix" TEXT NOT NULL,
"permission" TEXT NOT NULL,
"created_at" INTEGER NOT NULL, -- unix seconds
"revoked_at" INTEGER DEFAULT NULL
);
-- Resolving a subject is a lookup by handle, on the subscribe path.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_kv_share_handles_handle`
ON `kv_share_handles` (`handle`);
-- The owner's audit listing, and the scope check behind revoke.
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_owner`
ON `kv_share_handles` (`owner_user_id`, `id`);
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions` grew three hot queries with no index behind them:
--
-- - `(app_uid, handler_name)` : a handler's own rows (`listByHandler`) —
-- suspending/resuming everything one
-- publish or removal touches.
-- - `(expires_at)` : the expiry reaper's sweep.
-- - `(suspended_at, id)` : the suspended reaper's sweep. The range on
-- `suspended_at` is the win; `id` only keeps
-- the reaper's keyset page off the table.
CREATE INDEX IF NOT EXISTS `idx_event_subscriptions_app_handler`
ON `event_subscriptions` (`app_uid`, `handler_name`);
CREATE INDEX IF NOT EXISTS `idx_event_subscriptions_expires`
ON `event_subscriptions` (`expires_at`);
CREATE INDEX IF NOT EXISTS `idx_event_subscriptions_suspended`
ON `event_subscriptions` (`suspended_at`, `id`);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Unix seconds after which an admin-issued temporary password stops
-- authenticating, so an unused reset dies instead of becoming a standing
-- credential. NULL for every password the account chose itself.
ALTER TABLE `user` ADD COLUMN `temp_password_expires_at` INTEGER DEFAULT NULL;
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Whether apps acting for a member may read this team's member list.
-- Members can already enumerate each other through `/teams/:uid/members`,
-- which requires a user actor; this opens the same names to an app actor,
-- so it is off until the team owner turns it on.
ALTER TABLE `group` ADD COLUMN `directory_enabled` INTEGER NOT NULL DEFAULT 0;
@@ -441,6 +441,7 @@ export class DDBClient extends PuterClient {
index = '',
consistentRead = false,
options?: {
scanIndexForward?: boolean;
beginsWith?: { key: string; value: string };
select?: 'COUNT';
filter?: {
@@ -497,6 +498,9 @@ export class DDBClient extends PuterClient {
? { FilterExpression: options.filter.expression }
: {}),
...(options?.select ? { Select: options.select } : {}),
...(options?.scanIndexForward !== undefined
? { ScanIndexForward: options.scanIndexForward }
: {}),
ReturnConsumedCapacity: 'TOTAL',
});
@@ -511,6 +515,7 @@ export class DDBClient extends PuterClient {
expression: string,
expressionValues?: Record<string, unknown>,
expressionNames?: Record<string, string>,
options?: { condition?: string },
) {
const hasValues =
!!expressionValues && Object.keys(expressionValues).length > 0;
@@ -531,6 +536,9 @@ export class DDBClient extends PuterClient {
...(hasNames
? { ExpressionAttributeNames: expressionNames }
: {}),
...(options?.condition
? { ConditionExpression: options.condition }
: {}),
ReturnValues: 'ALL_NEW',
ReturnConsumedCapacity: 'TOTAL',
}),
+62 -14
View File
@@ -19,6 +19,7 @@
import { describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import { digestLines } from '../../services/share/shareNotifyTitle';
import { EmailClient } from './EmailClient';
const FROM = '"Puter" <[email protected]>';
@@ -262,14 +263,25 @@ describe('EmailClient — share notification templates', () => {
return captured;
};
// Built by the producer rather than hand-shaped, so a change to the digest
// wording can't leave these fixtures describing a shape it no longer emits.
const HOLDER = {
recipient: 'alice',
subject_line: 'bob shared notes.md with you',
shares: [
{ sender: 'bob', what: 'notes.md' },
{ sender: 'carol', what: '3 items — a.txt, b.txt, +1 more' },
],
link: 'https://puter.test',
shares: digestLines([
{
username: 'bob',
count: 1,
items: [{ name: 'notes.md', link: 'https://puter.test/?shared=%2Fbob%2Fu1%2Fnotes.md' }],
},
{
username: 'carol',
count: 3,
items: [{ name: 'a.txt' }, { name: 'b.txt' }],
},
]),
link: 'https://puter.test/?shared=%2Fbob%2Fu1%2Fnotes.md',
origin: 'https://puter.test',
unsubscribe_uuid: null,
};
@@ -293,13 +305,41 @@ describe('EmailClient — share notification templates', () => {
it('escapes item names in the html and leaves them raw in the text', async () => {
const { html, text } = await renderShare('file_shared_with_you', {
...HOLDER,
shares: [{ sender: 'bob', what: 'r&d "notes".md' }],
shares: digestLines([
{
username: 'bob',
count: 1,
items: [{ name: 'r&d "notes".md' }],
},
]),
});
expect(html).toContain('r&amp;d &quot;notes&quot;.md');
expect(text).toContain('r&d "notes".md');
});
// The name links to the item. The URL is ours, built from the origin and one
// encoded path, so it stays literal — an entity-escaped `=` would still
// resolve, but the text part has no parser to undo it.
it('links a named item to itself in both parts', async () => {
const link = 'https://puter.test/?shared=%2Fbob%2Fu1%2Fnotes.md';
const { html, text } = await renderShare('file_shared_with_you', HOLDER);
expect(html).toContain(`<a href="${link}"`);
expect(html).toContain(`>notes.md</a>`);
expect(text).toContain(link);
expect(html).not.toContain('&#x3D;');
});
// An item with no link — an invite, with no account to route to yet —
// renders as the plain name it always did.
it('leaves an unlinked item as plain text', async () => {
const { html } = await renderShare('file_shared_with_you', HOLDER);
expect(html).toContain('a.txt');
expect(html).not.toContain('>a.txt</a>');
});
it('renders as a responsive single column with no remote assets', async () => {
const { html } = await renderShare('file_shared_with_you', HOLDER);
@@ -325,15 +365,21 @@ describe('EmailClient — share notification templates', () => {
expect(preheader).toBeLessThan(html.indexOf('Hi alice,'));
});
it('points the call to action at the app and nowhere else', async () => {
const { html, text } = await renderShare(
'file_shared_with_you',
HOLDER,
);
// The button carries every item in the mail, so it is a query string with
// `&` and `=` in it — which must reach the client as written, in both parts.
it('points the call to action at Shared with every item picked out', async () => {
const link =
'https://puter.test/?shared=%2Fbob%2Fu1%2Fa.txt&shared=%2Fbob%2Fu2%2Fb.txt';
const { html, text } = await renderShare('file_shared_with_you', {
...HOLDER,
link,
});
expect(html).toContain('href="https://puter.test"');
expect(html).toContain(`href="${link}"`);
expect(html).toContain('>Open Puter</a>');
expect(text).toContain('Open Puter: https://puter.test');
expect(text).toContain(`Open Puter: ${link}`);
expect(html).not.toContain('&#x3D;');
expect(html).not.toContain('&amp;shared');
});
it('separates senders without a rule above the first', async () => {
@@ -364,7 +410,9 @@ describe('EmailClient — share notification templates', () => {
const { html, text } = await renderShare('file_shared_invite', {
email: '[email protected]',
subject_line: 'bob shared notes.md with you on Puter',
shares: [{ sender: 'bob', what: 'notes.md' }],
shares: digestLines([
{ username: 'bob', count: 1, items: [{ name: 'notes.md' }] },
]),
link: 'https://puter.test',
});
+13 -4
View File
@@ -28,10 +28,15 @@ import {
type EmailTemplateName,
} from './templates';
/** Attachment shape passed through to the underlying transport. */
/**
* Attachment shape passed through to the underlying transport. Give either
* `content` (held in memory) or `path` (a local file the transport streams on
* its own, re-read for every message that carries it).
*/
export interface EmailAttachment {
filename: string;
content: Buffer | string;
content?: Buffer | string;
path?: string;
contentType?: string;
encoding?: string;
}
@@ -161,19 +166,23 @@ export class EmailClient extends PuterClient {
* Render a template and send it to `to`. `options.replyTo` sets the
* Reply-To header (e.g. so a recipient can respond to the originator of the
* message rather than the no-reply From address).
*
* Returns the transport's send result, or `null` when none is configured —
* {@link sendRaw} drops the message instead of throwing. A caller that
* treats undelivered mail as a failure must check for `null` too.
*/
async send<T extends EmailTemplateName>(
to: string,
template: T,
values: Record<string, unknown> = {},
options: { replyTo?: string } = {},
): Promise<void> {
) {
const compiled = this.compiledTemplates[template];
if (!compiled) {
throw new Error(`Unknown email template: ${template}`);
}
await this.sendRaw({
return await this.sendRaw({
from: this.defaultFrom(),
to,
subject: compiled.subject(values),
+78 -6
View File
@@ -169,14 +169,22 @@ const headingRow = (text: string): string => `
* One row per sender, hairline-separated. The wording comes pre-composed from
* the digest (`digestLines`), so the list stays a list however many senders and
* items fold into it.
*
* Each named item links to itself where it has one; `lead`/`trail` are the
* wording around the names. An unlinked item renders as the plain name.
*
* `link` is triple-braced because we build it, so `?` and `=` stay literal;
* `name` is the owner's text and stays escaped.
*/
const SHARE_ITEM_LIST = `{{this.lead}}{{#each this.items}}{{#unless @first}}, {{/unless}}{{#if this.link}}<a href="{{{this.link}}}" style="color: ${ACCENT}; text-decoration: underline;">{{this.name}}</a>{{else}}{{this.name}}{{/if}}{{/each}}{{this.trail}}`;
const SHARE_LIST_ROW = `
<tr>
<td class="panel" style="padding: 6px 18px; background-color: ${PANEL_BG}; border: 1px solid ${PANEL_BORDER}; border-radius: 10px;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">
{{#each shares}}
<tr>
<td class="ink rule" style="padding: 12px 0;{{#unless @first}} border-top: 1px solid ${RULE};{{/unless}} font-family: ${FONT}; font-size: 16px; line-height: 24px; color: ${INK};"><strong style="font-weight: 600;">{{this.sender}}</strong> shared {{this.what}}</td>
<td class="ink rule" style="padding: 12px 0;{{#unless @first}} border-top: 1px solid ${RULE};{{/unless}} font-family: ${FONT}; font-size: 16px; line-height: 24px; color: ${INK};"><strong style="font-weight: 600;">{{this.sender}}</strong> shared ${SHARE_ITEM_LIST}</td>
</tr>
{{/each}}
</table>
@@ -193,6 +201,9 @@ const textRow = (text: string, padding = '18px 0 0'): string => `
* The call to action. Padding sits on the cell and the color on `bgcolor` so
* Outlook still draws a real button (square-cornered, which is fine); the table
* goes full width under 600px so the tap target spans the card.
*
* `link` is triple-braced for the same reason as an item's: we build it, and a
* deep link's `=` and `&` should read the same in the html as in the text.
*/
const buttonRow = (label: string): string => `
<tr>
@@ -200,7 +211,7 @@ const buttonRow = (label: string): string => `
<table role="presentation" class="btn" cellpadding="0" cellspacing="0" border="0" style="border-collapse: separate;">
<tr>
<td align="center" bgcolor="${ACCENT}" style="background-color: ${ACCENT}; border-radius: 10px; padding: 14px 26px;">
<a href="{{link}}" style="display: inline-block; font-family: ${FONT}; font-size: 16px; line-height: 20px; font-weight: 600; color: #ffffff; text-decoration: none;">${label}</a>
<a href="{{{link}}}" style="display: inline-block; font-family: ${FONT}; font-size: 16px; line-height: 20px; font-weight: 600; color: #ffffff; text-decoration: none;">${label}</a>
</td>
</tr>
</table>
@@ -333,6 +344,64 @@ [email protected] immediately.
<p>A password recovery request was issued for your account, please follow the link below to reset your password:</p>
<p><a href="{{link}}">{{link}}</a></p>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
team_account_created: {
subject: 'Your {{team_name}} account on Puter',
html: `
<p>Hi there,</p>
<p>{{team_name}} has created a Puter account for you: <b>{{username}}</b>.
They will send you a temporary password separately; you will be asked to
choose your own the first time you sign in.</p>
<p>What this means:</p>
<ul>
<li>This account belongs to {{team_name}}. They pay for it and can close it.</li>
<li>{{team_name}} <b>cannot</b> see your files, apps or data.</li>
<li>{{team_name}} <b>can</b> reset your password, which would let them sign in
as you. Choose your own password promptly.</li>
</ul>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
team_account_disabled: {
subject: 'Your {{team_name}} account has been disabled',
html: `
<p>Hi there,</p>
<p>{{team_name}} has disabled your Puter account <b>{{username}}</b>. You can no
longer sign in to it.</p>
<p>Nothing has been deleted. Your files, apps and data are as you left them, and
{{team_name}} can re-enable the account at any time. If you believe this is a
mistake, ask them.</p>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
team_closed: {
subject: '{{team_name}} has been closed',
html: `
<p>Hi there,</p>
<p>{{team_name}} has closed its Puter team, and your account
<b>{{username}}</b> has been disabled along with it. You can no longer sign in to
it.</p>
<p>Nothing has been deleted. Your files and data are still there; closing the
team does not destroy the accounts it created, and each one would have to be
deleted on its own request.</p>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
team_password_reset: {
subject: 'Your {{team_name}} account password was reset',
html: `
<p>Hi there,</p>
<p>An administrator of {{team_name}} reset the password on your Puter account
<b>{{username}}</b>. They will send you a temporary password separately, and you
will be asked to choose your own the next time you sign in.</p>
<p>If you did not expect this, the record of who reset it and when is in your
team activity, along with every sign-in to your account.</p>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
@@ -362,7 +431,9 @@ immediately</p>
/**
* A digest: shares to one recipient are held briefly and merged, so
* `shares` may carry several senders. The subject is composed by the
* service (see `digestSubject`), which owns the grouped wording.
* service (see `digestSubject`), which owns the grouped wording. `link`
* opens Shared with every item highlighted; `origin` is the bare site, for
* the unsubscribe link.
*/
file_shared_with_you: {
subject: '{{subject_line}}',
@@ -378,7 +449,7 @@ immediately</p>
'24px 0 0',
),
footer: `You're receiving this because someone shared with your Puter account.{{#if unsubscribe_uuid}}
<br /><a href="{{link}}/unsubscribe?user_uuid={{unsubscribe_uuid}}" style="color: inherit; text-decoration: underline;">Unsubscribe from notification emails</a>{{/if}}`,
<br /><a href="{{origin}}/unsubscribe?user_uuid={{unsubscribe_uuid}}" style="color: inherit; text-decoration: underline;">Unsubscribe from notification emails</a>{{/if}}`,
}),
text: `
Hi{{#if recipient}} {{recipient}}{{/if}},
@@ -386,7 +457,8 @@ immediately</p>
Shared with you on Puter:
{{#each shares}}
- {{this.sender}} shared {{this.what}}
{{/each}}
{{#each this.items}}{{#if this.link}} {{this.name}}: {{this.link}}
{{/if}}{{/each}}{{/each}}
Open Puter: {{link}}
@@ -395,7 +467,7 @@ immediately</p>
--
You're receiving this because someone shared with your Puter account.
{{#if unsubscribe_uuid}}Unsubscribe from notification emails:
{{link}}/unsubscribe?user_uuid={{unsubscribe_uuid}}{{/if}}
{{origin}}/unsubscribe?user_uuid={{unsubscribe_uuid}}{{/if}}
`,
},
// The only way to reach someone with no account. Same digest shape.
+188 -6
View File
@@ -32,6 +32,27 @@ type GuiEvent<R = Record<string, unknown>> = {
response: R;
};
// The entry arrives under several aliases, for handlers of any vintage.
type FsCreateEvent = { node: FSEntry; entry: FSEntry; uid: string };
/**
* Who pays, and for which team. Deliberately no payment identity: the payer's
* row is alive at emit time, so a consumer resolves it when it acts -- a
* snapshot taken here would be stale, and two events racing on it would each
* create their own customer.
*/
export type TeamBillingContext = {
team_uid: string;
owner_user_id: number;
};
/** A team event about one seat. */
export type TeamBillingEvent = TeamBillingContext & {
user_id: number;
user_uuid: string;
username: string;
};
/**
* Extension-augmentable half of {@link EventMap}. Extensions that emit their own
* events declare the payload here by declaration merging, so both the emitter
@@ -50,6 +71,15 @@ type GuiEvent<R = Record<string, unknown>> = {
// eslint-disable-next-line @typescript-eslint/no-empty-object-type
export interface IExtensionEventMap {}
/**
* What a KV mutation did to a key. `expire` is a lifetime change rather than a
* value change, which is why it is not folded into `set`.
*/
export type KvOp = 'set' | 'del' | 'expire';
/** A flush empties a whole namespace, so it is never reported per key. */
export type KvMutation = KvOp | 'flush';
export type EventMap = {
// ---- Server lifecycle ----
serverStart: Record<string, never>;
@@ -107,6 +137,8 @@ export type EventMap = {
};
// ---- Apps ----
/** Awaited before lookup; listeners may mutate or replace appNames. */
'app.recommended': { appNames: string[] };
'app.changed': {
app_uid: string;
action: string;
@@ -277,6 +309,14 @@ export type EventMap = {
publishable_key: string | null;
[key: string]: unknown;
};
// Side-effect-free "is the card gate usable?" probe. The extension stamps
// `enabled` from its own config and does nothing else — no provider call,
// no user state, nothing billed. It exists so the SMS-to-card fallback can
// default on only where a card gate actually works, which means it gets
// asked often and must stay cheap.
'puter.card-verification.status': {
enabled: boolean | null;
};
'puter.card-verification.confirm': {
user_id: number;
user_uid: string;
@@ -314,6 +354,23 @@ export type EventMap = {
stripe_customer_id?: string | null;
};
// ---- Team billing ---- the trigger, never the charge ----
'team.account.created': TeamBillingEvent;
/** `held_bytes` is what to bill storage on while the seat is off. */
'team.account.disabled': TeamBillingEvent & { held_bytes: number };
'team.account.enabled': TeamBillingEvent & { held_bytes: number };
/** Emitted pre-delete: the membership row cascades away with the user. */
'team.account.deleted': TeamBillingEvent;
/** Per-seat charges stop; byte charges do not, the accounts remain. */
'team.deleted': TeamBillingContext & { account_count: number };
/** A budget line was crossed. Emitted on transition only, never per request. */
'metering.credit-state': {
user_uuid: string;
state: 'near-limit' | 'exhausted';
allowance_used: number;
month_usage_allowance: number;
};
// ---- Filesystem ----
'fs.copy.node': {
source: unknown;
@@ -334,6 +391,18 @@ export type EventMap = {
};
'fs.remove.node': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.write.file': { node: FSEntry; entry: FSEntry; target: FSEntry };
/** A new entry, one key per flavor; `fs.write.file` is the overwrite. */
'fs.create.file': FsCreateEvent;
'fs.create.directory': FsCreateEvent;
'fs.create.shortcut': FsCreateEvent;
'fs.create.symlink': FsCreateEvent;
/** In-place name change. A move emits `fs.move.node` instead. */
'fs.rename': FsCreateEvent & {
old_name: string;
new_name: string;
old_path: string;
new_path: string;
};
'fs.storage.upload-progress': {
upload_tracker: unknown;
context: unknown;
@@ -347,6 +416,57 @@ export type EventMap = {
};
'storage.quota.bonus': { userId: number; extra: number };
// ---- Key-value ----
/**
* Keys of one namespace that changed together, announced after the write
* commits. `namespace` is `v1:<userUuid>:<appUid>` and `userId` is the same
* user by its row id — the only part a listener cannot derive from the
* namespace, and the one dispatch keys on.
*
* Distinct from `outer.kv.cacheInvalidated`, which carries encoded cache
* keys and is suppressed entirely when the read cache is off.
*/
'kv.mutated': {
namespace: string;
userId: number;
keys: string[];
op: KvOp;
};
/**
* A whole namespace was emptied. Namespace-level on purpose: `flush`'s own
* key enumeration is truncated for a large namespace, so the keys it saw
* are not the keys it removed.
*/
'kv.flushed': { namespace: string; userId: number };
// ---- Notifications ----
/**
* A notification row exists. Emitted after the insert commits — nothing
* pushes a uid before there is a row to name — and carries the scope tuple
* so a listener never has to read the row back.
*/
'notif.created': {
userId: number;
userUuid: string;
uid: string;
type: string;
audience: string;
appUid: string | null;
value: Record<string, unknown>;
createdAt: number;
};
/**
* One notification, addressed at whatever sockets each region holds for the
* recipient. `outer.*` reaches peer regions and never sibling nodes, so
* every region applies it exactly once — which is what keeps a socket from
* receiving the same notification twice.
*/
'outer.notif.delivery': {
userId: number;
wire: 'notif.message' | 'notif.unreads' | 'notif.ack';
response: unknown;
};
// ---- Metering ----
// Recurring charges are pure mechanism here: the metering service knows
// when to ask (once per user per month, the first time that month's usage
@@ -391,6 +511,14 @@ export type EventMap = {
worker: string;
};
// An app's events worker coming into being (its handler count going 0→1)
// and going away (1→0, whether by removing the last handler or by the
// destroy route). `actor.user` is the app's owner, not the caller — a
// developer session publishing for an app it owns is the common case, but
// billing follows ownership.
'events.worker.create': { actor: Actor; appUid: string };
'events.worker.destroy': { actor: Actor; appUid: string };
// ---- Outer / GUI broadcast ----
'outer.cacheUpdate': {
cacheKey: string[];
@@ -412,6 +540,37 @@ export type EventMap = {
'outer.permission.flatInvalidated': {
entries: Array<{ holderUserId: number; permission: string }>;
};
/**
* A user's subscription set changed, so every process must drop its cached
* "does this user have any subscriptions" answer. Carries the counter the
* bump produced so a listener can order two bumps it sees out of order.
*
* The dispatch hot path never reads the counter — it is the broadcast that
* invalidates, which is what keeps an unsubscribed write at zero Redis
* commands. Hence `outer.pubsub.*`: the cache is a per-process map, the
* webhook reaches one node per peer region, and only the Redis fan reaches
* that node's siblings — a node the bump never reached has nothing that
* expires.
*/
'outer.pubsub.events.generationBumped': {
userId: number;
generation: number;
/** Whether the table changed; only then does a peer need to re-read it. */
durable: boolean;
};
/**
* A user's sockets moved between regions, so every node must drop what it
* cached about where they are. Bumped on a first connect, a last
* disconnect, and every repair — never on a timer, which is what keeps
* presence reads proportional to session churn rather than to event volume.
* `outer.pubsub.*` for the same reason as the generation bump: a
* per-process cache that only the Redis fan reaches on a region's other
* nodes.
*/
'outer.pubsub.events.presenceBumped': {
userId: number;
generation: number;
};
'outer.fs.write-hash': { hash: string; uuid: string };
/**
* Cache keys the KV read cache must stop serving, because the entries
@@ -427,8 +586,8 @@ export type EventMap = {
'outer.gui.item.moved': GuiEvent;
'outer.gui.item.pending': GuiEvent;
'outer.gui.item.removed': GuiEvent;
'outer.gui.item.renamed': GuiEvent;
'outer.gui.notif.ack': GuiEvent<{ uid: string }>;
'outer.gui.notif.persisted': GuiEvent<{ uid: string }>;
'outer.gui.notif.message': GuiEvent<{ uid: string; notification: unknown }>;
'outer.gui.notif.unreads': GuiEvent<{
unreads: { uid: string; notification: unknown }[];
@@ -470,6 +629,30 @@ export type EventMap = {
'web.socket.connected': { socket: unknown; user: unknown };
'web.socket.user-connected': { socket: unknown; user: unknown };
/**
* Sessions were revoked for a user. Anything holding one of them open — a
* live socket, say — should drop it: `revoked_at` is otherwise only read on
* the next handshake.
*/
'auth.sessions.revoked': { user_id: number; session_uids: string[] };
/**
* A grant was withdrawn, so whatever was standing on it has to be settled
* rather than left to fail its next check.
*
* Local rather than `outer.*` on purpose: a listener settles shared state,
* and one region doing it covers every other. Emitted once per revoke call,
* not once per row it affects.
*/
'permission.revoked': {
/** Whose access went. */
holderUserId: number;
/** The app the grant was to, or `null` for a user-to-user grant. */
appUid: string | null;
/** The grant string, or `null` when every grant to the app went. */
permission: string | null;
};
// ---- Extension hooks / misc ----
'puter.gui.addons': {
prependHeadContent?: string[];
@@ -610,11 +793,10 @@ export type EventKey = keyof EventMap & string;
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
? | `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
@@ -0,0 +1,410 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* `DispatcherInvokeTransport` against an injected fetch, so the handled/error
* header contract is pinned without a real dispatcher on the other end.
*/
import type { fetch as undiciFetch } from 'undici';
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types.js';
import {
DispatcherInvokeTransport,
EVENTS_DEPLOYED_HEADER,
EVENTS_DISPATCH_ERROR_HEADER,
EVENTS_DISPATCH_PATH,
EVENTS_ERROR_HEADER,
EVENTS_HANDLED_HEADER,
EventsWorkerInvokerClient,
type WorkerInvokeRequest,
} from './EventsWorkerInvokerClient.js';
type FetchImpl = typeof undiciFetch;
const CALL = {
script: 'evw-test',
appUid: 'app-test',
key: 'k1:test',
body: '{}',
timeoutMs: 5_000,
};
/** A stub fetch that answers the same way on every call. */
const stubFetch = (
status: number,
headers: Record<string, string> = {},
): FetchImpl =>
(async () =>
new Response(null, { status, headers })) as unknown as FetchImpl;
/** A stub fetch that never answers until the abort signal fires. */
const hangingFetch: FetchImpl = ((_url: string, init?: RequestInit) =>
new Promise((_resolve, reject) => {
init?.signal?.addEventListener('abort', () =>
reject(new Error('This operation was aborted')),
);
})) as unknown as FetchImpl;
/** A stub fetch that records the URL it was called with. */
const capturingFetch = (
calls: string[],
status = 200,
): FetchImpl =>
(async (url: string) => {
calls.push(url);
return new Response(null, { status });
}) as unknown as FetchImpl;
describe('DispatcherInvokeTransport', () => {
it('reports a handled 400 with no error — the handler`s own refusal', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(400, { [EVENTS_HANDLED_HEADER]: '1' }),
});
expect(await transport.send(CALL)).toEqual({
status: 400,
handled: true,
});
});
it('reports an unmarked 404 with no dispatch-error header as unhandled', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(404),
});
expect(await transport.send(CALL)).toEqual({
status: 404,
handled: false,
});
});
it('turns a dispatch-error header into a null status with the reason', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(502, { [EVENTS_DISPATCH_ERROR_HEADER]: 'deploy-failed' }),
});
expect(await transport.send(CALL)).toEqual({
status: null,
error: 'dispatcher: deploy-failed (502)',
dispatchReason: 'deploy-failed',
});
});
it('adds the deployed header when the call says so', async () => {
const headers: Array<Record<string, string>> = [];
const fetchImpl = (async (_url: string, init?: RequestInit) => {
headers.push(init?.headers as Record<string, string>);
return new Response(null, { status: 200 });
}) as unknown as FetchImpl;
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl,
});
await transport.send(CALL);
await transport.send({ ...CALL, deployed: true });
expect(headers[0][EVENTS_DEPLOYED_HEADER]).toBeUndefined();
expect(headers[1][EVENTS_DEPLOYED_HEADER]).toBe('1');
});
it('reports a handled 200 as settled with no error', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(200, { [EVENTS_HANDLED_HEADER]: '1' }),
});
expect(await transport.send(CALL)).toEqual({
status: 200,
handled: true,
});
});
it('passes a 429 through as-is', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(429),
});
expect(await transport.send(CALL)).toEqual({
status: 429,
handled: false,
});
});
it('answers null with an error when the request times out', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: hangingFetch,
});
const result = await transport.send({ ...CALL, timeoutMs: 10 });
expect(result.status).toBeNull();
expect(result.error).toMatch(/abort/i);
});
it('surfaces the runtime`s own error header alongside its status', async () => {
const transport = new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(500, {
[EVENTS_HANDLED_HEADER]: '1',
[EVENTS_ERROR_HEADER]: 'handler-threw',
}),
});
expect(await transport.send(CALL)).toEqual({
status: 500,
handled: true,
error: 'handler-threw',
});
});
it('preserves a path prefix on the dispatcher URL', async () => {
const calls: string[] = [];
const transport = new DispatcherInvokeTransport(
'http://dispatcher/prefix/',
's',
{ fetchImpl: capturingFetch(calls) },
);
await transport.send(CALL);
expect(calls).toEqual([`http://dispatcher/prefix${EVENTS_DISPATCH_PATH}`]);
});
});
const REQUEST: WorkerInvokeRequest = {
script: 'evw-test',
appUid: 'app-test',
handler: 'h',
token: 't',
key: 'k1:test',
event: {},
ctx: {},
};
const makeClient = () => new EventsWorkerInvokerClient({} as unknown as IConfig);
/** Answers `missing` until it sees the deployed header, then settles. */
const missUntilDeployedFetch = (
headersSeen: Array<Record<string, string>>,
): FetchImpl =>
(async (_url: string, init?: RequestInit) => {
const headers = (init?.headers ?? {}) as Record<string, string>;
headersSeen.push(headers);
if (headers[EVENTS_DEPLOYED_HEADER] === '1')
return new Response(null, {
status: 200,
headers: { [EVENTS_HANDLED_HEADER]: '1' },
});
return new Response(null, {
status: 404,
headers: { [EVENTS_DISPATCH_ERROR_HEADER]: 'missing' },
});
}) as unknown as FetchImpl;
/** Answers `missing` no matter what, deployed header included. */
const alwaysMissingFetch = (calls: unknown[]): FetchImpl =>
(async () => {
calls.push(null);
return new Response(null, {
status: 404,
headers: { [EVENTS_DISPATCH_ERROR_HEADER]: 'missing' },
});
}) as unknown as FetchImpl;
/**
* Answers `deploy-timeout` once, then 200 once it sees the deployed header —
* the shape of a script that finished propagating between the two calls.
*/
const deployTimeoutThenOkFetch = (
headersSeen: Array<Record<string, string>>,
): FetchImpl =>
(async (_url: string, init?: RequestInit) => {
const headers = (init?.headers ?? {}) as Record<string, string>;
headersSeen.push(headers);
if (headers[EVENTS_DEPLOYED_HEADER] === '1')
return new Response(null, {
status: 200,
headers: { [EVENTS_HANDLED_HEADER]: '1' },
});
return new Response(null, {
status: 503,
headers: { [EVENTS_DISPATCH_ERROR_HEADER]: 'deploy-timeout' },
});
}) as unknown as FetchImpl;
describe('EventsWorkerInvokerClient deploy-on-miss', () => {
it('deploys the script and retries once with the deployed header', async () => {
const headersSeen: Array<Record<string, string>> = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: missUntilDeployedFetch(headersSeen),
}),
);
const missCalls: Array<[string, string]> = [];
client.setMissHandler(async (appUid, script) => {
missCalls.push([appUid, script]);
return 'deployed';
});
const result = await client.invoke(REQUEST);
expect(missCalls).toEqual([[REQUEST.appUid, REQUEST.script]]);
expect(headersSeen).toHaveLength(2);
expect(headersSeen[0][EVENTS_DEPLOYED_HEADER]).toBeUndefined();
expect(headersSeen[1][EVENTS_DEPLOYED_HEADER]).toBe('1');
expect(result).toEqual({ outcome: 'settled', status: 200 });
});
it('does not retry when the miss handler reports the script is stale', async () => {
const calls: unknown[] = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: alwaysMissingFetch(calls),
}),
);
client.setMissHandler(async () => 'stale');
const result = await client.invoke(REQUEST);
expect(calls).toHaveLength(1);
expect(result).toEqual({
outcome: 'retriable',
status: null,
error: 'deploy: stale',
});
});
it('never asks the miss handler twice for one invocation', async () => {
const calls: unknown[] = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: alwaysMissingFetch(calls),
}),
);
let handlerCalls = 0;
client.setMissHandler(async () => {
handlerCalls++;
return 'deployed';
});
const result = await client.invoke(REQUEST);
expect(handlerCalls).toBe(1);
// The first send plus the one retry — never a third.
expect(calls).toHaveLength(2);
expect(result.outcome).toBe('retriable');
expect(result.status).toBeNull();
});
it('leaves behavior unchanged with no miss handler set', async () => {
const calls: unknown[] = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: alwaysMissingFetch(calls),
}),
);
const result = await client.invoke(REQUEST);
expect(calls).toHaveLength(1);
expect(result).toEqual({
outcome: 'retriable',
status: null,
error: 'dispatcher: missing (404)',
});
});
it('does not trigger the miss handler for a forbidden dispatch reason', async () => {
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: stubFetch(403, {
[EVENTS_DISPATCH_ERROR_HEADER]: 'forbidden',
}),
}),
);
let handlerCalled = false;
client.setMissHandler(async () => {
handlerCalled = true;
return 'deployed';
});
const result = await client.invoke(REQUEST);
expect(handlerCalled).toBe(false);
expect(result).toEqual({
outcome: 'retriable',
status: null,
error: 'dispatcher: forbidden (403)',
});
});
it('retries a deploy-timeout with the deployed header instead of redeploying', async () => {
const headersSeen: Array<Record<string, string>> = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: deployTimeoutThenOkFetch(headersSeen),
}),
);
const deployCalls: unknown[] = [];
client.setMissHandler(async () => {
deployCalls.push(null);
return 'deployed';
});
const result = await client.invoke(REQUEST);
// The script was already deployed once — the one that led to this
// deploy-timeout answer — so this client must not add a second.
expect(deployCalls).toHaveLength(0);
expect(headersSeen).toHaveLength(2);
expect(headersSeen[0][EVENTS_DEPLOYED_HEADER]).toBeUndefined();
expect(headersSeen[1][EVENTS_DEPLOYED_HEADER]).toBe('1');
expect(result).toEqual({ outcome: 'settled', status: 200 });
});
it('gives up after one retry when propagation is still not done', async () => {
const calls: unknown[] = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: (async () => {
calls.push(null);
return new Response(null, {
status: 503,
headers: {
[EVENTS_DISPATCH_ERROR_HEADER]: 'deploy-timeout',
},
});
}) as unknown as FetchImpl,
}),
);
const deployCalls: unknown[] = [];
client.setMissHandler(async () => {
deployCalls.push(null);
return 'deployed';
});
const result = await client.invoke(REQUEST);
// The first send plus the one retry — never a third, and never a
// deploy. The delivery comes back on its own backoff instead.
expect(calls).toHaveLength(2);
expect(deployCalls).toHaveLength(0);
expect(result).toEqual({
outcome: 'retriable',
status: null,
error: 'dispatcher: deploy-timeout (503)',
});
});
});
@@ -0,0 +1,399 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
// Claims the global dispatcher slot for Node's own fetch before npm `undici`
// can — see the guard's own comment.
import '../../util/nodeFetchDispatcherGuard.js';
import { Agent, fetch as undiciFetch } from 'undici';
import { PuterClient } from '../types.js';
/**
* The call that leaves the platform and runs an app's own code.
*
* Events workers are unreachable from the internet: deployed into their own
* dispatch namespace and reached only through the events dispatcher, which
* answers on its own hostname behind the internal secret.
*
* POST <events dispatcher>/invoke
* x-puter-internal-auth: <internal secret>
* x-puter-events-script: <script the handler set deploys as>
* x-puter-events-app: <app uid, for the deploy-on-miss callback>
* x-puter-events-key: <key derived for that script>
* { handler, event, ctx, token }
*
* The delivery token rides in the body, not a header, so nothing between here
* and the isolate can treat it as its own authorization.
*
* On a miss, a handler set here may deploy the script itself and retry once
* with `x-puter-events-deployed: 1`, skipping the dispatcher's own callback.
*/
/** The one route an events worker answers. */
export const EVENTS_INVOKE_PATH = '/__events/invoke';
/** The route the events dispatcher takes an invocation on. */
export const EVENTS_DISPATCH_PATH = '/invoke';
/**
* Header the dispatcher marks its own failures with. Anything carrying it never
* reached a handler, so its status says nothing about the delivery.
*/
export const EVENTS_DISPATCH_ERROR_HEADER = 'x-puter-events-dispatch';
/**
* Header a script's own answer carries, whatever its status — set by the
* runtime itself and re-stamped by the dispatcher, so a 4xx with neither this
* nor the dispatch-error header came from neither (an edge 404, a WAF) and is
* retriable rather than a handler's refusal.
*/
export const EVENTS_HANDLED_HEADER = 'x-puter-events-handled';
/** Machine-readable reason on the runtime's own failure answers. */
export const EVENTS_ERROR_HEADER = 'x-puter-events-error';
/**
* Set on the retry after this backend deploys a missing script itself, so the
* dispatcher skips its own deploy-on-miss callback and negative cache and goes
* straight to retrying the namespace.
*/
export const EVENTS_DEPLOYED_HEADER = 'x-puter-events-deployed';
/**
* Dispatch-error reasons this backend can resolve itself by deploying, rather
* than wait on the rehydrate callback finding a backend behind the dispatcher's
* own hostname. `deploy-timeout` is not one of them: it is only ever answered
* after a deploy that succeeded, so the script exists and another upload of it
* cannot make the namespace see it any sooner.
*/
const SELF_DEPLOYABLE_MISS_REASONS = new Set([
'missing',
'deploy-failed',
'deploy-error',
]);
/**
* Dispatch-error reasons where the deploy already happened and only propagation
* is still catching up — one prompt retry with the deployed header is worth it,
* without spending another upload on it.
*/
const ALREADY_DEPLOYED_MISS_REASONS = new Set(['deploy-timeout']);
/** What a self-deploy attempt did, and to which app/script it applies. */
export type WorkerMissHandler = (
appUid: string,
script: string,
) => Promise<'deployed' | string>;
/** How long a handler has to answer before the attempt is abandoned. */
export const EVENTS_INVOKE_TIMEOUT_MS = 30_000;
/**
* Connections held open to the dispatcher, so a busy fleet is not
* re-handshaking. The 30s invoke abort starts when a request is queued, not
* when it gets a connection, so a saturated pool shows up as timeouts.
*/
const INVOKE_CONNECTIONS = 256;
const INVOKE_KEEP_ALIVE_MS = 30_000;
/** One invocation, as the events service hands it over. */
export interface WorkerInvokeRequest {
/** Script the app's current handler set deploys as. */
script: string;
appUid: string;
handler: string;
/** The subscriber's app session token. */
token: string;
/** Value the script compares against its own invoke-key binding. */
key: string;
event: unknown;
ctx: unknown;
}
/** One invocation, as a transport delivers it. */
export interface EventsInvokeCall {
script: string;
appUid: string;
/** Value the script compares against its own invoke-key binding. */
key: string;
/** Serialized `{ handler, event, ctx, token }`. */
body: string;
timeoutMs: number;
/** Set on the retry after this backend deployed the script itself. */
deployed?: boolean;
}
/**
* Where an invocation is delivered.
*
* `status` is what the handler answered with. A `null` status means nothing
* ran: no transport, no script, or a dispatcher that refused us — all of which
* are the platform's fault and so retriable. `handled` is set only when the
* answer is provably the script's own (the handled header); a 4xx without it
* came from something between here and the script, not a handler's refusal.
*/
export interface EventsInvokeTransport {
send(call: EventsInvokeCall): Promise<{
status: number | null;
handled?: boolean;
error?: string;
/** Set only by the dispatcher transport, naming its own failure. */
dispatchReason?: string;
}>;
}
/**
* What the call did.
*
* - `settled` — the handler took the delivery (2xx).
* - `terminal` — it refused it (a marked 4xx). Retrying sends the same body to
* the same code, so it is not retried.
* - `retriable` — it could not answer: 5xx, 429, a timeout, a transport failure,
* an unmarked 4xx (nothing said it was a handler's answer), or no way to
* reach the worker at all.
*/
export type WorkerInvokeOutcome = 'settled' | 'terminal' | 'retriable';
export interface WorkerInvokeResult {
outcome: WorkerInvokeOutcome;
/** The status the handler answered with, or `null` when it never did. */
status: number | null;
/** Why it could not answer, when nothing did. */
error?: string;
}
const NO_TRANSPORT =
'no events dispatcher configured (url and internal secret)';
export class EventsWorkerInvokerClient extends PuterClient {
/** Set only where invocations do not leave the process — local development. */
#transport: EventsInvokeTransport | null = null;
#dispatcher: DispatcherInvokeTransport | null = null;
/** Lets this backend resolve a dispatcher miss instead of waiting on it. */
#missHandler: WorkerMissHandler | null = null;
/** Replaces the dispatcher with an in-process runtime. */
setTransport(transport: EventsInvokeTransport): void {
this.#transport = transport;
}
/**
* Deploys a script the dispatcher reports missing, in place of the
* rehydrate callback — for the case where that callback lands on a backend
* other than this one. Set once; `invoke()` calls it at most once per
* delivery and never loops.
*/
setMissHandler(handler: WorkerMissHandler): void {
this.#missHandler = handler;
}
override onServerShutdown(): void {
this.#dispatcher?.close();
this.#dispatcher = null;
}
async invoke(request: WorkerInvokeRequest): Promise<WorkerInvokeResult> {
const transport = this.#pickTransport();
if (!transport)
return { outcome: 'retriable', status: null, error: NO_TRANSPORT };
const call: EventsInvokeCall = {
script: request.script,
appUid: request.appUid,
key: request.key,
body: JSON.stringify({
handler: request.handler,
event: request.event,
ctx: request.ctx,
token: request.token,
}),
timeoutMs: this.#timeoutMs(),
};
let result = await transport.send(call);
if (result.status === null && result.dispatchReason) {
if (
this.#missHandler &&
SELF_DEPLOYABLE_MISS_REASONS.has(result.dispatchReason)
) {
const outcome = await this.#missHandler(
request.appUid,
request.script,
);
if (outcome !== 'deployed')
return {
outcome: 'retriable',
status: null,
error: `deploy: ${outcome}`,
};
result = await transport.send({ ...call, deployed: true });
} else if (
ALREADY_DEPLOYED_MISS_REASONS.has(result.dispatchReason)
) {
// No deploy to trigger — just give propagation the one retry
// it usually needs, without spending an upload on it.
result = await transport.send({ ...call, deployed: true });
}
}
const { status, handled, error } = result;
if (status === null)
return { outcome: 'retriable', status: null, error };
const outcome = outcomeForStatus(status, handled === true);
// An unmarked 4xx (429 aside — that one is always retriable) is not
// the handler's own answer; say so even when the transport did not,
// so the reason a delivery kept retrying shows up in the log rather
// than reading as an unexplained retry.
const unmarkedFourxx =
status !== 429 && status >= 400 && status < 500 && !handled;
const reportedError =
error ?? (unmarkedFourxx ? 'unmarked 4xx' : undefined);
return {
outcome,
status,
...(reportedError ? { error: reportedError } : {}),
};
}
#pickTransport(): EventsInvokeTransport | null {
if (this.#transport) return this.#transport;
const { dispatcherUrl, internalSecret } = this.config.events ?? {};
// Without the secret the dispatcher would refuse every call, which is
// a misconfiguration worth reporting once per attempt rather than a
// round trip.
if (!dispatcherUrl || !internalSecret) return null;
this.#dispatcher ??= new DispatcherInvokeTransport(
dispatcherUrl,
internalSecret,
);
return this.#dispatcher;
}
#timeoutMs(): number {
const configured = this.config.events?.invokeTimeoutMs;
return typeof configured === 'number' && configured > 0
? configured
: EVENTS_INVOKE_TIMEOUT_MS;
}
}
/** Reaches an events worker the only way production can: through the dispatcher. */
export class DispatcherInvokeTransport implements EventsInvokeTransport {
readonly #url: string;
readonly #secret: string;
readonly #fetchImpl: typeof undiciFetch;
#agent: Agent | null = null;
constructor(
dispatcherUrl: string,
secret: string,
options: { fetchImpl?: typeof undiciFetch } = {},
) {
this.#url = dispatcherUrl;
this.#secret = secret;
this.#fetchImpl = options.fetchImpl ?? undiciFetch;
}
close(): void {
void this.#agent?.close().catch(() => {});
this.#agent = null;
}
async send(call: EventsInvokeCall): Promise<{
status: number | null;
handled?: boolean;
error?: string;
dispatchReason?: string;
}> {
try {
// `new URL(path, base)` resolves an absolute path against the
// base's origin, dropping any path prefix `base` already carries.
const url = `${this.#url.replace(/\/$/, '')}${EVENTS_DISPATCH_PATH}`;
const response = await this.#fetchImpl(url, {
method: 'POST',
headers: {
'content-type': 'application/json',
'x-puter-internal-auth': this.#secret,
'x-puter-events-script': call.script,
'x-puter-events-app': call.appUid,
'x-puter-events-key': call.key,
...(call.deployed ? { [EVENTS_DEPLOYED_HEADER]: '1' } : {}),
},
body: call.body,
signal: AbortSignal.timeout(call.timeoutMs),
dispatcher: this.#dispatcherFor(),
});
// The body is not read: a handler answers with a status, and
// whatever else it writes is its own business.
void response.body?.cancel().catch(() => {});
// The dispatcher never reached a handler, so its status is not the
// delivery's answer.
const dispatchError = response.headers.get(
EVENTS_DISPATCH_ERROR_HEADER,
);
if (dispatchError !== null)
return {
status: null,
error: `dispatcher: ${dispatchError} (${response.status})`,
dispatchReason: dispatchError,
};
const handled = response.headers.get(EVENTS_HANDLED_HEADER) === '1';
const errorHeader = response.headers.get(EVENTS_ERROR_HEADER);
return {
status: response.status,
handled,
...(errorHeader ? { error: errorHeader } : {}),
};
} catch (err) {
return {
status: null,
error: err instanceof Error ? err.message : String(err),
};
}
}
/** One pool per process, built on first use and closed with the server. */
#dispatcherFor(): Agent {
this.#agent ??= new Agent({
connections: INVOKE_CONNECTIONS,
keepAliveTimeout: INVOKE_KEEP_ALIVE_MS,
keepAliveMaxTimeout: INVOKE_KEEP_ALIVE_MS,
});
return this.#agent;
}
}
/**
* 2xx took it regardless. 429 is always "not now". Any other 4xx is a refusal
* only when `handled` says the answer is provably the script's own — an
* unmarked 4xx came from something between here and the handler.
*/
export const outcomeForStatus = (
status: number,
handled: boolean,
): WorkerInvokeOutcome => {
if (status >= 200 && status < 300) return 'settled';
if (status === 429) return 'retriable';
if (status >= 400 && status < 500)
return handled ? 'terminal' : 'retriable';
return 'retriable';
};
+2
View File
@@ -21,6 +21,7 @@ import { AlarmClient } from './alarm/AlarmClient';
import { DatabaseClientFactory } from './database';
import { EmailClient } from './email/EmailClient';
import { EventClient } from './event/EventClient';
import { EventsWorkerInvokerClient } from './events/EventsWorkerInvokerClient';
import { DDBClient } from './dynamodb/DDBClient';
import { RedisClient } from './redis/RedisClient';
import { S3Client } from './s3/S3Client';
@@ -32,6 +33,7 @@ export const puterClients = {
db: DatabaseClientFactory,
email: EmailClient,
event: EventClient,
eventsWorkerInvoker: EventsWorkerInvokerClient,
dynamo: DDBClient,
redis: RedisClient,
s3: S3Client,
@@ -21,6 +21,7 @@ import { isAccessTokenActor, isAppActor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { driversContainers } from '../../exports.js';
import {
APP_ICON_SIZES,
ICON_DATA_URL_MIME_ALLOWLIST,
isTrustedIconHost,
} from '../../util/appIcon.js';
@@ -414,8 +415,6 @@ export class AppController extends PuterController {
//
// ⚠ FLAG: Missing sharp-based resize pipeline; serves the original.
const ICON_SIZES = [16, 32, 64, 128, 256, 512];
// Neutering headers for any response that echoes an icon byte
// stream on the main origin. `image/svg+xml` is in our MIME
// allow-list — it's a legitimate image format, and our own
@@ -457,7 +456,7 @@ export class AppController extends PuterController {
res.status(400).send('Missing app_uid');
return;
}
if (!ICON_SIZES.includes(size)) {
if (!APP_ICON_SIZES.includes(size)) {
res.status(400).send('Invalid size');
return;
}
File diff suppressed because it is too large. Load diff
+415 -129
View File
@@ -20,6 +20,7 @@
import bcrypt from 'bcrypt';
import type { Request, RequestHandler, Response } from 'express';
import crypto from 'node:crypto';
import { posix as pathPosix } from 'node:path';
import { v4 as uuidv4, validate as validateUuid } from 'uuid';
import validator from 'validator';
import { Controller, Get, Post } from '../../core/http/decorators.js';
@@ -54,37 +55,55 @@ import {
} from '../../services/auth/OTPUtil.js';
import type { UserRow } from '../../stores/user/UserStore.js';
import { isOwnedEmailConflict } from '../../stores/user/UserStore.js';
import type { CardFallbackDeps } from '../../util/cardFallback.js';
import {
CARD_FALLBACK_OPEN_TTL_SECONDS,
SEND_PHONE_RATE_LIMIT,
SEND_PHONE_RATE_WINDOW_MS,
cardFallbackAfterAttempts,
cardFallbackFlagKey,
isCardFallbackEligible,
isCardFallbackEnabled,
phoneAttemptsKey,
} from '../../util/cardFallback.js';
import { sessionCookieFlags } from '../../util/cookieFlags.js';
import { cleanEmail, isBlockedEmail } from '../../util/email.js';
import { generate_identifier } from '../../util/identifier.js';
import { parsePhone } from '../../util/phone.js';
import { isTemporaryPasswordExpired } from '../../util/temporaryPassword.js';
import { getTaskbarItems } from '../../util/taskbarItems.js';
import {
generateDefaultFsentries,
promoteToVerifiedGroup,
} from '../../util/userProvisioning.js';
import {
assertBoundedManageGrant,
isKvSharePermission,
} from '../../services/events/kvShares.js';
import {
APP_DATA_PERMISSION_PREFIX,
appDataSharingAllowed,
parseAppDataPermission,
} from '../../services/permission/appDataScopes.js';
import {
assertCreatablePath,
fsCreateKindFor,
MAX_CREATED_ENTRIES_PER_GRANT,
parseCreateFlag,
parseFsPathPermission,
type FsCreateKind,
} from '../../services/permission/fsPathPermission.js';
import { normalizeAbsolutePath } from '../../services/fs/resolveNode.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import { PuterController } from '../types.js';
const USERNAME_REGEX = /^\w{1,}$/;
const USERNAME_MAX_LENGTH = 45;
export const USERNAME_REGEX = /^\w{1,}$/;
export const USERNAME_MAX_LENGTH = 45;
const FINGERPRINT_MAX_LENGTH = 128;
// One consent prompt covers a handful of scopes at most. The cap keeps a
// crafted request from turning a single grant call into a bulk write.
const MAX_PERMISSIONS_PER_REQUEST = 16;
const DISPATCH_ID_MAX_LENGTH = 128;
// Default SMS send attempts before the card fallback opens.
const DEFAULT_CARD_FALLBACK_ATTEMPTS = 2;
// /send-confirm-phone route rate limit. Also caps the fallback's
// `after_attempts`: requests past the route limit are rejected in middleware
// and never reach the attempt counter, so a higher threshold could never be
// crossed.
const SEND_PHONE_RATE_LIMIT = 10;
const SEND_PHONE_RATE_WINDOW_MS = 60 * 60_000;
// -- Post-login route limits -----------------------------------------
//
@@ -173,9 +192,6 @@ const SESSION_LIMIT = {
window: 60_000,
key: 'user',
} as const;
// Once the threshold is crossed the fallback stays open this long, so the
// user can finish the card flow without racing the attempt counter's expiry.
const CARD_FALLBACK_OPEN_TTL_SECONDS = 24 * 60 * 60;
// How long a failed-SMS-send record stays readable by its error_id — long
// enough to cover the typical support round-trip.
const SMS_SEND_ERROR_TTL_SECONDS = 7 * 24 * 60 * 60;
@@ -194,7 +210,7 @@ const cardAlreadyVerified = (user: {
requires_card_verification?: boolean;
}): boolean =>
Boolean(user.card_fingerprint) && !user.requires_card_verification;
const RESERVED_USERNAMES = new Set([
export const RESERVED_USERNAMES = new Set([
'admin',
'administrator',
'root',
@@ -239,9 +255,12 @@ export class AuthController extends PuterController {
],
})
async loginWait(req: Request, res: Response) {
const { session } = req.body;
// Destructuring an absent body is a TypeError, not the 400 a request
// with no session id deserves.
const session = (req.body as { session?: unknown } | undefined)
?.session;
// validate uuid to prevent ultra long key or listening on pubsub.login.*
if (!session || !validateUuid(session)) {
if (typeof session !== 'string' || !validateUuid(session)) {
throw new HttpError(400, 'session is required.', {
legacyCode: 'bad_request',
});
@@ -343,7 +362,7 @@ export class AuthController extends PuterController {
],
})
async loginSet(req: Request, res: Response) {
const { session, auth_token } = req.body;
const { session, auth_token } = req.body ?? {};
if (!session || !auth_token || !validateUuid(session)) {
throw new HttpError(400, 'session and auth_token are required.', {
legacyCode: 'bad_request',
@@ -382,7 +401,7 @@ export class AuthController extends PuterController {
],
})
async handleLogin(req: Request, res: Response): Promise<void> {
const { username, email, password } = req.body;
const { username, email, password } = req.body ?? {};
if (!username && !email) {
throw new HttpError(400, 'Username or email is required.', {
@@ -451,6 +470,15 @@ export class AuthController extends PuterController {
legacyCode: 'password_mismatch',
});
}
// An administrator-issued temporary password that was never used dies
// rather than becoming a standing credential the team holds.
if (isTemporaryPasswordExpired(user)) {
throw new HttpError(
401,
'This temporary password has expired. Ask your team administrator for a new one.',
{ legacyCode: 'temporary_password_expired' },
);
}
const reauthAuthId = this.#extractAuthIdFromReauthToken(
req.body.reauth_token,
@@ -499,7 +527,7 @@ export class AuthController extends PuterController {
],
})
async handleLoginOtp(req: Request, res: Response): Promise<void> {
const { token, code } = req.body;
const { token, code } = req.body ?? {};
if (!token)
throw new HttpError(400, 'token is required.', {
legacyCode: 'bad_request',
@@ -565,7 +593,7 @@ export class AuthController extends PuterController {
],
})
async handleLoginRecoveryCode(req: Request, res: Response): Promise<void> {
const { token, code } = req.body;
const { token, code } = req.body ?? {};
if (!token)
throw new HttpError(400, 'token is required.', {
legacyCode: 'bad_request',
@@ -841,6 +869,10 @@ export class AuthController extends PuterController {
// guarantee — everything between here and the insert widens the window,
// so the check runs again against the primary immediately before the
// write, and the unique index catches whatever still slips through.
const clientIp: string | null =
req.ip || req.socket?.remoteAddress || null;
const proxyIpChain = req.headers['x-forwarded-for'];
let pseudo_user = is_temp
? null
: await this.#resolveSignupEmailClaim(body.email);
@@ -855,12 +887,9 @@ export class AuthController extends PuterController {
const validateEvent = {
req,
data: body,
ip: ((req.headers?.['x-forwarded-for'] as string | undefined) ||
(req as unknown as { connection?: { remoteAddress?: string } })
.connection?.remoteAddress ||
req.ip ||
req.socket?.remoteAddress ||
null) as string | null,
// `req.ip` honors `trust proxy`; reading x-forwarded-for directly
// would let a client pick its own per-IP abuse bucket.
ip: clientIp,
email: body.email,
// The same canonical form `email.validate` was given, so a check
// in the abuse harness can look up the verdict that hook cached
@@ -1035,9 +1064,6 @@ export class AuthController extends PuterController {
});
} else {
// -- New user ----------------------------------------
const clientIp = req.ip || req.socket?.remoteAddress || null;
const proxyIpChain = req.headers['x-forwarded-for'];
try {
user = await this.stores.user.create({
username: body.username,
@@ -1058,13 +1084,16 @@ export class AuthController extends PuterController {
fingerprint,
},
signup_ip: clientIp,
signup_ip_forwarded: proxyIpChain,
// The abuse harness and the admin IP lookup both key on
// this column, so it holds the trusted client address; the
// raw forwarded chain stays in `audit_metadata.ip_fwd`.
signup_ip_forwarded: clientIp,
signup_user_agent: req.headers?.['user-agent'] ?? null,
signup_origin:
(req.headers?.origin as string | null) ?? null,
signup_server: (this.config as { serverId?: string })
.serverId,
referrer: req.body.referrer ?? null,
referrer: body.referrer ?? null,
last_activity_ts: signupSqlTs,
reputation: validateEvent.reputation,
// Phone collected later in the verification dialog (null now).
@@ -1120,8 +1149,9 @@ export class AuthController extends PuterController {
) {
const sendCode = body.send_confirmation_code ?? true;
try {
let sent;
if (sendCode) {
await this.clients.email.send(
sent = await this.clients.email.send(
user!.email!,
'email_verification_code',
{
@@ -1130,14 +1160,18 @@ export class AuthController extends PuterController {
);
} else {
const link = `${this.config.origin ?? ''}/confirm-email-by-token?token=${email_confirm_token}&user_uuid=${user!.uuid}`;
await this.clients.email.send(
sent = await this.clients.email.send(
user!.email!,
'email_verification_link',
{ link },
);
}
// `null` = dropped for want of a transport; silent otherwise.
if (sent === null) {
this.#confirmationEmailFailed('signup', user!, null);
}
} catch (e) {
console.warn('[signup] email send failed:', e);
this.#confirmationEmailFailed('signup', user!, e);
}
}
@@ -1157,17 +1191,10 @@ export class AuthController extends PuterController {
// a pseudo-user claim ends up with credentials, so it
// reports false here. Same signal completeLogin uses.
is_temp: user!.password === null && user!.email === null,
ip:
(req?.headers?.['x-forwarded-for'] as
string | undefined) ||
(
req as unknown as {
connection?: { remoteAddress?: string };
}
)?.connection?.remoteAddress ||
req?.ip ||
req?.socket?.remoteAddress ||
null,
// Same derivation as the validate event above — the two
// have to agree or per-IP counters are written under one
// key and read under another.
ip: clientIp,
} as never,
{},
);
@@ -1269,13 +1296,16 @@ export class AuthController extends PuterController {
if (this.clients.email) {
try {
await this.clients.email.send(
const sent = await this.clients.email.send(
user.email,
'email_verification_code',
{ code },
);
if (sent === null) {
this.#confirmationEmailFailed('resend', user, null);
}
} catch (e) {
console.warn('[send-confirm-email] send failed:', e);
this.#confirmationEmailFailed('resend', user, e);
}
}
res.json({});
@@ -1386,6 +1416,64 @@ export class AuthController extends PuterController {
res.json({ email_confirmed: true, original_client_socket_id });
}
/**
* Alarm on a confirmation email that did not reach the recipient. A
* `requires_email_confirmation` account is refused by
* `requireVerifiedAccount` everywhere, so a lost code leaves an account
* that cannot be used. `cause === null` is the silent case: `sendRaw` drops
* the message rather than throwing when no transport is configured.
*
* `sole_gate` reports that no phone/card gate is outstanding either, so
* this user is stuck on the email alone. `dedup` because one broken mail
* path fails once per signup and is still one thing to fix.
*/
#confirmationEmailFailed(
stage: 'signup' | 'resend',
user: {
uuid?: string | null;
username?: string | null;
email?: string | null;
requires_phone_verification?: unknown;
requires_card_verification?: unknown;
},
cause: unknown,
): void {
const email = user.email ?? null;
const detail =
cause instanceof Error
? cause.message
: cause === null
? 'no transport configured (message dropped)'
: String(cause);
console.warn(
`[${stage === 'signup' ? 'signup' : 'send-confirm-email'}] ` +
`confirmation email not delivered: ${detail}`,
);
// Best-effort: failing to alarm must not fail the signup.
try {
this.clients.alarm?.create(
`auth:confirmation-email-send-failed:${stage}`,
'Confirmation email could not be sent — gated accounts cannot be used until it arrives',
{
stage,
user_uid: user.uuid ?? null,
username: user.username ?? null,
email,
email_domain: email?.split('@')[1] ?? null,
sole_gate:
!user.requires_phone_verification &&
!user.requires_card_verification,
detail,
...(cause instanceof Error ? { error: cause } : {}),
},
'warning',
{ dedup: true },
);
} catch (e) {
console.warn(`[${stage}] confirmation-email alarm failed:`, e);
}
}
// -- Phone verification (SMS via Prelude) ------------------------
/**
@@ -1449,9 +1537,10 @@ export class AuthController extends PuterController {
// -- SMS-to-card fallback -----------------------------------------
//
// Once a user has made enough SMS send attempts in the rate-limit window
// without getting through, they can verify a card instead to clear the
// phone gate. Off unless config enables it.
// Once a user has used up their SMS send attempts for the window without
// getting through, they can verify a card instead to clear the phone gate.
// On wherever both gates work unless config opts out. The rule itself lives
// in ../../util/cardFallback.ts, because /whoami answers the same question.
//
// Two KV keys: a short-lived counter tied to the send rate-limit window
// triggers the fallback, and a longer-lived "open" flag holds eligibility
@@ -1460,30 +1549,11 @@ export class AuthController extends PuterController {
// user is mid-way through the card flow. Every KV failure fails closed
// (fallback unavailable), never open.
private cardFallbackConfig(): { enabled: boolean; afterAttempts: number } {
const cfg = this.config.phone_verification_card_fallback;
const afterAttempts = Math.min(
typeof cfg?.after_attempts === 'number' && cfg.after_attempts > 0
? cfg.after_attempts
: DEFAULT_CARD_FALLBACK_ATTEMPTS,
SEND_PHONE_RATE_LIMIT,
);
return { enabled: Boolean(cfg?.enabled), afterAttempts };
}
private phoneAttemptsKey(userId: number): string {
return `phone-verify-attempts:${userId}`;
}
private cardFallbackFlagKey(userId: number): string {
return `card-fallback-open:${userId}`;
}
// TTL ties the counter to the send rate-limit window, so it resets with it.
private async bumpPhoneAttempts(userId: number): Promise<number> {
try {
const { res } = await this.stores.kv.incr({
key: this.phoneAttemptsKey(userId),
key: phoneAttemptsKey(userId),
pathAndAmountMap: { attempts: 1 },
expireAt:
Math.floor(Date.now() / 1000) +
@@ -1507,16 +1577,15 @@ export class AuthController extends PuterController {
requires_phone_verification?: boolean | number | null;
}): Promise<boolean> {
const attempts = await this.bumpPhoneAttempts(user.id);
const { enabled, afterAttempts } = this.cardFallbackConfig();
const open =
enabled &&
Boolean(user.requires_phone_verification) &&
attempts >= afterAttempts;
attempts >= cardFallbackAfterAttempts(this.config) &&
(await isCardFallbackEnabled(this.config, this.cardFallbackDeps()));
if (open) {
try {
// Plain set, so each eligible attempt refreshes the window.
await this.stores.kv.set({
key: this.cardFallbackFlagKey(user.id),
key: cardFallbackFlagKey(user.id),
value: true,
expireAt:
Math.floor(Date.now() / 1000) +
@@ -1537,17 +1606,33 @@ export class AuthController extends PuterController {
id: number;
requires_phone_verification?: boolean | number | null;
}): Promise<boolean> {
const { enabled } = this.cardFallbackConfig();
if (!enabled || !user.requires_phone_verification) return false;
try {
const { res } = await this.stores.kv.get({
key: this.cardFallbackFlagKey(user.id),
});
return res === true;
} catch (e) {
console.warn('[card-verification] fallback flag read failed:', e);
return false;
}
return isCardFallbackEligible(
this.config,
user,
async (key) => (await this.stores.kv.get({ key })).res,
this.cardFallbackDeps(),
);
}
/**
* The two facts the fallback's default rests on: SMS can only work with a
* provider configured, and the card gate belongs to an extension, so the
* only honest way to ask whether it is on is to ask that extension. Nothing
* is listening on a stock build, which reads as "no card gate".
*/
private cardFallbackDeps(): CardFallbackDeps {
return {
smsConfigured: () => Boolean(this.clients.prelude?.isConfigured()),
probeCardVerification: async () => {
const statusEvent = { enabled: null as boolean | null };
await this.clients.event?.emitAndWait(
'puter.card-verification.status',
statusEvent,
{},
);
return statusEvent.enabled;
},
};
}
@Post('/send-confirm-phone', {
@@ -2356,7 +2441,8 @@ export class AuthController extends PuterController {
let result;
try {
result = await this.clients.db.write(
'UPDATE `user` SET `password` = ?, `pass_recovery_token` = NULL, `change_email_confirm_token` = NULL WHERE `id` = ? AND `pass_recovery_token` = ?',
'UPDATE `user` SET `password` = ?, `pass_recovery_token` = NULL, `change_email_confirm_token` = NULL, ' +
'`requires_password_change` = 0, `temp_password_expires_at` = NULL WHERE `id` = ? AND `pass_recovery_token` = ?',
[password_hash, user.id, decoded.token],
);
} catch (e) {
@@ -2384,6 +2470,13 @@ export class AuthController extends PuterController {
});
}
await this.stores.user.invalidateById(user.id);
// Best effort: the password is already committed, and an audit write
// must not skip the eviction below.
await this.services.team
.recordPasswordSelfChange(user.id as number)
.catch((err: unknown) => {
console.warn('[team] password self-change audit failed:', err);
});
// A password reset is the "I think someone else has access" flow —
// evict every interactive session so a hijacked one doesn't survive.
@@ -2421,11 +2514,20 @@ export class AuthController extends PuterController {
const user = req.userProtected!.user;
const password_hash = await bcrypt.hash(new_pass, 8);
// Clearing the forced-change gate is what lets a team seat back
// in; nothing else writes these two columns to their cleared state.
await this.stores.user.update(user.id, {
password: password_hash,
pass_recovery_token: null,
change_email_confirm_token: null,
requires_password_change: 0,
temp_password_expires_at: null,
});
await this.services.team
.recordPasswordSelfChange(user.id)
.catch((err: unknown) => {
console.warn('[team] password self-change audit failed:', err);
});
// Sign out every other web session (cascading to their derived
// rows); only the session that changed the password survives.
@@ -3146,14 +3248,108 @@ export class AuthController extends PuterController {
}
}
/**
* Whether a granted `fs:` path permission needs its target created before
* the grant can resolve, and if so where and what kind. Null when there is
* nothing to create: `create` was not requested, the entry does not name an
* `fs:` path, it is a `manage:` grant, or the path already exists.
*
* Side-effect-free, like `#prepareAppDataGrant` above — a pre-grant hook
* run in the same validation loop, so a bad entry elsewhere in the list
* still leaves nothing created.
*/
async #planFsPathCreate(
actor: Actor,
permission: string,
create: boolean | FsCreateKind,
): Promise<{ path: string; kind: FsCreateKind } | null> {
if (!create) return null;
const parsed = parseFsPathPermission(permission);
if (!parsed) return null;
// A `manage:` grant delegates re-sharing, not access itself — creating
// a path in order to delegate management of it is not a coherent
// request.
if (parsed.hasManage) return null;
const path = normalizeAbsolutePath(parsed.path);
const existing = await this.stores.fsEntry.getEntryByPath(path);
if (existing) {
if (
(create === 'dir' && !existing.isDir) ||
(create === 'file' && existing.isDir)
) {
throw new HttpError(409, `An entry already exists at ${path}`, {
legacyCode: 'conflict',
});
}
// Already there: the grant resolves the existing entry, nothing to
// create. A `true` heuristic guess never conflicts with reality.
return null;
}
// Checked only once the path is confirmed missing, so an existing
// shared path outside the caller's home is not a new rejection.
assertCreatablePath(path, actor.user!.username!);
const kind =
create === true
? fsCreateKindFor(pathPosix.basename(path))
: create;
return { path, kind };
}
/** Creates the target `#planFsPathCreate` planned, as the user actor. */
async #createGrantTarget(
actor: Actor,
plan: { path: string; kind: FsCreateKind },
): Promise<FSEntry> {
const userId = actor.user!.id;
// Missing intermediate directories are created along with the leaf.
// They are deliberately not rolled back if a later phase fails, so a
// failed grant can leave empty directories inside the user's home.
return plan.kind === 'dir'
? await this.services.fs.mkdir(userId, {
path: plan.path,
createMissingParents: true,
})
: await this.services.fs.touch(userId, {
path: plan.path,
createMissingParents: true,
});
}
/**
* Undoes what `#createGrantTarget` made for this request, newest first,
* when a later phase fails. Auto-created intermediate directories are not
* rolled back — they are left as empty directories inside the user's own
* home.
*/
async #rollbackCreatedGrantTargets(
actor: Actor,
created: FSEntry[],
): Promise<void> {
const userId = actor.user!.id;
for (const entry of [...created].reverse()) {
try {
await this.services.fs.remove(userId, {
entry,
recursive: false,
});
} catch {
// Best-effort: the error that triggered the rollback is what
// the caller sees, not a failure to undo it.
}
}
}
@Post('/auth/grant-user-app', {
subdomain: 'api',
requireUserActor: true,
rateLimit: GRANT_LIMIT,
})
async handleGrantUserApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body;
const { origin, permission, permissions, extra, meta } = req.body;
let { app_uid } = req.body ?? {};
const { origin, permission, permissions, extra, meta } = req.body ?? {};
this.#validateAppPermissionParams({
app_uid,
origin,
@@ -3170,26 +3366,73 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
}
// 400 on a bad value, before anything else runs.
const create = parseCreateFlag(req.body?.create);
// Validate every entry before writing any, so a bad one in the list
// cannot leave a partially-granted set behind: the dialog reads a 4xx as
// "nothing was written" and skips its withdrawal, so a partial commit
// leaves live access the user was told they refused. The rewrite running
// twice is cheaper than splitting the grant into prepare/commit.
// Validate every entry before writing or creating anything, so a bad
// one in the list cannot leave a partially-granted set behind: the
// dialog reads a 4xx as "nothing was written" and skips its
// withdrawal, so a partial commit leaves live access the user was
// told they refused.
const plans: Array<{
entry: string;
path: string;
kind: FsCreateKind;
}> = [];
for (const entry of list) {
await this.services.permission.assertUserAppPermissionWritable(
entry,
);
// A delegation over a whole key-value namespace has no bounded
// description, so it is refused where it is asked for rather than
// prompted for and then refused at use.
assertBoundedManageGrant(entry);
await this.#prepareAppDataGrant(req.actor!, entry);
}
for (const entry of list) {
await this.services.permission.grantUserAppPermission(
const plan = await this.#planFsPathCreate(
req.actor!,
app_uid,
entry,
extra ?? undefined,
meta ?? undefined,
create,
);
if (plan) {
// Checked on every planned entry, not just successful
// creations, so the cap bounds attempts in this request.
if (plans.length >= MAX_CREATED_ENTRIES_PER_GRANT) {
throw new HttpError(400, 'Too many paths to create', {
legacyCode: 'bad_request',
});
}
plans.push({ entry, ...plan });
} else {
// Nothing to create for this entry: the same pre-flight the
// rewrite always ran, measuring the width of the row it
// resolves to. Once a path is created, `grantUserAppPermission`
// re-runs the identical rewrite and check itself.
await this.services.permission.assertUserAppPermissionWritable(
entry,
);
}
}
// Two entries in the same batch can name the same path (different
// modes on one `fs:` target); create it once, not once per entry.
const uniquePlans = [
...new Map(plans.map((p) => [p.path, p])).values(),
];
const created: FSEntry[] = [];
try {
for (const plan of uniquePlans) {
created.push(await this.#createGrantTarget(req.actor!, plan));
}
for (const entry of list) {
await this.services.permission.grantUserAppPermission(
req.actor!,
app_uid,
entry,
extra ?? undefined,
meta ?? undefined,
);
}
} catch (err) {
await this.#rollbackCreatedGrantTargets(req.actor!, created);
throw err;
}
res.json({});
}
@@ -3197,9 +3440,8 @@ export class AuthController extends PuterController {
// -- Permission revokes ------------------------------------------
/**
* @deprecated Use `puter.fs.unshare()`, which withdraws the share row and
* the grant together. Kept for direct HTTP callers: the grant side is
* retired, but access it left behind has to stay withdrawable.
* @deprecated Use `puter.fs.unshare()`. Kept for direct HTTP callers:
* access left behind by an older client has to stay withdrawable.
*/
@Post('/auth/revoke-user-user', {
subdomain: 'api',
@@ -3207,7 +3449,7 @@ export class AuthController extends PuterController {
rateLimit: GRANT_LIMIT,
})
async handleRevokeUserUser(req: Request, res: Response): Promise<void> {
const { target_username, permission, meta } = req.body;
const { target_username, permission, meta } = req.body ?? {};
if (!target_username || !permission) {
throw new HttpError(
400,
@@ -3221,6 +3463,13 @@ export class AuthController extends PuterController {
permission,
meta,
);
// The share index is what listings read, so it has to go with the
// grant — otherwise the item keeps reporting itself as shared.
await this.services.share.onGrantRevoked(
req.actor!,
target_username,
permission,
);
res.json({});
}
@@ -3230,8 +3479,8 @@ export class AuthController extends PuterController {
rateLimit: GRANT_LIMIT,
})
async handleRevokeUserApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body;
const { origin, permission, permissions, meta } = req.body;
let { app_uid } = req.body ?? {};
const { origin, permission, permissions, meta } = req.body ?? {};
this.#validateAppPermissionParams({
app_uid,
origin,
@@ -3276,7 +3525,7 @@ export class AuthController extends PuterController {
rateLimit: AUTH_CHECK_LIMIT,
})
async handleCheckPermissions(req: Request, res: Response): Promise<void> {
const { permissions } = req.body;
const { permissions } = req.body ?? {};
if (!Array.isArray(permissions)) {
throw new HttpError(400, 'Missing or invalid `permissions` array', {
legacyCode: 'bad_request',
@@ -3317,7 +3566,7 @@ export class AuthController extends PuterController {
// mandatory: an access token must not be able to revoke its own
// issuing web session.
async handleRevokeSession(req: Request, res: Response): Promise<void> {
const { uuid } = req.body;
const { uuid } = req.body ?? {};
if (!uuid || typeof uuid !== 'string') {
throw new HttpError(400, 'Missing or invalid `uuid`', {
legacyCode: 'bad_request',
@@ -3394,8 +3643,8 @@ export class AuthController extends PuterController {
rateLimit: GRANT_LIMIT,
})
async handleGrantDevApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body;
const { origin, permission, extra, meta } = req.body;
let { app_uid } = req.body ?? {};
const { origin, permission, extra, meta } = req.body ?? {};
if (origin && !app_uid) {
// Registered apps only, for the same reason the user-app handlers
// insist on it: a synthesised `app-<uuidv5(origin)>` is resolved
@@ -3428,8 +3677,8 @@ export class AuthController extends PuterController {
rateLimit: GRANT_LIMIT,
})
async handleRevokeDevApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body;
const { origin, permission, meta } = req.body;
let { app_uid } = req.body ?? {};
const { origin, permission, meta } = req.body ?? {};
if (origin && !app_uid) {
// Registered apps only — see handleGrantDevApp.
app_uid = await this.#registeredAppUidFromOrigin(origin);
@@ -3513,14 +3762,21 @@ export class AuthController extends PuterController {
? JSON.parse(r.extra)
: (r.extra ?? {}),
})),
user_to_myself: (userPermsIn as Row[]).map((r) => ({
user: r.username,
permission: r.permission,
extra:
typeof r.extra === 'string'
? JSON.parse(r.extra)
: (r.extra ?? {}),
})),
// A key-value share grant names its owner's uuid and the exact
// key prefix it covers. The handle is the only name its holder is
// meant to have for that region, so the grant behind it is not
// part of what they are shown. The owner still sees their own side
// above, and in `GET /events/kv-handles`.
user_to_myself: (userPermsIn as Row[])
.filter((r) => !isKvSharePermission(r.permission))
.map((r) => ({
user: r.username,
permission: r.permission,
extra:
typeof r.extra === 'string'
? JSON.parse(r.extra)
: (r.extra ?? {}),
})),
});
}
@@ -3551,8 +3807,8 @@ export class AuthController extends PuterController {
rateLimit: { ...AUTH_CHECK_LIMIT, scope: 'app-token', limit: 120 },
})
async handleGetUserAppToken(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body;
const { origin } = req.body;
let { app_uid } = req.body ?? {};
const { origin } = req.body ?? {};
const resolvedFromOrigin = !app_uid && !!origin;
if (!app_uid && origin) {
app_uid = await this.services.auth.appUidFromOrigin(origin);
@@ -3673,8 +3929,8 @@ export class AuthController extends PuterController {
rateLimit: AUTH_CHECK_LIMIT,
})
async handleCheckApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body;
const { origin } = req.body;
let { app_uid } = req.body ?? {};
const { origin } = req.body ?? {};
if (!app_uid && origin) {
app_uid = await this.services.auth.appUidFromOrigin(origin);
}
@@ -3713,7 +3969,7 @@ export class AuthController extends PuterController {
rateLimit: CREDENTIAL_MINT_LIMIT,
})
async handleCreateAccessToken(req: Request, res: Response): Promise<void> {
const { permissions, expiresIn, label } = req.body;
const { permissions, expiresIn, label } = req.body ?? {};
if (!Array.isArray(permissions) || permissions.length === 0) {
throw new HttpError(400, 'Missing or empty `permissions` array', {
legacyCode: 'bad_request',
@@ -3732,6 +3988,24 @@ export class AuthController extends PuterController {
normalizedLabel = label.trim().slice(0, 64) || null;
}
// Whole seconds or a duration string ('30d'). A wrong type, a
// fraction, or an unparseable unit all reach the signer, which throws
// after the session row is already inserted.
if (expiresIn !== undefined && expiresIn !== null) {
const usable =
typeof expiresIn === 'number'
? Number.isInteger(expiresIn) && expiresIn > 0
: typeof expiresIn === 'string' &&
/^\d+\s*[smhdwy]?$/.test(expiresIn.trim());
if (!usable) {
throw new HttpError(
400,
'`expiresIn` must be a whole number of seconds or a duration string like `30d`',
{ legacyCode: 'bad_request' },
);
}
}
// Normalize specs: string → [string], [string] → [string, {}], [string, extra] → as-is
const normalized = permissions.map((spec) => {
if (typeof spec === 'string') return [spec];
@@ -3759,7 +4033,7 @@ export class AuthController extends PuterController {
// mandatory: a leaked access token must not be able to silently
// revoke its own siblings.
async handleRevokeAccessToken(req: Request, res: Response): Promise<void> {
let { tokenOrUuid } = req.body;
let { tokenOrUuid } = req.body ?? {};
if (!tokenOrUuid || typeof tokenOrUuid !== 'string') {
throw new HttpError(400, 'Missing `tokenOrUuid`', {
legacyCode: 'bad_request',
@@ -4165,6 +4439,9 @@ export class AuthController extends PuterController {
'/user-protected/change-password',
{
requireUserActor: true,
// The forced-change gate refuses everything else, so this is
// the one route an account owing a password change may reach.
allowUnconfirmed: true,
rateLimit: {
scope: 'passwd',
limit: 10,
@@ -4379,7 +4656,15 @@ export class AuthController extends PuterController {
): Promise<UserRow | null> {
const existing = await this.stores.user.findEmailOwner(email, opts);
if (!existing) return null;
if (existing.email_confirmed || existing.password !== null) {
// A provisioned account looks exactly like a claimable placeholder --
// no password, unconfirmed -- but claiming it hands a stranger that
// team's membership.
const orgSeat = await this.stores.team.getOrgSeat(existing.id);
if (
existing.email_confirmed ||
existing.password !== null ||
orgSeat !== null
) {
throw new HttpError(
400,
'This email already exists in our database. Please use another one.',
@@ -4562,6 +4847,7 @@ export class AuthController extends PuterController {
services: this.services,
apiBaseUrl: (this.config as { api_base_url?: string })
.api_base_url,
config: this.config,
} as never,
);
} catch (e) {
@@ -20,6 +20,7 @@
import type { Request, Response } from 'express';
import { Controller, Post } from '../../core/http/decorators.js';
import type { BroadcastService } from '../../services/broadcast/BroadcastService.js';
import type { ForwardBatch } from '../../services/events/forwardQueue.js';
import { PuterController } from '../types.js';
/**
@@ -55,18 +56,11 @@ export class BroadcastController extends PuterController {
return;
}
const headerOnce = (name: string): string | undefined => {
const value = req.headers[name];
if (Array.isArray(value)) return value[0];
return value;
};
const result = await broadcast.verifyAndEmit(req.rawBody, req.body, {
peerId: headerOnce('x-broadcast-peer-id'),
timestamp: headerOnce('x-broadcast-timestamp'),
nonce: headerOnce('x-broadcast-nonce'),
signature: headerOnce('x-broadcast-signature'),
});
const result = await broadcast.verifyAndEmit(
req.rawBody,
req.body,
broadcastHeaders(req),
);
if (result.ok) {
res.status(200).json({ ok: true, ...(result.info ?? {}) });
@@ -76,4 +70,57 @@ export class BroadcastController extends PuterController {
error: { message: result.message ?? 'Bad request' },
});
}
/**
* Event deliveries one region addressed at this one, on the same signed
* channel as the webhook above but carrying socket traffic rather than bus
* events. The answer names the pairs this region holds no socket for, which
* is what lets the sender correct its presence row.
*/
@Post('/events', { subdomain: '*' })
async events(req: Request, res: Response): Promise<void> {
const broadcast = this.services.broadcast as unknown as
| BroadcastService
| undefined;
if (!broadcast) {
res.status(503).json({
error: { message: 'Broadcast service not registered' },
});
return;
}
const verified = await broadcast.verifySignedRequest(
req.rawBody,
broadcastHeaders(req),
);
if (!verified.ok) {
res.status(verified.status ?? 400).json({
error: { message: verified.message ?? 'Bad request' },
});
return;
}
if (verified.info?.ignored) {
res.status(200).json({ ok: true, ...verified.info });
return;
}
const reply = await this.services.eventForward.receive(
(req.body ?? {}) as ForwardBatch,
);
res.status(200).json({ ok: true, ...reply });
}
}
const broadcastHeaders = (req: Request) => {
const headerOnce = (name: string): string | undefined => {
const value = req.headers[name];
if (Array.isArray(value)) return value[0];
return value;
};
return {
peerId: headerOnce('x-broadcast-peer-id'),
timestamp: headerOnce('x-broadcast-timestamp'),
nonce: headerOnce('x-broadcast-nonce'),
signature: headerOnce('x-broadcast-signature'),
};
};
@@ -263,19 +263,24 @@ describe('DriverController — concurrent acquire/release', () => {
});
it('does not attach release listeners when the driver declares no concurrent config', async () => {
// The optimisation that lets the existing test stubs in
// DriverController.test.ts get away without an EventEmitter-shaped
// `res`: skip the once() wiring entirely when there's no spec.
// With no spec there is nothing to release, so the gate must not
// wire `finish`/`close` release listeners. The client-disconnect
// hook is separate and always present: exactly one `close` listener.
const driver = makeSyntheticDriver();
(driver as { concurrent?: unknown }).concurrent = undefined;
const controller = buildController(driver);
const handler = captureCallHandler(controller);
// Bare object with no event-emitter surface — exposes the bug
// case where the gate would try to call `res.once`.
// Minimal `res` that only records which events get listeners.
const listened: string[] = [];
const bareRes = {
statusCode: 200,
body: undefined as unknown,
writableFinished: false,
once(event: string) {
listened.push(event);
return this;
},
status(code: number) {
this.statusCode = code;
return this;
@@ -301,5 +306,6 @@ describe('DriverController — concurrent acquire/release', () => {
bareRes as unknown as Response,
);
expect(bareRes.body).toMatchObject({ success: true, result: 'pong' });
expect(listened).toEqual(['close']);
});
});
@@ -35,9 +35,10 @@
/* eslint-disable @typescript-eslint/no-explicit-any */
import { Readable, Writable } from 'node:stream';
import type { Request, RequestHandler, Response } from 'express';
import { APIConnectionTimeoutError } from 'openai';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import type { DriverMethodLifecycleEvent } from '../../clients/event/types.js';
import { runWithContext } from '../../core/context.js';
import { Context, runWithContext } from '../../core/context.js';
import { configureRateLimit } from '../../core/http/middleware/rateLimit.js';
import { DriverController } from './DriverController.js';
@@ -325,6 +326,32 @@ describe('DriverController upstream error translation', () => {
expect(err).toBe('a bare string');
});
it('maps an SDK connection timeout, which carries no status, to a 504 upstream_timeout', async () => {
const raw = new APIConnectionTimeoutError();
const { err } = await callWith(throwing(raw));
expect(err).toMatchObject({
statusCode: 504,
legacyCode: 'upstream_timeout',
message: 'AI provider timed out',
cause: raw,
});
});
it('maps a fetch timeout that undici wraps in a `fetch failed` TypeError to a 504 upstream_timeout', async () => {
const raw = new TypeError('fetch failed', {
cause: Object.assign(new Error('Headers Timeout Error'), {
name: 'HeadersTimeoutError',
code: 'UND_ERR_HEADERS_TIMEOUT',
}),
});
const { err } = await callWith(throwing(raw));
expect(err).toMatchObject({
statusCode: 504,
legacyCode: 'upstream_timeout',
fields: { upstreamCode: 'UND_ERR_HEADERS_TIMEOUT' },
});
});
it('passes an error with a sub-400 status through untranslated', async () => {
const raw = { status: 302, message: 'redirected' };
const { err } = await callWith(throwing(raw));
@@ -435,3 +462,53 @@ describe('DriverController per-method rate limiting', () => {
expect(alarms).toEqual([]);
});
});
// -- Client disconnect --------------------------------------------------
describe('DriverController client disconnect', () => {
it('exposes an abort signal in the request context that fires when the client leaves early', async () => {
let seen: AbortSignal | undefined;
const { handler } = build({
run: async () => {
seen = Context.get('abortSignal');
await new Promise((r) => setImmediate(r));
return { ok: true };
},
});
const res = new MockRes();
const done = runWithContext({}, () =>
handler(
makeReq({ interface: 'test-iface', method: 'run' }),
res as unknown as Response,
() => {},
),
);
res.destroy();
await done;
expect(seen).toBeInstanceOf(AbortSignal);
expect(seen?.aborted).toBe(true);
});
it('does not abort when the response simply finished', async () => {
let seen: AbortSignal | undefined;
const { handler } = build({
run: () => {
seen = Context.get('abortSignal');
return { ok: true };
},
});
const res = new MockRes();
await runWithContext({}, () =>
handler(
makeReq({ interface: 'test-iface', method: 'run' }),
res as unknown as Response,
() => {},
),
);
res.end();
await new Promise((r) => setImmediate(r));
expect(seen?.aborted).toBe(false);
});
});
@@ -0,0 +1,186 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/* eslint-disable @typescript-eslint/no-explicit-any */
import { EventEmitter } from 'node:events';
import type { Request, RequestHandler, Response } from 'express';
import { beforeEach, describe, expect, it } from 'vitest';
import { runWithContext } from '../../core/context.js';
import { isHttpError } from '../../core/http/HttpError.js';
import { configureRateLimit } from '../../core/http/middleware/rateLimit.js';
import type { IConfig } from '../../types';
import { DriverController } from './DriverController.js';
// The reputation requirement a driver declares per method, exercised through
// `#handleCall` the same way the subscription test does: a synthetic driver
// carrying only the field under test, plus a config that says what its tiers
// are worth.
const captureCallHandler = (controller: DriverController): RequestHandler => {
let handler: RequestHandler | undefined;
const fakeRouter = {
post: (path: string, _opts: unknown, h: RequestHandler) => {
if (path === '/call') handler = h;
return fakeRouter;
},
get: () => fakeRouter,
use: () => fakeRouter,
};
controller.registerRoutes(
fakeRouter as unknown as Parameters<
typeof controller.registerRoutes
>[0],
);
if (!handler) throw new Error('failed to capture POST /call handler');
return handler;
};
class StubRes extends EventEmitter {
statusCode = 200;
body: unknown = undefined;
status(code: number) {
this.statusCode = code;
return this;
}
json(body: unknown) {
this.body = body;
return this;
}
setHeader() {
return this;
}
type() {
return this;
}
send() {
return this;
}
}
const makeReq = (
method: string,
tag: string,
reputation: number | undefined,
): Request =>
({
body: { interface: 'test-iface', method },
actor: { user: { uuid: `u-${tag}`, username: 'u', reputation } },
headers: { 'user-agent': tag },
query: {},
ip: '127.0.0.1',
socket: { remoteAddress: '127.0.0.1' },
}) as unknown as Request;
const syntheticDriver = {
driverInterface: 'test-iface',
driverName: 'test-driver',
isDefault: true,
requireReputation: {
methods: { gated: 'standard', strict: 'trusted', ungated: false },
},
onServerStart() {},
onServerPrepareShutdown() {},
onServerShutdown() {},
open: async () => 'open-ok',
ungated: async () => 'ungated-ok',
gated: async () => 'gated-ok',
strict: async () => 'strict-ok',
};
const buildController = (config: IConfig) =>
new DriverController(
config,
{ alarm: { create: () => {} } } as any,
{} as any,
{} as any,
{ syntheticDriver } as any,
);
const call = (
config: IConfig,
method: string,
tag: string,
reputation?: number,
) => {
const handler = captureCallHandler(buildController(config));
const res = new StubRes();
return runWithContext({}, () =>
handler(
makeReq(method, tag, reputation),
res as unknown as Response,
() => {},
),
).then(() => res);
};
const tiers = {
reputationGate: { tiers: { standard: 60, trusted: 90 } },
} as unknown as IConfig;
const expect403 = (promise: Promise<unknown>) =>
expect(promise).rejects.toSatisfy(
(e) =>
isHttpError(e) &&
(e as { statusCode: number }).statusCode === 403 &&
(e as { legacyCode?: string }).legacyCode === 'reputation_required',
);
describe('DriverController — per-method reputation requirement', () => {
beforeEach(() => {
configureRateLimit();
});
it('leaves a method that declares nothing open to a low-scoring account', async () => {
const res = await call(tiers, 'open', 'open', 10);
expect(res.body).toMatchObject({ success: true, result: 'open-ok' });
});
it('leaves a method that opts out open to the same account', async () => {
const res = await call(tiers, 'ungated', 'ungated', 10);
expect(res.body).toMatchObject({ success: true, result: 'ungated-ok' });
});
it('rejects an account below the tier a method asks for', async () => {
await expect403(call(tiers, 'gated', 'low', 30));
});
it('admits an account that clears it', async () => {
const res = await call(tiers, 'gated', 'high', 70);
expect(res.body).toMatchObject({ success: true, result: 'gated-ok' });
});
it('holds each method to its own tier', async () => {
await expect403(call(tiers, 'strict', 'strict-low', 70));
const res = await call(tiers, 'strict', 'strict-high', 95);
expect(res.body).toMatchObject({ success: true, result: 'strict-ok' });
});
it('enforces nothing on a deployment that defines no tiers', async () => {
const res = await call({} as IConfig, 'gated', 'no-tiers', 0);
expect(res.body).toMatchObject({ success: true, result: 'gated-ok' });
});
it('enforces nothing once the master switch is off', async () => {
const off = {
reputationGate: { enabled: false, tiers: { standard: 60 } },
} as unknown as IConfig;
const res = await call(off, 'gated', 'switched-off', 0);
expect(res.body).toMatchObject({ success: true, result: 'gated-ok' });
});
});
@@ -24,18 +24,21 @@ import { Context } from '../../core/context.js';
import { Controller } from '../../core/http/decorators.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import { assertNotUserSession } from '../../core/http/middleware/gates.js';
import { assertActorMeetsReputation } from '../../core/reputation.js';
import {
acquireDriverConcurrent,
checkDriverRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { DriverMeta } from '../../drivers/meta.js';
import { isUpstreamTimeoutError } from '../../drivers/util/upstreamErrors.js';
import {
isDriverStreamResult,
resolveCallableMethods,
resolveDriverMeta,
resolveDriverMethodConcurrent,
resolveDriverMethodRateLimit,
resolveDriverMethodRequireReputation,
resolveDriverMethodRequireSubscription,
} from '../../drivers/meta.js';
import { assertActorHasSubscription } from '../../services/metering/enforcement.js';
@@ -114,9 +117,20 @@ const translateProviderError = (err: unknown): unknown => {
message?: string;
error?: { code?: string; type?: string; message?: string };
code?: string;
cause?: unknown;
};
const status = extractUpstreamStatus(e);
if (typeof status !== 'number') return err;
if (typeof status !== 'number') {
if (isUpstreamTimeoutError(e)) {
const cause = e.cause as { code?: string } | undefined;
return new HttpError(504, 'AI provider timed out', {
legacyCode: 'upstream_timeout',
fields: { upstreamCode: e.code ?? cause?.code },
cause: err,
});
}
return err;
}
const msg = e.error?.message ?? e.message ?? 'Upstream provider error';
const upstreamCode = e.error?.code ?? e.code;
@@ -327,6 +341,25 @@ export class DriverController extends PuterController {
}
}
// Methods that ask for a trusted-enough account. Declared per-driver
// (`@Driver({ requireReputation })`) for the same reason the
// subscription block is. Checked ahead of the plan and rate-limit
// gates, matching the route chain: whether this account should be
// reaching the method at all is settled before what it pays for or how
// often it may ask. Inert unless the running config gives the named
// tier a score.
const reputationRequirement = resolveDriverMethodRequireReputation(
driverMeta?.requireReputation,
method,
);
if (reputationRequirement !== undefined) {
await assertActorMeetsReputation(
req.actor,
reputationRequirement,
this.config,
);
}
// Subscriber-only methods. Declared per-driver
// (`@Driver({ requireSubscription })`) because `/drivers/call` is a
// single route and a route option would apply to every driver at once.
@@ -412,6 +445,15 @@ export class DriverController extends PuterController {
// a stale value from a prior call.
Context.set('driverName', requestedDriver);
// A caller that hangs up mid-call gets nothing back, so long-running
// drivers watch this to stop working (and metering) as soon as it does.
// `close` after `finish` is the normal end of a response, not an abort.
const abort = new AbortController();
res.once('close', () => {
if (!res.writableFinished) abort.abort();
});
Context.set('abortSignal', abort.signal);
// Per-method lifecycle events, scoped to `driver.<iface>.<method>`.
// Subscribers can listen on `driver.*`, `driver.<iface>.*`, or the
// exact key. `before` is emitted via `emitAndWait` so a listener may
@@ -0,0 +1,463 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { timingSafeEqual } from 'node:crypto';
import type { Request, Response } from 'express';
import type { Actor } from '../../core/actor.js';
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { DURABLE_LIST_LIMIT_CAP } from '../../stores/events/DurableSubscriptionStore.js';
import { EVENTS_WORKERS_LIST_LIMIT_CAP } from '../../stores/events/EventHandlerStore.js';
import { KV_HANDLE_LIST_LIMIT_CAP } from '../../stores/events/KvShareHandleStore.js';
import { normalizeLimit } from '../../util/pagination.js';
import { PuterController } from '../types.js';
import { EVENTS_WORKER_PREFIX } from '../../services/events/workerRuntime.js';
import {
EVENTS_FETCH_LIMIT,
EVENTS_FETCH_LIMIT_CAP,
EVENTS_HANDLER_LIST_LIMIT,
EVENTS_LIST_LIMIT,
EVENTS_WORKER_LIST_LIMIT,
} from './limits.js';
import {
EventsWorkerDeployer,
LocalEventsInvokeTransport,
} from './workerDeploy.js';
/**
* The durable half of the events surface. Session subscriptions arrive over the
* socket that holds them and are not routable; these are rows that outlive
* every connection, so they need somewhere to be created, listed and revoked
* from without one.
*
* Gates are `subdomain: 'api'` (verb routes are root-origin by default) plus
* `allowAccessToken`, because API tokens are in the scoping matrix and access
* tokens are refused on authenticated routes otherwise. Deliberately **not**
* `allowedAppIds`: it does not restrict a route to app actors, and using it as
* though it did is the mis-gating this surface has to avoid. Scope is decided
* from `effectiveApp` inside the service, over the index the rows are stored
* under.
*/
/** What a dispatcher may ask to have deployed: an events script name. */
const EVENTS_SCRIPT_REGEX = new RegExp(`^${EVENTS_WORKER_PREFIX}[a-f0-9]{32}$`);
/** Matches the edge dispatcher's own validation of the app uid it forwards. */
const APP_UID_REGEX = /^app-[A-Za-z0-9_-]{1,64}$/;
@Controller('/events')
export class EventsController extends PuterController {
/**
* POST /events/subscribe — register a subscription that outlives the
* caller.
*/
@Post('/subscribe', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async subscribe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const { sub } = await this.services.events.subscribeDurable(
actor,
this.#body(req),
);
res.json(sub);
}
/** GET /events/subscriptions — what the caller holds, one page at a time. */
@Get('/subscriptions', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async list(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.services.events.listDurable(actor, {
limit: normalizeLimit(query.limit, { cap: DURABLE_LIST_LIMIT_CAP }),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
});
res.json({
items: page.items,
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
/**
* GET /events/fetch — what the caller missed, one page at a time.
*
* Stateless: no subscription, no stored position, nothing written. The
* cursor comes back to the caller and goes out again as `after`.
*/
@Get('/fetch', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_FETCH_LIMIT,
})
async fetch(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.services.events.fetchMissed(actor, {
subject: typeof query.subject === 'string' ? query.subject : '',
after: typeof query.after === 'string' ? query.after : undefined,
limit: normalizeLimit(query.limit, {
cap: EVENTS_FETCH_LIMIT_CAP,
}),
});
res.json({
items: page.items,
...(page.cursor ? { cursor: page.cursor } : {}),
});
}
/**
* POST /events/unsubscribe — an id the caller does not hold reads as
* absent.
*/
@Post('/unsubscribe', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async unsubscribe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
await this.services.events.unsubscribeDurable(actor, this.#body(req));
res.json({});
}
// -- Cross-user key-value handles --------------------------------
/**
* POST /events/kv-handles — hand another user a watchable region of this
* account's key-value data.
*
* An account session, or an app session holding a `manage:` delegation on
* the region — the service tells those apart and gates each rather than the
* route doing it, because `effectiveApp` is where app-ness actually lives.
* A token an app minted is refused there too: a delegation is the app's to
* hold, not to pass on. A user's own token still acts for the user.
*/
@Post('/kv-handles', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async mintKvHandle(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.mintKvHandle(actor, this.#body(req)),
);
}
/**
* GET /events/kv-handles — what this account has shared out. An app sees
* its own namespace only.
*/
@Get('/kv-handles', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async listKvHandles(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.services.events.listKvHandles(actor, {
limit: normalizeLimit(query.limit, {
cap: KV_HANDLE_LIST_LIMIT_CAP,
}),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
});
res.json({
items: page.items,
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
/**
* DELETE /events/kv-handles/:handle — take a shared region back. An app may
* take back one in its own namespace; a handle another account minted reads
* as absent.
*/
@Delete('/kv-handles/:handle', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async revokeKvHandle(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.revokeKvHandle(actor, req.params.handle),
);
}
// -- Handlers ----------------------------------------------------
//
// Deploying an app's code, so the gate is the same as the verbs above plus
// the ownership check the service makes: the app token's own app, or an app
// a user session names and that user owns.
/** POST /events/handlers/publish — create or update one named handler. */
@Post('/handlers/publish', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async publishHandler(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.publishHandler(actor, this.#body(req)),
);
}
/** POST /events/handlers/publishAll — a build step's whole set, in order. */
@Post('/handlers/publishAll', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async publishHandlers(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json({
handlers: await this.services.events.publishHandlers(
actor,
this.#body(req),
),
});
}
/** GET /events/handlers/list — names and hashes, never source. */
@Get('/handlers/list', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_HANDLER_LIST_LIMIT,
})
async listHandlers(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
res.json({
handlers: await this.services.events.listHandlers(actor, {
appUid:
typeof query.appUid === 'string' ? query.appUid : undefined,
}),
});
}
/** POST /events/handlers/remove — delete, and suspend what was bound. */
@Post('/handlers/remove', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async removeHandler(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.removeHandler(actor, this.#body(req)),
);
}
// -- Events workers ------------------------------------------------
//
// The billable artifact a published handler set implies. Account-scoped
// like the kv-handle surface above: this is the owner's own view of what
// it is paying for, so — unlike the handler routes — an app token cannot
// act here on its owner's behalf.
/** GET /events/workers — the caller's own events workers, one per app. */
@Get('/workers', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_WORKER_LIST_LIMIT,
})
async listEventsWorkers(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.services.events.listEventsWorkers(actor, {
limit: normalizeLimit(query.limit, {
cap: EVENTS_WORKERS_LIST_LIMIT_CAP,
}),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
});
res.json({
items: page.items,
...(page.cursor ? { cursor: page.cursor } : {}),
deployable: page.deployable,
});
}
/**
* POST /events/workers/destroy — remove every handler an app has published,
* taking its events worker down with the last one.
*/
@Post('/workers/destroy', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async destroyEventsWorker(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.destroyEventsWorker(
actor,
this.#body(req),
),
);
}
/**
* POST /events/worker/rehydrate — the events dispatcher asking for a script
* it could not find in its namespace, which is also how a handler set is
* deployed the first time. Internal: the shared secret is the only gate,
* and there is no actor.
*/
@Post('/worker/rehydrate', {
subdomain: 'api',
// The shared secret is what gates this route, so the limit is only a
// guessing-rate bound — hence per-IP and deliberately high. The caller
// fans in from every edge location behind a small set of egress
// addresses and asks once per missing script per isolate.
rateLimit: {
scope: 'events-worker-rehydrate',
limit: 6000,
window: 60_000,
key: 'ip',
},
})
async rehydrateWorker(req: Request, res: Response): Promise<void> {
this.#requireInternalAuth(req);
const body = this.#body(req);
const script = String(body.script ?? '');
const appUid = String(body.appUid ?? '');
if (!EVENTS_SCRIPT_REGEX.test(script) || !APP_UID_REGEX.test(appUid))
throw new HttpError(400, 'Missing or invalid `script`/`appUid`', {
legacyCode: 'bad_request',
});
const deployer = this.#eventsWorkerDeployer();
if (!deployer.enabled) {
res.status(404).json({ deployed: false, reason: 'disabled' });
return;
}
const outcome = await deployer.ensure(appUid, script);
if (outcome === 'deployed') {
res.json({ deployed: true });
return;
}
// `failed` and `throttled` are worth another attempt: the rest say
// this script is not what the app's handlers currently are.
res.status(
outcome === 'failed' || outcome === 'throttled' ? 502 : 404,
).json({
deployed: false,
reason: outcome,
});
}
// -- Internals ---------------------------------------------------
/**
* Local development has no events dispatcher, so invocations are handed
* straight to the local worker runtime — including the deploy-on-miss the
* dispatcher would otherwise ask for over the rehydrate route above.
*
* Elsewhere, this backend already knows the app and script a miss names, so
* it deploys the script itself rather than count on the rehydrate callback
* landing on a backend behind the dispatcher's own hostname.
*/
override onServerStart(): void {
const deployer = this.#eventsWorkerDeployer();
if (!deployer.enabled) return;
if (this.config.workers?.localServer) {
this.services.events.useWorkerTransport(
new LocalEventsInvokeTransport(this.services, deployer),
);
return;
}
this.services.events.useWorkerMissHandler((appUid, script) =>
deployer.ensure(appUid, script),
);
}
#deployer: EventsWorkerDeployer | null = null;
#eventsWorkerDeployer(): EventsWorkerDeployer {
this.#deployer ??= new EventsWorkerDeployer({
config: this.config,
clients: this.clients,
stores: this.stores,
services: this.services,
drivers: this.drivers,
});
return this.#deployer;
}
/**
* Constant-time, so the secret cannot be recovered a byte at a time.
* Compares Buffer byte lengths, not string lengths — a string length
* compares UTF-16 code units, which can differ from the byte length
* `timingSafeEqual` actually requires to match.
*/
#requireInternalAuth(req: Request): void {
const expected = Buffer.from(this.config.events?.internalSecret ?? '');
const offered = Buffer.from(
String(req.headers['x-puter-internal-auth'] ?? ''),
);
const ok =
expected.length > 0 &&
expected.length === offered.length &&
timingSafeEqual(expected, offered);
if (!ok)
throw new HttpError(403, 'Forbidden', { legacyCode: 'forbidden' });
}
#requireActor(req: Request): Actor {
const actor = req.actor;
if (!actor?.user)
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
return actor;
}
#body(req: Request): Record<string, unknown> {
const body = req.body;
if (!body || typeof body !== 'object' || Array.isArray(body))
throw new HttpError(400, 'body must be an object', {
legacyCode: 'bad_request',
});
return body as Record<string, unknown>;
}
}
@@ -0,0 +1,123 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* The event limits, and the page that publishes them. An undisclosed limit is
* one a developer meets as a service failure, so the numbers here and the
* numbers on the page are held against each other.
*/
import { describe, expect, it } from 'vitest';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
import {
EVENTS_BROADCAST_DELIVERY_LIMIT,
EVENTS_DURABLE_SUBSCRIPTIONS_MAX,
EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP,
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_USER,
EVENTS_SINGLE_DELIVERY_LIMIT,
EVENTS_WORKER_INVOCATION_LIMIT,
limitFor,
type TieredLimit,
} from './limits.js';
const tiers: Array<[string, TieredLimit]> = [
['durable subscriptions per account', EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER],
['durable subscriptions per app', EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP],
['live share handles per account', EVENTS_KV_HANDLES_PER_USER],
['live share handles per app', EVENTS_KV_HANDLES_PER_APP],
];
describe('the tiered subscription quotas', () => {
it.each(tiers)('%s never lets a free tier exceed paid', (_name, tier) => {
for (const n of Object.values(tier.bySubscription))
expect(n).toBeLessThanOrEqual(tier.limit);
});
it.each(tiers)('%s gives a temporary account none', (_name, tier) => {
expect(tier.bySubscription[DEFAULT_TEMP_SUBSCRIPTION]).toBe(0);
});
it('keeps what one app may take below what the account may hold', () => {
for (const plan of [
null,
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
]) {
expect(
limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP, plan),
).toBeLessThanOrEqual(
limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER, plan),
);
}
});
it('keeps what one app may hold below what the account may hold, for share handles', () => {
for (const plan of [
null,
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
]) {
expect(
limitFor(EVENTS_KV_HANDLES_PER_APP, plan),
).toBeLessThanOrEqual(limitFor(EVENTS_KV_HANDLES_PER_USER, plan));
}
});
it('holds an unrecognised plan to the paid base', () => {
expect(limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER, 'some_plan')).toBe(
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit,
);
expect(limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER, null)).toBe(
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit,
);
});
it('reads its structural maximum off the paid cap', () => {
expect(EVENTS_DURABLE_SUBSCRIPTIONS_MAX).toBe(
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit,
);
});
});
describe('the delivery budgets', () => {
it('keeps `single` well under broadcast — each one costs far more', () => {
expect(EVENTS_SINGLE_DELIVERY_LIMIT.limit).toBeLessThan(
EVENTS_BROADCAST_DELIVERY_LIMIT.limit,
);
expect(EVENTS_WORKER_INVOCATION_LIMIT.limit).toBeLessThan(
EVENTS_SINGLE_DELIVERY_LIMIT.limit,
);
});
it('pins an explicit scope on each, so two call sites share one counter', () => {
for (const spec of [
EVENTS_BROADCAST_DELIVERY_LIMIT,
EVENTS_SINGLE_DELIVERY_LIMIT,
EVENTS_WORKER_INVOCATION_LIMIT,
]) {
expect(spec.scope).toBeTruthy();
expect(spec.window).toBe(60_000);
}
});
});
+395
View File
@@ -0,0 +1,395 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { RouteRateLimit } from '../../core/http/types';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
// -- Shared event limits ---------------------------------------------
//
// One cheap write can fan out into (matched subscriptions × connected
// sockets), and every number here caps a term of that product. Storage quota
// and the FS write limits bound how many events can be *produced*; nothing
// else bounds how many deliveries one of them turns into.
//
// The verbs arrive over the socket rather than over HTTP, so these are not
// mounted as route gates — they are spent imperatively at the point each one
// protects, with the `scope` naming the counter the same way a route gate
// would. Every number below is published in `rate-limits-and-quotas.md`.
/** Per-user sliding window, spent imperatively via `checkRateLimit`. */
const userWindow = (
scope: string,
limit: number,
window = 60_000,
): RouteRateLimit => ({ scope, limit, window, key: 'user' });
/**
* A cap that varies by plan, in the shape route gates already declare theirs
* in: the base is what a subscribed account sees, and `bySubscription` carves
* the free tiers out beneath it. A plan nobody enumerated falls through to the
* base, so a new one is generous rather than accidentally throttled.
*/
export interface TieredLimit {
limit: number;
bySubscription: Record<string, number>;
}
const tiered = (paid: number, free: number, temp: number): TieredLimit => ({
limit: paid,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: free,
[DEFAULT_TEMP_SUBSCRIPTION]: temp,
},
});
/** The cap one plan sees. An unresolved plan is held to the base. */
export const limitFor = (
tier: TieredLimit,
subscriptionId: string | null,
): number =>
(subscriptionId === null
? undefined
: tier.bySubscription[subscriptionId]) ?? tier.limit;
/**
* What a plan-tiered quota resolves to: what an account may hold, and what one
* app may take of it.
*/
export interface TieredQuota {
perUser: number;
perApp: number;
}
// -- Subscription surface --------------------------------------------
/**
* Live subscriptions one socket may hold.
*
* Session subscriptions are Redis set members keyed to a socket that can go
* away without saying so, so the cap is really on how much a disconnect can
* leave behind. A client watching more than a few dozen distinct anchors wants
* one subscription on their common parent instead.
*/
export const EVENTS_SESSION_SUBSCRIPTIONS_PER_SOCKET = 50;
/**
* Durable subscriptions one account may hold, across every app.
*
* These are table rows that keep costing after the client that made them is
* gone — a delivery each time their anchor changes, and a cache entry in every
* region that sees a write. Counted over the holder index. A temporary account
* holds none: nothing outlives its connection to deliver to.
*/
export const EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER = tiered(500, 100, 0);
/**
* Durable subscriptions one app may hold for one account.
*
* Below the per-account cap so that one app cannot spend an account's whole
* budget: the account-wide number is what the watched-token set costs, and this
* is what any single app may take of it.
*/
export const EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP = tiered(100, 25, 0);
/** Most rows any account can hold, whatever its plan — a read bound, not a gate. */
export const EVENTS_DURABLE_SUBSCRIPTIONS_MAX =
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit;
/**
* How long a suspended durable subscription is kept before it is deleted.
*
* A suspension caused by a revoked grant never resumes — consent to watch is
* re-established by subscribing again — so the row survives only long enough
* for its holder to see, in `list`, that it stopped and why.
*/
export const SUSPENDED_ROW_TTL_DAYS = 30;
/**
* Subscribe + unsubscribe calls per minute, per user.
*
* Both resolve a path and take a write, so a loop over them is a write loop.
* Sized like the sharing verbs, which are the closest existing analogue: a
* client sets its subscriptions up once and then leaves them alone.
*/
export const EVENTS_SUBSCRIBE_LIMIT = userWindow('events:subscribe', 60);
/**
* Subscription listings per minute, per user. Reads an index and returns one
* page, so it is budgeted well above the verbs that write.
*/
export const EVENTS_LIST_LIMIT = userWindow('events:list', 120);
/**
* Delivery acknowledgements per minute, per user.
*
* One ack per `single` delivery, so this sits level with what one subscription
* may be delivered — a client acking faster than that is acking things it was
* never sent.
*/
export const EVENTS_ACK_LIMIT = userWindow('events:ack', 600);
/**
* Share-handle mint + revoke calls per minute, per user.
*
* Each one issues or withdraws a grant and settles what stood on it, so it is
* budgeted with the subscribe verbs rather than the listings.
*/
export const EVENTS_KV_HANDLE_LIMIT = userWindow('events:kvHandles', 60);
/**
* Live share handles one account may hold out at a time, across every app.
*
* Each is a standing grant on part of the account's data, and revoking marks
* rather than deletes, so without a ceiling the rate limit alone lets the rows
* grow forever. A temporary account holds none: the grant outlives the session
* that made it, and no account is left to take it back.
*/
export const EVENTS_KV_HANDLES_PER_USER = tiered(500, 200, 0);
/**
* Live share handles one app may hold out for one account.
*
* Below the per-account cap so that one app cannot spend an account's whole
* budget: a handle names one namespace, so without this a second app on the
* same account would be left with nothing.
*/
export const EVENTS_KV_HANDLES_PER_APP = tiered(100, 50, 0);
// -- Handler surface -------------------------------------------------
/**
* Named handlers one app may have published.
*
* A handler is a row read on the delivery path, so the cap is on how many
* distinct pieces of code an app asks the system to keep addressable — not on
* how often it changes them. An app past this is describing events by name
* where a `match` filter belongs.
*/
export const EVENTS_HANDLERS_PER_APP = 100;
/** Longest a serialized handler may be. */
export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
/**
* Longest an app's generated events worker may be, all handlers combined. Above
* this a publish is refused with `events_worker_too_large` before it lands, and
* a set that got here some other way fails its deploy instead of shipping a
* script too big for the runtime to load.
*/
export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
/**
* Deploys one app's events worker may trigger per hour.
*
* A deploy provisions upstream, so a set that flaps — publish, remove,
* republish — must not turn into an unbounded provisioning loop. Rehydration (a
* namespace miss) counts the same as a publish-triggered deploy.
*/
export const EVENTS_WORKER_DEPLOYS_PER_HOUR = 30;
/**
* Handlers one `publishAll` may carry. A build step publishes its whole set in
* one call, and the set is capped by what an app may hold anyway.
*/
export const EVENTS_HANDLER_PUBLISH_BATCH = 50;
/**
* Handler publishes and removals per minute, per user.
*
* A build step publishes its whole set in one call and a developer iterating
* publishes a handful; level with the subscribe budget, which is the closest
* analogue — a write a client makes deliberately, never in a loop.
*/
export const EVENTS_HANDLER_PUBLISH_LIMIT = userWindow(
'events:handlers:publish',
60,
);
/**
* Missed-event fetches per minute, per user.
*
* A keyset page off one index, so it is budgeted like the other listings — a
* client catching up after a disconnect walks a handful of pages once, not a
* page per event.
*/
export const EVENTS_FETCH_LIMIT = userWindow('events:fetch', 120);
/** Rows one fetch page may carry. Defaults to a quarter of it. */
export const EVENTS_FETCH_LIMIT_CAP = 200;
export const EVENTS_FETCH_LIMIT_DEFAULT = 50;
/** Handler listings per minute, per user. Reads an index, so budgeted higher. */
export const EVENTS_HANDLER_LIST_LIMIT = userWindow(
'events:handlers:list',
120,
);
/**
* Events worker listings per minute, per user. Same shape as the handler
* listing.
*/
export const EVENTS_WORKER_LIST_LIMIT = userWindow('events:workers:list', 120);
// -- Dispatch fan-out ------------------------------------------------
/**
* Subscriptions one event may deliver to before dispatch stops and reports a
* gap. The amplification ceiling: without it, one write costs as many
* deliveries as an account cared to register.
*/
export const EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT = 50;
/**
* Broadcast deliveries per minute, per subscription.
*
* Ten a second sustained is far past what a UI can render and well past what a
* human is watching for; a subscription over it is looping, and the gap marker
* tells it so rather than letting it silently miss events.
*/
export const EVENTS_BROADCAST_DELIVERY_LIMIT = userWindow(
'events:delivery',
600,
);
/**
* `single` deliveries per minute, per subscription.
*
* A fifth of the broadcast budget: each one is leased, acknowledged and may run
* an app's handler, so it costs an order of magnitude more than a socket copy.
* Over it the event is not queued — a gap marker takes its place, so the
* consumer learns it fell behind instead of inheriting a backlog it can never
* work through.
*/
export const EVENTS_SINGLE_DELIVERY_LIMIT = userWindow(
'events:delivery:single',
120,
);
/**
* Handler invocations per minute, per (account, app).
*
* The one term of the fan-out product that costs real compute, so it is capped
* per app rather than per subscription — an app cannot widen it by holding more
* subscriptions. A delivery that arrives over the budget is not failed: it
* stays owed, and its lease is the backoff.
*/
export const EVENTS_WORKER_INVOCATION_LIMIT = userWindow(
'events:worker:invoke',
60,
);
/**
* Filter evaluations one event may spend. Lives with the matcher because the
* primitive that enforces it does; re-exported here so every published number
* is readable in one place.
*/
export { FILTER_EVALUATIONS_PER_EVENT } from '../../services/events/matcher.js';
// -- Undelivered backlog ---------------------------------------------
/**
* Deliveries one subscription may hold undelivered.
*
* A `single` delivery waits until something takes it, so a subscription whose
* consumer is gone accumulates. Over the cap the oldest go and one gap marker
* takes their place, which is what keeps "at-least-once" honest: what was lost
* is visible rather than silent.
*/
export const EVENTS_PENDING_DELIVERIES_PER_SUBSCRIPTION = 10_000;
/**
* Undelivered deliveries one region may hold across every subscription.
*
* The per-subscription cap bounds one backlog and nothing in aggregate —
* multiply it by the subscriptions that can exist and the region's memory is
* the only remaining limit. Over this, the oldest deliveries in the region are
* shed first, each shedding subscription gets a gap marker, and an alarm says
* it happened.
*/
export const EVENTS_REGION_PENDING_CEILING = 1_000_000;
// -- Handler retries -------------------------------------------------
//
// A handler that answers "not now" — a 5xx, a timeout, a 429 — is retried, and
// a handler that answers "no" is not. Retrying on a fixed cadence turns one
// broken deploy into a permanent load on whatever is failing, so the wait
// doubles per attempt up to a ceiling, and a run of failures stops the
// subscription rather than retrying it forever.
/** Wait before the first retry of a delivery a handler could not take. */
export const EVENTS_RETRY_BASE_MS = 2_000;
/** Longest wait between retries, however many have failed. */
export const EVENTS_RETRY_MAX_MS = 5 * 60 * 1000;
/**
* Failures in a row before a subscription is suspended. Counted per
* subscription and reset by the first delivery a handler takes, so an
* occasional failure never accumulates into one.
*/
export const EVENTS_CONSECUTIVE_FAILURES = 5;
/**
* How long a run of failures is remembered. Five failures at the capped wait
* span well under this, so a counter nothing has touched for an hour describes
* a run that ended.
*/
export const EVENTS_FAILURE_COUNTER_TTL_MS = 60 * 60 * 1000;
/** How long to hold a delivery whose handler has failed `attempts` times. */
export const deliveryBackoffMs = (attempts: number): number =>
Math.min(
EVENTS_RETRY_MAX_MS,
EVENTS_RETRY_BASE_MS * 2 ** Math.max(0, attempts - 1),
);
// -- Suspended backlog -----------------------------------------------
//
// A suspended subscription stops metering but keeps what it is owed, and that
// pair is a free memory hold: removing one widely-subscribed handler would
// otherwise turn every dependent into a full backlog nobody pays for. So a
// suspension trims to a much smaller cap and stamps an expiry, and the pending
// sweeper drops what is left over with a gap marker in its place.
/** Deliveries a suspended subscription may keep, whatever the reason. */
export const EVENTS_SUSPENDED_PENDING_CAP = 100;
/**
* How long a backlog held for a handler that may come back is kept. A bad
* deploy is recoverable within a day; past that the events are stale anyway.
*/
export const EVENTS_SUSPENDED_BACKLOG_TTL_MS = 24 * 60 * 60 * 1000;
/**
* How long a backlog held for an account out of credit is kept. The resume
* condition is a top-up, which is usually minutes.
*/
export const EVENTS_NO_CREDIT_BACKLOG_TTL_MS = 60 * 60 * 1000;
// -- Coalescing ------------------------------------------------------
/**
* Debounce window per (subscription, subject). A multipart upload or a
* recursive delete is one intent that lands as many writes; this is what makes
* it arrive as one event.
*/
export const EVENTS_COALESCE_WINDOW_MS = 250;
@@ -0,0 +1,147 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* `EventsWorkerDeployer` against stubbed layers — no real Cloudflare, no real
* database — pinning the in-flight keying and the per-app deploy throttle
* without booting the full test server.
*/
import MockRedis from 'ioredis-mock';
import { describe, expect, it } from 'vitest';
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
import { EventsWorkerDeployer } from './workerDeploy.js';
import { eventsWorkerScript } from '../../services/events/workerRuntime.js';
import { handlerSetHash } from '../../services/events/workerSource.js';
import type { IConfig } from '../../types.js';
const SOURCE_HASH = 'a'.repeat(64);
const SCRIPT = eventsWorkerScript(
handlerSetHash([{ name: 'h', sourceHash: SOURCE_HASH }]),
'',
);
interface Owner {
ownerUserId: number;
suspended?: boolean;
}
/** The private layers shape `EventsWorkerDeployer` takes — not exported. */
type Layers = ConstructorParameters<typeof EventsWorkerDeployer>[0];
/** Stub layers whose only real dependency is a fresh redis mock. */
const makeLayers = (owners: Record<string, Owner | null>) => {
const redis = new MockRedis.Cluster(['redis://localhost:7001']);
const created: string[] = [];
const layers = {
config: {
events: { workerRuntime: true, internalSecret: 'sekret' },
// `#namespace()` needs a local-worker shape to resolve without
// throwing — this stub never reaches a real Cloudflare or
// Miniflare deploy either way.
workers: { localServer: true },
} as IConfig,
clients: { redis },
stores: {
eventHandler: {
allForApp: async (appUid: string) => [
{
appUid,
name: 'h',
source: 'async () => {}',
sourceHash: SOURCE_HASH,
createdAt: 0,
updatedAt: 0,
},
],
},
app: {
getByUid: async (appUid: string) => {
const owner = owners[appUid];
return owner === null || owner === undefined
? null
: { owner_user_id: owner.ownerUserId };
},
},
user: {
getById: async (id: number) => {
const owner = Object.values(owners).find(
(o) => o?.ownerUserId === id,
);
if (!owner) return null;
return {
id,
uuid: `uuid-${id}`,
username: `user-${id}`,
suspended: owner.suspended ?? false,
};
},
},
},
services: {},
drivers: {
workers: {
create: async (args: { workerName: string }) => {
created.push(args.workerName);
return { success: true, errors: [] };
},
},
},
};
return { layers: layers as unknown as Layers, created, redis };
};
describe('EventsWorkerDeployer.ensure', () => {
it('keys in-flight deploys by app and script, not script alone', async () => {
// Two apps whose handler sets hash identically, and so deploy as the
// same script; only one has a live owner. Both `ensure()` calls run
// synchronously back to back — keying the in-flight map by script
// alone would collapse the second call into the first's outcome.
const { layers } = makeLayers({
'app-a': { ownerUserId: 1 },
'app-b': { ownerUserId: 2, suspended: true },
});
const deployer = new EventsWorkerDeployer(layers);
const [a, b] = await Promise.all([
deployer.ensure('app-a', SCRIPT),
deployer.ensure('app-b', SCRIPT),
]);
expect(a).toBe('deployed');
expect(b).toBe('no-owner');
});
it('refuses once the app has deployed past the hourly cap', async () => {
const { layers, redis } = makeLayers({ 'app-c': { ownerUserId: 3 } });
const deployer = new EventsWorkerDeployer(layers);
const hour = new Date().toISOString().slice(0, 13);
await redis.set(
`ev:deploys:{app-c}:${hour}`,
String(EVENTS_WORKER_DEPLOYS_PER_HOUR),
);
expect(await deployer.ensure('app-c', SCRIPT)).toBe('throttled');
});
it('deploys normally under the cap', async () => {
const { layers } = makeLayers({ 'app-d': { ownerUserId: 4 } });
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure('app-d', SCRIPT)).toBe('deployed');
});
});
@@ -0,0 +1,366 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { puterClients } from '../../clients/index.js';
import type { puterDrivers } from '../../drivers/index.js';
import {
INTERNAL_ADMISSION_BYPASS,
INTERNAL_DEPLOY_TARGET,
} from '../../drivers/workers/WorkerDriver.js';
import type { EventsInvokeCall } from '../../clients/events/EventsWorkerInvokerClient.js';
import {
EVENTS_HANDLED_HEADER,
EVENTS_INVOKE_PATH,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import type { puterServices } from '../../services/index.js';
import {
eventsInvokeKey,
eventsWorkerScope,
eventsWorkerScript,
} from '../../services/events/workerRuntime.js';
import { generateEventsWorkerSource } from '../../services/events/workerSource.js';
import type { puterStores } from '../../stores/index.js';
import type { IConfig, LayerInstances } from '../../types.js';
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
/**
* Deploys an app's events worker from its published handlers.
*
* Deployed the first time a delivery needs it and again on eviction — a publish
* only writes rows. Scripts carry no worker token and no `subdomains` row: not
* one of the owner's workers, not addressable from the internet, not listed or
* deleted as if it were. A republished set leaves the old script behind rather
* than deleting it; redeploying a still-live one is always safe.
*/
/** Why a deploy could not happen, for the callers that answer differently. */
export type EventsDeployOutcome =
| 'deployed'
| 'stale'
| 'no-handlers'
| 'no-owner'
| 'throttled'
| 'failed';
interface DeployLayers {
config: IConfig;
clients: LayerInstances<typeof puterClients>;
stores: LayerInstances<typeof puterStores>;
services: LayerInstances<typeof puterServices>;
drivers: LayerInstances<typeof puterDrivers>;
}
/**
* `ev:deploys:{<appUid>}:<hour>` bucket TTL — comfortably past the hour it
* counts.
*/
const DEPLOY_THROTTLE_TTL_SECONDS = 2 * 60 * 60;
interface DeployResult {
success?: boolean;
errors?: unknown[];
}
export class EventsWorkerDeployer {
readonly #layers: DeployLayers;
/** Collapses the burst of misses one cold app produces into one deploy. */
readonly #inFlight = new Map<string, Promise<EventsDeployOutcome>>();
constructor(layers: DeployLayers) {
this.#layers = layers;
}
/**
* Whether handler sets deploy at all. Off means handler CRUD is storage
* only.
*/
get enabled(): boolean {
return this.#layers.config.events?.workerRuntime === true;
}
/**
* Deploy `script`, which must be what the app's handlers currently hash to.
* A request for anything else is `stale` — the set moved on, and the next
* delivery resolves the new script rather than this one being revived.
*/
ensure(appUid: string, script: string): Promise<EventsDeployOutcome> {
const key = `${appUid}|${script}`;
const existing = this.#inFlight.get(key);
if (existing) return existing;
const started = this.#deploy(appUid, script).finally(() =>
this.#inFlight.delete(key),
);
this.#inFlight.set(key, started);
return started;
}
async #deploy(
appUid: string,
script: string,
): Promise<EventsDeployOutcome> {
const { stores, drivers } = this.#layers;
const handlers = await stores.eventHandler.allForApp(appUid);
if (handlers.length === 0) return 'no-handlers';
const generated = generateEventsWorkerSource(handlers);
// Two apps that published byte-identical sets share one script, which
// is sound: the source is the same, and everything per-app arrives with
// the invocation. Whoever deploys first owns it.
if (
eventsWorkerScript(
generated.setHash,
eventsWorkerScope(this.#layers.config),
) !== script
)
return 'stale';
if (generated.tooLarge) {
console.error(
`[events] ${appUid}: generated events worker exceeds the size cap`,
);
return 'failed';
}
if (generated.broken.length > 0)
console.warn(
`[events] ${appUid}: handlers not runnable as published:`,
generated.broken.join(', '),
);
if (await this.#tooManyDeploysThisHour(appUid)) return 'throttled';
const owner = await this.#owner(appUid);
if (!owner) return 'no-owner';
const secret = this.#layers.config.events?.internalSecret;
if (!secret) {
console.error(
'[events] no internalSecret configured: an events worker deployed without an invoke key would refuse every delivery',
);
return 'failed';
}
// Deployed as the owner, so the deploy is theirs: their app binding,
// their namespace quota, their name in whatever reads the deploy.
// Admission is bypassed because this is not a deploy they asked for —
// publishing is the gate, and an owner whose email lapsed afterwards
// must not silently stop delivering.
try {
const result = (await runWithContext({ actor: owner }, async () =>
drivers.workers.create({
appId: appUid,
workerName: script,
filePath: '',
[INTERNAL_ADMISSION_BYPASS]: true,
[INTERNAL_DEPLOY_TARGET]: {
scriptName: script,
namespace: this.#namespace(),
runtime: 'events',
source: generated.source,
omitOwnerToken: true,
// What the script compares an invocation against.
// Derived from the script name, so a redeploy of
// the same set binds the same key.
secrets: {
events_invoke_key: eventsInvokeKey(secret, script),
},
// Names are hashes and do not reverse, so tags are
// the only way to answer "whose scripts are these?"
// when cleaning up superseded sets.
tags: [
`events-app:${appUid}`,
`user:${owner.user.uuid}`,
],
},
}),
)) as DeployResult | undefined;
// A deploy the platform rejected comes back as a result rather
// than a throw, and must not be reported as a success.
if (result?.success === false)
throw new Error(
(result.errors ?? []).map(String).join('; ') ||
'deploy rejected',
);
} catch (err) {
console.error(
`[events] ${appUid}: events worker deploy failed`,
err instanceof Error ? err.message : err,
);
return 'failed';
}
return 'deployed';
}
/**
* Whether this app has already deployed `EVENTS_WORKER_DEPLOYS_PER_HOUR`
* times this hour. A flapping set (publish, remove, republish) provisions
* upstream on every miss, and this is the ceiling on that loop.
*/
async #tooManyDeploysThisHour(appUid: string): Promise<boolean> {
const redis = this.#layers.clients.redis;
const hour = new Date().toISOString().slice(0, 13);
const key = `ev:deploys:{${appUid}}:${hour}`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
return count > EVENTS_WORKER_DEPLOYS_PER_HOUR;
}
/** The app owner, as an actor the deploy can run as. */
async #owner(
appUid: string,
): Promise<
| (Actor & { user: { id: number; uuid: string; username: string } })
| null
> {
const { stores } = this.#layers;
const app = await stores.app.getByUid(appUid);
const ownerUserId = Number(app?.owner_user_id);
if (!app || !Number.isFinite(ownerUserId)) return null;
const user = await stores.user.getById(ownerUserId);
if (!user?.id || !user.uuid || !user.username || user.suspended)
return null;
return { user } as Actor & {
user: { id: number; uuid: string; username: string };
};
}
/**
* Where events workers are deployed. Deliberately has no default: without a
* value there is nowhere to put them that is not somewhere else's, and the
* default namespace is the one the public dispatcher serves.
*/
#namespace(): string | undefined {
const namespace = this.#layers.config.events?.workerNamespace;
if (namespace) {
// The public dispatcher resolves any script in its namespace off
// the request hostname, with no further check, so sharing that
// namespace would put every events worker on the public worker
// domain — the one thing this deploy target exists to prevent.
if (namespace === this.#layers.config.workers?.namespace)
throw new HttpError(
503,
'events.workerNamespace must not be the public worker namespace',
{ legacyCode: 'response_timeout' },
);
return namespace;
}
// A local worker runtime has no namespaces to keep apart, and no
// dispatcher that could serve one either way.
if (
this.#layers.config.workers?.localServer &&
!this.#layers.config.workers?.ACCOUNTID
)
return undefined;
throw new HttpError(503, 'Events workers are not configured', {
legacyCode: 'response_timeout',
});
}
}
/**
* The local-development transport: no dispatcher, no namespace, no HTTP hop —
* the invocation goes straight into the local worker runtime, and a script that
* is not resident is deployed on the spot the way the dispatcher's miss path
* would have asked for.
*/
export class LocalEventsInvokeTransport {
readonly #services: Pick<DeployLayers['services'], 'localworkerservice'>;
readonly #deployer: EventsWorkerDeployer;
constructor(
services: Pick<DeployLayers['services'], 'localworkerservice'>,
deployer: EventsWorkerDeployer,
) {
this.#services = services;
this.#deployer = deployer;
}
async send(call: EventsInvokeCall): Promise<{
status: number | null;
handled?: boolean;
error?: string;
}> {
const request = (): Request =>
new Request(
`http://${call.script}.workers.puter.localhost${EVENTS_INVOKE_PATH}`,
{
method: 'POST',
headers: {
'content-type': 'application/json',
'x-puter-events-key': call.key,
},
body: call.body,
},
);
try {
const resident = await this.#dispatch(request(), call.timeoutMs);
if (resident !== null) return resident;
const outcome = await this.#deployer.ensure(
call.appUid,
call.script,
);
if (outcome !== 'deployed')
return { status: null, error: `deploy: ${outcome}` };
const retried = await this.#dispatch(request(), call.timeoutMs);
if (retried === null)
return { status: null, error: 'script not resident' };
return retried;
} catch (err) {
return {
status: null,
error: err instanceof Error ? err.message : String(err),
};
}
}
/**
* `null` when the script is not deployed here. The timeout is ours to
* apply: an in-process dispatch has no socket to abort.
*/
async #dispatch(
request: Request,
timeoutMs: number,
): Promise<{ status: number; handled: boolean } | null> {
let timer: ReturnType<typeof setTimeout> | undefined;
const timeout = new Promise<'timeout'>((resolve) => {
timer = setTimeout(() => resolve('timeout'), timeoutMs);
timer.unref();
});
try {
const response = await Promise.race([
this.#services.localworkerservice.dispatchEventsWorker(request),
timeout,
]);
if (response === 'timeout') throw new Error('invocation timed out');
if (response === null) return null;
return {
status: response.status,
handled: response.headers.get(EVENTS_HANDLED_HEADER) === '1',
};
} finally {
clearTimeout(timer);
}
}
}
@@ -226,4 +226,131 @@ describe('GET /fs/readdir over HTTP', () => {
const body = (await response.json()) as Array<{ name: string }>;
expect(body.map((e) => e.name)).toContain('Documents');
});
describe('the share flag', () => {
const post = async (path: string, token: string, body: unknown) => {
const response = await fetch(new URL(path, env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
expect(response.status).toBe(200);
return response.json() as Promise<Record<string, unknown>>;
};
const mkdir = (path: string, token: string) =>
post('/fs/mkdir', token, { path });
const readdir = async (path: string, token: string) => {
const response = await fetch(readdirUrl({ path, auth_token: token }));
expect(response.status).toBe(200);
const entries = (await response.json()) as Array<{
name: string;
isShared: boolean | null;
}>;
return new Map(entries.map((e) => [e.name, e.isShared]));
};
const base = () =>
`/${env.users.user.username}/Documents/flag-${crypto.randomUUID().slice(0, 8)}`;
it('marks a shared child in a listing, and clears it on revoke', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const parent = base();
await mkdir(parent, owner.token);
const shared = await mkdir(`${parent}/shared`, owner.token);
await mkdir(`${parent}/private`, owner.token);
// A write response says nothing about sharing.
expect(shared).not.toHaveProperty('isShared');
await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: shared.uuid }],
mode: 'read',
});
expect(await readdir(parent, owner.token)).toEqual(
new Map([
['shared', true],
['private', false],
]),
);
await post('/share/revoke', owner.token, {
recipients: [recipient.username],
items: [{ uid: shared.uuid }],
});
expect(await readdir(parent, owner.token)).toEqual(
new Map([
['shared', false],
['private', false],
]),
);
});
it('names the recipients in stat only when asked', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const parent = base();
await mkdir(parent, owner.token);
const shared = await mkdir(`${parent}/shared`, owner.token);
await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: shared.uuid }],
mode: 'read',
});
const plain = await post('/fs/stat', owner.token, {
uid: shared.uuid,
});
expect(plain.isShared).toBe(true);
expect(plain).not.toHaveProperty('shares');
const withShares = await post('/fs/stat', owner.token, {
uid: shared.uuid,
return_shares: true,
});
const shares = withShares.shares as Array<Record<string, unknown>>;
expect(shares).toHaveLength(1);
expect(shares[0]).toMatchObject({
holder: recipient.username,
issuer: owner.username,
mode: 'read',
inherited_from: null,
});
for (const key of ['holder_user_id', 'issuer_user_id', 'fsentry_id']) {
expect(shares[0]).not.toHaveProperty(key);
}
});
it('tells a recipient nothing about who else can reach the item', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const parent = base();
await mkdir(parent, owner.token);
const shared = await mkdir(`${parent}/shared`, owner.token);
await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: shared.uuid }],
mode: 'read',
});
const stat = await post('/fs/stat', recipient.token, {
uid: shared.uuid,
return_shares: true,
});
expect(stat.isShared).toBeNull();
// Asked for, so the key is there — but it names nobody.
expect(stat.shares).toEqual([]);
});
});
});
+82 -53
View File
@@ -26,8 +26,9 @@ import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import {
assertNormalized,
expandTildePath,
isOwnersTrash,
normalizeAbsolutePath,
} from '../../services/fs/resolveNode.js';
import {
maskEntryPath,
@@ -45,6 +46,7 @@ import {
runWithConcurrencyLimitSettled,
} from '../../util/concurrency.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { listClientShares } from '../share/clientShare.js';
import { PuterController } from '../types.js';
import { STORAGE_OP_COSTS } from '../../services/metering/costs.js';
import {
@@ -151,7 +153,10 @@ export class FSController extends PuterController {
) {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
const requestBody = this.#withGuiMetadata(req.body, req.body);
const requestBody = this.#withGuiMetadata(
this.#requireObjectBody(req.body),
req.body,
);
requestBody.fileMetadata = await this.#normalizeFileMetadataPath(
req,
requestBody.fileMetadata,
@@ -219,7 +224,7 @@ export class FSController extends PuterController {
? await Promise.all(
req.body.map(async (requestBody) => {
const normalizedRequestBody = this.#withGuiMetadata(
requestBody,
this.#requireObjectBody(requestBody),
req.body,
);
normalizedRequestBody.fileMetadata =
@@ -318,7 +323,10 @@ export class FSController extends PuterController {
res: Response<ClientCompleteWriteResponse>,
) {
const userId = this.#getActorUserId(req);
const requestBody = this.#withGuiMetadata(req.body, req.body);
const requestBody = this.#withGuiMetadata(
this.#requireObjectBody(req.body),
req.body,
);
this.#assertNoInlineSignedThumbnailData(requestBody.thumbnailData);
const response = await this.services.fs.completeUrlWrite(
@@ -356,7 +364,10 @@ export class FSController extends PuterController {
const userId = this.#getActorUserId(req);
const requests = Array.isArray(req.body)
? req.body.map((requestBody) => {
return this.#withGuiMetadata(requestBody, req.body);
return this.#withGuiMetadata(
this.#requireObjectBody(requestBody),
req.body,
);
})
: [];
for (const requestBody of requests) {
@@ -442,7 +453,10 @@ export class FSController extends PuterController {
) {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
const requestBody = this.#withGuiMetadata(req.body, req.body);
const requestBody = this.#withGuiMetadata(
this.#requireObjectBody(req.body),
req.body,
);
requestBody.fileMetadata = await this.#normalizeFileMetadataPath(
req,
requestBody.fileMetadata,
@@ -840,7 +854,7 @@ export class FSController extends PuterController {
? await Promise.all(
req.body.map(async (requestBody) => {
const normalizedRequestBody = this.#withGuiMetadata(
requestBody,
this.#requireObjectBody(requestBody),
req.body,
);
normalizedRequestBody.fileMetadata =
@@ -959,9 +973,14 @@ export class FSController extends PuterController {
// -- Read-side routes ------------------------------------------------
// The read-side routes admit scoped access tokens: `#assertAccess` runs the
// ACL against the token's grant intersected with its issuer's, so a token
// reaches exactly what it was minted for. A background events handler
// acts through one — its `user` is scoped to the subscription's grant.
@Post('/stat', {
subdomain: 'api',
requireVerified: true,
allowAccessToken: true,
rateLimit: FS_STAT_LIMIT,
})
async statEntry(req: Request, res: Response) {
@@ -972,17 +991,28 @@ export class FSController extends PuterController {
await this.#assertAccess(actor, entry.path, 'see');
const wantsSize = this.#toBoolean(body.return_size);
const subtreeSize =
entry.isDir && wantsSize
? await this.services.fs.getSubtreeSize(userId, entry.path)
: undefined;
entry.suggestedApps =
await this.services.suggestedApps.getSuggestedApps(entry);
const [subtreeSize, suggestedApps, shareFlags, shares] =
await Promise.all([
entry.isDir && wantsSize
? this.services.fs.getSubtreeSize(userId, entry.path)
: undefined,
this.services.suggestedApps.getSuggestedApps(entry),
this.services.share.shareFlags(actor, [entry]),
this.#toBoolean(body.return_shares)
? listClientShares(
this.services.share,
this.clients.event,
actor,
entry.uuid,
)
: undefined,
]);
entry.suggestedApps = suggestedApps;
res.json({
...this.#toClientEntry(entry),
...this.#toClientEntry(entry, shareFlags.get(entry.uuid) ?? null),
...(subtreeSize !== undefined ? { size: subtreeSize } : {}),
...(shares !== undefined ? { shares } : {}),
});
}
@@ -994,7 +1024,7 @@ export class FSController extends PuterController {
* (with public folders enabled) any authenticated user. The legacy read
* path already curates its output; this does the same for the v2 routes.
*/
#toClientEntry(entry: FSEntry): ClientFSEntry {
#toClientEntry(entry: FSEntry, isShared?: boolean | null): ClientFSEntry {
// Allowlist, not a denylist: a denylist silently ships every column
// added to `fsentries` later. Omits the numeric primary keys (`id`,
// `parentId`, `associatedAppId`), the storage columns, the owning
@@ -1026,6 +1056,8 @@ export class FSController extends PuterController {
workers: entry.workers ?? [],
hasWebsite: entry.hasWebsite ?? subdomains.length > 0,
suggestedApps: entry.suggestedApps ?? [],
// Read paths pass a value; write responses leave it off entirely.
...(isShared === undefined ? {} : { isShared }),
};
}
@@ -1080,6 +1112,7 @@ export class FSController extends PuterController {
@Get('/readdir', {
subdomain: 'api',
requireVerified: true,
allowAccessToken: true,
rateLimit: FS_READDIR_LIMIT,
})
async readdirEntriesViaGet(req: Request, res: Response) {
@@ -1089,6 +1122,7 @@ export class FSController extends PuterController {
@Post('/readdir', {
subdomain: 'api',
requireVerified: true,
allowAccessToken: true,
rateLimit: FS_READDIR_LIMIT,
})
async readdirEntries(req: Request, res: Response) {
@@ -1134,7 +1168,7 @@ export class FSController extends PuterController {
child.suggestedApps = rootSuggestions[index] ?? [];
}
}
const rootItems = await this.#toReaddirEntries(rootChildren);
const rootItems = await this.#toReaddirEntries(actor, rootChildren);
if (paginated) {
res.json({
items: rootItems,
@@ -1207,7 +1241,7 @@ export class FSController extends PuterController {
)
: undefined;
res.json({
items: await this.#toReaddirEntries(page.entries),
items: await this.#toReaddirEntries(actor, page.entries),
...(page.cursor ? { cursor: page.cursor } : {}),
...(total !== undefined ? { total } : {}),
});
@@ -1227,7 +1261,7 @@ export class FSController extends PuterController {
? await this.services.fs.countDirectory(parent.uuid)
: undefined;
res.json({
items: await this.#toReaddirEntries(page.entries),
items: await this.#toReaddirEntries(actor, page.entries),
...(page.cursor ? { cursor: page.cursor } : {}),
...(total !== undefined ? { total } : {}),
});
@@ -1241,7 +1275,7 @@ export class FSController extends PuterController {
sortOrder,
});
await this.#attachSuggestedApps(children);
res.json(await this.#toReaddirEntries(children));
res.json(await this.#toReaddirEntries(actor, children));
}
/**
@@ -1249,14 +1283,20 @@ export class FSController extends PuterController {
* the three fields the SDK cannot reconstruct on its own so it can rebuild
* the v1 shape: `type` (MIME), a signed `thumbnail`, and `associatedApp`.
*/
async #toReaddirEntries(entries: FSEntry[]): Promise<ClientReaddirEntry[]> {
const appsById = await loadLegacyAssociatedApps(
this.stores.app,
entries,
);
async #toReaddirEntries(
actor: Actor,
entries: FSEntry[],
): Promise<ClientReaddirEntry[]> {
const [appsById, shareFlags] = await Promise.all([
loadLegacyAssociatedApps(this.stores.app, entries),
this.services.share.shareFlags(actor, entries),
]);
return Promise.all(
entries.map(async (entry) => ({
...this.#toClientEntry(entry),
...this.#toClientEntry(
entry,
shareFlags.get(entry.uuid) ?? null,
),
// Fields the client cannot derive on its own.
type: fsEntryMimeType(entry),
thumbnail: await signEntryThumbnail(
@@ -1320,6 +1360,7 @@ export class FSController extends PuterController {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
allowAccessToken: true,
rateLimit: FS_READ_LIMIT,
concurrent: FS_READ_CONCURRENT,
})
@@ -2220,33 +2261,7 @@ export class FSController extends PuterController {
}
#normalizePath(path: string, username?: string): string {
const trimmedPath = path.trim();
if (trimmedPath.length === 0) {
throw new HttpError(400, 'Path cannot be empty', {
legacyCode: 'bad_request',
});
}
let pathToNormalize = trimmedPath;
if (pathToNormalize === '~' || pathToNormalize.startsWith('~/')) {
if (!username) {
throw new HttpError(400, 'Unable to resolve home path', {
legacyCode: 'bad_request',
});
}
pathToNormalize = `/${username}${pathToNormalize.slice(1)}`;
}
assertNormalized(pathToNormalize);
let normalizedPath = pathToNormalize;
if (!normalizedPath.startsWith('/')) {
normalizedPath = `/${normalizedPath}`;
}
if (normalizedPath.length > 1 && normalizedPath.endsWith('/')) {
normalizedPath = normalizedPath.slice(0, -1);
}
return normalizedPath;
return normalizeAbsolutePath(expandTildePath(path, username));
}
#expandFileMetadataPath(
@@ -2342,6 +2357,20 @@ export class FSController extends PuterController {
return guiMetadata;
}
/**
* The write handlers build on `req.body` being an object. A request whose
* body never parsed leaves it undefined, and the first field read after
* that is a 500 where the request deserves a 400.
*/
#requireObjectBody<T>(body: T | undefined): T {
if (!body || typeof body !== 'object' || Array.isArray(body)) {
throw new HttpError(400, 'A request body is required', {
legacyCode: 'bad_request',
});
}
return body;
}
#withGuiMetadata<T extends { guiMetadata?: WriteGuiMetadata }>(
value: T,
fallbackSource: unknown,
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import type { Request, Response } from 'express';
@@ -542,6 +543,74 @@ describe('FSController.write', () => {
// false makes the ceiling free disk space), so this group runs its own
// server with the limit switched on.
describe('write handlers with no parsable body', () => {
// A request whose body never parsed leaves `req.body` undefined, and the
// handlers read fileMetadata straight off it.
it.each([
[
'write',
(c: typeof controller, r: Request, res: Response) =>
c.write(r as never, res as never),
],
[
'startWrite',
(c: typeof controller, r: Request, res: Response) =>
c.startWrite(r as never, res as never),
],
[
'completeWrite',
(c: typeof controller, r: Request, res: Response) =>
c.completeWrite(r as never, res as never),
],
])('%s answers 400', async (_label, call) => {
const { actor } = await makeUser();
const req = makeReq({ actor });
(req as { body?: unknown }).body = undefined;
const { res } = makeRes();
await expect(
withActor(actor, () => call(controller, req, res)),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
});
});
});
describe('batch write handlers with a malformed element', () => {
// The body is an array, so the array check passes; the elements are what
// the handlers then read fileMetadata / thumbnailData off.
it.each([
[
'startBatchWrites',
(c: typeof controller, r: Request, res: Response) =>
c.startBatchWrites(r as never, res as never),
],
[
'batchWrites',
(c: typeof controller, r: Request, res: Response) =>
c.batchWrites(r as never, res as never),
],
[
'completeBatchWrites',
(c: typeof controller, r: Request, res: Response) =>
c.completeBatchWrites(r as never, res as never),
],
])('%s answers 400 for a null element', async (_label, call) => {
const { actor } = await makeUser();
const req = makeReq({ actor });
(req as { body?: unknown }).body = [null];
const { res } = makeRes();
await expect(
withActor(actor, () => call(controller, req, res)),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
});
});
});
describe('FSController.write storage allowance', () => {
let limitedServer: PuterServer;
let limitedController: FSController;
@@ -331,6 +331,55 @@ describe('LegacyFSController inline routes', () => {
expect(entry).not.toHaveProperty('owner_user_id');
});
it('pairs a recent app icon with a direct subdomain URL at the asked size', async () => {
const { actor, userId } = await makeUser();
const app = (await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; name: string }>
)(
{
name: `recent-${uuidv4()}`,
title: 'Recent App',
index_url: 'https://recent.example.test/',
// An http(s) icon column is what the resize pipeline leaves
// behind, so the sized PNG is on the icons subdomain.
icon: 'https://api.puter.com/app-icon/x',
},
{ ownerUserId: userId },
))!;
await server.clients.db.write(
'INSERT INTO `app_opens` (`app_uid`, `user_id`, `ts`) VALUES (?, ?, ?)',
[app.uid, userId, Math.floor(Date.now() / 1000)],
);
const runWithIconSize = async (query: Record<string, unknown>) => {
const { res, captured } = makeRes();
await routeHandler('get', '/get-launch-apps')(
makeReq({ actor, query }),
res,
(() => {}) as never,
);
const body = captured.body as {
recent: Array<Record<string, unknown>>;
};
return body.recent.find((item) => item.uuid === app.uid)!;
};
const { protocol, static_hosting_domain } = controller.config;
const base = `${protocol}://puter-app-icons.${static_hosting_domain}`;
expect((await runWithIconSize({ icon_size: '64' })).iconCdnUrl).toBe(
`${base}/${app.uid}-64.png`,
);
// An unusable size falls back to the default rather than pointing at a
// file the pipeline never wrote.
expect((await runWithIconSize({ icon_size: '65' })).iconCdnUrl).toBe(
`${base}/${app.uid}-256.png`,
);
});
it('reports a zero cache timestamp when unauthenticated', async () => {
const { res, captured } = makeRes();
await routeHandler('get', '/cache/last-change-timestamp')(
@@ -373,6 +373,47 @@ describe('LegacyFSController.stat', () => {
expect(body.size).toBe(0);
});
it('reports the share flag, and the recipients when asked', async () => {
const { actor } = await makeUser();
const recipient = await makeUser();
const username = actor.user!.username!;
const path = `/${username}/Documents/shared-folder`;
await withActor(actor, () =>
controller.mkdir(makeReq({ body: { path }, actor }), makeRes().res),
);
const before = makeRes();
await withActor(actor, () =>
controller.stat(makeReq({ body: { path }, actor }), before.res),
);
expect(before.captured.body).toMatchObject({ is_shared: false });
await withActor(actor, () =>
server.services.share.share(actor, {
path,
recipient: { username: recipient.actor.user!.username! },
mode: 'read',
}),
);
const after = makeRes();
await withActor(actor, () =>
controller.stat(
makeReq({ body: { path, return_shares: true }, actor }),
after.res,
),
);
const body = after.captured.body as Record<string, unknown>;
expect(body.is_shared).toBe(true);
expect(body.shares).toMatchObject([
{
holder: recipient.actor.user!.username,
issuer: username,
mode: 'read',
},
]);
});
it('throws 401 when the request has no actor', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
@@ -650,6 +691,44 @@ describe('LegacyFSController.readdir', () => {
expect(names).toContain('beta');
});
it('flags a shared child in the listing', async () => {
const { actor } = await makeUser();
const recipient = await makeUser();
const username = actor.user!.username!;
const parent = `/${username}/Documents/flagged`;
for (const name of ['', '/shared', '/private']) {
await withActor(actor, () =>
controller.mkdir(
makeReq({ body: { path: `${parent}${name}` }, actor }),
makeRes().res,
),
);
}
await withActor(actor, () =>
server.services.share.share(actor, {
path: `${parent}/shared`,
recipient: { username: recipient.actor.user!.username! },
mode: 'read',
}),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdir(makeReq({ body: { path: parent }, actor }), res),
);
const entries = captured.body as Array<{
name: string;
is_shared: boolean | null;
}>;
expect(new Map(entries.map((e) => [e.name, e.is_shared]))).toEqual(
new Map([
['shared', true],
['private', false],
]),
);
});
it('returns the root listing when path = "/"', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
@@ -41,12 +41,15 @@ import {
NON_OWNER_SIGNATURE_TTL_SECONDS,
verifySignature,
} from '../../util/fileSigning.js';
import { APP_ICON_SIZES, getAppIconCdnUrl } from '../../util/appIcon.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import { maskEntryPath } from '../../services/fs/sharePathMask.js';
import {
buildHostedBackingDenial,
hostedIndexUrlBackingIsUnavailable,
} from '../../util/hostedAppBacking.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { listClientShares } from '../share/clientShare.js';
import { PuterController } from '../types.js';
import {
FS_BATCH_CONCURRENT,
@@ -141,14 +144,16 @@ export class LegacyFSController extends PuterController {
// draw from the same per-user budget as their v2 counterparts rather
// than handing a caller a second allowance for the same operation.
const mutate = { ...apiOptions, rateLimit: FS_MUTATE_LIMIT };
router.post(
'/stat',
{ ...apiOptions, rateLimit: FS_STAT_LIMIT },
this.stat,
);
// Read-side routes admit scoped access tokens: the ACL check each
// handler runs already intersects a token's grant with its issuer's,
// so the token reaches exactly what it was minted for. That is what a
// background events handler acts through — its `user` is a token
// scoped to the subscription's grant.
const reads = { ...apiOptions, allowAccessToken: true } as RouteOptions;
router.post('/stat', { ...reads, rateLimit: FS_STAT_LIMIT }, this.stat);
router.post(
'/readdir',
{ ...apiOptions, rateLimit: FS_READDIR_LIMIT },
{ ...reads, rateLimit: FS_READDIR_LIMIT },
this.readdir,
);
router.post('/mkdir', mutate, this.mkdir);
@@ -170,6 +175,7 @@ export class LegacyFSController extends PuterController {
'/read',
{
...spends,
allowAccessToken: true,
rateLimit: FS_READ_LIMIT,
concurrent: FS_READ_CONCURRENT,
},
@@ -291,6 +297,14 @@ export class LegacyFSController extends PuterController {
? await recommendedSvc.getRecommendedApps()
: [];
// The direct icon URL names a size; honour the one the
// caller asked for so the client isn't handed a 256px PNG for
// a 64px slot. `icon` keeps the raw column either way.
const requestedIconSize = Number(req.query.icon_size);
const iconSize = APP_ICON_SIZES.includes(requestedIconSize)
? requestedIconSize
: undefined;
let recent: unknown[] = [];
const userId = req.actor?.user?.id;
if (userId) {
@@ -344,6 +358,13 @@ export class LegacyFSController extends PuterController {
name: app.name,
title: app.title,
icon: app.icon ?? null,
// Direct subdomain URL for the client to try
// before `icon`.
iconCdnUrl: getAppIconCdnUrl(
app,
this.config,
iconSize,
),
godmode: Boolean(app.godmode),
maximize_on_start: Boolean(app.maximize_on_start),
index_url: backingGone ? null : app.index_url,
@@ -437,12 +458,12 @@ export class LegacyFSController extends PuterController {
'see',
);
entry.suggestedApps =
await this.services.suggestedApps.getSuggestedApps(entry);
const appsById = await loadLegacyAssociatedApps(this.stores.app, [
entry,
const [suggestedApps, appsById, shareFlags] = await Promise.all([
this.services.suggestedApps.getSuggestedApps(entry),
loadLegacyAssociatedApps(this.stores.app, [entry]),
this.services.share.shareFlags(actor, [entry]),
]);
entry.suggestedApps = suggestedApps;
const shaped = await toLegacyEntry(this.clients.event, entry, {
fsEntryStore: this.stores.fsEntry,
@@ -452,6 +473,7 @@ export class LegacyFSController extends PuterController {
) => Promise<Record<string, unknown> | null>;
},
appsById,
isShared: shareFlags.get(entry.uuid) ?? null,
});
// Optional hydrations:
@@ -461,13 +483,17 @@ export class LegacyFSController extends PuterController {
entry.path,
);
}
// Legacy clients sometimes ask for `return_versions`, `return_shares`.
// We don't have parity for these yet — return empty arrays to avoid
// breaking `response.x.forEach(...)` patterns. `return_owner` is a
// no-op flag here: the `owner` field is already populated by
// `toLegacyEntry` as `{ username }`.
// `return_versions` has no parity yet; the empty array keeps `forEach`
// callers working. `return_owner` is a no-op — `owner` is always set.
if (getBoolean(body, 'return_versions')) shaped.versions = [];
if (getBoolean(body, 'return_shares')) shaped.shares = [];
if (getBoolean(body, 'return_shares')) {
shaped.shares = await listClientShares(
this.services.share,
this.clients.event,
actor,
entry.uuid,
);
}
res.json(shaped);
};
@@ -500,14 +526,15 @@ export class LegacyFSController extends PuterController {
child.suggestedApps = rootSuggestions[index] ?? [];
}
}
const rootAppsById = await loadLegacyAssociatedApps(
this.stores.app,
rootChildren,
);
const [rootAppsById, rootShareFlags] = await Promise.all([
loadLegacyAssociatedApps(this.stores.app, rootChildren),
this.services.share.shareFlags(actor, rootChildren),
]);
const shaped = await Promise.all(
rootChildren.map((c) =>
toLegacyEntry(this.clients.event, c, {
appsById: rootAppsById,
isShared: rootShareFlags.get(c.uuid) ?? null,
}),
),
);
@@ -581,14 +608,17 @@ export class LegacyFSController extends PuterController {
}
}
const appsById = await loadLegacyAssociatedApps(
this.stores.app,
children,
);
const [appsById, shareFlags] = await Promise.all([
loadLegacyAssociatedApps(this.stores.app, children),
this.services.share.shareFlags(actor, children),
]);
const shaped = await Promise.all(
children.map((c) =>
toLegacyEntry(this.clients.event, c, { appsById }),
toLegacyEntry(this.clients.event, c, {
appsById,
isShared: shareFlags.get(c.uuid) ?? null,
}),
),
);
@@ -628,10 +658,7 @@ export class LegacyFSController extends PuterController {
typeof body.parent === 'string' &&
(body.parent.startsWith('/') || body.parent.startsWith('~'))
) {
parentPath = this.#expandTilde(
body.parent,
actor.user?.username,
);
parentPath = expandTildePath(body.parent, actor.user?.username);
} else {
const parent = await resolveV1Selector(
this.stores.fsEntry,
@@ -832,7 +859,9 @@ export class LegacyFSController extends PuterController {
// Trash, and `null`/`{}` when restoring. See
// `src/gui/src/helpers.js` → `window.move_items`.
newMetadata: (body.new_metadata ?? undefined) as
Record<string, unknown> | null | undefined,
| Record<string, unknown>
| null
| undefined,
});
const oldPath = source.path;
await this.#emitGuiEvent('outer.gui.item.moved', moved, {
@@ -1279,7 +1308,8 @@ export class LegacyFSController extends PuterController {
}
type SignedOrEmpty =
(SignedFile & { path?: string }) | Record<string, never>;
| (SignedFile & { path?: string })
| Record<string, never>;
const result: { signatures: SignedOrEmpty[]; token?: string } = {
signatures: [],
};
@@ -1900,7 +1930,10 @@ export class LegacyFSController extends PuterController {
const subjectRef = body.subject;
const appRef = body.app;
const mode = (getString(body, 'mode') ?? 'read') as
'see' | 'list' | 'read' | 'write';
| 'see'
| 'list'
| 'read'
| 'write';
if (!subjectRef || !appRef)
throw new HttpError(400, '`subject` and `app` are required', {
legacyCode: 'bad_request',
@@ -2192,7 +2225,7 @@ export class LegacyFSController extends PuterController {
});
}
const parentPath = getString(record, 'path') ?? '';
const expandedParent = this.#expandTilde(
const expandedParent = expandTildePath(
parentPath,
username,
);
@@ -2257,7 +2290,7 @@ export class LegacyFSController extends PuterController {
legacyCode: 'bad_request',
});
}
const expandedParent = this.#expandTilde(
const expandedParent = expandTildePath(
parentPath,
username,
);
@@ -2310,7 +2343,7 @@ export class LegacyFSController extends PuterController {
this.stores.fsEntry,
{ uid: shortcutToUid },
);
const expandedParent = this.#expandTilde(
const expandedParent = expandTildePath(
parentPath,
username,
);
@@ -2544,16 +2577,6 @@ export class LegacyFSController extends PuterController {
return [];
}
#expandTilde(path: string, username: string | undefined): string {
if (!path) return path;
if (path !== '~' && !path.startsWith('~/')) return path;
if (!username)
throw new HttpError(400, 'Unable to resolve home path', {
legacyCode: 'bad_request',
});
return `/${username}${path.slice(1)}`;
}
#serializeBatchError(err: unknown): Record<string, unknown> {
if (err instanceof HttpError) {
const payload: Record<string, unknown> = {
@@ -449,6 +449,8 @@ export async function toLegacyEntry(
getById: (id: number) => Promise<Record<string, unknown> | null>;
};
appsById?: Map<number, Record<string, unknown>>;
/** Omitted from the response when undefined. */
isShared?: boolean | null;
} = {},
): Promise<Record<string, unknown>> {
// Someone else's entry is published under its masked path; the owner's
@@ -496,6 +498,7 @@ export async function toLegacyEntry(
? (opts.appsById.get(entry.associatedAppId) ?? null)
: null,
appdata_app,
...(opts.isShared === undefined ? {} : { is_shared: opts.isShared }),
};
// `is_empty` — only meaningful for directories. Single-row probe so we
+26
View File
@@ -243,3 +243,29 @@ export const DAV_CONCURRENT: NonNullable<RouteOptions['concurrent']> = {
limit: 10,
key: 'fingerprint',
};
/**
* Failed HTTP Basic verifications on the DAV host.
*
* `DAV_LIMIT` counts requests, and a working DAV client resends its credentials
* on every one of them, so it cannot double as a credential ceiling — these
* count only verifications that failed, which a working client never produces.
* The gate reads them before the bcrypt compare, so an exhausted bucket costs
* nothing to serve.
*
* Two axes, sized like `/login`'s, and for the same reasons: per-account so one
* account can't be ground down from many networks, plus a coarser per-IP
* backstop, because the network fingerprint `DAV_LIMIT` buckets on rotates with
* client-controlled headers.
*/
export const DAV_AUTH_FAILURE_LIMIT: RouteRateLimit = {
scope: 'dav-auth',
limit: 10,
window: 15 * 60_000,
};
export const DAV_AUTH_FAILURE_IP_LIMIT: RouteRateLimit = {
scope: 'dav-auth-ip',
limit: 50,
window: 15 * 60_000,
key: 'ip',
};
@@ -177,6 +177,7 @@ export interface ClientFSEntry {
workers: FSEntrySubdomain[];
hasWebsite: boolean;
suggestedApps: unknown[];
isShared?: boolean | null;
id?: never;
userId?: never;
@@ -278,6 +278,47 @@ describe('HomepageController GET /app/:name', () => {
expect(html).toContain('Cool App');
});
it('caps description, og:description and twitter:description at 150 chars', async () => {
const { userId } = await makeUser();
const name = `long-${Math.random().toString(36).slice(2, 10)}`;
const longDescription =
'This drawing app has brushes, layers, filters and more. '.repeat(
8,
);
await server.stores.app.create(
{
name,
title: 'Wordy App',
description: longDescription,
index_url: `https://example.com/${name}/`,
},
{ ownerUserId: userId },
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/app/:name',
makeReq({ params: { name }, path: `/app/${name}` }),
res,
);
const html = String(captured.body);
for (const tag of [
'name="description"',
'property="og:description"',
'name="twitter:description"',
]) {
const match = new RegExp(`${tag} content="([^"]*)"`).exec(html);
expect(match, `no ${tag} tag`).toBeTruthy();
const content = match![1];
expect(content.length).toBeLessThanOrEqual(150);
expect(content.endsWith('…')).toBe(true);
// Cut at a word boundary, not mid-word.
expect(longDescription).toContain(content.slice(0, -1));
}
});
it('does not leak a private app index_url or owner id to an anonymous visitor', async () => {
const { userId } = await makeUser();
const name = `priv-${Math.random().toString(36).slice(2, 10)}`;
Loaded 100 of 821 files, more files were not shown because too many files have changed in this diff. Show more