Compare commits

...
Author SHA1 Message Date
Daniel Salazar 8c31f9177c ci: the AI provider workflow runs only the provider tests and passes every provider's key 2026-11-13 20:31:22 -07:00
Daniel Salazar 5531f381f5 docs(ai): the image reference and quotas page cover Replicate's safety reruns 2026-11-13 20:31:22 -07:00
Daniel Salazar 8a198b6f0a style: restore the repo formatting in files the AI stack reflowed 2026-11-13 20:31:22 -07:00
Daniel Salazar 17dc879ff5 fix(ai): the native Gemini request type accepts the xhigh effort Hydrogen takes 2026-11-13 20:31:22 -07:00
Daniel Salazar d3321b1d0b fix(ai): a model id with a colon after its vendor keeps its name when it is qualified 2026-11-13 20:31:21 -07:00
Daniel Salazar a52f32e0d2 test(ai): a fuzz case keeps its input and route when the catalog changes, and a suite that reaches no provider fails 2026-11-13 20:31:21 -07:00
Daniel Salazar 086e3a0028 fix(ai): OpenAI speech to text answers 400 to a prompt that is a list or an object 2026-11-13 20:31:21 -07:00
Daniel Salazar aabbafdaab fix(ai): the image type hint only has to be a string when a request carries an input image 2026-11-13 20:31:21 -07:00
404oops 97ab0ee12b Forward Claude inference speed from SDK chat options 2026-11-13 17:12:31 +01:00
404oops 358b680b69 Qualify published chat model names while preserving routing aliases 2026-11-13 14:07:30 +01:00
404oops e534a3d09a Merge branch 'feat/gemini-carbon-kling-4' into filipkujundzic/put-2351-backend-deprecated-models-check-2026 2026-11-13 13:53:25 +01:00
404oops 5fd07e9ed7 Cap Mistral Large 4 output at the 262,144 tokens Mistral accepts
The catalog listed `mistral-large-4` with a 1,000,000-token output
limit, equal to its context window. Mistral rejects a request for more
than 262,144 completion tokens, so a request that left `max_tokens`
unset, or set it higher, failed with a 400 whenever the account's
credit covered more than 262,144 output tokens.

- `max_tokens` for `mistral-large-4` is 262,144. The context window
  stays at 1,000,000 tokens.
- A test sends `mistral-large-4` and `mistral-large-latest` through the
  chat driver with ample credit and checks the cap that reaches
  Mistral.
2026-11-13 13:51:47 +01:00
404oops e138f1e9d8 Move the reasoning_effort model list out of the chat options table
The `reasoning_effort` row in chat.md listed the models that read the
option. Model-specific details belong under Model-specific features,
which already gives each family's accepted values.

- The row no longer names models.
- The Gemini and Moonshot Kimi sections say that their other models
  ignore the option.
2026-11-13 13:44:06 +01:00
404oops 7886907bdf Add Gemini 4 Hydrogen and cap unary Gemini 4 Carbon output
Google released `gemini-4-hydrogen` on 2026-11-12. It uses the same
native request shape as Carbon and Argon.

- Hydrogen is served natively at $1.50 input, $7.50 output and $0.075
  cached input per million tokens, with fast service at 1.8 times
  those rates. It accepts `low`, `medium`, `high` and `xhigh` reasoning
  effort, ignores temperature, and takes text and images.
- A Carbon response that isn't streamed is capped at 32,768 output
  tokens, as Argon's already is, and the credit hold is sized to that
  cap. Google reports this limit for both models. The cap is now a
  field of the native model options instead of a check on Argon's id.
- chat.md and rate-limits-and-quotas.md list Hydrogen's reasoning
  levels and limits and Carbon's unary cap.
2026-11-13 11:31:10 +01:00
404oops 0eec08df98 Point Mistral's floating Large aliases at Large 4 and apply its GA price
Mistral made Large 4 generally available on 2026-11-05, moved
`mistral-large-latest` to it, and deprecated Large 3 and Medium 3.5,
which both retire on 2027-04-30.

- `mistral-large-latest`, `mistral-large` and
  `mistralai/mistral-large-latest` resolve to `mistral-large-4`. They
  resolved to `mistral-large-2512`, which stays callable by its own id
  until Mistral retires it.
- Large 4 uses Mistral's current rates: $0.70 input, $0.07 cached input
  and $2.10 output per million tokens, in place of the preview rates.
  It is marked open-weight.
- The Large 3 and Medium 3.5 entries carry their retirement date.
2026-11-13 11:30:34 +01:00
404oops 4b5d6e034c Document the Kling 4 prompt limit for non-English text
Kling 4.0 and 4.0 Flash take 26,000 characters of English but 11,000
of other languages, with mixed text sharing the limit in proportion.
The video page and the quotas page said 26,000 for every prompt.

- txt2vid.md and rate-limits-and-quotas.md give both limits, the rule
  for mixed text, and the error each one returns.
- Puter still rejects only a prompt over 26,000 characters. Kling
  enforces the lower limit, and its refusal reaches the caller as
  `upstream_bad_request` with nothing charged. A test covers that path.
2026-11-13 11:00:50 +01:00
404oops 7d53cc7ed6 Include Kling 4 models in the video driver catalog test 2026-11-13 07:53:16 +01:00
404oops 86f871c9a8 Add Gemini 4 Carbon and Kling 4 video models 2026-11-13 07:53:16 +01:00
404oops c038a6983f Hold the AI concurrency slot until a detached chat completion settles
A chat completion keeps running for up to 10 minutes after the caller
hangs up, so the upstream can report usage. It now counts against the
caller's concurrency cap for as long as the upstream runs, instead of
until the response closes.

- The chat driver registers its upstream run with the request's
  concurrency slot, so `/drivers/call` keeps the slot until the
  upstream is done. The cleanup after it does not hold the slot.
- The route concurrency gate now uses the same CallSlot as
  `/drivers/call`: the slot is held until the response is done and the
  work the handler registered has settled. The OpenAI- and
  Anthropic-compatible chat routes register the completion, so a
  non-streaming call counts until the upstream answers too.
- CallSlot moves to core/http, since the route gate below the
  controllers uses it. The context key is renamed keepConcurrencySlot.
- rate-limits-and-quotas.md says a completion counts until the model
  finishes it, even after a disconnect.
2026-11-13 07:53:16 +01:00
404oops f044d8d071 Restore the CODEOWNERS default owner and wildcard patterns
The default owner line read `- @Salazareo`, which only matches a file
named `-`, so files without a specific owner had none. Three patterns
had their `*` escaped, which GitHub rejects as invalid, so their lines
assigned no owner.
2026-11-13 07:52:52 +01:00
404oops aea6e662f6 Fuzz the AI drivers with seeded requests and upstream answers
Two seeded suites send malformed input through the real drivers and
providers. The vendors are played at the HTTP boundary, so each
provider's request and response code runs.

- ChatCompletionDriver.fuzz.test.ts covers the 17 chat providers that
  have a static catalog. Requests with a few values changed must
  resolve or reject below 500 without marking a route unhealthy.
  Valid requests against wrong statuses, bodies of the wrong shape and
  broken event streams must end with an answer or a classified error.
  A stream only has to end; the error line it carries is not checked.
- mediaDrivers.fuzz.test.ts covers the image, video, speech and OCR
  drivers, called as a user with unlimited metering so a request gets
  past file input and the credit hold. Every vendor refuses the
  request, and every call must end with a classified error.

shared/fuzzTestUtil.ts holds the seeded random source, the hostile JSON
values, the mutation, and the stand-in for fetch, undici and axios. An
SDK that calls https.request itself fails the suite. PUTER_FUZZ_CASES
raises the case count for a longer run. doc/ai-drivers.md describes the
suites.
2026-11-13 07:52:52 +01:00
404oops 31f6dc2d37 Answer malformed AI requests with a 400 where they enter
A value that used to end in an unclassified error, in a request or in
an upstream answer, now gets a classified one before provider code runs
on it. A value a driver or provider ignored or refused before is
handled as it was.

Every driver:
- shared/scalarOptions.ts answers 400 for a value that cannot be
  converted to text, such as { "toString": null }, in the options a
  driver reads as text, a number or a flag. The chat, video, text to
  speech, speech to text, voice changer and OCR drivers call it first.
- An unknown provider name that is not text gets the same 400 as any
  other unknown name.

Chat requests:
- A falsy `tools` means no tools on every provider.
- A tool call id or a content block type that cannot be converted to
  text is a 400.
- A tool result that is an object is sent as its JSON, like the content
  of a `tool` message.
- `include` that is not an array is a 400.
- `anthropicBetas` that is not a list of names is dropped.
- Gemini's native route answers 400 when no user or assistant message
  is left to send.
- The prompt estimate reads a compaction block whose content is not a
  string.

Image, video, speech and OCR requests:
- The image driver checks `input_image_mime_type` with `quality` and
  `resolution`.
- The video driver answers 400 for a model that is not a string.
- The voice changer answers 400 for an `output_format` that is not a
  string.
- Mistral OCR answers 400 when the SDK refuses the options before
  sending them, as Mistral chat does.

Upstream answers to chat:
- A choice without a message is the existing 400 bad_response, which
  still falls back.
- A usage count that is not a number is reported as zero in the cost
  event, as pricing already skips it.
2026-11-13 07:52:52 +01:00
404oops 2b03bc53b1 Rerun Replicate predictions an output safety checker rejects
Some community models check the generated image and fail the
prediction with "Try running it again" for a share of seeds, whatever
the prompt. adirik/realvisxl-v3.0-turbo failed 5 of 24 runs of "a small
red cube on a white table" that way.

A prediction that fails with that message is now run again, up to two
more times, inside the same deadline and credit hold. A request that
pins a seed is not rerun, because the same seed is rejected again.
Only the prediction that succeeds is metered, as before.
2026-11-13 07:52:52 +01:00
404oops 3ea0010220 Remove the unused ImageProvider base class
Every image provider implements IImageProvider directly; nothing extended
or imported the abstract class in interfaces/image/ImageProvider.ts.

The guide's import rules now say that only an interface's driver file
imports providers, which holds since the tool helpers stopped importing
the Claude policy.
2026-11-13 07:52:52 +01:00
404oops 395e4bd54a Split the tool and compaction helpers by dialect
FunctionCalling.js and compaction.js each held Puter-facing parsing next
to OpenAI and Anthropic request shapes. Each dialect now owns its mapping:

  interfaces/chat/utils     normalize_tools_object, readCompaction and the
                            other dialect-neutral checks
  protocols/openaiCompatible/openaiParams.ts
                            toolsToWire (was make_openai_tools) and
                            toOpenAiContextManagement
  providers/claude/chat/contextManagement.ts
                            toAnthropicContextManagement

toolsToWire takes the dialect as its second argument, like the other
mappers in that module, and takes an array; every caller already guarded
the empty case.

make_claude_tools is removed. Nothing but its tests called it: the Claude
provider uses claudeToolPolicy directly. The three cases only those tests
had are added to the claudeToolPolicy tests. The chat interface helpers no
longer import from providers/claude.
2026-11-13 07:52:52 +01:00
404oops b55cb715ca Move the AI test helpers into the AI tree
drivers/integrationTestUtil.ts held helpers for AI provider tests, but a
store test and the realtime relay test imported from it too.

  drivers/ai/shared/integrationTestUtil.ts   the provider test helpers
  testEnv.ts                                 optionalEnv and skipUnlessEnv,
                                             for any env-gated suite
  testUtil.ts                                withAiCostFactorHook, next to
                                             the server it hooks

No test under services/ or stores/ imports from the driver layer's AI
code now. The typecheck baseline entry for the moved file is renamed, not
regenerated.
2026-11-13 07:52:52 +01:00
404oops 2e4d918900 Remove the aiUserIdentifier alias
shared/aiUserIdentifier.ts re-exported upstreamUserIdentifier under a
second name, and only its own test imported it. The alias is gone; its
tests, which cover truncation and actor shapes the other file didn't,
replace the four cases in upstreamIdentifier.test.ts.
2026-11-13 07:33:27 +01:00
404oops 8c799a433a Move the AI billing helpers the realtime relay uses below the driver layer
OpenAIRealtimeRelayService is a service, and it imported the cost factor,
the provider key reader and a transcription price from drivers/ai. Those
now live in src/backend/util, which both layers can import:

  util/aiCostFactor.ts    withAiCostFactor, withAiCreditHold and the
                          request cost helpers, unchanged
  util/aiProviderKey.ts   readProviderKey, cut from providerRegistry.ts
  util/aiPrices.ts        the gpt-transcribe per-second price, read by the
                          OpenAI speech-to-text cost table and the relay

withStreamedAiCharge returns a DriverStreamResult, so it stays in the
driver layer as drivers/ai/shared/streamedAiCharge.ts with its tests.

MeteringService is untouched. No prices or billing behavior change.
2026-11-13 07:33:27 +01:00
404oops 5f84d5897b Move the ElevenLabs voice changer into a provider class
VoiceChangerDriver built the ElevenLabs request, measured the converted
audio and metered it, and imported the output formats from the ElevenLabs
text-to-speech provider. That code is now in providers/elevenlabs:

  outputFormats.ts                         formats both speech interfaces use
  speechToSpeech/ElevenLabsSpeechToSpeechProvider.ts
  speechToSpeech/measuredOutput.ts         WAV, Opus and byte-rate timing
  speechToSpeech/costs.ts

The driver keeps provider resolution, test mode and the auth check. With
one provider it holds the class directly; there is no provider interface.

No behavior change: same validation order, error messages, usage keys,
holds and charges.
2026-11-13 07:33:27 +01:00
404oops c0aff857a4 Move the Textract and Mistral OCR calls into provider classes
OCRDriver held both vendors' clients, request building, response
normalizing and metering. That now lives in TextractOcrProvider
(providers/awsTextract/ocr) and MistralOcrProvider (providers/mistral/ocr),
both implementing IOcrProvider. The driver keeps the provider catalog,
model resolution, auth and credit gates, and input resolution.

The page estimate, document detection and credit hold both providers use
are in interfaces/ocr/common.ts.

No behavior change: same validation order, error messages, usage keys,
holds and charges.
2026-11-13 07:33:26 +01:00
404oops ff606abf3c Move catalogPrice into ai/shared
Chat providers and the Cloudflare and xAI image catalogs all parse vendor
prices with catalogPrice, so it lives in shared/ instead of the chat
interface's pricing helpers. The image catalogs no longer import from
the chat interface.
2026-11-13 07:33:26 +01:00
404oops 8159998830 Document the AI driver layout
doc/ai-drivers.md explains what interfaces, providers, protocols and
shared each own, the import rules between them, where a new file goes,
and the steps for adding a provider or an AI interface. It also lists
the three places that don't follow the layout yet.

doc/architecture.md and AGENTS.md link to it.
2026-11-13 07:33:26 +01:00
404oops 395f7f6adb Move the AI drivers into drivers/ai
The seven drivers/ai-* folders and the AI-only helpers in drivers/util
become one tree:

  drivers/ai/interfaces/<interface>/            the Puter-facing drivers
  drivers/ai/providers/<vendor>/<interface>/    vendor adapters, catalogs, tests
  drivers/ai/protocols/openaiCompatible/        the shared OpenAI wire protocol
  drivers/ai/shared/                            helpers several interfaces use

Files are moved and their relative imports rewritten; no implementation
changes. Driver names, aliases, interfaces, config keys, model ids and
usage keys are unchanged.

The chat catalog registration test scanned providers/*/ next to itself.
It now scans providers/*/chat/ and fails if the scan finds no catalog, so
a wrong directory can't pass it.

General helpers that non-AI code also uses stay in drivers/util: file
input, data URIs, upstream errors, stream piping, CRUD limits.
2026-11-13 07:33:26 +01:00
Juan Fernando CastroandDaniel Salazar 1c5909331c fix(fs): a path lookup of an old row returns its healed path while the replica catches up (#4612)
* fix(fs): a healed entry carries its written path, and an entry with no path answers 404

The lineage heal now returns the entry with the path it just wrote
instead of re-reading the row, so a replica that hasn't caught up can't
hand back the old NULL path. Entries that still have no path are kept
out of the cache, resolve to the usual 404 in the fs routes, and an ACL
check on a resource with no path refuses instead of throwing.

* fix(fs): heal paths along one parent chain, fill only empty paths, and return the healed row from the primary

The path lineage now picks the target row and walks up its own
parent_uid links, so each row in a healed path is the parent of the
next. The backfill only fills a NULL path and never replaces one that
is set. Both lineage routes return the row read back from the primary,
so every column is current. Path-less entries are kept out of the
cache by the cache writers themselves, and a missing or empty path is
unresolvable both in resolveOrNotFound and in ACLService.check.

* fix(fs): a path lineage resolves only to a row still at that path, and the backfill runs its updates concurrently

The lineage lookup now answers a miss when the row read back from the
primary sits at a different path than the one walked, and doesn't cache
it. The backfill runs its per-row updates through
runWithConcurrencyLimit and drops cached entries only for rows it
actually filled. ACLService.check refuses only a resource with no
string path, so an empty client path keeps its existing 400.

* fix(fs): build healed paths from the primary's ancestor chain, and invalidate the paths the heal wrote

Both lineage routes now read the ancestor chain from the primary and
write the path it spells, so a lagging replica's names never end up in
a path column. A path lookup whose replica walk no longer matches the
primary's chain is a miss and writes nothing. The backfill drops the
cached entries of the rows it filled in one call, keyed by the path it
wrote. The lazy-chain comparison now lets a missing path reach
ACLService.check on both sides.

* fix(fs): leave path-less uid and id lookups to resolveNode

resolveNode already answers a uid or id lookup whose entry has no path
as not found, and a path lookup always returns an entry with its path,
so resolveOrNotFound goes back to trusting what it resolves. The
readdir case for a broken parent chain is covered by the existing HTTP
test.

* fix(fs): the ancestor walk stops at a fixed depth and updates only rows with an empty path

The primary's ancestor walk now selects each row's path and hands only
the rows whose path is NULL to the backfill, so filled ancestors cost no
UPDATE. The walk stops after LINEAGE_MAX_DEPTH steps; a chain that has
not reached a row without a parent by then is treated as broken and the
row is left alone.

* fix(fs): a listing heals its path-less entries a bounded number at a time

* test(fs): a stored path the parent chain no longer spells heals from the primary, and a listing leaves a path-less entry uncached

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 22:24:37 -07:00
Juan Fernando CastroandDaniel Salazar 5d541cd8e2 fix(auth): a just-minted read URL or token is found while the replica catches up (#4556)
* fix(auth): a just-minted read URL or token is found while the replica catches up

* fix(auth): a token's grants aren't cached when it's revoked mid-mint

* fix(permissions): a mint succeeds when its grants cannot be read back to cache

The grants are already stored at that point, so a failed read-back drops
the cache key and the next read fills it.

* test(auth): assert the grant list cached at mint; skip the sqlite-copy replica test on postgres

* fix(auth): test that a token row is read from the primary only when both replica lookups miss

* fix(auth): one-line comments on the replica-then-primary token row lookup

* perf(auth): a mint caches the grants it stored without reading them back

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 22:22:42 -07:00
Juan Fernando CastroandDaniel Salazar fb526bfb7e perf(share): skip the ancestor read for owners with nothing shared in their own home (#4597)
* fix(share): look up a cross-owner move's origin by its former owner

The move listener now passes the event's fromUserId down to the origin
side of the fan-out, so the shares at the former location are looked up
under the owner they belong to, and re-marks the new owner as shared
before the fan-out runs instead of after it starts.

Neither changes what recipients hear today: the origin's ancestors
already carry the former owner, and the fan-out reads ancestors before
it consults the owner-shares cache. Both are needed once the fan-out
checks the entry's owner before reading ancestors. Tests cover a real
cross-owner move with a cold owner-shares cache, and a hand-over to an
owner with nothing shared.

* perf(share): skip the ancestor read when the entry's owner shares nothing

Every fs write, create, rename and move resolved the entry's ancestors
(a cache batch) before asking whether any owner on that chain had
anything shared. Ask about the entry's owner first, plus the former
owner for the origin side of a cross-owner move, and return no audience
without reading ancestors when none of them has a share. Otherwise the
lookup runs as before.

Self-contained so it can be reverted on its own. It assumes every entry
in a shared subtree carries the subtree owner's user id. Entries that
older writers created under someone else's shared folder with the
writer's own id would stop notifying that folder's recipients when the
writer has nothing shared, so this only holds once existing data has no
such mixed-owner subtrees.

* fix(share): read ancestors whenever an entry sits outside its owner's home

The early return in the share fan-out assumed every entry in a subtree
carries the subtree owner's id. Rows from older writers don't always:
an app writing into another user's AppData stamped itself as the owner,
so an entry in a shared folder can name an owner with nothing shared.

Skip the ancestor read only when the entry's owner has nothing shared
and the path sits in that owner's own home directory; anything under
another user's home takes the full lookup as before, whatever the data
holds. The owner's username comes from a per-process map (60s TTL,
10k entries) backed by the user store's cache, so a hit costs nothing
and a miss one user cache read, against the ancestor batch it replaces.

* refactor(share): drop the former-owner hint from the move fan-out

With the home-directory check, the origin side of a cross-owner move
sits under the former owner's home, not the entry's new owner's, so it
always takes the full ancestor lookup and finds the former owner's
shares through the chain. Passing fromUserId down and re-marking the new
owner before the fan-out no longer change anything, so both go back to
how they were. The test for a hand-over to an owner with nothing shared
stays; the slower duplicate that waited out the cache is removed.

* perf(share): keep the chain owner check after the ancestor read

The early return replaced the check that ran after the ancestor read,
so an entry outside its owner's home went on to the share-table lookup
even when nobody on its chain had anything shared. Ask about every owner
on the chain again once the ancestors are in hand, as before.

* fix(share): mark the new owner shared before a hand-over's fan-out

The fan-out now reads the owner-shares cache before it reads ancestors.
On a move into another user's home, a "nothing shared" answer cached
for the new owner shortly before the move made the destination side
skip the entry's re-pointed share, so its recipient was told the item
was removed rather than moved. Mark the new owner shared before the
fan-out starts whenever the move changed the owner.

The earlier "drop the former-owner hint" commit said this ordering no
longer mattered; it does for the destination side.

* perf(share): cache missing usernames and share in-flight lookups

The home check now uses isOwnHomePath with the owner's username. The
per-process username map also keeps a null answer for an owner with no
account or no username, and a burst of writes by one owner shares a
single user lookup, as the owner-shares lookups already do. A failed
lookup is not cached and reads as someone else's home, so the full
ancestor lookup runs. Tests cover an owner with no account, a home that
differs only in case and a renamed owner, each taking the full lookup.

* test(share): run the hand-over's revoke for real and assert the destination read

* perf(share): check the home path before asking whether the owner shares

An entry outside its owner's home goes straight to the full read, which asks about every owner on the chain together. The username cache is read once per call and the single-caller home helper is inlined.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 22:22:26 -07:00
Thibaud-VdbandDaniel Salazar 1abd25eb31 ci: take release-please from its current home, scope three workflow tokens and lock the actions (#4521)
* ci: take release-please from its current home

google-github-actions/release-please-action is archived: the action
moved to googleapis/release-please-action, same code, same inputs,
same major version, and that is where the fixes now land. The
release-please workflow references the maintained repository.

* ci: declare what the three unscoped workflows may do with the token

Docs Build, Maintain Release Merge PR and Notify HeyPuter had no
permissions block, so they ran with the repository's default token
scope. Docs Build only checks out and builds, Notify HeyPuter only
calls the dispatch endpoint with its own token: both declare
contents: read. Maintain Release Merge PR pushes the
auto-update/release branch and opens the release pull request, so it
declares contents: write and pull-requests: write. The other
workflows already declared their scopes and are left as they are.

* ci: stop persisting the checkout credentials in the jobs that never push

actions/checkout keeps the job token in the clone's git config, where
every later step can read it. Twelve checkouts now set
persist-credentials: false: the backend and puter.js test jobs, the
coverage jobs, the type checks, the docs build, the AI provider tests
and the Docker image build read the clone and push nothing with git.
The two checkouts of Maintain Release Merge PR keep the token: that
job resets the release branch and pushes it through
create-pull-request.

* ci: lock the workflow actions to the commits they resolve to

GitHub's workflow dependency lockfile holds the commit each action
reference resolves to, transitive actions included, and the runner uses
those commits even if a tag moves later. .github/workflows/actions.lock
comes from gh actions-lock (v0.4.1), which also narrowed every reference
to the exact release it locked, so Dependabot keeps bumping versions and
its security alerts keep matching. Re-run the same command after adding
or bumping an action.

* ci: drop the unused token scope from Notify HeyPuter, match workflow indentation

* ci: remove the disabled release workflows

release-please and Maintain Release Merge PR have been disabled since 2024; releases are cut from calver tags. Their entries leave the lockfile with them.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-13 03:59:06 +00:00
Daniel Salazar b5e4f20087 fix: KV size and credit edge cases, a metering double count, site name caching, /token-read content type (#4618)
* fix(fs): /token-read serves a file with no known extension as application/octet-stream

* fix(kv): a value too large once stored rejects with value_too_large, and a consistent read of several keys is consistent

* fix(kv): remove on an account with no credits left removes the paths and returns null

* fix(metering): a buffered increment whose base read fails is not also written through

* fix(hosting): site lookups by name share one cache entry for every spelling

* fix(metering): an increment whose base read fails is written through and taken back out of the buffer

* fix(hosting): create stores a site name trimmed and lowercased
2026-11-12 20:49:51 -07:00
Reynaldi Chernando bdb88507df throw error for get usage APIs (#4608) 2026-11-12 20:47:03 -07:00
Juan Fernando Castro 0694b76751 feat(email): sends name a stream, and streams add headers, a from address and message tags from config (#4520)
* feat(email): sends name a stream, and streams add headers, a from address and message tags from config

`email.streams` maps a stream name to optional headers, a from address and a
tags header. A send that names a configured stream gets that stream's
headers (caller headers win) and from address (when the caller gives none),
and its tags rendered as `name=value` pairs into the tags header. Unknown or
missing streams change nothing, and `streams` never reaches the transport.

Account mail is sent on `auth`, share and app feedback mail on `notify`, and
the signup confirmation on `signup` unless a `puter.signup.validate` listener
names another stream, which is remembered in the account metadata for
resends, deferred sends and save-account.

The signup response carries `confirmation_email_sent` when a confirmation
email was due at signup, and `/send-confirm-email` answers `{ sent }`.

PUT-2346

* fix(email): review fixes for stream naming

* fix(email): contact form mail names the notify stream, streams resolve from own config entries, the no-transport log shows the stream

Templated sends take only replyTo and stream; the header and tag options had no caller.

* fix(signup): a claimed row drops its earlier confirmation stream, and save-account reports a failed send under its own stage
2026-11-12 22:46:34 -05:00
Juan Fernando Castro e7ee778985 perf(notifications): notify and mark-ack skip their primary read-backs (#4547)
* perf(notifications): notify returns the row it wrote instead of reading it back

The service hands the driver the row the store built on insert, and a
group replacement builds the row it leaves from the one it looked up, so
neither path reads the row back from the primary.

* perf(notifications): mark-ack reads the row's fixed columns from a replica

The type, app uid and audience the dismissal event carries never change,
so they are read from a replica alongside the update, and from the
primary only when the replica has no row yet.

* test(notifications): cover a repeat dismissal and tidy notify docs

* fix(notifications): a repeat mark-ack returns without waiting for the replica read, and a failed replica read is logged

* refactor(notifications): NotificationRow spells out the row's columns, and a replacement passes the app uid it already has
2026-11-12 22:46:25 -05:00
Juan Fernando Castro c3ff8ca3b2 fix(permissions): an app opened on a file in a shared folder can read and save it (#4555)
* fix(permissions): an app opened on a file in a shared folder can read and save it

* perf(permissions): app scans check implied options before scanning the user

* test(permissions): cover app grants on folders and revoked shares

* fix(permissions): an app's folder grant counts only while its user holds that folder

The app branch of the ACL check counted a user-to-app grant on any folder above the file. It now counts a grant on the file itself, or on a folder above it that the user holds at the asked mode, so the grant and the user's own access meet on the same node.

* test(fs): the other-app case pins 404 beside the opened app's 200
2026-11-12 22:46:15 -05:00
Juan Fernando Castro 39d931306d fix(auth): sign-in relays a token set between waits (#4572)
* fix(auth): sign-in relays a token set between waits

* fix(auth): sign-in relay skips another app's stored token and survives a failed store

* fix(auth): sign-in relay always clears the stored copy and logs only the error message

* fix(auth): the sign-in relay keeps only signed app tokens, per app, and a subscribed wait answers without Redis

/login/set stores a token only when this server signed it for an app, under a
key that includes the app, so a later set for another app or an unsigned value
leaves a pending token in place. /login/wait takes the stored copy once and
falls back to listening when Redis cannot be read. A token that arrives by
event is delivered to every subscribed wait for that app; the stored copy is
then removed best-effort, and left in place when the wait's client has gone.
A relayed token for another app no longer ends the wait. The event payload is
back to { authtoken }.

* chore(events): event types file matches main again
2026-11-12 22:46:04 -05:00
Juan Fernando Castro 70e823c5af feat(email): notification emails ask listeners before each recipient (#4596)
* feat(email): notification emails ask listeners before each recipient

Share notification emails (account holders and invites) and the app
feedback email to the owner now emit `email.notify.validate` with
`{ to, kind, allow }` once per recipient address, next to the existing
unsubscribe and blocked-domain checks. A listener that sets `allow` to
false skips that recipient's email; the others are sent as before. The
invite check runs before its send budget is claimed and the feedback
check before the per-app cap slot, so a skipped address uses neither.
When no listener answers or the hook fails, the email is sent.

* refactor(email): type the notify kind as a template name, drop the dead catch

`emitAndWait` already catches a listener that throws, so the try/catch
around it never ran; a failed listener leaves `allow` true and the email
is sent. `kind` is now `EmailTemplateName`, so renaming a template breaks
the type check instead of the listener.

* fix(share): ask email.notify.validate when the digest is sent

A digest entry can wait out its window or a retried send, so the
listener is asked about the recipient right before the mail goes.
Invites are still asked once more before their budget is claimed.

* fix(email): any falsy allow on email.notify.validate skips the mail

Matches how email.validate reads its answer. The helper also takes the
event client as required, since every caller has one.

* feat(email): email.notify.validate carries every recipient of one kind of mail

The payload is { kind, recipients: [{ to, allow }] }. Share invites ask
once for all their addresses before any budget is claimed; the digest
send and the feedback mail ask with one entry. kind is narrowed to the
three templates that ask.
2026-11-12 22:45:56 -05:00
dependabot[bot] 1bc1953344 chore(deps): bump ws from 8.22.0 to 8.22.2 (#4567)
Bumps [ws](https://github.com/websockets/ws) from 8.22.0 to 8.22.2.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](https://github.com/websockets/ws/compare/8.22.0...8.22.2)

---
updated-dependencies:
- dependency-name: ws
  dependency-version: 8.22.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:35:51 -07:00
dependabot[bot] 0f83bb9b56 chore(deps): bump nodemailer from 10.1.4 to 10.1.7 (#4478)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 10.1.4 to 10.1.7.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v10.1.4...v10.1.7)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 10.1.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:15:05 -07:00
dependabot[bot] 1ae95d97e8 build(deps-dev): bump @babel/core from 7.29.7 to 7.29.13 (#4565)
Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.29.7 to 7.29.13.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.29.13/packages/babel-core)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-version: 7.29.13
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:13:34 -07:00
dependabot[bot] 31f8936da2 build(deps): bump undici from 7.30.1 to 7.30.2 (#4566)
Bumps [undici](https://github.com/nodejs/undici) from 7.30.1 to 7.30.2.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.30.1...v7.30.2)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.30.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:12:08 -07:00
dependabot[bot] aab559c9c1 chore(deps): bump zod from 4.4.3 to 4.6.7 (#4465)
Bumps [zod](https://github.com/colinhacks/zod) from 4.4.3 to 4.6.7.
- [Release notes](https://github.com/colinhacks/zod/releases)
- [Commits](https://github.com/colinhacks/zod/compare/v4.4.3...v4.6.7)

---
updated-dependencies:
- dependency-name: zod
  dependency-version: 4.6.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:08:45 -07:00
dependabot[bot] 7f1d46028a chore(deps): bump fastify from 5.12.5 to 5.12.7 (#4464)
Bumps [fastify](https://github.com/fastify/fastify) from 5.12.5 to 5.12.7.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/compare/v5.12.5...v5.12.7)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:08:27 -07:00
dependabot[bot] 39d9482409 chore(deps): bump body-parser from 2.3.0 to 2.4.0 (#4463)
Bumps [body-parser](https://github.com/expressjs/body-parser) from 2.3.0 to 2.4.0.
- [Release notes](https://github.com/expressjs/body-parser/releases)
- [Changelog](https://github.com/expressjs/body-parser/blob/master/HISTORY.md)
- [Commits](https://github.com/expressjs/body-parser/compare/v2.3.0...v2.4.0)

---
updated-dependencies:
- dependency-name: body-parser
  dependency-version: 2.4.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-12 20:08:06 -07:00
Juan Fernando CastroandDaniel Salazar 4a958b3ca1 fix(gui): desktop push status counts only Puter's devices and follows subscription changes (#4550)
* fix(gui): desktop push status counts only Puter's devices and follows subscription changes

- fix(gui): desktop push status counts only Puter's own devices
- fix(gui): the desktop keeps its push subscription when the browser replaces it
- docs(gui): shorten push comments to one line
- fix(gui): removed browsers stay off after a subscription change, and websites can reach Puter push
- docs(puter-js): fit the desktop push request comment on one line
- fix(gui): background push registration checks every earlier endpoint and waits out a 429
- fix(puter-js): inside the desktop, requestPermission leaves a held push grant to the desktop
- fix(gui): a browser that stays off after being removed forgets its receipt
- fix(gui): push status holds right after enable(), and a subscription change is picked up after a run in flight
- fix(gui): an app's push grant counts whenever the app list can't be read

* fix(puter.js): a held push grant is left to the desktop only when it says it answers push

The desktop lists `push-permission` in `puter.gui_features` on the app
iframe's URL. Without it the SDK settles a held `notifications:push` from
its own check, so a desktop whose requestPermission handler has no held
check shows no dialog for a grant the app already holds.

* fix(gui): a held push grant isn't asked again beside other access, and never counts without an app uid

requestPermission drops a push grant held for Puter's devices from the
dialog and asks only for what is missing. The app list lookup answers no
for a missing app uid, whatever the list says; an app the list leaves out
still keeps its grant, since the list is capped and can trail a grant.

* test(puter.js): e2e for a push grant the app already holds inside the desktop

A held grant resolves true with no dialog, and a request for it beside
access the app is missing shows only the missing access.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 20:06:23 -07:00
Juan Fernando Castro 4653ff6dae fix(gui): a removed browser stays off, and launches reuse the page's push status (#4549)
* fix(gui): a browser removed from push stays off until its user turns it on

The silent registration on load and app launch no longer replaces a
subscription the user removed in Puter. When the server answers
push_device_removed, the browser's subscription and its receipt are
dropped and the browser is recorded as turned off, so later loads leave
it alone. Turning push on from Settings or a push dialog still replaces
the removed subscription.

* perf(gui): launches reuse the page's last push status

The built-in push client keeps the last registered status it read in
memory, for the same user and subscription. App launches and an app's
puter.notifications.status() answer from it instead of listing devices
again. It is dropped on enable(), disable(), an unregistered answer and
the worker's subscription-changed message, and the Notifications tab
and push dialogs still read fresh, so a new page load always asks the
server.

* fix(notifications): a background enable opens no sign-in, and a page keeps its last push on for five minutes

_enableWebPush takes { background } in place of { renewRemoved }: a call
without a click leaves a removed browser removed and sends its requests
without interactive sign-in.

The desktop's push client keeps the on that enable() read, so the next
cached status lists no devices, and asks the server again once that on is
five minutes old.

* fix(gui): the kept push on belongs to the user whose devices were listed, and a failed enable() keeps none

A status read or enable() that another user's sign-in overlapped keeps no
on, and an enable() that fails drops what a read begun meanwhile kept.
2026-11-12 20:06:23 -07:00
Juan Fernando Castro a5c7b427b8 fix(puter-js): a confirmed push subscription registers for another account without a new confirmation (#4548)
* fix(puter-js): a confirmed push subscription registers for another account without a new confirmation

The server's verify and active subscribe answers can carry a receipt for
the subscription. It is kept per worker scope and sent with the next
announce of the same endpoint, so the Puter desktop registers the same
browser for the next signed-in account without another confirmation push.
The receipt is forgotten when the subscription is ended, replaced or
removed, and when the app is turned off in Puter. A server without
receipts behaves as before.

* docs(puter-js): shorten a push helper comment

* fix(notifications): keep receipts only in the Puter GUI and store the new one as soon as the server answers
2026-11-12 20:06:22 -07:00
Daniel Salazar cd3600bae9 fix(fs): overwrite a file in place when a write spells its name in another case or normalization form (#4620)
* fix(fs): overwrite a file in place when a write spells its name in another case or normalization form

* docs(fs): say a write needs overwrite off to get a new name beside a look-alike one
2026-11-12 20:02:39 -07:00
Juan Fernando Castro 2db55bd636 perf(share): read share owners from the entry owner column and backfill missing ones (#4598)
* perf(share): skip the legacy owner probe once a recorded share answers

ownerHasShares ran its recorded-owner and legacy NULL-owner queries side
by side, so the join to fsentries ran even when the first query already
found a share. Run the legacy half only when the recorded half finds
nothing; the answers stay the same.

* perf(share): drop the NULL-owner fallbacks from owner and outbound reads

ownerHasShares, listOutbound, countOutbound, the outbound apps summary
and the team-row filter behind listByHolder/countByHolder each carried a
second half for share rows written before entry_owner_user_id existed,
matched by joining fsentries on a NULL owner. The column migration
backfilled those rows from fsentries, so the joins only add work: each
costs about min(owner's files, NULL-owner rows) per call.

Every remaining read goes through entry_owner_user_id alone. Invite rows
(no fsentry, no owner) are matched as before: they were never reachable
through the joins.

Self-contained so it can be reverted on its own. It assumes existing
data has no share row that names an fsentry but has a NULL
entry_owner_user_id; if any remain, they stop showing up in these reads.

* fix(share): backfill the entry owner on share rows written without it

The share reads now match on entry_owner_user_id alone. Add a migration
for each engine that fills that column from fsentries on every row that
names an entry but still has a NULL owner, so the backfill runs before
the reads stop looking at NULL-owner rows and their answers don't depend
on what existing data holds. Rows that name no entry (invites) keep a
NULL owner, and a replay finds nothing to fill.

* fix(share): record the entry owner when a claim names its entry

applyPending can point a claimed invite at an entry, but left
entry_owner_user_id NULL, so the row was missing from the owner's reads
that now match on that column alone. Fill it from fsentries with the
same subquery the upserts use. The share service claims without an
entry today; the parameter stays for callers that pass one.

* refactor(share): read the app from the data column directly

Every caller of the app filter and the app expression passes the share
row's own data column now, so the column argument goes. Also names the
owner probe test after what it checks and puts back an unrelated line
the earlier formatting pass had reflowed.

* chore(share): drop the redundant MySQL and Postgres owner backfills

MySQL and Postgres replay every migration file on boot, and mysql_mig_47
and postgres_mig_36 already end with the same owner backfill, so they
re-run it on every boot. Only SQLite, which tracks a schema version and
runs each file once, needs its own backfill migration (0103).

* fix(share): list a team row for its members when no entry owner is recorded

* refactor(share): claiming an invite leaves its entry and owner as written

* test(db): run the share owner backfill against the full prior schema

* fix(share): ship the owner backfill as its own MySQL and Postgres migration

The backfill at the end of mysql_mig_47 and postgres_mig_36 replays on boot
only when database.migrationPaths is set. A new file gives installs that apply
migrations another way a step to run; a replay finds nothing to fill.
2026-11-12 18:06:29 -07:00
Daniel Salazar 36e2d199c8 fix(fs): refuse instead of replacing an entry whose name differs only by an emoji or accent (#4589)
* fix(fs): refuse instead of replacing an entry whose name differs only by an emoji or accent

MySQL compares fsentries names and paths under collations that also fold
accents and most emoji, so a lookup for one name could return a sibling
with another, and writes then overwrote and renamed it.

Path lookups now return only an entry whose stored path matches the
requested one after NFC normalization and lower-casing; case-only and
NFC/NFD spellings still resolve as before. Name-collision checks opt in
to the folded match and treat it as taken by a different entry: write,
mkdir, touch, rename, shortcut, move and copy dedupe when asked and
otherwise answer 409, and never replace or return that entry.

* fix(fs): leave a sibling's subtree alone when a folder its path folds onto is renamed or moved

The subtree rewrite matches descendants with `path LIKE 'old/%'`, and LIKE
compares under the path collation (ASCII case on SQLite). With a sibling
folder whose path differs only that way, renaming or moving one folder also
rewrote the paths below the other.

When another row sits at such a path, the rewrite now follows parent links
down from the moved folder instead. Without one, the rewrite is unchanged,
after one indexed lookup.

* fix(fs): resolve an entry whose stored path is spelled unlike its names, and heal that path on the first read

Older rows can store a `path` whose spelling differs from the names on their
parent chain in ways the path collation ignores. A lookup that finds such a row
by its real spelling now confirms it through the parent chain, writes the path
from the stored names, and returns it, so overwriting, touching or copying over
it by name works and the next read takes the single indexed query again.

The lineage walk now writes a row's path only when it differs byte for byte
from the one its names give, and drops only those rows' cache keys, so a walk
over rows that are already right issues no writes. Whether an entry found at a
name is a different, folded name is decided by its own name and its path after
that walk, not by a stale path column. With `foldedNames`, a row only the path
collation matches still counts as holding the name.

* fix(fs): match entry names under full case mapping

Names are compared after upper- then lower-casing, with a dotted capital I
taken as I, so a request in another case finds the stored entry for Greek
final sigma (`οδος.txt` for `ΟΔΟΣ.txt`), German sharp s (`STRASSE.txt` for
`Straße.txt`) and Turkish dotted I (`istanbul.txt` for `İstanbul.txt`).
Accent, emoji, kana and width differences still make two names different.

* fix(fs): name the existing entry when a folded name is already taken, and show why a confirmed replace failed

`item_with_same_name_exists` from write, move and copy now carries the
existing entry's own name in `entry_name` when it differs from the requested
one only by accents or emoji, so the prompt names the file the folder shows.

In the GUI, a replace confirmed from that prompt that still fails is reported
with its error message and counts as not written, instead of rejecting out of
`handle_same_name_exists` and leaving the caller waiting.

* fix(webdav): answer MKCOL with 405 and COPY or MOVE with Overwrite: F with 412 on a folded name

The existence checks before MKCOL, COPY and MOVE now see an entry whose name
differs from the target only by accents or emoji, so those requests answer as
for an existing target instead of reaching the filesystem and coming back as
409.

* docs(fs): list the errors write, move and copy return when a name is taken

`puter.fs.write()` gets an Errors section with `item_with_same_name_exists`,
`cannot_overwrite_a_directory` and `conflict`, the last for a name that differs
from an existing one only by accents or emoji. The move and copy pages say how
they reject in that case.

* fix(fs): walk the parent chain on the primary when a path lookup reads hard

A lookup that asks for a try-hard read (the checks after a create or move hits
the unique key) now also runs its parent-chain walk on the primary, and falls
back to the primary for the entry it finds. Before, the walk read a replica, so
a name the replica had not seen yet came back as missing and the mkdir or move
surfaced the duplicate-key error as a 500 instead of a 409.

Lookups without a try-hard read are unchanged. A try-hard lookup issues the
same number of queries; its walk reads the primary instead of a replica.

* fix(fs): treat a folder only the collation matches as missing instead of naming a file in it

A name-collision lookup lets only the last segment of the path match a
differently spelled name. When a folder above it matches only under the
collation (`😁dir` for `😀dir`), the lookup reports no occupant, so a write,
mkdir or touch there fails as for a missing folder: 404, or with
`createMissingParents` a 409 that names the folder. Before, it reported the
file inside the other folder and answered 409 naming that file. A folder whose
stored path is spelled that way still counts as the same folder, so lookups
built from stored paths find what they did before.

A rename onto a name the unique key already holds now answers 409 when the
lookup before it missed the occupant, instead of a 500.

* fix(fs): name the existing entry when a rename's name is folded, and leave a copy onto itself out of the copy docs

A rename refused because the folder holds a name that differs only by accents
or emoji now names that entry in its 409, as write, move and copy do.

A copy onto a folded spelling of the source's own name answers as a copy onto
itself (400 with `overwrite`), so the copy page's paragraph on folded names
now covers only items other than the source.

* fix(fs): walk the parent chain on the primary only when a hard read found another spelling of the path
2026-11-12 17:26:22 -07:00
Daniel Salazar 8616b00f7b fix(gui): save a URL from showSaveFilePicker, and always answer the app when the save dialog closes (#4587)
* fix(gui): save a URL from showSaveFilePicker and always answer the picker

The save picker downloads an http(s) URL itself, without cookies or a
referrer, up to 100 MiB, and writes it where the user chose, in apps and on
websites. A failed download shows an error. In apps, a save dialog that
closes without saving (failed download, failed write, close button) answers
fileSaveCancelled once instead of leaving the call pending.

* fix(gui): answer the save picker with the save's real outcome when it closes mid-save

Closing the save dialog while a URL is downloading stops the download,
writes nothing and answers fileSaveCancelled, in apps and on websites. In
apps, closing it once a write has started leaves the answer to the write:
fileSaved if it lands, fileSaveCancelled if it fails, never both. A save
that throws shows an error and answers fileSaveCancelled instead of leaving
the dialog stuck, and a second Enter during a save is ignored.

On an https desktop only https URLs are accepted, with a message saying so,
since the browser blocks http downloads there.

* fix(gui): retry signing a saved file once, and say so when the app can't get it

The app save picker signs the written file for the app with one retry. If
both tries fail, the user is told the file was saved but couldn't be passed
back, and the app gets fileSaveCancelled. The picker's one-answer state
(answered, closed, saving, writing) lives in helpers/savePickerAnswer.js so
the website picker can share it.

* fix(gui): answer the website save picker once, with the save's real outcome

The website save picker runs one save at a time, so a second Enter is
ignored. Closing it while a URL downloads stops the download, writes nothing
and answers fileSaveCancelled; closing it once the write has started leaves
the answer to the write, so the page hears fileSaved or fileSaveCancelled,
never both. A failed download shows an error, answers fileSaveCancelled and
closes the picker, as in apps, and so does a save that throws. A saved file
that can't be signed is reported to the user and answered as cancelled.

* fix(gui): show apostrophes in the save picker's failure messages
2026-11-12 17:26:05 -07:00
Daniel Salazar 873303a7c4 fix(events): offer a queued delivery only to connections that run its handler (#4590)
* fix(events): offer a queued delivery only to connections that run its handler

A queue subscription with an events worker was sent to every connection of
the app. A connection without the handler declined it at once, which moved it
to the worker while the tab with the handler was still running it.

puter.js now sends `events.handle` with the persistent subscriptions a
connection runs a handler for, on connect and whenever that set changes. The
server puts the connection in a room for each queue row it holds, and offers
the row's deliveries only to that room. With nobody in it anywhere in the
region, the delivery goes to the worker at once. A peer region handed such a
delivery declines it straight back when none of its connections run the
handler. Rows without a worker are still sent to the whole app room.

* fix(events): probe the handler room only when the app has a connection in the region

A queued delivery for a pair with no connection here goes to the worker on
the same local lookup and presence read as before, with no request to the
other nodes.

* fix(events): probe a forwarded batch's handler rooms together, within one timeout

A region receiving a batch now looks up every handler room its queued
deliveries need before it sends any of them: each distinct room once, up to
16 at a time, all under one shared 500 ms deadline. A room not answered by
then counts as empty, so its delivery is declined home with `no_handler` and
never sent. Deliveries still go out in the batch's order, and a batch with no
such delivery probes nothing.

* fix(events): resend a handler announce refused for the call budget

puter.js now sends `events.handle` with an ack and a 10 second timeout. When
the server refuses it with `too_many_requests`, the client sends the current
set again after 10 seconds, until one goes through or the connection is
replaced. No answer, as from a server without the verb, and any other refusal
are left alone. A connection that never had a persistent handler sends no
announce; one that had sends an empty set when the last is removed.

The rate limits page now says that announces count toward the
acknowledgement budget.

* docs(events): say when a client starts getting first try at queued events

A client counts once the server has registered its handler, shortly after it
connects or calls `onPersistent()`. Before then, an event for a subscription
with an events worker goes to another client with the handler or to the
worker.

* fix(socket): settle a cluster room count on the first node that holds a socket

A count ask now hears each node's answer as it arrives: the first node holding
a socket answers it at once, every node saying none answers it once all have
said so, and a node that never answers only decides it when no answering node
holds one. The request sent is unchanged, so every node answers it as before.

* fix(events): look up handler connections in the region's record, not every node

Each connection that announces a handler is kept in a per-subscription
sorted set in the region's Redis, scored by renewal: added after it joins
the handler room, removed before it leaves or on disconnect, renewed every
minute while it runs one, and counted only for three minutes after its last
renewal. Handing out a queued delivery, and receiving a forwarded one, now
check this node's sockets and then that set with one read, instead of
asking every node in the region to count the room.

A connection the set names but that is gone gets the delivery sent to an
empty room, and the socket lease hands it to the worker as before.

* fix(events): try the next region with the app open after one declines for want of a handler

A queued delivery's region is picked by app presence, so the first region
tried can have the app open without the handler. Its no_handler decline now
hands the delivery to the next region with the app open that hasn't had it,
and the worker runs it once none is left. A lease that lapses still goes to
the worker.

A decline now counts only when it comes from the region the current attempt
went to (this region for a local attempt). The region goes into the entry in
the same write as the attempt's lease, so a second decline from a region
already tried can't take the next region's lease. An attempt made after a
decline stamps its lease in its own field, which a server that doesn't make
such attempts never reads, so it can't take that lease back either.
2026-11-12 17:25:54 -07:00
Juan Fernando CastroandDaniel Salazar 11107f3108 fix(permissions,share): revoked team shares and removed links aren't read back from a lagging replica (#4554)
* fix(permissions): a revoked group grant isn't read back from a lagging replica

* fix(share): links removed by trash or a move aren't read back from a lagging replica, in any region

* test(permissions): start each test server in the file on an empty redis mock

* fix(permissions): node deletes find just-made grants on the primary; add a group membership hook

* fix(share): link revokes reach peer regions even when the local marker fails, and subtree retires read the primary

* fix(permissions): read a group's members after the grant is deleted

* fix(permissions): mark group members before a group grant is deleted

The member read and the marker now come before the delete, so a failed
member read leaves the grant in place for a retry and member reads are
already on the primary when the row goes. Members are read again after
the delete, best effort, to include someone who joined meanwhile.

The node-delete path returns those members, so ShareService bumps the
same list instead of reading each group again from a replica.
readUserGroupPerms checks the marker without the unused base-key GET.

* perf(share): trash and delete read grants from a replica unless the owner shared just now

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 17:58:05 -05:00
Juan Fernando CastroandDaniel Salazar 2c3b69011f fix(puter-js): unshare lists refused pairs in failed and rejects when all were refused (#4557)
* fix(puter-js): unshare rejects when every revoke was refused

/share/revoke reports refused pairs in a 200 body, and unshare() only
read `revoked`, so a call where every pair was refused resolved as if it
had worked. It now rejects with the first pair's { message, code }, the
same way share() does; a call where some pairs went through still
resolves with { revoked }.

GUI tests drive the SDK's unshare against a refused body and check that
the share dialogs and Remove from Shared show the error and keep the
row. The unshare docs list the per-pair codes.

* fix(puter-js): unshare and share reject with the first refused pair that names a code

A pair the server could not complete carries no code and a generic message,
so it no longer comes ahead of a refusal that says what happened. Both
operations pick the error through one helper. The unshare docs describe which
pair's error is surfaced and the request-level rejections.

* fix(gui): share modal says how many items a person was removed from when some were refused

After removing someone from several items, the refreshed listing is compared
with the items the removal covered, and the status reports the count that
went through instead of all of them.

* test(gui): unshare fixture swaps XMLHttpRequest per call; alert mock cleared in hooks

* feat(puter-js): unshare resolves with the refused pairs; share is as it was

unshare resolves { revoked, failed }, with one failed entry per refused
recipient/item pair. A pair refused because the recipient or the item is gone
counts as nothing to withdraw: the call rejects only when every pair was
refused and at least one for another reason, with the first such refusal that
names a code. paths and recipients are typed and documented.

share() picks its rejection as before; the error choice lives in unshare.

* fix(gui): sharing dialogs report the items an unshare was refused for

The share modal counts refused items from the unshare result across every
run of a large selection, going on past a run refused whole, for removing a
person and for closing the link on several items. The share window and
Remove from Shared show the refusal when the call resolves with the pair
refused.

* fix(gui): sharing dialogs keep the unshare outcome on screen after the listing refresh

The share modal shows the partial count once its refresh has settled, for
removing a person and for closing the link, so a listing that fails for the
refused item does not replace it. The share window does the same when closing
the link.

* feat(puter-js): an unshare rejected for refused pairs lists them in failed; failed[].path is documented as the absolute path

* test(puter-js): API suite covers unshare's refused pairs

* fix(gui): share modal stops at a revoke request that failed and counts only refused items

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 17:57:59 -05:00
Juan Fernando CastroandDaniel Salazar eefc1f2f2e fix(auth): the session list leaves out expired sessions, and revoking one works (#4573)
* fix(auth): the session list leaves out expired sessions

* test(auth): cover revoking expired sessions and their children

* fix(auth): the session list follows what authentication accepts

A token minted under a session that has expired is left out of the
list along with that session. The check runs in listSessions with the
rule authentication uses, so getByUserId returns expired rows again
and the revoke callers need no option.

* fix(auth): revoke-session answers 404 for another user's session

Missing, already revoked and other users' rows now share one answer.

* fix(auth): type the session list rows instead of casting them

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 17:57:51 -05:00
Juan Fernando CastroandDaniel Salazar 7e093d2ba7 perf(puter-js): reading the user at load joins the boot lookup (#4574)
* perf(puter-js): reading the user at load joins the boot lookup

* fix(puter-js): keep the token fallback to the user lookup resend

The fallback bearer is opt-in on initXhr, so other requests and their
replays send no bearer after sign-out, as before. A failed join (the
lookup or the copy) falls back to sending the request. Adds tests for
the legacy callback form and a resend after a network failure.

* fix(puter-js): join the boot user lookup only at the same API origin

A user lookup sent after setAPIOrigin() joined the boot /whoami still in
flight for the old origin and resolved its user instead of asking the new
one. The in-flight marker now records the origin too.

* fix(puter-js): a joined user read signs in from the boot lookup's answer and leaves on its first retry

- The boot lookup keeps one in-flight record (token, origin, user, refusal), made and cleared in cacheWhoami_.
- getUser(), puter.getUser() and whoami() share one join. A refused boot lookup drives the sign-in from its own response, so the token is not sent again; a sign-out meanwhile fails with 401 and sends nothing.
- The join ends when the boot lookup first has to retry, so the caller's own request goes out at once (new fetchUrl onRetry option).
- The joined user is copied with a JSON round trip; initXhr is back to its previous signature.

* fix(puter-js): whoami() keeps its refused-request error when it joined the boot lookup

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-11-12 17:56:42 -05:00
velzie bd7614372a fix(peer): grant forgery bug and stale comment 2026-11-12 17:48:10 -05:00
Daniel Salazar 61b5aa4900 fix(fs): treat path-less entries found by uid or id as missing (#4611)
A uid or id lookup can return a legacy row with no path when its parent
chain is broken. resolveNode now reports it as not found, so readdir,
stat and the other uid-addressed routes answer 404 instead of crashing
in the ACL check.
2026-11-12 15:44:14 -07:00
Wei-Lun Li a6e75b46cb fix(install): turn off plan gates in the Windows installer config (#4599)
install.sh writes "meteringEnforcement": { "subscriptions": false } into
config.json because a self-hosted install has no paid plans, so with the
gates on the OpenAI/Anthropic-compatible AI endpoints answer 402
subscription_required for every account. install.ps1 never got that
key, so a Windows install made with the README one-liner still has the
gates on. Write the same key from install.ps1.
2026-11-12 22:43:22 +00:00
Daniel Salazar dd32c7f9b2 fix(auth): a token whose lookup fails gets the outage error, not token_auth_failed (#4586)
The auth probe used to mark a token as failed whenever authenticate threw,
so a database error while checking a valid token came back as 401
token_auth_failed, which clients read as "discard this token".

The probe now records the thrown error on the request instead. Routes
that require auth hand it to the error handler (503 db_unavailable with
Retry-After for a database outage, a generic 500 otherwise); routes that
don't require auth serve the request anonymously, as before. Only a token
that authenticate reports invalid still gets 401 token_auth_failed.

The periodic socket re-check likewise keeps a connection up when its
lookup fails, instead of disconnecting it.
2026-11-12 15:39:51 -07:00
Nariman Jelveh aaa5c82236 Redirect /show/* to the dashboard and remove leftover desktop code (#4610)
* Redirect /show/* to the dashboard's Files tab

/show/<path> redirects to /dashboard#files/<path> with its query intact,
instead of booting the desktop and opening Explorer there.

Also removes what only that route used: the shell's launch options, the
launch_options GUI param, and the GUI service that ran them.

* Remove the unused component window and Flexer

Nothing opens UIComponentWindow or uses Flexer, and JustHTML was only the
component window's default body. Also drops the attach() shim UIQRCode kept
for component containers.

* Remove translation keys left by removed features

Drops 36 keys nothing references in every language: the keyboard shortcuts
guide, desktop shortcuts, personalization, the toolbar's fullscreen, GitHub
and refer entries, the referral strings, and the publishing option prompt.

* Inline the toolbar height, which is always zero

The toolbar is gone, so window.toolbar_height only added or subtracted zero
in window sizing and placement.

* Remove window edge snapping

Dragging a window to a screen edge or corner no longer snaps it to half or a
quarter of the screen. Double-clicking the head and the scale button still
maximize.

* Remove the window color theme service

Nothing can set a window theme, so the GUI stops reading the saved theme
file on boot and windows keep the default colors. puter.ui.on() still
accepts themeChanged, which is never sent; its docs entry is removed.

* Remove styles and icons nothing references

Drops the stylesheet rules of the removed Settings window, launch menu,
intro menu and a few other removed features, and 16 icons whose names
appear nowhere in the GUI.

* Remove more translation keys nothing references

Drops 56 keys in every language that no GUI code, extension or backend
message uses, such as the system information labels, the old 2FA setup and
confirm-code steps, and the auth token copy dialog strings.

* Remove the global menubar leftovers and more unused styles

Nothing creates a global menubar, so the window code stops hiding, showing
and removing one. Also drops stylesheet rules for classes nothing uses,
including the old auth dialog, the responsive hidden-* and visible-*
helpers, and two keyframes only those rules animated.
2026-11-12 15:31:38 -07:00
Daniel Salazar 8b68c103c5 fix(auth): mint under a full-access token without re-reading its session row, and look token rows up by uid through their indexes (#4588)
* fix(auth): mint under a full-access token from the row auth loaded, and look up token rows by uid with one index per column

A token minted by a full-access token now parents to the session row that
authentication already loaded and checked for the minting token, instead of
reading it again by token uid. The same applies to the session a relay token
is bound to. Tokens whose claims name no row of their own still look it up.

findActiveByAccessTokenUid now queries access_token_uid first and the legacy
uid only on a miss, each through its own index, instead of one OR across both
columns.

* fix(auth): send both access-token row lookups together, so a miss on the first costs no second round trip
2026-11-12 15:17:45 -07:00
Daniel Salazar 454f94fc54 fix(hosting): serve a launched public app's assets from its billing cookie without re-minting it (#4585)
Same-origin assets of a launched public app carry the launch token in the
Referer. When that token's signature verifies and it names the same user
and session as a valid public billing cookie (and, for an app token, the
same app), the cookie identity is used directly: no re-authentication, no
new cookie, no Set-Cookie. Any other combination resolves exactly as
before. Responses that do set the public cookie now carry
`Cache-Control: private`.
2026-11-12 14:28:35 -07:00
Daniel Salazar 8b7d380de1 fix(metering): a settle whose balance read fails still charges the allowance (#4584)
When the purchased-credit read or the near-allowance exact read fails while
settling a metered call, the call is now settled against the month record it
already has: the share that fits the monthly allowance is charged, and
purchased credit is left untouched since its balance could not be read.
Previously the whole settle was skipped, leaving the cost in the month total
but in neither pool.

Such settles skip the cached-balance update and overuse check, and raise one
info alarm id with the amount credit would have covered.
2026-11-12 14:27:29 -07:00
Daniel Salazar 8052a4287e fix(auth): close a revoked access token's sockets on every node (#4583)
Access-token revocation is announced as outer.auth.access-token.revoked,
so peer clusters evict the token's socket.io connections and realtime
relay sessions as the revoking cluster does. Peer payloads without a
string token_uid are ignored.
2026-11-12 14:27:10 -07:00
Daniel Salazar f114b13306 perf(gui): boot keeps a same-session token and skips /get-gui-token when the page already answers it (#4591)
* perf(gui): keep the stored token when /get-gui-token returns one for the same session

/get-gui-token signs a new string on every call, so every signed-in boot
stored a token that differed from the one it held only in `iat` and passed
it to puter.setAuthToken. That opened a second FS socket, re-sent /whoami
and the cache timestamp, and every other open tab picked the new token up
from storage and did the same.

The boot now keeps the stored token when the returned one carries exactly
the same claims apart from `iat`; any other difference replaces it as
before. A kept token still starts the boot from an empty FS cache, as the
swap did. A boot that keeps the persisted cache (no session cookie) now
waits for the cache timestamp check before the first render, which it
started before reading the user.

* perf(gui): skip /get-gui-token at boot when the page already says what it would return

Every signed-in boot waited on /get-gui-token before it read the user. The
shell is rendered from the same session cookie that route reads, so it now
carries `guiTokenDigest` in gui_params: a digest of the claims (minus
`iat`) of the GUI token the cookie's session would get, or null when the
request has no session cookie. A cookie that doesn't resolve to a user
session leaves it out.

The GUI skips the request when the digest matches its stored token (same
session, user and token type), or when it is null (the route would refuse).
It still asks when the parameter is missing, the digest names another
session, nothing is stored, the page and the route are on different
origins, or Navigation Timing doesn't show this page as freshly served for
this load (a history navigation or restored tab can reuse a page rendered
for an older cookie).
2026-11-12 14:26:09 -07:00
Nariman Jelveh ce70728eeb Remove the desktop's icons and right-click menu, and redirect /desktop (#4609)
* Remove the desktop's icons and right-click menu

The desktop no longer lists the Desktop folder's items, and right-clicking
its background opens no menu. It stays as the surface windows open on.

Also removes what only the icons used: file drops and item drops on the
desktop, rubber-band selection, auto arrange and saved icon positions, and
their two keys in puter.js's GUI boot cache. Clicking the desktop clears the
active item container so keyboard shortcuts stop acting on a window's items.

* Redirect /desktop and /settings to the dashboard

/desktop redirects to the root with its query intact, and /settings to the
dashboard's Home tab. /desktop drops out of the OIDC return_to allowlist,
sign-out returns to the root, and the auth cover page's logo links there.

Also removes eight GUI globals nothing references.

* Remove code paths that only ran for the desktop's item container

The desktop is no longer an item container, so the Trash item that
refresh_item_container added to it, the empty-folder message exemption,
and the context menu parent check for it can't be reached.
2026-11-12 14:00:29 -07:00
Nariman Jelveh 8518023b0b Remove more desktop-only features and entry points (#4607)
* Remove the desktop's Publish Website window

The right-click menu on desktop folders and on an Explorer window's
background no longer has a Publish as Website item. This removes
UIWindowPublishWebsite, the publishAsWebsite helper that opened it, both
menu items, the window's CSS, window.extractProtocol (only the window used
it), and the dir_published_as_website and pick_name_for_website i18n keys.

The dashboard keeps publishing websites through its own dialog.
generateFileContextMenu no longer falls back to the removed window, so it
offers Publish as Website only when the caller passes onPublishWebsite,
which the dashboard's Files tab does. The website badge on desktop items
and the backend hosting API are unchanged.

* Remove the desktop's Hide Desktop Icons toggle

The desktop's right-click menu no longer has a Hide/Show Desktop Icons
item. Without the wallpaper, hiding the icons left an empty page. This
removes the menu item, toggleDesktopIcons/hideDesktopIcons/
showDesktopIcons, the boot-time read of the saved setting, and the
.desktop-icons-hidden CSS rule.

puter.js's GUI boot cache no longer batch-fetches desktop_icons_hidden,
menubar_style or user_preferences.clock_visible, which nothing reads
anymore. Values already saved under those keys stay in KV, unread.

Also drops the desktop's BroadcastChannel, whose message handler was
empty and which nothing else referenced.

* Remove the desktop's ?download= link handler

Opening puter.com/?download=<url> or /desktop?download=<url> no longer
asks to save that remote file to the Desktop. The parameter is ignored,
and a root URL that carries it boots the dashboard like any other root
URL instead of forcing the desktop.

This removes the handler in UIDesktop, its UIWindowProgress import, the
download entry in the GUI's dashboard routing, and the
confirm_download_file_to_desktop, downloading, downloading_file,
uploading_file and error_download_failed i18n keys only it used.

The decision of whether a page load boots the dashboard moves from an
inline block in initgui into helpers/bootsDashboard.js, so it is tested.

* Stop granting Camera and Recorder Desktop access on desktop boot

The desktop no longer grants Camera and Recorder write access to the
user's Desktop folder the first time it loads. Those apps now ask
through the normal permission prompt, as they already did for users
who only used the dashboard. Grants already given stay in place.

puter.js's GUI boot cache no longer batch-fetches the
has_set_default_app_user_permissions flag that kept the grant to one
run. Values already saved under it stay in KV, unread.

* Redirect /@username links to the dashboard

puter.com/@<username> no longer opens that user's Public folder on the
desktop. The backend now redirects /@:username to /, so old links land
on the dashboard instead of booting the desktop.

This removes the GUI's /@ handler and the error_invalid_username i18n
key only it used. error_user_or_path_not_found stays, since share links
still use it.

* Redirect /desktop/app links and stop opening share links on the desktop

/desktop/app/<name> now redirects to /app/<name>, keeping its query
string, so the app opens in the dashboard. The backend no longer serves
an app shell there, OIDC sign-in no longer returns to it, and the GUI no
longer strips a leading /desktop from its route segments. The rate-limits
page lists only /app/<name>, the route the limit still applies to.

/desktop?shared=<path> no longer opens the shared item on the desktop.
The desktop ignores ?shared=: it doesn't ask a guest to sign in for it,
doesn't switch to the account a share email names, and OIDC sign-in from
/desktop doesn't carry it along. Share emails link to /?shared=, which
the dashboard handles as before. Opening a share notification on the
desktop still opens the item.

* Remove the desktop's ?app=explorer link handler

?app=explorer[&path=<folder>] no longer boots the desktop and opens an
Explorer window. ?app= is ignored and no longer forces the desktop, so a
root URL that carries it opens the dashboard, and /app/<name>?app=... now
launches its app in the dashboard instead of booting the desktop and
launching nothing.

* Stop ?puter.fullpage= from booting the desktop

The root and /app/<name> with ?puter.fullpage= now open in the dashboard
like a plain /app/<name>, with the app maximized. The dashboard always
shows apps full page, so the parameter no longer overrides that. Iframe
embeds and popups, which still boot the desktop, read it as before.

bootsDashboard no longer takes the query string, since no parameter
decides the interface anymore.
2026-11-12 11:54:40 -07:00
Nariman Jelveh 33988ac84c Remove the desktop taskbar (#4605)
* Remove the desktop taskbar

The desktop no longer has a taskbar. Windows on /desktop can't be
minimized, since nothing could bring them back; the dashboard still
minimizes into app tiles. The desktop and maximized windows use the
full page height.

Backend: POST /update-taskbar-items is removed, and /whoami and the
login response no longer include taskbar_items. The user table's
taskbar_items column stays until a migration drops it.

puter.ui.createWindow() still accepts show_in_taskbar, which has no
effect and is marked deprecated.

* Keep desktop backdrops above ordinary windows

A backdrop's z-index used to take the taskbar's as a floor, which kept
modal dialogs on /desktop above every ordinary window, including ones
focused after the dialog opened. Keep that floor on the desktop; the
dashboard and fullpage mode stack as before.

Also drops commented-out geometry that referenced window.taskbar_height,
and a dashboard comment about the removed fullpage minimize rule.

* Keep dropped desktop icons fully on screen, and fill the phone popover

A desktop icon drop is accepted only where the whole item fits above the
bottom edge. Phone popovers use the full viewport height.
2026-11-11 20:16:13 -07:00
Nariman Jelveh 72de6e16b2 Remove the desktop wallpaper (#4604)
The desktop no longer shows a wallpaper. It shows the page's plain
background color: white, or #1e1e1e in dark mode. The sign-in dialogs
and account picker that opened over the wallpaper now open over that
color too.

This removes the default wallpaper image and its credit, its preload in
both page shells, the "Set as Desktop Background" item on images'
right-click menu, the GUI's set_desktop_background and
refresh_desktop_background, and the set_as_background i18n key.
Desktop icon labels were white with a shadow to read on the wallpaper;
they now use the same gray as labels inside windows, and a light gray
in dark mode.

The backend drops POST /set-desktop-bg, and /whoami no longer returns
desktop_bg_url, desktop_bg_color or desktop_bg_fit. The user table's
desktop_bg_* columns stay; dropping them needs a migration.
2026-11-11 17:40:52 -07:00
Nariman Jelveh 234994d3af Remove the desktop toolbar and nine desktop features, and use puter.js's color picker on the desktop (#4602)
* Remove the desktop's QR login button

The desktop toolbar no longer shows a QR code that signs another device
into the current session. This removes the toolbar button, its click
handler, the UIWindowQR window (its only caller), the qr.svg icon, the
window's CSS, and the four i18n keys only that window used.

UIQRCode and the qrcode library stay, since 2FA setup still uses them.
The ?auth_token= handler in initgui also stays, because the Playwright
e2e harness signs in through it.

* Remove the desktop's My Websites window

The user menu no longer has a My Websites item. This removes the
UIWindowMyWebsites window, its menu item, its CSS, the plug.svg icon only
it used, and the i18n keys only it used.

When publishing a website or worker hits the subdomain limit, the error
now shows only the server's message, without the "Manage your subdomains"
link that opened the window. The trash handlers keep fading the website
badge on a trashed folder and no longer touch the window's rows.

* Remove the desktop's Task Manager window

The user menu no longer has a Task Manager item. This removes the
UIWindowTaskManager window, its menu item, its CSS, the cog.svg icon only
it used, and the i18n keys only it used, including the process_type_*
and process_status_* keys it built at runtime.

The process model in definitions.js, including END_SOFT and END_HARD,
stays: windows still register processes, and a process still forwards
an end signal to its children.

* Remove the desktop's Change Desktop Background window

The desktop's context menu no longer has a "Change Desktop Background…"
item, or the separator above it. This removes the
UIWindowDesktopBGSettings window, its CSS, the palette.svg icon only it
used, and the i18n keys only it used.

Desktop backgrounds still work: the saved background loads at boot, and
an image's "Set as Desktop Background" item still applies and saves it
through /set-desktop-bg.

* Remove the desktop's Start menu

The taskbar no longer has a Start button, so it begins at Explorer. This
removes the Start item and its app popover (recent and recommended apps,
search, tile context menu, and drag-to-taskbar), along with:

- The taskbar's startup /get-launch-apps request and the desktop's
  app.opened socket handler, which only kept the menu's lists current
- The Start popover's search, click and keyboard handlers in UIDesktop.js,
  keyboard.js, initgui.js and UITaskbarItem.js
- The taskbar's handling of items dropped from outside it, and the
  UITaskbarItem append_to_taskbar option only that path used
- The Start CSS, the start.svg icon, and the i18n keys only the menu used

The dashboard's recently used apps still load, since the dashboard fetches
its own list and launchApp.js still records launches. UITaskbar.test.js now
covers the taskbar's item order and that it doesn't fetch launch apps.

* Remove the desktop's Search window

The desktop toolbar no longer has a Search button, and Ctrl/Cmd+F on the
desktop no longer opens the Search window, so it falls through to the
browser's find as it already did on the dashboard. This removes
UIWindowSearch, the toolbar button and its handler, the window's
keyboard handling (arrow keys, Enter, Escape) and click-outside close,
its CSS, the search.svg and magnifier-outline.svg icons only it used, and
the i18n keys only it used.

The backend /search endpoint is unchanged.

* Remove the desktop toolbar

The desktop no longer has the toolbar at the top of the screen, so its
Puter logo, guest Save Account button, and account menu are gone from the
desktop. Log out, account switching, language, settings, and contact all
remain on the dashboard.

Removed:
- The toolbar markup, its auto-hide logic and mouse tracking, its
  right-click menu, and the account menu with its click handlers
- The modify-user-options-menu GUI extension, which only added a
  Developer item to that menu
- UIWindowFeedback, the desktop's Contact Us window, which only the menu
  opened; the dashboard keeps its own contact dialog
- Toolbar hooks elsewhere: the hide_toolbar() call on window maximize,
  pointer-events toggles during window and item drags, the click guard in
  initgui, the account-button toggle in helpers.js, and the avatar sync
  for the toolbar's profile image in TabAccount
- The toolbar and feedback CSS, the toolbar's two KV keys from the
  puter.js GUI read cache, and the i18n keys only these used

window.toolbar_height stays: it was already 0, so window and desktop
layout are unchanged. The welcome window keeps its backdrop stacking.

* Remove the desktop's Share window

The right-click menu on desktop and Explorer items, and on items in open
and save file dialogs, no longer has a Share item. This removes
UIWindowShare, that menu item, the share-outline.svg icon only it used, its
CSS, and the share_with and share_row_team i18n keys only it used.

The Dashboard keeps sharing through UIShareModal. generateFileContextMenu
no longer falls back to the removed window, so it offers Share only when
the caller passes onShare, which the Dashboard's Files tab does. The share
dialog layout classes the Blocked Senders window uses, and the recipient
suggestion styles the share modal uses, stay.

* Remove the desktop's Welcome window

New desktop users no longer see the Welcome window on their first visit.
This removes UIWindowWelcome, the first-visit check in UIDesktop that
opened it, its CSS, the welcome_* i18n keys only it used, and the
UIWindow backdrop_covers_toolbar option only it passed.

has_seen_welcome_window is also dropped from the puter.js GUI boot read
cache. Values already saved under that key stay in KV, unread.

* Replace the desktop's color picker with puter.js's picker

puter.ui.showColorPicker() inside Puter now opens puter.js's own
<puter-color-picker>, the one it already shows on websites outside Puter,
instead of the desktop's iro-based window. Apps get the same answer
everywhere: a lowercase 6-digit hex string, or null when the user cancels.
Inside Puter it used to resolve to an 8-digit hex with alpha, or undefined.

This removes UIWindowColorPicker, UIColorPickerWidget, iro.min.js and its
lib-list entries, and the select_color i18n key only that window used.

The GUI loads puter.js from js.puter.com, so the IPC handler doesn't rely
on this branch's component: it drops the alpha from an 8-digit default,
leaves out a default that isn't a 3- or 6-digit hex, and removes the
picker once it answers.

<puter-color-picker> drops the alpha from an 8-digit default instead of
failing to open, opens on its own color when the default isn't valid,
and removes itself when Escape closes it. showColorPicker()'s type and
docs now give the 6-digit return value and null on cancel.

* Remove the desktop's Publish Worker window

The right-click menu on .js files on the desktop and in Explorer, and in
open and save file dialogs, no longer has a Publish as Worker item. This
removes UIWindowPublishWorker, that menu item, its CSS, and the
pick_name_for_worker i18n key only it used.

The Dashboard keeps publishing workers through its own dialog.
generateFileContextMenu no longer falls back to the removed window, so it
offers Publish as Worker only when the caller passes onPublishWorker,
which the Dashboard's Files tab does.

* Add a Language setting to the dashboard's Account tab

The desktop toolbar's account menu was the only place to change Puter's
language, so removing the toolbar left no way to change it. The Account
tab now has a Language card that shows the current language and opens a
dialog listing every supported language by its own name. Saving a new
one stores it on the account and reloads the page in it.

changeLanguage() now returns false when the language can't be saved, so
the dialog can show an error instead of doing nothing. The
change_language key comes back in every locale, and
language_save_failed is new.
2026-11-11 17:17:30 -07:00
Nariman Jelveh eef2d089a8 Remove the dashboard's Open Desktop card and the unused theme dialog (#4601)
* Remove the unused theme dialog and its translations

UIWindowThemeDialog has had no importers since the settings window was
removed, and the reset_colors, ui_colors and change_ui_colors strings
are referenced only by it or by nothing at all.

* Remove the Open Desktop card from the dashboard Home tab

Drops the card's markup, styles, color tokens, and the open_desktop and
open_desktop_hint strings, which nothing else uses.
2026-11-11 15:12:34 -07:00
Nariman Jelveh 711ece5800 feat(gui): let a first-party page frame the custom domain dialog (#4581)
* feat(gui): let a first-party page frame the custom domain dialog

`/embed/attach-custom-domain` joins the framed popups. On a signed-in page
one label under the Puter origin, `puter.hosting.attach()` now shows the
custom domain dialog in a frame over the page instead of a popup window.
Apps, the desktop and every other website keep their current behavior.

The frame covers the page with a transparent GUI page, so the dialog
floats over it with its own scrim as it does on the desktop. The GUI
serves only the framing page's own user, with no account check waiting,
and otherwise reports `unavailable`, which leaves the page to open the
popup as before. It runs the popup's action unchanged: the framing page
is the requester the dialog names, and gets the same
`customDomainAttached` answer.

Escape is left to the dialog, which doesn't close while a request is in
flight, and the frame allows `clipboard-write` for the DNS record copy
buttons. A call made while the frame shows resolves null without
replacing it, as a second call does on the desktop.

* Update RecommendedAppsService.ts

* fix(gui): let the framed custom domain dialog finish closing before it answers

The answer removes the frame from the page, so the dialog's exit transition
never showed. The framed action waits for the overlay to leave first, with the
wait the feedback embed already had, now one shared helper.
2026-11-11 11:44:39 -07:00
Filip Kujundžić 40ea3686da Merge pull request #4594 from HeyPuter/fix/chat-served-tier-pricing
Price native service tiers and served premium capacity
2026-11-11 18:02:29 +01:00
404oops db7ca8382b Price native service tiers and served premium capacity 2026-11-11 12:53:12 +01:00
404oops 0f2c325598 Meter Responses usage before validating empty output 2026-11-11 12:53:12 +01:00
404oops ffcfb85749 Validate voice changer models against the price table 2026-11-11 12:53:11 +01:00
Nariman Jelveh 44bf99f808 Update recommended apps 2026-11-10 20:36:06 -07:00
Nariman Jelveh d392b6857d fix(dashboard): drop the repeated add-on credits remainder from the Usage tab 2026-11-10 17:38:03 -07:00
Daniel Salazar d8c3e01dd9 fix(http): parse large JSON bodies in slices; store WebDAV uploads labeled as JSON (#4563)
JSON request bodies over 32 KiB are parsed in slices, with the event loop
running between them, instead of in one JSON.parse call. Reading, limits,
decompression, req.rawBody and the errors stay as express.json has them.
Large bodies parse one at a time.

Gemini TTS and video replies use the same parser, and their base64 is
checked and decoded 1 MiB at a time.

A profile picture is sized before its pattern runs, so one of several
megabytes gets a 413.

Body parsers skip the dav subdomain: a PUT labeled as JSON or form data
was read by them and stored empty, or refused.

puter.js sends auth_token as the first field of a driver call. A chat body
over 50 MiB that opens with a valid token is accepted up to 128 MiB, as
one with a header credential is.

JSON in a charset other than UTF-8 is now limited to 1 MiB, and a body
over 32 KiB nested deeper than 10,000 levels is refused.
2026-11-10 16:36:52 -07:00
Nariman Jelveh 4bc679956f fix(dashboard): label the Usage tab's credit bar "Monthly credits" 2026-11-10 16:28:49 -07:00
Daniel Salazar b751f75467 fix(fs): keep a file readable when two overwrites create the same path at once (#4579)
Two writers that both start before the row exists each upload under their
own object key. The second one found the first writer's row and updated its
bucket and size while keeping the first writer's uuid, so the row could name
an object that was never written under that key.

The store now leaves a row untouched when an overwrite arrives with a
different object key, and the service moves the uploaded bytes onto the
row's key before updating it. The last writer wins and the uuid is stable.
2026-11-10 16:15:42 -07:00
Daniel Salazar 224b2f2367 fix(backend): null driver args, non-string chat model, and long app-stats uid lists no longer answer 500 (#4576)
- drivers: a null `args` in a driver call is treated as no arguments
- ai-chat: a `model` that is not a string is answered as model not found (400)
- apps: app stats split a long uid list across ClickHouse queries, which cap the size of one query parameter
2026-11-10 16:08:55 -07:00
Nariman Jelveh 80bcc4ba14 feat(dashboard): show add-on credit as its own bar on the Usage tab (#4578)
The Usage tab told an account how much add-on credit it had left in a
line of text under the monthly meter. It now gets a meter of its own
there: a heading, how much of the credit bought has been spent, and a
bar, with the line saying what is left and when it is used kept under
it. An account that never had add-on credit sees the tab as before.
2026-11-10 15:11:21 -07:00
Nariman Jelveh f26201649a feat(gui): let a first-party page frame the deployment's top-up dialog (#4570)
* feat(gui): let a first-party page frame the deployment's top-up dialog

`/embed/top-up` joins the framed popups. Like the upgrade, it covers the
framing page with a transparent GUI page, serves only the page's own
user, and reports `unavailable` unless the deployment defines
`window.UITopUp` with `embeddable === true` and no account check is
waiting. It runs `new UITopUp({ onClose }).open()` and posts
`closed { toppedUp }` once the dialog closes. The frame's lifecycle is the
upgrade's, now shared by both.

`<puter-user-menu>` gets an Add credits item under Usage, shown only when
whoami reports `subscribed: true`. It frames the top-up on a page that can
frame the GUI, and otherwise opens the dashboard with `?topup=` in a new
tab, within the click. Hidden items are skipped by the menu's arrow keys.

`puter.ui.showTopUp_()` is internal; no public SDK method is added.

* fix(puter-js): show no usage-limit dialog over an open top-up

A call refused for funds while the framed top-up was open put the
usage-limit dialog in the top layer over it, covering the payment the
user was making to clear that refusal. The prompt now stays out of the
way while a top-up shows, as it does for the framed upgrade.

* fix(puter-js): leave Add credits out of the user menu for a team seat

whoami reports a seat as subscribed through its team's plan, so the menu
offered Add credits to an account that can't buy any: the dialog only
answers that the team pays. The item now stays hidden when whoami names a
team paying for the account, as the desktop's own menu already does.

* fix(puter-js): open the top-up tab on the page's own deployment

The tab the Add credits item falls back to went to puter.com whatever
GUI origin the page belongs to, so on another deployment it opened a
different one, signed in as whoever is signed in there. It now opens the
dashboard of the GUI origin, like the menu's Usage link beside it.

* fix(puter-js): return keyboard focus to the account button after Add credits

Choosing Add credits closed the menu with focus on the item it had just
hidden, and the frame restored focus to the menu's shadow host, which
can't take it, so a keyboard user was left on the page body once the
dialog closed. The menu now moves focus to its button first, and a
framed popup restores focus to the element inside a shadow root.

* docs(embed): say when the top-up dialog's open() resolves

It resolves after the account has loaded, not as soon as the dialog
shows.

* fix(puter-js): offer Add credits after an upgrade made on the page

The user menu showed Add credits by the whoami it loaded with the page.
A user who subscribed in the framed upgrade therefore found no Add
credits in the menu until a reload. A framed upgrade that reports a plan
now marks the page's cached whoami as subscribed, as a confirmed email
already is, and the menu reads it again each time it opens.

* fix(puter-js): let a top-up that failed to start be asked for again

showTopUp_ cleared its in-flight marker in a finally inside the async
function it stored. A failure before that function's first await, such
as building the dashboard address from an origin with no scheme, ran the
finally before the marker was set, so the marker kept the rejected
promise for good: every later call got the same rejection, and
topUpShowing_ went on suppressing the usage-limit dialog. The marker is
now cleared once the stored promise settles.

* fix(puter-js): keep requestUpgrade from framing the upgrade over an open top-up

The usage-limit dialog already stays away while the top-up shows, but a
page's own requestUpgrade() call, such as one in the handler of a call
refused mid-payment, still mounted the upgrade frame. Frames are single
per path, so it covered the top-up and the payment being typed in it,
and the two dialogs' own guard can't see across documents. The call now
waits for the top-up and resolves { upgraded: false } once it closes.

* fix(puter-js): offer Add credits after an upgrade when whoami was read again

The user menu keeps the whoami object it loaded and ignores a token set
again unchanged, but the SDK reads whoami into a new object each time a
token is set. A framed upgrade then marked the new object as subscribed
and the menu went on reading the old one, so Add credits stayed hidden
until a reload. The menu now reads the SDK's cached whoami when it is
the same account's.

* docs(embed): list the top-up tests among the checks to run
2026-11-10 14:53:34 -07:00
Nariman Jelveh 1550be358d fix(explorer): navigation, keyboard, drag-and-drop, realtime and layout fixes (#4569)
* fix(explorer): record typed address-bar paths in navigation history and canonicalize them

Pressing Enter in the path input now pushes the destination onto the
window's Back/Forward history like every other navigation, and resolves
the entry first (trim, ~ and home-relative expansion, trailing or doubled
slashes) so the window's path matches the form the rest of the GUI compares
against.

* fix(explorer): keep the address-bar input in step with the Shared view and the root

The path input was only updated after a successful stat, so in views that
are never statted it kept the previous folder's path and Enter navigated
back there.

* fix(explorer): drop the previous folder's identity when a folder fails to load

A rejected stat was never handled, so the window kept the previous folder's
uid, title and item count while showing the load error, and layout or sort
changes were saved to the wrong folder. The window now clears them, and the
failed request no longer surfaces as an unhandled rejection.

* fix(explorer): release a window's selection listeners and history when it closes

Each folder window left its rubber-band selection instance, a document
mouseup handler and its navigation history behind after closing, so every
later mouse press ran once per window ever opened. Windows that are not
folders no longer bind the handler at all.

* fix(explorer): anchor a Ctrl/Cmd rubber-band selection where the drag started

The rectangle's origin was only recorded for unmodified drags, so adding to
a selection drew it from the previous drag's start point or the top-left
corner.

* fix(explorer): keep arrow-key navigation on visible items and a valid anchor

Arrow keys could land on a hidden dotfile and then stop moving, threw when
the remembered anchor belonged to another window, and left the footer's
selected count stale.

* fix(explorer): ignore non-character keys in type-to-select

Shift, arrows and Enter were appended to the search term by name, so typing
a capital letter or typing right after navigating matched nothing. Hidden
files are no longer matched, and the footer count follows the container the
selection changed in.

* fix(explorer): make the keyboard permanent-delete confirmation work in every language

The confirmation compared the dialog's answer with the English button label,
so in any other language confirming did nothing.

* fix(explorer): paste once per Ctrl+V press

The shortcut ran on both keydown and keyup, so releasing V before Ctrl
pasted the clipboard twice.

* fix(explorer): keep selection shortcuts working after navigating, and out of text fields

Select All, Copy, Cut and Delete looked up the folder from the element last
pressed, which is a removed item or a sidebar entry after navigating, so the
shortcuts did nothing. Select All also skips hidden files, and Delete keys
typed in a text field are left to the field.

* fix(explorer): count only visible items in the footer and drop hidden files from the selection

The item count included dotfiles that were not shown, and hiding hidden
files left them selected, so a later Delete or Cut acted on files the user
could not see.

* fix(explorer): paste cut items after their folder is no longer displayed

Paste resolved the cut paths to rows on screen, so cutting a file and then
navigating the same window to the destination moved nothing and silently
emptied the clipboard. Items without a row are now looked up and moved the
same way.

* fix(explorer): show a moved item once in its destination

The tab that moves an item renders the row and also receives the move
event, so the destination showed the item twice whenever the event arrived
first. Both paths now replace any row the other already rendered.

* fix(explorer): sort names the way the dashboard does, with a stable tie-break

Folder windows and the desktop compared raw lowercase strings, so file10
came before file2 and accented names landed after z, and equal sizes, dates
or types kept whatever order they arrived in. They now use the dashboard's
comparator, so the same folder reads the same in both.

* fix(explorer): keep the Trash window's states when Trash is emptied elsewhere

Emptying Trash from another tab cleared the whole container, including its
empty, error and loading elements, and left the item count stale.

* fix(explorer): remove rows deleted elsewhere instead of hiding them

A deletion from another tab or an app only faded the row out, so it still
counted in the footer, in Select All and in keyboard actions. The row is now
removed, and windows showing the deleted folder close even when its own row
is not displayed anywhere.

* fix(explorer): refresh size and date when a file is overwritten elsewhere

An overwrite arrives as an update event, which only applied the name, icon
and path, so the size and modified columns and sorting kept old values.

* fix(explorer): download files whose names contain &, #, % or +

The path went into the download URL unencoded, so the server received a
truncated path and the download failed.

* fix(explorer): allow moving into folders whose name starts with Trash

The deleted-folder check matched any sibling of Trash sharing its prefix,
so moving into a folder such as Trash-old was refused.

* fix(explorer): end the extract operation when an archive is corrupt or has no files

A corrupt zip threw before its error could be shown, and an archive with
nothing to write or a failed upload never closed the progress window. A
progress update arriving before the window existed also threw.

* fix(explorer): follow renamed and moved folders in Back/Forward and on undo

A window's navigation history kept the old path after a folder was renamed
or moved, so Back led to a load error. Undoing a folder rename also left
windows and items inside it on the renamed path, because the undo retargeted
from the path it was restoring.

* fix(explorer): keep one progress window per copy or move

The progress timer is re-armed after every conflict dialog, and it opened a
second window when one was already showing, leaving the first on screen for
good.

* fix(explorer): show file names once-escaped in move alerts

The name was escaped before being passed to an alert that escapes its whole
message, so names with &, quotes or angle brackets showed entity text.

* fix(explorer): don't flash the empty-folder message while a folder refreshes

Clearing the rows for a refresh showed the message until the listing came
back, even for a folder full of items.

* fix(explorer): accept drops into a window that covers a desktop folder

A desktop folder hidden under a window still reacted to the drag and
disabled the window as a drop target, so the drop was ignored. A drop that
bailed out early also left every folder view disabled as a target until the
next hover; targets are now restored when the drag ends.

* fix(explorer): release each row's document click handler when the row is removed

Every file row bound a click handler on document that was never unbound, so
browsing folders grew the number of handlers run on every click and kept the
removed rows alive.

* fix(explorer): let Delete work on a file after its rename was cancelled

Cancelling a rename with Escape, or leaving the editor without a change,
left the item marked as being renamed, and the Delete shortcut skips such
items.

* fix(explorer): offer a download when no app can open a file

The prompt called a translation import that is not a function, so opening
a file with no suitable app threw and did nothing.

* fix(explorer): use an item's current path and name in its context menu

Cut, Copy, Download, Create Shortcut and the rename editor read the path and
name captured when the row was created. After a rename made from another
window or tab they acted on the old path, and closing the editor repainted
the old name.

* fix(explorer): offer Cut only where the item can be moved

The item menu on the desktop and in folder windows offered Cut on trashed
items and on shared items the user may not move, unlike the same menu built
for the dashboard.

* fix(explorer): give a folder named *.app a folder icon

The app-shortcut branch ran before the directory branch, so such a folder
was read as a file on every listing, failed, and showed the app icon.

* fix(explorer): make New > JPEG Image create an actual JPEG

The blank image was encoded with the canvas default, so the .jpg file held
transparent PNG data.

* fix(explorer): handle a drop onto a folder that is part of the dragged selection

Dragging a selection onto one of its own selected folders also tried to move
that folder into itself. On the desktop the drop also cleared the saved
position of the target folder instead of the items that left the desktop.

* fix(explorer): don't highlight a window as a drop target for its own items

The check meant to skip the folder an item is already in compared the wrong
path, so dragging inside a window outlined it although dropping there does
nothing.

* fix(explorer): keep desktop rearranging to the desktop itself

Dropping an item from a window showing the Desktop folder onto the desktop
background positioned the window's row absolutely and saved that position.

* fix(explorer): empty Trash from its window in one request

The Trash window's own Empty Trash sent one delete request per item and
reported Trash as empty before any of them finished. It now runs the same
single operation as the taskbar and desktop.

* fix(explorer): label the root and Shared in the Back/Forward menus

The long-press history menus showed an empty row for the root folder and a
lowercase path segment for Shared, and the Forward menu's Home entry was not
translated.

* fix(explorer): keep sidebar labels translated and titled after reordering

A reordered sidebar rendered the labels saved at the time of the drag, so
they stayed in the old language, and every item's tooltip was empty.

* fix(explorer): enable the Open dialog's button only for a selected file

The button stayed enabled after navigating to another folder, and a
rubber-band selection of folders alone enabled it, although clicking it then
did nothing.

* fix(explorer): show the slow-load message only for a load that is slow

Once any folder took over a second to load, the spinner text switched to
the slow-load message three seconds later even after the load finished, and
every later load in that window opened with it. The text is also translated
now.

* fix(explorer): keep the current folder visible in a long breadcrumb

A path too long for the bar was cut at its end, hiding the folder being
viewed and its nearest parents. It is now cut at the root end.

* fix(explorer): grey out the whole selected row in an unfocused details view

Only the name lost its highlight when the window lost focus, leaving a grey
pill inside a blue row.

* fix(explorer): align the details header with its rows and mark it clickable

The header ended 10px before the rows and the selection highlight, and its
sortable column titles kept the default cursor.

* fix(explorer): give the layout button a tooltip and a pointer cursor

It was the only navbar control without either, so it did not read as
clickable.

* fix(explorer): remove the stray tooltip from the loading spinner

The spinner's SVG carried an internal title that showed as a tooltip and
was read out by screen readers.

* fix(explorer): refuse uploads, copies and shortcuts into the Shared view

Dropping files on Shared in the sidebar, or Ctrl/Alt-dropping items onto a
window showing it, sent requests against a path that is not a directory and
ended in an error or a stuck progress window.

* fix(explorer): open an item's context menu where it was requested

The menu is built after permission and app lookups, and it opened wherever
the pointer had moved to by the time they finished.

* fix(explorer): close a folder's window when the folder is deleted, however it was opened

Windows opened by path, such as with Ctrl-click on a breadcrumb or sidebar
entry, never received the marker used to find a deleted folder's windows,
so they stayed open on a folder that no longer existed.
2026-11-10 11:23:30 -07:00
Nariman Jelveh 8a15f09f48 Send one request, not four, after the account pick in the sign-in popup (#4564)
* Send one request, not four, after the account pick in the sign-in popup

A sign-in popup exchanges for the opener's app token as soon as the user
picks an account. postAuthActions now hands over that result when the
opener origin and session token still match, and exchanges again otherwise.

The relationship check is skipped when an account was picked, since the
pick already decides both the second picker and consent. Signup in the
popup is not a pick and still runs it.

The /whoami read before the picker is reused only when the same user and
session token are still in place, the exchange just accepted that session,
no verification gate is pending, and the read is under a minute old.

* Test picking a different account in the sign-in popup

The popup has to read the picked account once and hand over a token for
it, not for the account it opened with. The second-account helper moves
to the shared e2e helpers so both specs that need one use the same copy.
2026-11-10 08:31:12 -07:00
Daniel Salazar 2084587885 fix(apps): list only the caller's own apps (#4562)
The `puter-apps` `select` method returned every app that was not
protected unless the caller sent `predicate: ['user-can-edit']`. It now
always scopes to the caller's own apps. The predicate is still accepted
and no longer changes the result.

Apps owned by someone else stay reachable by name or uid through `read`
and `/query/app`.

`AppStore.count` drops `visibleToUserId`, which only the unscoped
listing used.
2026-11-09 17:43:28 -07:00
Daniel Salazar 222ffcfe39 fix(dev-center): accept only a hostname for puter.domain and digits for puter.port; escape the saved app link (#4561) 2026-11-09 17:27:42 -07:00
Nariman Jelveh 7494aa1cbc Reload a tab when another tab switches to a different account (#4560)
* Reload a tab when another tab switches to a different account

Every tab shares one stored session token. A tab that sees a new token
appear in storage adopted it whoever it was issued to, so after an account
switch in one tab the others kept showing the previous account while
sending requests as the new one.

A tab now reads the user out of the stored token. A token for the account
it already shows is adopted as before. A token for a different account
reloads the tab, which comes back up as that account. A tab with no
signed-in user, or a token whose user can't be read, behaves as before.

* Store a new account's token only after its session cookie is set

A tab that sees another account's token in storage reloads and signs in
from the session cookie. update_auth_data() stored the token before it
synced the cookie, so a tab that reloaded faster than the sync came back
up as the previous account and stored that account's token again, which
the switching tab then adopted under the new account.

The token is now stored after the cookie sync, in the same task as the
stored user.

* Keep the new account's token when another tab stores one during the cookie sync

update_auth_data() sets window.auth_token, then waits for the session cookie sync. A token another tab stored during that wait was adopted over it, so the tab finished the switch holding the previous account's token and saved it under the new account in logged_in_users. The token is set again once the sync has settled.

* Create one visitor account for the account switch e2e tests

Each test signed up its own visitor, and sign-ups are limited to 10 per 15 minutes, so a few runs in a row failed in setup.
2026-11-09 16:49:50 -07:00
Nariman Jelveh 1adca8a794 Show a shared item once, with its strongest access, when it has more than one share (#4559)
* Show a shared item once, with its strongest access, when it has more than one share

The Shared listing returns one row per grant, so an item shared by two
people, or directly and through a team, appeared twice and took its
actions from whichever grant was listed last. The GUI now merges the
rows for an item and keeps the strongest mode.

* Keep a shared item under each person who shared it in the people filter

A merged row carried only the strongest grant's issuer, so filtering the
Shared view to someone whose grant was the weaker one hid an item they
shared, and their chip left it out of its count. A merged row now lists
every issuer. The filter and its counts match any of them, Everyone
counts the item once, and the Shared by tooltip names them all.

* Keep an item listed after Remove from Shared when a team still shares it

Leaving an item drops the grants made to the user directly. One a team
also holds stays shared, but the desktop took its icon away until the
next refresh and the dashboard removed the row before re-listing it. A
merged row now records that a team holds the item too. The desktop
re-lists the Shared folder so the item shows as the team share it has
become, and the dashboard leaves the row in place for its refresh.

* Show an item as shared by the person the Shared view is filtered to

An item several people shared showed the strongest grant's issuer in
the Shared by column even when the listing was filtered to one of the
others. A filtered row now shows the person filtered to, which also
keeps the Shared by sort in step with what the column displays.
2026-11-09 14:55:24 -07:00
Nariman Jelveh 07b19888a4 Match desktop items and windows by their path value, not by a selector (#4558)
* Match desktop items and windows by their path value, not by a selector

Path lookups on the desktop compare data-path values through one helper, so
a name holding &, ' or " is found like any other. Explorer windows keep the
raw path in data-path after navigating, the same as when they open.

* Read data-path straight from the element in path lookups

atPath() and underPath() built two jQuery objects for every element they
looked at. On a desktop with 5,000 items a lookup took about 1.4 ms,
against 0.08 ms for the selector it replaced. Reading the attribute from
the element brings it to about 0.3 ms.

* Match no element when a path lookup has no path

atPath() with a missing path matched every element whose attribute is
empty, which is every item that is not a shortcut and every app window.
An item.moved event without old_path would point all of those items at
the moved file. underPath() with an empty path matched every path.

* Use atPath() to find the items a share badge belongs to

mark_item_shared() had its own copy of the same lookup.

* Keep the raw folder name in an explorer window's data-name

update_window_path() and refresh_item_container() wrote the HTML-encoded
name there, so after navigating into "R&D" the tab title and the publish
dialog showed "R&amp;D". A newly opened window already held the raw name.
2026-11-09 14:24:52 -07:00
Nariman Jelveh 2394221745 Answer requestEmailConfirmation() once, and show it in a frame on first-party pages (#4551)
* Answer requestEmailConfirmation() once, and show it in a frame on first-party pages

puter.ui.requestEmailConfirmation() in an app opened a second code dialog
after the first: a confirmed user got a dialog they couldn't complete and
two answers, and an unconfirmed one entered the code and then got a second
dialog, whose result was the only answer. The desktop handler now runs one
walk-through and answers once with its result.

That walk-through is ensureConfirmedEmail: save a temporary account, then
ask an unconfirmed email for its code, each window at most once. Publishing
a website or worker uses it too, replacing two copies of the same steps.
The code window now resolves false when it closes without a confirmation,
so dismissing it answers the app instead of leaving it waiting, and it
marks window.user confirmed before resolving, so a caller that checks next
doesn't ask again before the refresh lands. A logout from its footer still
leaves it unsettled, so the boot gates don't reopen it. In the dashboard
the windows open above the app's window, which stays on top and covered
them.

On a signed-in page one label under the Puter origin, the call now frames
/embed/request-email-confirmation over the page, served like the upgrade
frame, and resolves { response }, the field an app reads from the
desktop's answer. The GUI shows it only for the framing page's own user
with no phone, card or password check waiting. It skips the boot's
non-closable email gate for this action, so the user gets one code window
they can close or Escape. An email already confirmed resolves true without
showing the frame. A frame that can't show it resolves { response: false }.

* Skip the email confirmation frame when the page knows the email is confirmed

A confirmed email never goes back to unconfirmed, so requestEmailConfirmation()
on a first-party page resolves { response: true } from the page's whoami
instead of booting the GUI in a frame behind a loading scrim.

* Close the framed email confirmation on Escape in the save-account window

A temporary account sees the save-account window first, where Escape did
nothing, while it closed the code window that follows.

* Let the email code window recover from a failed code check with no response body

A network failure has no JSON body, so reading its error threw, leaving the
window spinning with its inputs disabled. It now shows an unknown-error message
and lets the user try again.

* Answer an email confirmation asked for while one is showing with the windows already open

A second walk-through opened a second code window over the first, and for a
temporary account it never answered, since the save-account window opens once.

* Read the email code from the code window it was typed in

A second code window read the inputs of the first one on the page, so typing
in it never filled in the code or enabled its button.

* Keep the email code window's answer in step with a code check it was closed during

Closing the window while a code was being checked answered false, and the
check went on to confirm the email. A code not sent yet is no longer sent, and
the window waits for one already sent before it closes.

* Show why a new email code couldn't be sent

A failed re-send, such as one past the hourly limit or a network failure,
showed nothing, so the link seemed to do nothing.

* Skip the email confirmation frame on calls after the user confirmed in one

The page's cached whoami kept reading the email as unconfirmed, so each later
call framed the GUI again and flashed its loading scrim before resolving true.

* Leave the email code window unsettled when any logout closes it

Only a logout from its footer did. One from elsewhere, such as after a 401,
answered false, so the boot's email gate opened a new code window while the
page was navigating away.

* Hide the save-account window while its email code window shows

A save-account window with a backdrop stayed under the code window's own, so
the page behind was dimmed twice, and on a phone the window showed around the
code window's edge.

* Answer a save-account request made while the save-account window is open

The window opens once, so the second request got no window and never
answered, which left an email confirmation that asked for it waiting too.

* Answer a framed email confirmation closed with Escape with the code check it was closed during

Escape in the framed code or save-account window closes that window, and the window's answer closes the frame. A code already sent is waited for, as it is when the window's close button is used. Before, Escape closed the frame and answered false while the check went on to confirm the email.
2026-11-09 13:48:10 -07:00
Filip Kujundžić ee2cdbfe1a Merge pull request #4552 from HeyPuter/feat/claude-fable-5-2
Add Claude Fable 5.2 with published pricing and fast mode
2026-11-09 20:50:25 +01:00
404oops 9b6350beab Remove redundant Fable model documentation 2026-11-09 20:36:58 +01:00
404oops b649d9ada7 Apply published Fable 5.2 pricing and fast mode rates 2026-11-09 20:03:16 +01:00
404oops 50b0639d02 Add Claude Fable 5.2 with provisional Fable 5.1 pricing 2026-11-09 20:02:49 +01:00
Filip Kujundžić ab96702718 Merge pull request #4546 from HeyPuter/feat/chat-service-speed
Add service speed selection and fallbacks to AI chat
2026-11-09 20:01:58 +01:00
404oops 47f9b0bdf3 Use the common service speed rate shape for Claude 2026-11-09 19:54:54 +01:00
404oops 8c4b0eb1fc Keep chat service speed rates in provider model catalogs 2026-11-09 19:31:25 +01:00
404oops a88c1b9d65 Fall back to standard Claude speed on fast capacity overload 2026-11-09 19:04:31 +01:00
Daniel Salazar 9caaf7162c fix(fs): follow S3 region redirects (#4544)
A file row without a stored region is read with the server's configured
region. When its bucket lives elsewhere, S3 answers 301 PermanentRedirect
and the read failed with a 500. The SDK now retries once in the region the
redirect names.
2026-11-09 10:31:30 -07:00
404oops 0191f2bfec Update chat.md 2026-11-09 16:11:51 +01:00
404oops 7aadc48cca Add service speed selection and billing to AI chat 2026-11-09 15:59:08 +01:00
Daniel Salazar 6b06793750 fix(gui): send picker results only to the popup opener's origin (#4543)
File, directory and save pickers opened in a popup now post their results,
cancellations and the save-data request to window.openerOrigin instead of
'*', matching the token hand-off. A test checks that opener messages in the
GUI only target '*' when the opener's origin is not yet known.
2026-11-08 23:11:57 -07:00
Neal Shah e441880f13 add more data to user audit events (#4542) 2026-11-08 19:22:09 -05:00
Nariman Jelveh 9a86803418 Show the upgrade in a frame on first-party pages (#4541)
* Show the upgrade in a frame on first-party pages

puter.ui.requestUpgrade() on a signed-in page one label under the Puter
origin now frames /embed/upgrade over the page and resolves { upgraded }
once it closes. The usage limit dialog's Upgrade button does the same on
those pages instead of opening the dashboard in a new tab. Apps, the
desktop and every other page keep their current behavior.

/embed/upgrade is served like the sign-in and picker frames: the GUI
shell with the same framing rules, query and exact-path redirects and
no-store. The GUI shows the deployment's upgrade dialog
(window.UIUpgradeAccount) only for the framing page's own user, with
nothing left to verify, and only when the dialog says it reports its
close (UIUpgradeAccount.embeddable). Otherwise it reports unavailable and
the page opens the dashboard upgrade in a tab, through a consent dialog
when the click has expired. The app it credits is resolved from the
framing page's origin, never read from the fragment.

The frame covers the page with a transparent GUI page so the dialog's own
scrim shows, allows payment, and leaves Escape to the dialog. A call made
while it shows settles with it. If every dialog and window in the frame
goes without a close being reported, the frame closes itself rather than
staying over the page.

* Open the framed upgrade only once the frame has a viewport

The page keeps a loading frame hidden, which Chromium and WebKit lay out at
0x0. The team-seat alert, a GUI window that centers itself when it opens,
landed at (-175, 0) with its message and OK button cut off. A hidden frame
now tells the page it's ready first, and opens the dialog on the resize that
follows, or after a second if none comes.

* Document when the framed upgrade posts ready

* Retry a framed popup's focus() that the frame ignored

Firefox and WebKit ignore focus() inside a frame for a moment after the
page shows it, and Firefox sends no focus event afterwards, so the framed
upgrade dialog and its team-seat alert opened with nothing focused there:
Enter didn't press OK and screen readers didn't land in the dialog. A
focus() that didn't take is now retried briefly, until the user presses a
key or the pointer.

* Keep the framed upgrade's GUI in the light color scheme

The transparent upgrade frame put `color-scheme: light dark` on body as
well as the root, so in dark mode form fields the GUI leaves at browser
defaults turned dark. The email confirmation code digits showed slate
text on a dark grey field inside a white window. Only the root needs to
match the frame's scheme to stay transparent; body now keeps the light
scheme the GUI uses everywhere else.

* Fall back to the upgrade tab when the framed dialog fails to open

If the deployment's upgrade dialog threw while opening, the frame reported
a close, so requestUpgrade() resolved { upgraded: false } and the user's
click did nothing. It now reports itself unavailable, and the page opens
the dashboard's upgrade page as it does for any frame that can't show it.

* Say which plan changes count as upgraded in UpgradeResult

The upgrade dialog sets `upgraded` only for a subscribe or an upgrade;
a downgrade, cancellation or resume closes with false. The type said any
plan change counted.

* Stop the page scrolling behind a framed GUI embed

Firefox and WebKit pass a wheel or touch scroll over a frame on to the
page behind it, whatever the dialog's overscroll-behavior, so scrolling
over the framed upgrade, sign-in or a picker scrolled the page under the
scrim. The page's root now stops scrolling while an embed shows, taking
a classic scrollbar's width as padding so its content doesn't shift, and
gets its own styles back once the last embed closes.

* Skip the usage limit dialog while the framed upgrade shows

A refused call while the upgrade shows in a frame over a first-party
page opened the usage limit dialog on top of it, which made the upgrade
dialog inert until the user closed it. The refusal still rejects.

* Restore the page's root styles after an embed the page removed

An embed dialog the page removed itself never unlocked the page scroll,
and the next embed read the locked styles as the page's own, so the
page stayed unscrollable once it closed. The page's styles now go back
before the next lock reads them.

* Open the upgrade tab when the page can no longer frame it

The usage limit dialog decided to frame the upgrade when it opened, so
an Upgrade click after the page signed out did nothing. onUpgrade now
returns false when the page can't frame it, and the dialog opens the
dashboard tab as it does without onUpgrade.

* Keep the upgrade showing until its tab consent is answered

When the framed upgrade fell back to a consent dialog, requestUpgrade
settled as soon as the dialog showed. A refusal meanwhile opened a usage
limit dialog over it, and a second call asked again. The upgrade now
counts as showing until the user answers, so both join it.

* Say where requestUpgrade never settles

Its JSDoc named only apps, though on any website that can't frame the
upgrade the promise doesn't settle either.

* Treat a GUI embed the page removes as dismissed

A page that re-renders its body took the embed's dialog with it, and the
embed's promise never settled. For the upgrade that left it counted as
showing for good: later requestUpgrade() calls joined the dead one, no
refused call opened the usage limit dialog again, and the page stayed
unscrollable.

* Let the framed upgrade credit the remembered prompt when it knows none

When the framing page's origin resolved to no app and the call named no
reason or method, the frame handed the dialog an empty attribution. The
dialog then remembered that as the latest prompt, erasing another app's
credit, and sent the purchase with no attribution at all. It now gets
null, as from every other entry point.

* Pad the page by the width its scrollbar gives back behind an embed

A page with scrollbar-gutter: stable keeps its gutter once the root stops
scrolling, so padding it by the scrollbar's width as well shifted its
content left while any GUI embed showed. The lock now measures how much
width the root actually gets back.

* Keep the page's scroll position behind an embed when its body scrolls it

A page whose body carries the viewport's overflow (html and body at 100%
height, body with overflow-x hidden) jumped to the top while an embed
showed, and stayed there after: hiding the root's overflow made the body
a scroller of its own. The lock now hides the overflow of whichever
element the viewport uses, and restores a single-axis inline overflow.
2026-11-08 14:27:41 -07:00
Nariman Jelveh 09c6ce6f46 Dashboard: fix 30 bugs, UX, a11y and perf issues from an adversarial review (#4540)
* fix(dashboard): middle-truncate long names in the default Files list view

In the list view's default auto-width name column, a long name was cut at
the end by CSS with no tooltip, so names that differ only near the end
("report … (1).txt", "report ….txt") looked identical and the extension
was hidden. The name is now cut in the middle to the room its cell has,
like a resized column and grid tiles already are, and the full name shows
on hover. The phone layout's two-row rows get the same treatment.

* fix(dashboard): toast the newest notifications of a catch-up burst

A burst of more than four notifications toasts three and folds the rest
into an "N new notifications" summary. The split kept the first three it
was given, assuming newest first, but the reconnect catch-up and the
events-feed replay hand notifications over oldest first, so after a tab
woke up the toasts showed the oldest three and hid the newest. The split
now sorts the burst itself.

* fix(dashboard): act on a regrouped notification's current content when its toast is clicked

When more shares from the same person fold into a notification that's
already toasted, the toast's text updates ("alice shared 3 items") but
its click still acted on the notification as first shown, so it selected
only the first file instead of opening Shared. The click now looks up the
notification's current content.

* fix(dashboard): show browsers on Puter's own Web Push as browsers in Your devices

A device registered through Puter's own Web Push client reports platform
"webpush", which the Notifications tab didn't know: it got the phone icon
and no "Web" label. It's now treated like the other browser platform.

* fix(dashboard): clear the push preference's load error once it loads

If the Puter notifications preference couldn't be read, its card's hint
was replaced with a load error. Coming back to the tab once the read
worked re-enabled the switch but left the error text in place. A
successful read now puts the hint back.

* fix(dashboard): keep push off for this browser when removing it half-fails

Removing "This browser" from Your devices turns its push off and then
deletes the server record. If the delete failed (a dropped connection),
the handler returned before saving the opt-out, so the card still read
"on" and the next dashboard load quietly turned push back on. The opt-out
is now saved as soon as push is off, and the card is redrawn on failure.

* fix(dashboard): don't let an older device or app listing redraw a removed row

The Notifications tab lists devices and apps each time it opens. If one
of those listings was still in flight when the user removed a device or
turned an app off, it landed afterwards and drew the removed row again.
Only the newest listing draws now, and a removal outdates any listing
already in flight, as the browser card already did.

* fix(dashboard): report the items that were shared when a large share fails partway

Sharing more than 50 items sends one request per 50. If a later request
failed (the daily share limit, say), the items earlier requests had
already shared were forgotten: the dialog showed a flat failure and none
of their rows got the shared badge until a reload. The dialog now says
"Shared with bob on 50 of 60 items" along with the error, badges those
rows, and refreshes the shared listing.

* fix(dashboard): drop the site address from files inside a disassociated folder

Disassociating a folder's website from its Properties cleared the
folder's own badge, but the files inside it kept their website-address
badge and URL, which now led to a site that no longer exists. Their
addresses are now worked out again without that site, so a file still
served by a site on a folder within keeps its address.

* fix(dashboard): keep an app's own "(N) " title prefix beside the unread badge

The dashboard puts the unread count in front of the page title and keeps
it there as app windows rewrite the title. It removed any leading
"(N) " before adding its own, so an app titled "(3) Inbox" lost its
count ("(1) Inbox"). Only the badge the dashboard itself put there is
replaced now ("(1) (3) Inbox").

* perf(dashboard): redraw the notification list once per burst of acks from other tabs

"Mark all read" in one tab sends the other open dashboards one ack per
notification, and each ack rebuilt the whole list. With 150 unread that
was about half a second of main-thread work in every other tab (more on
slower devices). Acks and removals from other tabs now redraw once per
50ms beat: 150 acks take about 5ms.

* fix(dashboard): let the keyboard reach "Also delete data" in the Uninstall dialog

The Apps tab's dialogs keep Tab inside the card, but the trap only
cycled buttons and text areas, so Tab and Shift+Tab bounced between
Cancel and Uninstall and the delete-data checkbox couldn't be reached
without a mouse. Inputs, selects and links are part of the cycle now.

* fix(dashboard): don't leave the app request form on an IME composition's Escape

In the "Request an app" form, Escape steps back to the options. While
typing with an input method (Chinese, Japanese, Korean), Escape is how
the composition is cancelled, and that Escape also left the form. It's
now ignored while composing, as the Apps grid and folder name already do.

* fix(dashboard): keep focus out of an Apps folder while it closes

A folder's card stays on screen for its 240ms close animation. During
that time its tiles could still take focus: an arrow key pressed right
after Escape moved into the closing card, and focus then fell to the
page when the card was removed, losing the user's place. The closing
card is now inert and the arrow keys skip it.

* fix(dashboard): save the Apps layout one write at a time

The app order and folders were saved with a write per change, without
waiting for the one before. Changes close together (ejecting an app from
a folder saves, and the drop right after saves again) put two writes of
the same key in flight, and if they landed out of order the older layout
was what the next load showed. Each key now has one write in flight at a
time, and the next carries only the newest layout.

* fix(dashboard): close an open dialog when browser Back switches tabs

Back and Forward between dashboard tabs closed the drawer, menus, Share
and Properties, but not the dashboard's dialogs (Edit Profile, Change
password, confirmations) or the photo crop. The tab changed underneath a
dialog that stayed up with focus inside it. They now close through their
own close buttons, like the rest.

* fix(dashboard): count the trial countdown in calendar days

The Home tab's trial badge rounded the time left up to whole 24-hour
blocks, so it disagreed with the date shown under it: a trial ending at
3pm today read "1 day left" beside today's date and never "Ends today",
and one ending tomorrow morning read "2 days left". It now counts
calendar days in the user's time zone.

* perf(gui): rewrite paths after a username change only on elements that carry one

Changing the username from the dashboard's Account tab rewrote paths on
every element in the document (the selector matched them all) and set
every username label once per element, which grows with the size of the
Files listing on screen. It now visits only elements with a path
attribute and sets the labels once. With no earlier username (signing
in) there's nothing to rewrite, which also stops it throwing then.

* fix(dashboard): make long resource names in the Usage table readable

Resource names in Usage details are cut off with an ellipsis, and on a
phone the resource column is the one that shrinks, so a key like
"openai:gpt-4o-mini-2024-07-18:prompt_tokens" read "openai:gpt-4o-mi…"
with no way to see the rest. The full name now shows on hover, and on
phones, which have no hover, the name wraps instead of being cut.

* fix(dashboard): show Usage units in the dashboard's language

In Usage details the Cost column followed the dashboard's language but
the Units column beside it always used English separators, so a German
row read "1,234,567 | 1,23 $", where the units look like 1.2. Units now
use the language's separators too ("1.234.567").

* fix(gui): translate the 2FA setup wizard's step labels and messages

The two-factor setup opened from the dashboard's Security tab had
English hardcoded in its progress steps (Scan, Backup, Verify, Done),
the "Enabling…" spinner, the success message and its Done button, the
copy-secret button's label, the "Copied!" confirmation and the printed
recovery codes' heading. They go through i18n now.

* fix(dashboard): leave Alt and Cmd arrow shortcuts to the browser in Files

Files took every arrow, Home/End and Page key to move the selection,
whatever modifier was held, so Alt+Left (Back) and Cmd+Up/Down moved the
selection instead of doing what the browser does with them. Arrow keys
with Alt or Cmd are left alone now.

* fix(dashboard): explain in the user's language why a Trash item can't be restored

An item moved into the Trash by an app or the API has no record of where
it came from. Choosing Restore on it showed "Cannot restore: original
path not found in metadata", in English and in internal terms. The
message now says which item it is and why, through i18n.

* perf(dashboard): delete Trash items permanently a few at a time

"Delete permanently" on a Trash selection deleted one item after another,
a round trip each: 60 items took about 8s at 100ms of latency, and a few
hundred took most of a minute. Six run at once now (60 items: about
1.3s). The bounded runner the notification center used for "Mark all
read" moves to a shared helper for both.

* fix(dashboard): don't leave a "folder is empty" notice over the next folder's rows

Opening a folder clears the pane while its listing loads. If something
finished in that time and found the pane empty (a Trash restore, a
failed new folder), it added the "This folder is empty" notice, and the
listing's rows were then drawn under it. The listing now clears a stale
notice before drawing its rows.

* fix(dashboard): restore create_file when two "New" creates overlap in Files

Each "New > Text Document" (or HTML, Web Link…) in Files swaps in its own
window.create_file while it runs and put back whatever it found when it
finished. When two overlapped and the first finished first, the second
put back the first's stand-in for good. Any later create that reached it
counted a create that never ended, and from then on files added by apps
or other tabs stopped showing in the listing until a reload. The stand-ins
are now tracked together, and the real function comes back once none is
left.

* fix(dashboard): don't let a slow "New" file pull the user back or land in another listing

After "New > Text Document" (or HTML, Web Link…) in Files, the new file's
row is drawn once its upload finishes. If the user opened another folder
meanwhile, then:
- if that folder was still loading, the row (with its rename box) was
  drawn into the loading pane and the spinner removed, and the folder
  then lost its "empty" notice;
- if it had loaded, the user was taken back to the folder the file was
  made in.
The row is now drawn only if the folder is still on screen, and the user
is only taken to the folder whose menu made it if they haven't moved.

* fix(dashboard): keep an uninstalled app off the Apps grid until its uninstall lands

Uninstalling an app takes its tile away at once, but the server keeps
listing the app as installed until the revoke request finishes. A grid
refresh in that time (switching tabs, coming back to the browser tab)
put the tile back, and it stayed until the next refresh; clicking it
reinstalled the app. Refreshes now leave the app out until the revoke
settles, and ones started before it landed are discarded.

* fix(dashboard): keep keyboard focus when uninstalling an app from inside a folder

After uninstalling an app from the Apps grid, focus goes to the tile that
took its place. An app uninstalled from inside an open folder had no
place on the grid, so focus fell to the page, and a keyboard user had to
start over from the top. Focus now goes to the folder's next app, to the
folder itself, or, when only one app was left and the folder dissolved,
to that app.

* fix(gui): show sizes under 1 KB as whole bytes

The size formatter wrote a size under 1 KB with two decimals and a
mismatched unit: a new account's storage on the dashboard read "0 Byte
Used", and a small file "149.00 Bytes". Bytes are now whole, with the
unit in the right number: "0 Bytes", "1 Byte", "149 Bytes".

* fix(dashboard): show the bottom edge of the Files drop outline in list view

Dragging files over the Files listing outlines it with a dashed border,
but in list view the bottom edge was missing: the listing cleared the
footer with padding, so its box ran under the footer and the footer hid
the outline's bottom edge. It now clears the footer with a margin, as
the grid view already did, so the outline shows all the way round.
2026-11-08 09:55:22 -07:00
Nariman Jelveh 0fdf40b8da Fill the send-feedback popup with the feedback dialog; fix <puter-user-menu> arrow focus (#4539)
* Fill the send-feedback popup with the feedback dialog

The popup opened by puter.ui.showFeedbackDialog() drew the floating modal
over a scrim. It now fills the window like the custom domain popup, with
the actions pinned to the bottom and the message box taking the spare
height. The full-page styles move from .custom-domain-fullpage to a shared
.app-feedback-fullpage on the modal shell.

* Land the first ArrowDown on the first <puter-user-menu> item after a click

Opening the menu with the pointer focused the first item without a visible
focus ring, so ArrowDown appeared to skip it. A pointer open now focuses the
menu itself; opening from the keyboard still focuses the first item.
2026-11-07 21:43:21 -07:00
Nariman Jelveh 5b850113ff Show sign-in and file pickers in a frame on first-party pages (#4537)
* Show sign-in and file pickers in a frame on first-party pages

puter.auth.signIn(), showOpenFilePicker(), showSaveFilePicker() and
showDirectoryPicker() on a page one label under the Puter origin now
show the GUI popup in a frame over the page instead of a window. Every
other page, and any frame that can't serve the request, keeps the popup
exactly as before, so apps need no changes.

/embed/sign-in and /embed/show-{open-file,save-file,directory}-picker
render the normal GUI shell with the framing rules, query redirect and
no-store of /embed/apps, and name the framing page (read from the
referrer, pinned with frame-ancestors) as embed_host_origin. The GUI
runs the popup boot unchanged with that page as window.opener; close()
asks it to remove the frame. Popup parameters come from the fragment
(msg_id, options, request_auth, and user for a picker).

Sign-in has a close button that rejects with auth_window_closed, as
closing the popup does. A federated provider can't be framed, so
choosing one continues in the popup. attempt_temp_user_creation and
cross-origin isolated pages keep the popup.

Pickers frame only for a signed-in page whose user owns the GUI
session; otherwise the GUI reports unavailable and the page opens the
popup, through a consent dialog if the click has expired. The page
keeps its own token rather than adopting the one sent with the answer.

The feedback embed and the new frames share one loading/ready state
machine in guiEmbed.js.

* Close a framed picker when Escape cancels it

Escape closes only the picker window inside the frame, so the frame now
closes with it. The mismatched-user e2e test waits for sign-in's whoami
before naming another user, which it could otherwise overwrite.

* Close a framed sign-in on Escape

The sign-in shows as a page in the frame, which ignored Escape. Escape
now asks the framing page to close it, as its close button does, unless
a request is in flight.

* Focus a framed sign-in once the page focuses the frame

A cross-origin frame's focus() calls do nothing until the framing page
focuses it, so the sign-in opened with focus on the body: keys missed
the form and Escape never reached the dialog. The screen now takes
focus when the frame gains it, unless something already has it.

* Skip the frame on subdomains the server refuses to let frame

The SDK framed embeds from any page one label under the GUI origin, but
the server refuses its own subdomains (api, docs, apps, ...). On those,
sign-in and the pickers waited on a refused frame before opening the
popup, which Safari could then block for want of a recent click. They
open the popup straight away now, and the user menu hides Apps there,
as its docs say.

* Keep a framed picker on the framing page's user

With several saved accounts, a framed picker could still show the
account picker (no grant yet, or request_auth in its fragment). Picking
another account switched the session inside the frame, and the picker
then ran with the first user's details and the second user's token,
saving that token under the first account. A framed picker already
holds the framing page's user, so it no longer offers to switch, and
its user check no longer writes the boot's shared whoami.

* Take a save picker's file only from the page that asked

The save picker opened a dialog for every showSaveFilePickerPopup
message from any window. In a frame, a sibling frame of the framing
page can reach it, so it could supply the bytes the user then saved,
or stack extra dialogs. The picker now takes the file once, and only
from its opener.

* Reopen a framed embed that reloads itself into a refused frame

A reload or navigation the embed starts itself names the embed as its
referrer, so the server refuses the framing: logging out at an account
gate, a sign-out in another tab, or a reset session left a blank box.
The page now shows its spinner and opens the embed again itself, once
per ready, and falls back as for a refused frame if that fails too.

* Correct why a temporary-account sign-in keeps the popup

notes.puter.com and puter.com are the same site, so browsers don't
partition the frame's storage. The popup is kept because only it
creates a temporary account.

* Remove the picker's consent dialog once answered

The consent dialog a picker shows when its click has expired was
closed but left in the page after Continue or Cancel, one element per
fallback. It is removed now, as the other consent dialogs are.

* Name the embed dialog for assistive tech

The modal dialog around a framed embed had no accessible name, so
screen readers announced an unnamed dialog. It is labelled with the
frame's title.

* Expose an embed's loading status only while it shows

The spinner is faded out rather than removed, so assistive tech still
read a Loading status over a frame that was ready. The status is
hidden from it whenever the spinner isn't showing.

* Let a popup frame fill in texted codes

Signup and the account checks fill a texted code in through WebOTP,
which a cross-origin frame may only use when allowed. Popup frames now
allow otp-credentials, so the code fills in as it does in the popup.

* Close a popup frame on Escape from the page too

Once a popup frame was ready, Escape with focus on the page (after a
click on the scrim) only moved focus back into the frame, so it took a
second Escape to close, and none if the frame stopped responding. It
now closes the frame, as Escape inside it does. The feedback embed
still leaves dismissal to the embed.

* Keep scrolling over an embed's scrim from scrolling the page

A wheel or swipe over the scrim around a framed embed chained to the
page behind the modal. The dialog now contains its overscroll.

* Frame a GUI popup only at its exact path

Routes match case-insensitively and with a trailing slash, so another
spelling of /embed/<action> was served frameable while the shell booted
the desktop for it. Any other spelling now redirects to the exact path.

* Prefill a web save picker with the suggested name

The popup and framed save pickers ignored suggestedName and opened with
an empty filename, unlike the save picker an app gets in the desktop.

* Open a provider picked in a framed sign-in straight in the popup

Picking a provider in the framed sign-in fell back to the sign-in popup,
where the user had to pick the provider again. The frame now hands the
page the provider's start URL for its sign-in, and the page opens the
popup there after checking it is the GUI's OIDC start for that sign-in.

* Open the popup straight away for a page framed by another site

frame-ancestors on the embeds allows only the GUI and the framing page,
so a first-party page inside another site's frame waited about a second
on a refused frame before sign-in or a picker opened the popup, by which
time Safari asks for another click. Such a page opens the popup straight
away now, as it did before the frame.

* Close a framed sign-in on Escape at its account checks

Escape closed the framed sign-in on its auth screens but did nothing once
signup moved on to confirming the email, phone or card, which show as
windows. Escape there now closes the frame as well, as its close button
does.

* Leave Escape that cancels an IME conversion to the IME in sign-in

In a framed sign-in, Escape pressed to cancel an IME conversion in a
field also closed the sign-in, losing what was typed. The auth screens
and the account checks now leave an Escape mid-composition to the IME,
as the dashboard's text fields do.

* Fade in a popup frame's scrim and box

The scrim behind a framed sign-in or picker, and the box's white
background, appeared at full opacity in one frame while the GUI's content
faded in over them. Both now fade in briefly, unless the user prefers
reduced motion.

* Keep a page's own styles off a popup frame's dialog, frame and close button

The dialog, its frame and the close button live in the page's document, so
a page's global dialog, iframe and button rules reached them: a transform
that centers the page's own dialogs moved the overlay half off screen, an
iframe max-width shrank the frame inside its box, and button margins and
min sizes turned the close button into a pill. The dialog and frame now
reset those properties inline, and the close button starts from all:unset.

* Fall back to the popup when a reopened embed frame never loads

After a refused self-reload the page opens the embed again, but nothing
timed that second start: a stalled request left the loading spinner up
until the user dismissed it, which settled the request as closed instead
of opening the popup. The reopened frame now gets the same ready timeout
as the first start.

* Hand a save picker frame its file again after it reloads

The page answered a picker's sendMeFileData once and then stopped
listening. A framed save picker that reloads asks again, and without an
answer it never opened its dialog, so the page waited out the ready
timeout and fell back to the popup. A frame is now answered until it
closes; a popup is still answered once.

* Keep a popup frame's white backing off its rounded edge in dark mode

The box's white background shared the frame's anti-aliased corner, so a
light arc showed around the dark sign-in. The backing now stops 1px inside
the edge, which leaves the curve to the frame and the scrim.

* Center a popup frame's box with margins so it fills a phone edge to edge

On a phone with a fractional pixel ratio the translate that centered the
full-screen box left it a fraction of a pixel short of the top, so a line
of scrim showed above the sign-in. Margins snap to device pixels.

* Focus a framed save picker's file name as the popup does

A cross-origin frame's focus() does nothing until the page focuses it, and
the page focusing it again resets it, so the framed save picker opened with
nothing focused and typing went nowhere. A framed popup now gets its latest
focus() back whenever the frame gains focus with nothing focused, until the
user presses a key or the pointer in it.

* Replace a showing popup frame instead of stacking a second one

A double click on a page's picker button framed two GUIs, one over the
other under two scrims, and the user had to cancel twice. A popup reuses
its named window instead. Now a newer popup frame dismisses the one
showing, which settles it as closed.

* Share one framed sign-in between calls made while it's open

A double click on a page's sign-in button started a second framed
sign-in, which replaced the first and rejected it as closed while the
user was still signing in. A call made while a framed sign-in with the
same request_auth is open now settles with it.

* Close only a context menu on Escape over a file dialog

Escape to dismiss a right-click menu in a file dialog also closed the
dialog, which cancels a picker and, in a framed picker, removes the whole
frame. An open context menu now takes the Escape on its own.

* Document that one popup frame shows at a time

* Keep a save dialog's file name and buttons on a phone's screen

The save dialog's bottom bar couldn't shrink below the file name's own
width, so on a phone it spilled off both edges: the name's start and
the Save button were cut off. A framed save picker fills the phone, so it
always hit this. The name now shrinks to fit.

* Keep a file dialog's buttons on one line on a small phone

On a 320px-wide screen the open picker's Open button wrapped under
Cancel and ran into the screen's edge, and Upload wrapped under its
icon. The buttons now stay on one line, with less side padding up to
480px wide so the save dialog's file name keeps room to show.

* Say what a picker is for when its popup needs a click to open

When a framed picker falls back to the popup after the click expired,
the consent dialog said the page used Puter for AI and Cloud features,
not that it wanted a file or folder. It now names the picker, as the
push notification dialog does.

* Share a framed sign-in with a call that differs in request_auth

Implicit auth asks with request_auth, so an explicit signIn() made while
it was open replaced its frame and rejected every puter.* call waiting
on it. Any framed sign-in in progress is now joined.

* Keep a popup frame open when a frame for another popup shows

A sign-in replaced a showing picker frame, cancelling the picker, and
the other way round, though their popups have different window names and
stay open together. Only a frame for the same popup is replaced now.

* Reopen a refused embed frame in WebKit

WebKit took the refused URL as a fragment navigation of its error page
and loaded nothing, so a frame that reloaded itself fell back to the
popup. The frame is reopened through the server's query redirect, which
keeps the fragment.

* Go back to the save picker on Escape at its replace prompt

Escape closed the alert without an answer, so the save waited on it
forever behind the picker's busy mask. In a frame nothing could close
the picker after that. The alert closing now counts as Cancel.

* Leave Safari's IME-ending Escape alone at a framed sign-in's account checks

Safari sends the Escape that ends an IME conversion with keyCode 229 and
isComposing false, so it closed the framed sign-in. The auth screens
already check for it.

* Keep an open dialog's buttons whole on a small phone in German

At 320px, German's longer labels pushed the Open button past the screen
edge, since the upload label couldn't shrink and the buttons could. The
upload label now gives way, cut off with an ellipsis.
2026-11-07 20:56:19 -07:00
Nariman Jelveh 51bf8697a8 Drop the alert after a provider signup lands on an existing account (#4538)
Signing up with a provider whose account already exists signs the user
in to it, then showed a blocking "You have been logged in to an existing
account" alert. In a sign-in popup it held back the token until the user
clicked OK, right before the popup closed. The sign-in now completes
without it, and the callback no longer adds oidc_switched to the return
URL.
2026-11-07 17:59:54 -07:00
Nariman Jelveh 8c14e7de30 Add account links and Send feedback to <puter-user-menu> (#4536)
The signed-in menu now shows the user's picture, username and email
(hidden when whoami leaves the email out), links to Manage account,
Usage and Open Puter on the GUI origin in a new tab, and Send feedback,
which calls puter.ui.showFeedbackDialog(). Each new item takes a label
attribute like the existing ones. The menu opens on its first item, and
the arrow, Home and End keys move between items.
2026-11-07 11:40:42 -07:00
Filip Kujundžić 84faf5c40c refactor(ai): rename convo API to realtime (#4524) 2026-11-07 10:55:35 -07:00
Rasad Regmi aba2666f26 fix(gui): improve dark mode input visibility (#4450) 2026-11-07 09:32:33 -07:00
tgarver-gdm abe6882de9 Import AGENTS.md from CLAUDE.md so Claude Code loads it (#4529) 2026-11-07 09:21:28 -07:00
jelveh 770af49528 Show a loading scrim while the feedback embed checks the session
The framed feedback dialog stayed hidden until the embed had checked the
GUI session, so on a slow network a click on Send feedback showed
nothing for seconds, and a second click silently resolved false. A
check that takes longer than 200ms now shows a scrim with a spinner over
the page, matching the embed's own scrim, which takes its place once
the dialog is ready. Escape or a click on it cancels, and a refused
session still falls back to the popup.
2026-11-07 02:12:59 -07:00
jelveh 47b4cc1158 Follow the OS theme in the feedback embed on Safari
WebKit gives a frame the color-scheme of its iframe element as its
preferred scheme, and the feedback embed's frame was set to normal, so
Safari in dark mode showed the light dialog. The frame and the embed's
root now both declare light dark: the embed follows the OS in every
engine, and the frame stays transparent because the two match.
2026-11-07 02:12:06 -07:00
Nariman Jelveh e253a7f862 Show the app feedback dialog in a frame on first-party pages (#4534)
* Show the app feedback dialog in a frame on first-party pages

puter.ui.showFeedbackDialog() on a page one label under the Puter origin
now frames /embed/feedback in a modal over the page, instead of opening a
puter.com popup. The embed uses the same framing rules, query redirect,
and host-user binding as /embed/apps, and sends the feedback for the
framing page's origin, which the server reads from the referrer and pins
with frame-ancestors. Without the named user's Puter session, or with an
email still to confirm, the embed reports itself unavailable and the SDK
falls back to the popup, which can sign in. A frame that never answers
(such as one the host's CSP blocks) falls back too.

The embed session check moves out of appsEmbed.js into a shared helper,
and the first-party page check out of <puter-user-menu> into puter.js's
lib, so both embeds use one copy of each.

* Keep the framing page from scrolling behind the feedback embed

A wheel or touch scroll over the feedback embed's scrim chained out of
the frame and scrolled the first-party page underneath the open dialog.
The embed's root now stops overscroll from leaving the frame.

* Name the feedback embed's user from the cached whoami

showFeedbackDialog() on a first-party page fetched /whoami before it
started loading the embed, adding a round trip to every call. It now
uses the user puter.js already cached for the session and asks the API
only when there is none.
2026-11-07 01:57:47 -07:00
Nariman Jelveh af35783df4 Test native app hosts with apps still served natively (#4535)
editor, markus and pdf moved to standard host routing, but the host
redirect and embed framing tests still used editor as their native app,
so they failed. They now use about, and a test checks that editor,
markus and pdf are passed through even with a folder under the native
apps root.
2026-11-07 01:39:45 -07:00
908 changed files with 43695 additions and 25030 deletions

No files matched your search

+8 -5
View File
@@ -2,7 +2,7 @@
# The last matching rule takes precedence.
- @Salazareo
* @Salazareo
# GUI
@@ -18,7 +18,10 @@
# Backend AI
/src/backend/controllers/puterai/ @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/ai-\*/ @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/ai/ @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/util/aiCostFactor* @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/util/aiPrices* @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/util/aiProviderKey* @404oops @ProgrammerIn-wonderland @Salazareo
# Backend: AI, hosting, workers, networking, and WebDAV
@@ -29,7 +32,7 @@
/src/backend/drivers/workers/ @ProgrammerIn-wonderland @Salazareo
/src/backend/services/localworker/ @ProgrammerIn-wonderland @Salazareo
/src/backend/stores/subdomain/ @ProgrammerIn-wonderland @Salazareo
/extensions/workerSandbox\* @ProgrammerIn-wonderland @Salazareo
/extensions/workerSandbox* @ProgrammerIn-wonderland @Salazareo
/src/worker/ @ProgrammerIn-wonderland @Salazareo
/src/worker-types/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/peer/ @ProgrammerIn-wonderland @Salazareo
@@ -42,7 +45,7 @@
/src/backend/services/permission/ @jfcastro92 @Salazareo
/src/backend/services/acl/ @jfcastro92 @Salazareo
/src/backend/services/team/ @jfcastro92 @Salazareo
/src/backend/services/apps/AppPermissionService\* @jfcastro92 @Salazareo
/src/backend/services/apps/AppPermissionService* @jfcastro92 @Salazareo
/src/backend/stores/share/ @jfcastro92 @Salazareo
/src/backend/stores/permission/ @jfcastro92 @Salazareo
/src/backend/stores/group/ @jfcastro92 @Salazareo
@@ -59,7 +62,7 @@
/src/puter-js/tests/api/suites/hosting.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/workers.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/net.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Peer\* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Peer* @ProgrammerIn-wonderland @Salazareo
# SDK: sharing and permissions
+106
View File
@@ -0,0 +1,106 @@
# This file is machine-generated by `gh actions-lock`.
# Do not edit by hand; run `gh actions-lock` to update.
# Docs: https://gh.io/actions-lockfile
version: 'v0.0.3'
workflows:
'.github/workflows/ai-provider-integration-tests.yaml':
- 'actions/[email protected]'
- 'actions/[email protected]'
'.github/workflows/backend-tests.yaml':
- 'actions/[email protected]'
- 'actions/[email protected]'
- 'actions/[email protected]'
- 'actions/[email protected]'
- 'davelosert/[email protected]'
- 'dorny/[email protected]'
'.github/workflows/dependabot-automerge.yml':
- 'dependabot/[email protected]'
'.github/workflows/docker-image.yaml':
- 'actions/[email protected]'
- 'docker/[email protected]'
- 'docker/[email protected]'
- 'docker/[email protected]'
- 'docker/[email protected]'
- 'docker/[email protected]'
- 'peter-evans/[email protected]'
'.github/workflows/docs-build.yml':
- 'actions/[email protected]'
- 'actions/[email protected]'
'.github/workflows/notify-dev.yaml': []
'.github/workflows/notify-prod.yaml': []
'.github/workflows/puterjs-tests.yaml':
- 'actions/[email protected]'
- 'actions/[email protected]'
- 'actions/[email protected]'
- 'actions/[email protected]'
- 'davelosert/[email protected]'
- 'dorny/[email protected]'
'.github/workflows/tag-alias.yaml':
- 'docker/[email protected]'
- 'docker/[email protected]'
dependencies:
'actions/[email protected]':
ref: 'v7.0.1'
commit: 'sha1-3d3c42e5aac5ba805825da76410c181273ba90b1'
owner_id: 44036562
repo_id: 197814629
'actions/[email protected]':
ref: 'v8.0.2'
commit: 'sha1-9000827ccba6bdab643e8b6fd33ac0654aef8333'
owner_id: 44036562
repo_id: 192626254
'actions/[email protected]':
ref: 'v7.1.0'
commit: 'sha1-949feb2413d6458794dcd2491c4babbbce0c15c1'
owner_id: 44036562
repo_id: 189476904
'actions/[email protected]':
ref: 'v7.0.2'
commit: 'sha1-cf430e030ddbb5b0abf93d22962f4752f3646cd9'
owner_id: 44036562
repo_id: 192625955
'davelosert/[email protected]':
ref: 'v2.13.2'
commit: 'sha1-74c2d9210b4fc44c84d8213f61d7ec99837b7aa0'
owner_id: 4287128
repo_id: 501176456
'dependabot/[email protected]':
ref: 'v3.1.0'
commit: 'sha1-25dd0e34f4fe68f24cc83900b1fe3fe149efef98'
owner_id: 27347476
repo_id: 371068214
'docker/[email protected]':
ref: 'v7.4.0'
commit: 'sha1-c3c9e263c25d99ce0380d002d59b67737d91b0dc'
owner_id: 5429470
repo_id: 241092383
'docker/[email protected]':
ref: 'v4.8.0'
commit: 'sha1-8aa5831fb8d48d8f76594c7ddde61ae58019f550'
owner_id: 5429470
repo_id: 287743349
'docker/[email protected]':
ref: 'v6.2.0'
commit: 'sha1-dc802804100637a589fabce1cb79ff13a1411302'
owner_id: 5429470
repo_id: 306769011
'docker/[email protected]':
ref: 'v4.4.1'
commit: 'sha1-f87e5991a6d7451dcb8d9637bfbc97413f497069'
owner_id: 5429470
repo_id: 288485773
'docker/[email protected]':
ref: 'v4.4.0'
commit: 'sha1-99012661954931238ded8c8b007157a8430204e1'
owner_id: 5429470
repo_id: 288487894
'dorny/[email protected]':
ref: 'v4.0.3'
commit: 'sha1-ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d'
owner_id: 1588572
repo_id: 265601040
'peter-evans/[email protected]':
ref: 'v5.0.0'
commit: 'sha1-1b9a80c056b620d92cedb9d9b5a223409c68ddfa'
owner_id: 18365890
repo_id: 193685475
@@ -17,10 +17,12 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -29,10 +31,9 @@ jobs:
run: npm ci
- name: Run AI provider integration tests
# Vitest treats positional args as filename substring filters.
# The pattern matches every `*.integration.test.ts` file and
# nothing else — the regular backend test suite stays out.
run: npm run test:backend -- integration.test
# Only the provider tests: the other `*.integration.test.ts`
# files need no credentials and run with the backend suite.
run: npm run test:backend -- $(git ls-files 'src/backend/drivers/ai/**/*.integration.test.ts')
env:
CI: 'true'
PUTER_TEST_AI_CLAUDE_API_KEY: ${{ secrets.PUTER_TEST_AI_CLAUDE_API_KEY }}
@@ -47,13 +48,22 @@ jobs:
PUTER_TEST_AI_MOONSHOT_API_KEY: ${{ secrets.PUTER_TEST_AI_MOONSHOT_API_KEY }}
PUTER_TEST_AI_ZAI_API_KEY: ${{ secrets.PUTER_TEST_AI_ZAI_API_KEY }}
PUTER_TEST_AI_ELEVENLABS_API_KEY: ${{ secrets.PUTER_TEST_AI_ELEVENLABS_API_KEY }}
PUTER_TEST_AI_ALIBABA_API_KEY: ${{ secrets.PUTER_TEST_AI_ALIBABA_API_KEY }}
PUTER_TEST_AI_AZURE_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_AZURE_OPENAI_API_KEY }}
PUTER_TEST_AI_AZURE_OPENAI_API_URL: ${{ secrets.PUTER_TEST_AI_AZURE_OPENAI_API_URL }}
PUTER_TEST_AI_BYTEPLUS_API_KEY: ${{ secrets.PUTER_TEST_AI_BYTEPLUS_API_KEY }}
PUTER_TEST_AI_HOONIFY_API_KEY: ${{ secrets.PUTER_TEST_AI_HOONIFY_API_KEY }}
PUTER_TEST_AI_INFRON_API_KEY: ${{ secrets.PUTER_TEST_AI_INFRON_API_KEY }}
PUTER_TEST_AI_KLING_API_KEY: ${{ secrets.PUTER_TEST_AI_KLING_API_KEY }}
PUTER_TEST_AI_META_API_KEY: ${{ secrets.PUTER_TEST_AI_META_API_KEY }}
PUTER_TEST_AI_MINIMAX_API_KEY: ${{ secrets.PUTER_TEST_AI_MINIMAX_API_KEY }}
# Realtime conversations through the SDK against live OpenAI. Builds
# the SDK bundle these tests load itself, so it runs even when the
# step above fails; the tests skip themselves without the key.
- name: Run live realtime conversation tests
if: ${{ !cancelled() }}
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/convoLive.test.ts
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/realtimeLive.test.ts
env:
CI: 'true'
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
+18 -12
View File
@@ -23,8 +23,10 @@ jobs:
outputs:
backend: ${{ steps.filter.outputs.backend }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
- uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- uses: dorny/[email protected]
id: filter
with:
filters: |
@@ -52,10 +54,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -80,13 +84,14 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
ref: ${{ github.event.pull_request.head.sha || github.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -106,7 +111,7 @@ jobs:
- name: Upload blob report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v7.0.2
with:
name: backend-blob-${{ matrix.shard }}
path: .vitest-reports/
@@ -146,13 +151,14 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
ref: ${{ github.event.pull_request.head.sha || github.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -161,7 +167,7 @@ jobs:
run: npm ci
- name: Download blob reports
uses: actions/download-artifact@v8
uses: actions/download-artifact@v8.0.2
with:
pattern: backend-blob-*
path: .vitest-reports
@@ -173,7 +179,7 @@ jobs:
CI: 'true'
- name: Upload coverage artifact
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v7.0.2
with:
name: backend-coverage-${{ github.event_name == 'push' && 'main' || 'pr' }}
path: src/backend/coverage/
@@ -204,7 +210,7 @@ jobs:
- name: Report coverage on PR
if: github.event_name == 'pull_request'
uses: davelosert/vitest-coverage-report-action@v2
uses: davelosert/vitest-coverage-report-action@v2.13.2
with:
json-summary-path: src/backend/coverage/coverage-summary.json
json-final-path: src/backend/coverage/coverage-final.json
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
steps:
- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v3
uses: dependabot/fetch-metadata@v3.1.0
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
+10 -8
View File
@@ -45,30 +45,32 @@ jobs:
fi
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v4
uses: docker/login-action@v4.8.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to Docker Hub
uses: docker/login-action@v4
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
uses: docker/setup-qemu-action@v4.4.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@v4.4.1
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
uses: docker/metadata-action@v6.2.0
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
@@ -82,7 +84,7 @@ jobs:
type=raw,value=main,enable=${{ github.event_name == 'push' || inputs.push_main }}
- name: Build and push Docker image
uses: docker/build-push-action@v7
uses: docker/build-push-action@v7.4.0
with:
platforms: linux/amd64,linux/arm64
context: .
@@ -95,7 +97,7 @@ jobs:
cache-to: type=gha,mode=max
- name: Sync README to Docker Hub
uses: peter-evans/dockerhub-description@v5
uses: peter-evans/dockerhub-description@v5.0.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
+7 -2
View File
@@ -5,16 +5,21 @@ on:
paths:
- 'src/docs/**'
permissions:
contents: read
jobs:
build-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
-40
View File
@@ -1,40 +0,0 @@
name: Maintain Release Merge PR
on:
push:
branches:
- main
jobs:
update-release-pr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Setup Git
run: |
git config user.name github-actions
git config user.email [email protected]
- name: Check for existing PR
id: find-pr
uses: juliangruber/find-pull-request-action@v1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
branch: release
- uses: actions/checkout@v7
if: steps.find-pr.outputs.number == ''
with:
ref: release
- name: Reset release branch
if: steps.find-pr.outputs.number == ''
run: |
git fetch origin main:main
git reset --hard main
- name: Create/Update PR
if: steps.find-pr.outputs.number == ''
uses: peter-evans/create-pull-request@v8
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: Update release branch
title: Merge Release Auto-PR
body: Merging this PR will invoke release actions
branch: auto-update/release
+2
View File
@@ -5,6 +5,8 @@ on:
branches:
- main
permissions: {}
jobs:
notify:
runs-on: ubuntu-latest
+23 -14
View File
@@ -22,8 +22,10 @@ jobs:
outputs:
puterjs: ${{ steps.filter.outputs.puterjs }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
- uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- uses: dorny/[email protected]
id: filter
with:
filters: |
@@ -52,10 +54,12 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -73,10 +77,12 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -130,14 +136,15 @@ jobs:
steps:
- name: Checkout ${{ matrix.ref }}
if: needs.changes.outputs.puterjs == 'true'
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
ref: ${{ matrix.ref }}
repository: ${{ matrix.repository }}
- name: Set up Node.js
if: needs.changes.outputs.puterjs == 'true'
uses: actions/setup-node@v7
uses: actions/setup-node@v7.1.0
with:
node-version: '26'
cache: 'npm'
@@ -161,7 +168,7 @@ jobs:
- name: Upload coverage artifact
if: needs.changes.outputs.puterjs == 'true' && !cancelled()
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v7.0.2
with:
name: puterjs-coverage-${{ matrix.artifact }}
path: src/puter-js/coverage/coverage-*.json
@@ -174,13 +181,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v7.0.1
with:
persist-credentials: false
# Missing when the PR leg failed; `coverage (pr)` already reports that.
- name: Download PR coverage
id: pr-coverage
continue-on-error: true
uses: actions/download-artifact@v8
uses: actions/download-artifact@v8.0.2
with:
name: puterjs-coverage-pr
path: src/puter-js/coverage
@@ -191,14 +200,14 @@ jobs:
id: base-coverage
if: steps.pr-coverage.outcome == 'success'
continue-on-error: true
uses: actions/download-artifact@v8
uses: actions/download-artifact@v8.0.2
with:
name: puterjs-coverage-base
path: coverage-base
- name: Report coverage on PR
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome == 'success'
uses: davelosert/vitest-coverage-report-action@v2
uses: davelosert/vitest-coverage-report-action@v2.13.2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
@@ -207,7 +216,7 @@ jobs:
- name: Report coverage on PR (no base comparison)
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome != 'success'
uses: davelosert/vitest-coverage-report-action@v2
uses: davelosert/vitest-coverage-report-action@v2.13.2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
-18
View File
@@ -1,18 +0,0 @@
on:
push:
branches:
- main
permissions:
contents: write
pull-requests: write
name: release-please
jobs:
release-please:
runs-on: ubuntu-latest
steps:
- uses: google-github-actions/release-please-action@v4
with:
release-type: node
+2 -2
View File
@@ -45,14 +45,14 @@ jobs:
fi
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v4
uses: docker/login-action@v4.8.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@v4.4.1
- name: Create alias tag
env:
+6 -1
View File
@@ -9,6 +9,7 @@ Use these as the source of truth before exploring further:
- [README.md](README.md) — project overview and quickstart.
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), actors, and extensions.
- [doc/contributing-apis.md](doc/contributing-apis.md) — adding and maintaining public APIs end to end (backend surface → puter.js → types → docs → tests). Follow it for any API work.
- [doc/ai-drivers.md](doc/ai-drivers.md) — where AI driver code lives (interfaces, providers, protocols, shared), the import rules between those folders, and how to add a provider or an AI interface.
- [doc/pagination.md](doc/pagination.md) — the one pagination convention for list APIs (limit/cursor/offset/includeTotal, envelope shape, cursor semantics).
- [doc/alarms.md](doc/alarms.md) — raising alarms and picking a severity (what pages, what only gets recorded), plus the config that routes them.
- [doc/self-hosting.md](doc/self-hosting.md) — running Puter outside hosted infra.
@@ -70,7 +71,7 @@ Commit messages and PR descriptions never credit anyone or mention a bug report.
- **Test new behavior.** Every new function, endpoint, driver method, or logic branch gets a test; every bug fix gets a regression test that fails before the fix. If something is genuinely hard to test, skip it but say so in the PR.
- **Boy Scout Rule, proportional to the change.** In files you're already touching, fix the typo, the dead import, the missing test, the bit you had to read twice. Don't ride a refactor along with a bug fix.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't have written, debugged, or defended in review is not.
- **A limit change is not done until the docs change with it.** Every rate limit, concurrency cap, quota, and allowance in the code is published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) — an undisclosed limit is one developers discover as a service failure. If a PR moves any of these numbers, the same PR updates that page. The numbers live in `src/backend/controllers/fs/limits.ts` (filesystem), `src/backend/drivers/util/aiLimits.ts` (all AI drivers), per-driver `rateLimit`/`concurrent` configs, and `src/backend/data/subPolicies/` (free-tier allowances).
- **A limit change is not done until the docs change with it.** Every rate limit, concurrency cap, quota, and allowance in the code is published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) — an undisclosed limit is one developers discover as a service failure. If a PR moves any of these numbers, the same PR updates that page. The numbers live in `src/backend/controllers/fs/limits.ts` (filesystem), `src/backend/drivers/ai/shared/aiLimits.ts` (all AI drivers), per-driver `rateLimit`/`concurrent` configs, and `src/backend/data/subPolicies/` (free-tier allowances).
---
@@ -97,6 +98,10 @@ Layer rules:
- **Drivers and controllers stay thin.** Domain rules a second entry point needs (signup, app CRUD, worker deploy, AI routing/billing) live in a service, not in the first driver or controller that needed them.
- **Prefer explicit arguments over [`Context`](src/backend/core/context.ts) (ALS).** Use it only for request-scoped values that would otherwise thread through many layers; today that's mostly `actor` and `req`.
### AI drivers
AI drivers live in [src/backend/drivers/ai/](src/backend/drivers/ai/): `interfaces/` holds the Puter-facing drivers, `providers/<vendor>/<interface>/` holds vendor code, `protocols/` holds wire protocols several vendors speak, and `shared/` holds helpers more than one interface uses. Follow [doc/ai-drivers.md](doc/ai-drivers.md) when adding or moving AI code.
### Extensions
[extensions/](extensions/) is for **non-crucial parts of the system**: Puter must still work with the extension removed. If core needs to call it, it belongs in core ([whoami](extensions/whoami.ts) is the cautionary example). Inside an extension, follow the same layered structure, unless it only needs a few route handlers, in which case the `extension.get/post/...` helpers are enough on their own. The `extension` API is in [doc/architecture.md](doc/architecture.md#extension-api).
+1 -1
View File
@@ -40,7 +40,7 @@ The following have already been reviewed and determined **not to be vulnerabilit
- **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
- **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
- **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
- **App metadata or app user-count "leaks".** This information is currently public by design. This covers **private apps** too: private apps are an unreleased, admin-only beta for paid listings in the Puter App Center. Their `index_url` and other metadata are not secret, the flag is not user-settable, and no production app relies on it. Reports that a private app's metadata is reachable through a sibling route (taskbar items, `/get-launch-apps`, `index_url` drift on the hosting gate, etc.) are normalization work, not vulnerabilities.
- **App metadata or app user-count "leaks".** This information is currently public by design. This covers **private apps** too: private apps are an unreleased, admin-only beta for paid listings in the Puter App Center. Their `index_url` and other metadata are not secret, the flag is not user-settable, and no production app relies on it. Reports that a private app's metadata is reachable through a sibling route (`/get-launch-apps`, `index_url` drift on the hosting gate, etc.) are normalization work, not vulnerabilities.
- **Worker hot-reload driven by a share recipient.** Sharing a worker's source file with write access is how you let a collaborator change that worker's code and have it deploy live. The redeploy running as the file owner, with the owner's worker credentials, is the intended behavior of that grant — the owner chose to hand out write access to executable code. If you do not want that, do not share the source file with write access.
- **Re-claiming a site or worker name its owner deleted.** A hosted site or worker name that its owner deletes is treated as abandoned by that owner. If one of their apps still points at it, whoever claims the name next serves that app's `index_url` and receives what the app is launched with, and a new app on the same origin may share the old one's identity. Guarding app identities against being recycled this way is ongoing hardening, not a security boundary, so new ways to re-claim an abandoned name, or to inherit an app's tokens or data through one, are not eligible.
- **Share-mode downgrades that do not cascade.** Demoting a delegate does not retroactively revoke the grants that delegate already issued downstream. Shares are independent grants once issued, not a live-evaluated chain. Revoking the downstream grant is a separate, supported action.
+1 -1
View File
@@ -1 +1 @@
FOLLOW ./AGENTS.md
FOLLOW @AGENTS.md
+10
View File
@@ -254,6 +254,16 @@
"user": "",
"pass": ""
}
// Each send names a stream: "auth", "signup" or "notify". A stream
// listed here adds its headers to every message on it, and its "from"
// when the sender gives none. Tags are written to "tagsHeader".
// "streams": {
// "notify": {
// "from": "\"Puter\" <[email protected]>",
// "headers": { "X-Example-Stream": "notify" },
// "tagsHeader": "X-Example-Tags"
// }
// }
},
// ── Inbound mail (separate process) ─────────────────────────────────
+153
View File
@@ -0,0 +1,153 @@
# AI Drivers
Where the AI driver code lives under [src/backend/drivers/ai/](../src/backend/drivers/ai/), what each folder owns, and how to add a provider or a new AI interface. The general layer rules are in [architecture.md](architecture.md) and [AGENTS.md](../AGENTS.md#backend). This page covers what is specific to AI.
## Layout
```text
src/backend/drivers/ai/
├── interfaces/ Puter-facing drivers, one folder per AI interface
├── providers/ Vendor code, as providers/<vendor>/<interface>/
├── protocols/ Wire protocols that several vendors speak
└── shared/ Helpers that more than one interface uses
```
| Folder | Owns | Example |
| --- | --- | --- |
| `interfaces/<interface>/` | The driver class, the interface's contracts in `types.ts`, its provider base class where one exists, and the helpers and tests that apply to every provider of that interface. | [interfaces/chat/ChatCompletionDriver.ts](../src/backend/drivers/ai/interfaces/chat/ChatCompletionDriver.ts) |
| `providers/<vendor>/<interface>/` | One vendor's implementation of one interface: the adapter, its model catalog and prices, its fixtures, and its tests. | [providers/gemini/video/](../src/backend/drivers/ai/providers/gemini/video/) |
| `providers/<vendor>/` | Helpers that several of that vendor's interfaces use. | [providers/gemini/geminiRest.ts](../src/backend/drivers/ai/providers/gemini/geminiRest.ts) |
| `protocols/openaiCompatible/` | The OpenAI wire protocol: the base provider class, message and usage conversion, and parameter mapping for Chat Completions and Responses. | [protocols/openaiCompatible/OpenAICompatProvider.ts](../src/backend/drivers/ai/protocols/openaiCompatible/OpenAICompatProvider.ts) |
| `shared/` | Helpers that more than one interface uses: limits, the provider registry, the remote catalog cache, job polling, streamed charging, and media input. | [shared/aiLimits.ts](../src/backend/drivers/ai/shared/aiLimits.ts) |
Helpers that non-AI code also uses stay in [src/backend/drivers/util/](../src/backend/drivers/util/): file input, data URIs, upstream error mapping, and stream piping.
AI code that a service also needs lives in [src/backend/util/](../src/backend/util/), because a service can't import from the driver layer. The realtime relay in `services/realtime/` bills through the same cost factor as the drivers, so these three files are there:
| File | Holds |
| --- | --- |
| [aiCostFactor.ts](../src/backend/util/aiCostFactor.ts) | `withAiCostFactor`, the metering view every AI driver and the realtime relay record usage through, and `withAiCreditHold`. |
| [aiProviderKey.ts](../src/backend/util/aiProviderKey.ts) | `readProviderKey`, which reads an API key from `config.providers` blocks. |
| [aiPrices.ts](../src/backend/util/aiPrices.ts) | Vendor prices that a driver and a service both bill. |
### Interfaces
| Folder | Driver | `driverInterface` | `driverName` | Key in `puterDrivers` |
| --- | --- | --- | --- | --- |
| `interfaces/chat/` | `ChatCompletionDriver` | `puter-chat-completion` | `ai-chat` | `aiChat` |
| `interfaces/image/` | `ImageGenerationDriver` | `puter-image-generation` | `ai-image` | `aiImage` |
| `interfaces/video/` | `VideoGenerationDriver` | `puter-video-generation` | `ai-video` | `aiVideo` |
| `interfaces/textToSpeech/` | `TTSDriver` | `puter-tts` | `ai-tts` | `aiTts` |
| `interfaces/speechToText/` | `SpeechToTextDriver` | `puter-speech2txt` | `ai-speech2txt` | `aiSpeech2Txt` |
| `interfaces/speechToSpeech/` | `VoiceChangerDriver` | `puter-speech2speech` | `ai-speech2speech` | `aiSpeech2Speech` |
| `interfaces/ocr/` | `OCRDriver` | `puter-ocr` | `ai-ocr` | `aiOcr` |
Folder names are not runtime identifiers. `driverInterface`, `driverName`, driver aliases, the keys under `providers` in config, provider ids, model ids, and usage keys are public contracts. Moving or renaming a file never changes them.
## Request flow
A call goes from the interface to a provider, and from there to a shared protocol when the vendor speaks one:
```text
interfaces/chat/ChatCompletionDriver.ts
validates the call, picks the model and provider, holds credits
→ providers/deepseek/chat/DeepSeekProvider.ts
DeepSeek's catalog, endpoint, and request differences
→ protocols/openaiCompatible/OpenAICompatProvider.ts
the OpenAI-compatible request, response, and stream handling
```
A provider whose vendor has its own API, such as `ClaudeProvider` or `ElevenLabsTTSProvider`, implements the interface's contract directly and uses no protocol.
## Import rules
- `shared/` imports nothing from `interfaces/`, `providers/`, or `protocols/`.
- Nothing in `services/`, `stores/`, or `clients/` imports from `drivers/ai/`. Code that a service and a driver both need goes in `src/backend/util/`.
- `providers/` and `protocols/` import an interface's `types.ts` and helpers. They never import a driver class. Their tests can, to drive a provider through the real driver.
- Only an interface's driver file imports providers, to register them. The interface's helpers don't.
- An interface doesn't import another interface. When a second interface needs a helper, move the helper to `shared/`, or to `providers/<vendor>/` when it belongs to one vendor.
- A vendor folder imports another vendor's folder only when it serves that vendor's models. `providers/azure/` reads the OpenAI and xAI catalogs for that reason.
## Where a file goes
| The code is | It goes in |
| --- | --- |
| Input validation, gates, routing, credit holds, or anything else every provider of the interface gets | `interfaces/<interface>/` |
| A request or response contract that providers implement | `interfaces/<interface>/types.ts` |
| A vendor's endpoint, request shape, catalog, prices, or quirks | `providers/<vendor>/<interface>/` |
| A vendor's auth, REST plumbing, or constants that two of its interfaces use | `providers/<vendor>/` |
| Behavior of a wire protocol that several vendors speak | `protocols/<protocol>/` |
| A helper that two or more interfaces use and that belongs to no vendor | `shared/` |
| A helper that non-AI drivers or controllers also use | `src/backend/drivers/util/` |
| AI code that a service also uses | `src/backend/util/`, in a file named `ai<Something>.ts` |
Keep a helper next to its only consumer. Move it up when a second consumer appears, not before.
When one Puter option becomes a different request shape per vendor, the interface reads the option and each dialect maps it. The chat `compaction` option is read in `interfaces/chat/utils/compaction.js`, mapped for OpenAI by `toOpenAiContextManagement` in `protocols/openaiCompatible/openaiParams.ts`, and mapped for Anthropic by `toAnthropicContextManagement` in `providers/claude/chat/contextManagement.ts`. Tools work the same way: `normalize_tools_object` in the interface, then `toolsToWire` for OpenAI and `claudeToolPolicy` for Anthropic.
Tests sit next to the code they test. Fixtures go in a `fixtures/` folder next to the test that reads them.
## Adding a provider
1. Create `providers/<vendor>/<interface>/`. Use the vendor's existing folder if it has one, so all of a vendor's interfaces stay together.
2. Implement the interface's contract from `interfaces/<interface>/types.ts`:
| Interface | Contract | Base class |
| --- | --- | --- |
| chat | `IChatProvider` | `OpenAICompatProvider` for an OpenAI-compatible vendor |
| image | `IImageProvider` | none |
| video | `IVideoProvider` | `VideoProvider` |
| textToSpeech | `ITTSProvider` | none |
| speechToText | `ISpeechToTextProvider` | `SpeechToTextProvider` |
| ocr | `IOcrProvider` | none |
Speech to speech has one provider, so `VoiceChangerDriver` uses `ElevenLabsSpeechToSpeechProvider` directly. Add a contract to its `types.ts` when a second provider needs one.
3. Put the model catalog and prices in the same folder, as `models.ts` or `costs.ts`. A provider that loads its catalog from the vendor wraps the fetch in `cachedRemoteCatalog` from `shared/`.
4. Register the provider in the interface's driver: in `#registerProviders()`, or in `onServerStart()` for OCR and speech to speech. A chat provider built from an API key and an optional base URL is one row in `KEYED_PROVIDERS` instead. Text to speech and speech to text also list the provider id and its aliases in `providerAliases.ts`.
5. Read credentials from `config.providers[<key>]`. The key is part of the self-hosting config contract, so reuse the vendor's existing key when it has one.
6. Add tests next to the provider. Mock the vendor SDK or `fetch` at the network boundary and drive the provider through the real driver where you can. A live test is a `*.integration.test.ts` file that skips itself, with `skipUnlessEnv` from [testEnv.ts](../src/backend/testEnv.ts), unless its `PUTER_TEST_AI_*` variable is set. Helpers for provider tests are in [shared/integrationTestUtil.ts](../src/backend/drivers/ai/shared/integrationTestUtil.ts).
7. For a chat provider, add its static catalog to `CATALOGS` in [modelCatalogs.test.ts](../src/backend/drivers/ai/interfaces/chat/modelCatalogs.test.ts) and the provider to [providerConsistency.test.ts](../src/backend/drivers/ai/interfaces/chat/providerConsistency.test.ts). The catalog test fails when a `providers/*/chat/` file with "model" in its name exports a catalog that isn't listed.
8. Add the provider's config key to the `providers` config of the interface's [fuzz suite](#fuzz-tests), and its id to the list of providers the suite expects.
9. Update the published docs for the models and limits the provider adds, following [contributing-apis.md](contributing-apis.md).
### OpenAI-compatible vendors
A vendor that speaks the OpenAI Chat Completions protocol extends `OpenAICompatProvider` and overrides only what differs. It still gets its own `providers/<vendor>/chat/` folder.
Vendors share one constants-only folder only when at least six of them differ in nothing but base URL and model catalog. Each then becomes one data file under `providers/openaiCompatible/providers/`, read by a single implementation. No vendor qualifies today, because every subclass overrides request, catalog, or billing behavior.
## Fuzz tests
Two seeded suites send malformed input through the real drivers and providers. They run with the rest of the backend tests.
| Suite | Covers |
| --- | --- |
| [interfaces/chat/ChatCompletionDriver.fuzz.test.ts](../src/backend/drivers/ai/interfaces/chat/ChatCompletionDriver.fuzz.test.ts) | Chat requests with a few values changed, and valid requests against wrong statuses, bodies of the wrong shape, and broken event streams. |
| [interfaces/mediaDrivers.fuzz.test.ts](../src/backend/drivers/ai/interfaces/mediaDrivers.fuzz.test.ts) | Image, video, speech and OCR requests with a few values changed. Every vendor refuses the request. |
What the suites assert:
- A malformed request never fails with an error that has no status, which `/drivers/call` would report as an unclassified 500.
- For chat, with the vendors answering, a malformed request also never fails with a 5xx and never marks a route unhealthy.
- When the upstream misbehaves, chat ends with an answer or an `HttpError`. A stream only has to end: the error line a stream carries isn't checked, so a malformed event can still end one with a 500.
- Requests reach the vendors, and with the vendors answering some chat requests are answered, so a suite that stops exercising the providers fails.
The vendors are played at the HTTP boundary by `upstream` in [shared/fuzzTestUtil.ts](../src/backend/drivers/ai/shared/fuzzTestUtil.ts), which stands in for global `fetch`, undici `fetch` and axios, and makes `https.request` throw so an SDK with its own client fails the suite. The vendors' SDKs run, except Groq's, which is swapped for the OpenAI SDK, and the AWS clients, whose `send` is stubbed. The media suite calls the drivers as a user with unlimited metering, so a request gets past file input and the credit hold to the vendor. A provider that loads its catalog from the vendor registers no models there, so the suites skip it.
A failure lists each distinct finding with its case number and input. Each case is generated from the seed and its own number, so the same case fails again on the next run. Routes are picked by rendezvous hashing: a model added to a catalog takes only the cases it wins, and every other case keeps its route. To search further, raise the case count:
```sh
PUTER_FUZZ_CASES=100000 npx vitest run --config src/backend/vitest.config.ts src/backend/drivers/ai/interfaces/chat/ChatCompletionDriver.fuzz.test.ts
```
When a suite finds a crash, validate the value where the request enters, in the driver or in `normalize_single_message`, so every provider is covered. Reject only what crashed: a value a provider ignored or refused before must still reach it. An option a driver reads as text, a number or a flag goes in the driver's `SCALAR_OPTIONS`, and `assertScalarOptions` from `shared/scalarOptions.ts` answers 400 for a value there that can't be converted to text. Add a unit test for the case next to the fix.
## Adding an interface
1. Create `interfaces/<interface>/` with the driver class and a `types.ts` for the contracts providers implement.
2. Register the driver in [src/backend/drivers/index.ts](../src/backend/drivers/index.ts).
3. Use `AI_RATE_LIMIT` and `AI_CONCURRENT` from `shared/aiLimits.ts`, and publish any new limit in [rate-limits-and-quotas.md](../src/docs/src/rate-limits-and-quotas.md).
4. Meter through `withAiCostFactor` from `src/backend/util/aiCostFactor.ts`, and route between providers with `ProviderRegistry` from `shared/providerRegistry.ts`.
5. List the options the providers read as text, a number or a flag in a `SCALAR_OPTIONS` array, and call `assertScalarOptions` from `shared/scalarOptions.ts` first in each method. Add the driver to [mediaDrivers.fuzz.test.ts](../src/backend/drivers/ai/interfaces/mediaDrivers.fuzz.test.ts).
6. Follow [contributing-apis.md](contributing-apis.md) for the puter.js method, types, docs, and tests.
+13
View File
@@ -42,6 +42,19 @@ block-beta
Extensions sit alongside this stack and can register into any layer; see [Extensions](#extensions).
### AI drivers
The AI drivers have their own layout under [src/backend/drivers/ai/](../src/backend/drivers/ai/):
| Folder | Holds |
| --- | --- |
| `interfaces/` | The Puter-facing drivers, one folder per AI interface (chat, image, video, text to speech, speech to text, speech to speech, OCR). |
| `providers/` | Vendor code, as `providers/<vendor>/<interface>/`. |
| `protocols/` | Wire protocols that several vendors speak, such as the OpenAI-compatible one. |
| `shared/` | Helpers that more than one interface uses. |
A call goes from an interface's driver to a provider, and from there to a shared protocol when the vendor speaks one. [ai-drivers.md](ai-drivers.md) has the ownership rules, the import rules, and the steps for adding a provider.
## Entry point: `PuterServer`
`PuterServer` is the bootstrap. It:
+221 -3
View File
@@ -124,7 +124,7 @@ localization system.
</script>
```
## Implementation and checks
## Implementation
`createAppBrowser` in `src/gui/src/UI/Dashboard/AppBrowser.js` owns the shared UI
and data behavior; its `guestApps` option is the signed-out list. `TabApps.js`
@@ -134,14 +134,232 @@ injects desktop actions. `appsEmbed.js` supplies new-tab launching through
`launch_app`, and loads only the embed entry, its SDK, and shared styles. `embedHostUser` in
`src/gui/src/helpers/embedHostUser.js` reads the host's user from the fragment, and
`embedHostTheme.js` beside it applies the host's theme.
The normal GUI webpack build emits `bundle.min.js` and `apps-embed.min.js`.
The normal GUI webpack build emits `bundle.min.js`, `apps-embed.min.js`, and `feedback-embed.min.js`.
The `/embed/apps` route puts the recommended list in the page's config as
`guestApps`. No puter.js API or app-list endpoint changes are needed.
## Feedback dialog
`GET /embed/feedback` is the app feedback dialog (`puter.ui.showFeedbackDialog()`)
for a first-party page, with the same framing rules, query redirect, and
host-user binding as `/embed/apps`. It sends the feedback for the framing
page's origin, which the server reads from the referrer and pins with
`frame-ancestors`; opened top-level, it does nothing. Its page is transparent
so the dialog's own scrim covers the host.
puter.js uses it on a page one label under the GUI origin, other than the
subdomains the server refuses, when the page is signed in, framing it in a
modal `<dialog>` that stays hidden until the embed is ready. A session check that takes longer than 200ms shows a loading scrim
with a spinner in its place; Escape or a click on it cancels. The embed posts `{ msg: 'feedbackEmbed', state }` to its parent
(`loading`, then `ready` or `unavailable`) and finally
`{ msg: 'feedbackDialogClosed', sent }`. It reports `unavailable` without the
named user's session, or while that user's email still needs confirming, and
puter.js then uses the `/action/send-feedback` popup, which can sign in. So does
a frame that never answers, such as one the host's CSP blocks. `feedbackEmbed.js`
is the entry, sharing `embedSession` in `src/gui/src/helpers/embedSession.js`
with the app browser; `showFeedbackEmbed` in `src/puter-js/src/lib/guiEmbed.js`
is the host side.
## Sign-in and file pickers
`GET /embed/sign-in`, `/embed/show-open-file-picker`, `/embed/show-save-file-picker`
and `/embed/show-directory-picker` serve the same GUI popups as `/action/<action>`
for a first-party page, with the same framing rules, query redirect, and
`no-store` as `/embed/apps`. Any other spelling of the path (another case, a
trailing slash) redirects to it, since the shell boots the desktop for it. They render the normal GUI shell, which names the
framing page (read from the referrer, pinned with `frame-ancestors`) as
`gui_params.embed_host_origin`. The GUI then runs the popup boot unchanged, with
the framing page as `window.opener`, and `window.close()` asks the page to
remove the frame (`src/gui/src/helpers/popupFrame.js`). Without a framing page
to answer (opened top-level, or framed by a refused page) it does nothing. The
popup's parameters come from the fragment instead of the query: `msg_id`,
`options` and `request_auth`, plus `user` for a picker.
The frame posts `{ msg: 'puterPopupFrame', state }` to its parent: `loading` as
soon as the shell boots, `ready` once it shows a window or dialog, `unavailable`
when the page has to use the popup, `provider` when the user picks a sign-in
provider (below), and `closed` where a popup would close. Its
answers (`puter.token`, `fileOpenPicked`, `directoryPicked`, `fileSaved`, the
cancel messages, `sendMeFileData`) are the popup's.
puter.js uses them on a page one label under the GUI origin, other than the
subdomains the server refuses, unless the page is cross-origin isolated or in a
frame of another site (which `frame-ancestors` refuses too).
`showPopupFrame` in `src/puter-js/src/lib/guiEmbed.js` shows the frame in a box
the size of the popup window, over a scrim, hidden until `ready` and behind the
loading spinner after 200ms. Escape closes it, on the page or in the frame. One shows per popup, as each
popup has its own window name: a second picker request of the same kind
replaces the one showing, and a `signIn()` call made while a framed sign-in is
open settles with it. On
screens up to 600px wide the box fills the screen. A frame that is refused,
never answers, or reports `unavailable` falls back to the popup, opened through
a consent dialog when the click that asked for it has expired. A reload or navigation the frame
starts itself (logging out, a sign-out in another tab) names the frame as its
referrer, so the server refuses it; the page then shows its spinner and opens
the frame again, once per `ready`.
- **Sign-in** (`puter.auth.signIn()`) has no host-user binding: it is how the
page gets a user. The box has a close button, which rejects with
`auth_window_closed` as closing the popup does. Providers refuse to be
framed, so choosing one posts `{ state: 'provider', url }` with the
provider's start URL for this sign-in's `msg_id`, then `unavailable`. The
page opens the popup straight at that URL, once it checks the URL is the
GUI's `/auth/oidc/<provider>/start` for this `msg_id`, and the provider
returns it as this sign-in's popup. `attempt_temp_user_creation` keeps the popup, since
only the popup creates a temporary account.
- **Pickers** frame only for a signed-in page. The GUI reports `unavailable`
unless its session belongs to the user named in `user`, and the page then uses
the popup, which can sign in and switch accounts. The token the GUI sends
before a picker's answer isn't adopted, since it is for the page's own user.
Dismissing the loading scrim settles the picker as cancelled.
## Upgrade
`GET /embed/upgrade` is the deployment's upgrade dialog for a first-party page,
served like the popups above: the GUI shell with the same framing rules, query
redirect, exact-path redirect and `no-store`, naming the framing page as
`gui_params.embed_host_origin`. It runs only when framed by an allowed page.
puter.js frames it on a signed-in first-party page (the same pages as the
pickers) for `puter.ui.requestUpgrade(details)`, and for the Upgrade button of
the usage limit dialog a refused call shows there. Elsewhere both keep their
behavior: inside an app `requestUpgrade()` asks the desktop, and on any other
page the usage limit dialog opens the dashboard's upgrade page in a new tab.
The frame covers the page with no box or close button, and the GUI page is
transparent (`html.popup-frame-transparent`) so the dialog's own scrim covers
the host. The frame allows `payment` as well as `otp-credentials`. Escape on the
page is left to the dialog, since one there shouldn't drop a payment in
progress. Its fragment names `user` (the page's signed-in user) and, when the
refusal gave them, `reason` (`funds`, `subscription` or `storage`) and `method`.
The GUI reports `unavailable` unless its session belongs to `user`, the account
has no verification or password change waiting (which the frame couldn't be
closed from), and the deployment defines `window.UIUpgradeAccount` with
`UIUpgradeAccount.embeddable === true`. Otherwise it runs:
```js
new UIUpgradeAccount({ attribution: { appUid, reason, method }, onClose }).open_as_window();
```
`appUid` is the app the framing page's origin resolves to, never one the
fragment names. `open_as_window()` resolves once the dialog shows, and the GUI
then posts `ready`. While the page still hides the frame, which Chromium and
WebKit lay out at 0x0, the GUI posts `ready` first and opens the dialog once
the frame has a viewport, so a window it opens doesn't center on 0x0.
`onClose({ upgraded })` must run exactly once, when the
dialog closes or when it declines to open for the account; the GUI then posts
`{ msg: 'puterPopupFrame', state: 'closed', upgraded }` and the page removes
the frame. If `open_as_window()` rejects, the GUI posts `unavailable` instead. `requestUpgrade()` resolves `{ upgraded }`. A call made while the
frame shows settles with it rather than replacing it, and a call refused while
it shows still rejects but opens no usage limit dialog over it.
When the frame can't show the upgrade (refused, never ready, `unavailable`, or
no user to name), puter.js opens the dashboard's upgrade page in a new tab with
what was refused, through a consent dialog when the click has expired or the
tab is blocked, and `requestUpgrade()` resolves `{ upgraded: false }`.
## Top-up
`GET /embed/top-up` is the deployment's top-up dialog, where a user with a plan
buys add-on credit, served and framed like the upgrade: a transparent GUI page
covering the host with no box or close button, allowing `payment` and
`otp-credentials`, with Escape left to the dialog. Its fragment names `user`.
puter.js frames it for the **Add credits** item of `<puter-user-menu>`, which
shows only when whoami reports `subscribed: true` and no `team` paying for the
account. On a page that can't frame the GUI, the item opens the dashboard with
`?topup=` in a new tab instead.
The GUI reports `unavailable` unless its session belongs to `user`, the account
has no verification or password change waiting, and the deployment defines
`window.UITopUp` with `UITopUp.embeddable === true`. Otherwise it runs:
```js
new UITopUp({ onClose }).open();
```
`open()` resolves once the dialog has loaded the account and shows what it can
buy, and `onClose({ toppedUp })` must run exactly once, when the dialog closes
or when it declines to open for the account. The GUI then posts `{ msg: 'puterPopupFrame', state: 'closed', toppedUp }`
and the page removes the frame. When the frame can't show the top-up, puter.js
opens the dashboard's top-up in a new tab, as it does the upgrade.
While the top-up shows, a call refused for funds still rejects but opens no
usage limit dialog over it, and `requestUpgrade()` frames nothing over it: it
resolves `{ upgraded: false }` once the top-up closes.
## Email confirmation
`GET /embed/request-email-confirmation` walks a first-party page's user through
confirming their email, served like the upgrade: the GUI shell with the same
framing rules, redirects and `no-store`, run only when framed by an allowed
page. puter.js frames it on a signed-in first-party page for
`puter.ui.requestEmailConfirmation()`. Inside an app the desktop shows the same
windows over the app instead.
Like the upgrade, the frame covers the page with no box or close button, over a
transparent GUI page, and its fragment names `user`. Escape closes it, on the
page or on the save-account or code window. On a window, Escape closes that
window as its close button does, so a code already sent still answers. The GUI
reports `unavailable` unless its session belongs to `user` and the account has
no phone, card or password check waiting.
An email check is fine: the boot skips its own email gate for this action, so
the user gets one code window they can close.
The GUI then runs `ensureConfirmedEmail` (`src/gui/src/helpers/ensureConfirmedEmail.js`),
which the desktop's handler uses too: a temporary account gets the save-account
window first, then an unconfirmed email gets the code window, each at most
once. When the page's whoami already reads the email as confirmed,
`requestEmailConfirmation()` resolves `{ response: true }` without framing
anything, and a confirmation in the frame marks that whoami confirmed for the
calls after it. An email the GUI reads as confirmed shows nothing. Before
opening a window the GUI waits for the frame to have a viewport, as the upgrade
does. It then posts
`{ msg: 'puterPopupFrame', state: 'closed', confirmed }`, and
`requestEmailConfirmation()` resolves `{ response: confirmed }`, the field an
app reads from the desktop's answer. A call made while the frame shows settles
with it. A frame that is refused, never ready, or `unavailable` resolves
`{ response: false }`, since there's no popup to fall back to.
## Custom domains
`GET /embed/attach-custom-domain` is the custom domain dialog of
`puter.hosting.attach()`, the popup at `/action/attach-custom-domain`, served
like the upgrade: the GUI shell with the same framing rules, redirects and
`no-store`, run only when framed by an allowed page. puter.js frames it on a
signed-in first-party page (the same pages as the pickers).
The frame covers the page with no box or close button, over a transparent GUI
page, so the dialog floats over the host with its own scrim as it does on the
desktop. The frame allows `clipboard-write`, for the dialog's copy buttons, as
well as `otp-credentials`. Escape on the page is left to the dialog, which
doesn't close while a request is in flight. Its fragment names `user` (the
page's signed-in user), `subdomain` (the site to attach a domain to) and
`msg_id`.
The GUI reports `unavailable` unless its session belongs to `user` and the
account has no verification or password change waiting. It then runs the
popup's action unchanged, with the framing page as the requester the dialog
names: it posts `ready` once the dialog is on the page, answers
`{ msg: 'customDomainAttached', domain, original_msg_id }` once the dialog has
closed and its exit has played, and then posts `closed`. `domain` is the domain the user chose, or
null. `attach()` resolves it. A frame that closes without an answer, or a
loading scrim the user dismisses, resolves null.
A call made while the frame shows resolves null without replacing it, as a
second call does on the desktop. A frame that is refused, never ready, or
`unavailable` falls back to the popup, which can sign in, through a consent
dialog when the click has expired.
## Checks
Run the GUI component/browser tests (with the SDK bundle built and Playwright
Chromium installed), and the backend route tests:
```sh
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser src/feedbackEmbed
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/modules/UI.showTopUp.test.js src/puter-js/src/modules/UI.attachCustomDomainFrame.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
npx vitest run --config src/gui/vitest.config.js src/gui/src/helpers/popupFrame.test.js
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
```
+6 -6
View File
@@ -16,7 +16,7 @@ Core subscribes the browser to a push service only when the server also publishe
`uid` is the notification's uid, a later push with the same `group` replaces the one showing, and `target` and `data` reach the app when the user taps it.
When the browser replaces the subscription (`pushsubscriptionchange`), `sw.js` posts `{ type: 'puter-push-subscription-changed' }` to every open window on Puter's origin. The worker has no token or session, so the page's push client registers the new subscription.
When the browser replaces the subscription (`pushsubscriptionchange`), `sw.js` keeps the new one, or subscribes again with Puter's stored key when the browser gave none and the event, the receipt or an earlier mark names the subscription that went. It writes `puter:subscription-changed:<scope>` to the `puter-push` IndexedDB as `{ at, oldEndpoint }`, the old endpoint kept from an earlier mark no page handled yet, else taken from the event or from the receipt, forgets the receipt kept for that scope, and posts `{ type: 'puter-push-subscription-changed' }` to every open window on Puter's origin. The worker has no token or session, so an open page runs `helpers/pushSync.js` on that message, after any run already in flight, and a later load does the same through the marker.
## Receipts
@@ -59,9 +59,9 @@ Apps on their own origin do the same through `push-sw.js`, which they import int
```ts
interface PushClient {
enable(): Promise<{ deviceUid: string }>;
enable(options?: { background?: boolean }): Promise<{ deviceUid: string }>;
disable(): Promise<void>;
status(): Promise<{
status(options?: { cached?: boolean }): Promise<{
permission: 'default' | 'granted' | 'denied';
registered: boolean;
deviceUid?: string;
@@ -69,12 +69,12 @@ interface PushClient {
}
```
- `enable()` may ask for notification permission, so core calls it from a click: Turn on in the Notifications tab, Allow on a permission prompt that includes `notifications:push` while this browser is off, and the "Turn on notifications" button of the `/action/enable-push` popup and the matching desktop dialog (`UIEnablePushDialog`). Core also calls it without a click, from `helpers/pushSync.js`, when `Notification.permission` is already `granted`, `status()` reports the browser as not registered, the user has not turned push off in this browser, and an app holds `notifications:push`; the browser does not prompt then. The built-in client runs `puter.notifications.enable()`, which subscribes with the `/sw.js` registration and registers the browser through the Web Push flow above (`platform: 'webpush'`). It then resolves the `uid` of this browser's `webpush` device from `listDevices()`. It rejects when permission is refused or the browser wasn't registered.
- `enable()` may ask for notification permission, so core calls it from a click: Turn on in the Notifications tab, Allow on a permission prompt that includes `notifications:push` while this browser is off, and the "Turn on notifications" button of the `/action/enable-push` popup and the matching desktop dialog (`UIEnablePushDialog`). Core also calls it without a click, from `helpers/pushSync.js`, when `Notification.permission` is already `granted`, `status()` reports the browser as not registered, the user has not turned push off in this browser, and an app may push to Puter's devices (`puterDevices` in `GET /push/apps`); the browser does not prompt then. That call passes `{ background: true }`: a browser the user removed in Puter stays removed, and the built-in client rejects with code `push_device_removed`, after which core records the browser as turned off. Before a background call the built-in client asks `POST /push/web/subscriptions/status` about every earlier endpoint it knows: the marker's `oldEndpoint` and the last endpoint the server answered `active` for (`puter.push.last-endpoint:<user uuid>` in `localStorage`). On a 429 it registers nothing and rejects with `too_many_requests`, which the sync reports as `later`, keeping the marker for the next run. When any answer is `removed`, it ends the current subscription in the browser, clears the marker and rejects with `push_device_removed`, so a subscription change or a dropped subscription never brings back a browser removed from another device. A click renews it as before. For a click the built-in client runs `puter.notifications.enable()`, which subscribes with the `/sw.js` registration and registers the browser through the Web Push flow above (`platform: 'webpush'`); in the background it runs the same registration without replacing a removed subscription or opening Puter's sign-in. It then resolves the `uid` of this browser's `webpush` device from `listDevices()` and clears the subscription-changed marker. It rejects when permission is refused or the browser wasn't registered.
- `disable()` runs `puter.notifications.disable()`, which sends `DELETE /push/web/subscriptions { endpoint }` and ends the browser's subscription. The tab calls it for Turn off, and for Remove on the device marked as this browser. Remove on any other device is `DELETE /push/devices/:uid`.
- `status()` reports the state without prompting. `permission` is `Notification.permission`. `registered` is true when this browser has a device registered for the signed-in user, and `deviceUid` is that device's `uid`.
- `status()` reports the state without prompting. `permission` is `Notification.permission`. `registered` is true when this browser has a device registered for the signed-in user and `POST /push/web/subscriptions/status` answers `active` for the current subscription's endpoint, and `deviceUid` is that device's `uid`. A server that can't answer leaves the device match alone, and so does an `unknown` for an endpoint `enable()` registered in this page in the last 5 minutes, since that check can trail a registration. While the subscription-changed marker is set, `registered` is false. With `{ cached: true }` a client may answer with the last registered state it read in this page. Core passes it for the silent registration on app launch and for an app's `puter.notifications.status()`; the Notifications tab and the push dialogs read fresh. The built-in client keeps that state in memory for the same user and subscription for up to 5 minutes, from `enable()` or once the server answered `active` for the endpoint, and answers from it without either request. Reads of the same user and endpoint share one real check while it is in flight. It drops it on `disable()`, a failed `enable()`, an unregistered answer, a set subscription-changed marker, and the worker's `puter-push-subscription-changed` message, so a new page load always asks the server. Whether the calling app may push is checked on every call, against a `GET /push/apps` answer the page reuses for 30 seconds and drops on `enable()`, `disable()`, a grant, a revoke and a user change. When it can't be read, or leaves the app out, the grant alone counts.
After `enable()` resolves, the tab calls `POST /push/devices/:uid/test` and asks the user whether a notification showed up.
The tab also reads `GET /push/apps` and `GET`/`POST /push/preferences`.
Core keeps its own record of a browser the user turned off, `puter.push.browser-off:<user uuid>` in `localStorage`, written by Turn off and by Remove on this browser's device, and cleared when `enable()` resolves. The silent registration above skips a browser with that record, so a device removed here stays removed while one removed from another device, or a new sign-in in a browser that already allows notifications, registers again.
Core keeps its own record of a browser the user turned off, `puter.push.browser-off:<user uuid>` in `localStorage`, written by Turn off, by Remove on this browser's device, and by a silent registration the server refuses with `push_device_removed`, and cleared when `enable()` resolves. The silent registration above skips a browser with that record, so a device removed here or from another device stays removed until the user turns it on from a click, while a new sign-in in a browser that already allows notifications registers again.
+14
View File
@@ -326,6 +326,20 @@ Used for password resets, email confirmation, and notifications. Without it thos
To require email confirmation before login, also set `"strict_email_verification_required": true`.
Account and notification mail is sent on a named stream: `auth` (password, address and 2FA changes), `signup` (address confirmation; a `puter.signup.validate` listener may name another: lowercase letters, digits, `:`, `_` and `-`, up to 64 characters, starting with a letter or digit) and `notify` (share, app feedback and contact form mail). Add `"streams"` inside `"email"` to configure one, for instance to let your mail provider route or report each kind of mail separately:
```json
"streams": {
"notify": {
"from": "\"Puter\" <[email protected]>",
"headers": { "X-Example-Stream": "notify" },
"tagsHeader": "X-Example-Tags"
}
}
```
`headers` are added to every message on the stream, `from` replaces the default sender, and `tagsHeader` names the header that carries a message's tags as `name=value` pairs joined by `, `. A stream that is not listed adds nothing: the message goes out with the default from address and the caller's headers. `streams` is never handed to the transport.
To read the mail yourself instead of sending it, point the transport at a local SMTP sink — [MailHog](https://github.com/mailhog/MailHog) catches everything and serves it at `http://localhost:8025`. Publish both ports; a container that only exposes them is unreachable from the host:
```sh
+40 -10
View File
@@ -257,8 +257,41 @@ describe('whoami extension — handleWhoami', () => {
expect(body.created_ts).toBeGreaterThan(0);
// `directories` is only sent to user actors — confirm it’s present.
expect(body.directories).toBeDefined();
// taskbar_items is only sent to user actors.
expect(body).toHaveProperty('taskbar_items');
});
it('leaves out taskbar items saved on the account', async () => {
const user = await seedUser();
await server.stores.user.update(user.id as number, {
taskbar_items: JSON.stringify([{ type: 'app', name: 'editor' }]),
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
expect(captured.body).not.toHaveProperty('taskbar_items');
});
it('leaves out a desktop background saved on the account', async () => {
const user = await seedUser();
await server.stores.user.update(user.id as number, {
desktop_bg_url: 'https://cdn.test/wallpaper.png',
desktop_bg_color: '#000000',
desktop_bg_fit: 'cover',
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body).not.toHaveProperty('desktop_bg_url');
expect(body).not.toHaveProperty('desktop_bg_color');
expect(body).not.toHaveProperty('desktop_bg_fit');
});
it('reports the SMS-to-card fallback only once a send has opened it', async () => {
@@ -322,7 +355,7 @@ describe('whoami extension — handleWhoami', () => {
expect(flags.payment_bypass).toBeUndefined();
});
it('strips desktop_bg_*, created_ts and human_readable_age fields for app actors', async () => {
it('strips created_ts and human_readable_age fields for app actors', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
@@ -340,9 +373,6 @@ describe('whoami extension — handleWhoami', () => {
expect(body.app_name).toBe('app-test-actor');
// is_user_token is stripped for app actors.
expect(body.is_user_token).toBeUndefined();
expect(body.desktop_bg_url).toBeUndefined();
expect(body.desktop_bg_color).toBeUndefined();
expect(body.desktop_bg_fit).toBeUndefined();
expect(body.human_readable_age).toBeUndefined();
// Account age, in either form, is not exposed to apps.
expect(body.created_ts).toBeUndefined();
@@ -432,7 +462,7 @@ describe('whoami extension — handleWhoami', () => {
});
});
it('cannot add taskbar_items for an app actor, which core omits', async () => {
it('cannot add directories for an app actor, which core omits', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
@@ -440,11 +470,11 @@ describe('whoami extension — handleWhoami', () => {
app: { uid: 'app-test-actor' },
}),
(_key, event) => {
event.details.taskbar_items = [{ name: 'injected' }];
event.details.directories = { '/injected': 'uuid' };
},
);
expect(body).not.toHaveProperty('taskbar_items');
expect(body).not.toHaveProperty('directories');
});
});
@@ -525,7 +555,7 @@ describe('whoami extension — handleWhoami', () => {
expect(body.otp_secret).toBeUndefined();
expect(body.signup_ip).toBeUndefined();
// Not just absent as a key — the values must not appear anywhere
// in the payload (nested under metadata, taskbar items, …).
// in the payload (nested under metadata, …).
const serialized = JSON.stringify(body);
expect(serialized).not.toContain('5551234567');
expect(serialized).not.toContain('fp_ABC123');
+2 -16
View File
@@ -1,6 +1,5 @@
import { Context } from '@heyputer/backend/src/core';
import { extension } from '@heyputer/backend/src/extensions';
import { APP_ICON_SIZES } from '@heyputer/backend/src/util/appIcon.js';
import {
buildUserDetails,
scrubSensitive,
@@ -54,19 +53,10 @@ export const handleWhoami = async (
return;
}
const rawIconSize =
typeof req.query?.icon_size === 'string'
? Number(req.query.icon_size)
: undefined;
const iconSize =
rawIconSize !== undefined && APP_ICON_SIZES.includes(rawIconSize)
? rawIconSize
: undefined;
const details = await buildUserDetails(
user,
{ config: extension.config, clients, stores, services },
{ isUser, iconSize, noIcons: !iconSize },
{ isUser },
);
details.human_readable_age = user.timestamp
? timeago.format(new Date(user.timestamp as string))
@@ -81,9 +71,6 @@ export const handleWhoami = async (
delete details.email;
delete details.unconfirmed_email;
}
delete details.desktop_bg_url;
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.created_ts;
delete details.is_user_token;
@@ -121,8 +108,7 @@ export const handleWhoami = async (
}
const subscription = details.subscription as
| { offering?: Record<string, unknown> }
| undefined;
{ offering?: Record<string, unknown> } | undefined;
if (subscription?.offering) {
delete subscription.offering.group;
delete subscription.offering.benefits;
+7
View File
@@ -190,6 +190,13 @@ S3_BUCKET=puter-local
providers = [ordered]@{
ollama = [ordered]@{ enabled = $false }
}
# Off because a self-hosted install has no paid plans: every account
# is on the free policy, so the plan gates would 402 the surfaces that
# declare one (the OpenAI/Anthropic-compatible AI endpoints) for
# everybody. Same as install.sh.
meteringEnforcement = [ordered]@{
subscriptions = $false
}
trust_proxy = 1
}
$configJson = $config | ConvertTo-Json -Depth 10
+86 -77
View File
@@ -897,21 +897,21 @@
}
},
"node_modules/@babel/core": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/core/-/core-7.29.7.tgz",
"integrity": "sha512-RgHBCvtjbOK2gXSNBNIkNoEc9qoVEtau3hj8gEqKQuL3HZAibKarWFEI3Lfm6EYKkLalOh8eSrj9b+ch9H/VBA==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/core/-/core-7.29.13.tgz",
"integrity": "sha512-j1nvOQCZYSgp7HLmisnCXFTpVvFODiijQ9QHkORfGKWH6YDtskTEeCJS7eV545GZ9UBzrq0e2xsOd5QQ5PjIqw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/code-frame": "^7.29.7",
"@babel/generator": "^7.29.7",
"@babel/helper-compilation-targets": "^7.29.7",
"@babel/helper-module-transforms": "^7.29.7",
"@babel/helpers": "^7.29.7",
"@babel/parser": "^7.29.7",
"@babel/generator": "^7.29.13",
"@babel/helper-compilation-targets": "^7.29.13",
"@babel/helper-module-transforms": "^7.29.13",
"@babel/helpers": "^7.29.13",
"@babel/parser": "^7.29.13",
"@babel/template": "^7.29.7",
"@babel/traverse": "^7.29.7",
"@babel/types": "^7.29.7",
"@babel/traverse": "^7.29.13",
"@babel/types": "^7.29.11",
"@jridgewell/remapping": "^2.3.5",
"convert-source-map": "^2.0.0",
"debug": "^4.1.0",
@@ -938,14 +938,14 @@
}
},
"node_modules/@babel/generator": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.29.7.tgz",
"integrity": "sha512-DkXD5OJQaAQIdZ1bt3UZdEnHAn9Imd3IVBdX03UFe+ony9Ojw5pzr9YVKGDY1jt+Gcn/FnGkNf8r+Vj5NOJWtQ==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.29.13.tgz",
"integrity": "sha512-T5FRo+RzVHvGjWOuF1Ljc44b+NXAOazodHkGUHp4I3zCkQ4/sNVJm56h1S278LMR0bY5xBhztGziSbli8zObIw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/parser": "^7.29.7",
"@babel/types": "^7.29.7",
"@babel/parser": "^7.29.13",
"@babel/types": "^7.29.11",
"@jridgewell/gen-mapping": "^0.3.12",
"@jridgewell/trace-mapping": "^0.3.28",
"jsesc": "^3.0.2"
@@ -955,9 +955,9 @@
}
},
"node_modules/@babel/helper-compilation-targets": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/helper-compilation-targets/-/helper-compilation-targets-7.29.7.tgz",
"integrity": "sha512-wem6WaBj4NaVYVdNhLPPVacES6ZJ+KBBfSkTMD3YZxbP3rm3Di85tJU5ljaUNhaOynt+Aj0xruhYuzQBt8n71g==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/helper-compilation-targets/-/helper-compilation-targets-7.29.13.tgz",
"integrity": "sha512-8WxJ4/Vmes3ghacj8VOJy3NW8eyHiFY/4thU/LSBuBE470+igjR/j1ZTNn8fV5VdNrLx6iXUwig526UdOvfP0w==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -982,22 +982,15 @@
}
},
"node_modules/@babel/helper-compilation-targets/node_modules/semver": {
"version": "6.3.1",
"resolved": "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz",
"integrity": "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==",
"version": "6.3.2",
"resolved": "https://registry.npmjs.org/semver/-/semver-6.3.2.tgz",
"integrity": "sha512-GYrCADwpu6yV5OAvchkQ27Z5wwMxWAEMg5NNWjyHh/5DboFFqbdbvYNIvZouQpxXyeMPoTOh0vdPYKlf0ZswdQ==",
"dev": true,
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
}
},
"node_modules/@babel/helper-compilation-targets/node_modules/yallist": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz",
"integrity": "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==",
"dev": true,
"license": "ISC"
},
"node_modules/@babel/helper-globals": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/helper-globals/-/helper-globals-7.29.7.tgz",
@@ -1023,15 +1016,15 @@
}
},
"node_modules/@babel/helper-module-transforms": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/helper-module-transforms/-/helper-module-transforms-7.29.7.tgz",
"integrity": "sha512-UPUVSyXbOh627KiCIGQSgwWzGeBKLkaJ9PJEdrngIwMSzxLR4jS4+f1f1jb7VzBbg8nFLaYotvVPFCTqdrmTAg==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/helper-module-transforms/-/helper-module-transforms-7.29.13.tgz",
"integrity": "sha512-/U96wOz+SCoZ7Da/8kjDGltcNY280JPQcRNPK0Hk3PTbdpaiBDQImtC6JsbVwb1HGMr3usZQU2WsWVjhYNmRzw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/helper-module-imports": "^7.29.7",
"@babel/helper-validator-identifier": "^7.29.7",
"@babel/traverse": "^7.29.7"
"@babel/traverse": "^7.29.13"
},
"engines": {
"node": ">=6.9.0"
@@ -1051,9 +1044,9 @@
}
},
"node_modules/@babel/helper-string-parser": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.29.7.tgz",
"integrity": "sha512-Pb5ijPrZ89GDH8223L4UP8i6QApWxs04RbPQJTeWDV0/keR2E36MeKnyr6LYmUUvqRRI+Iv87SuF1W6ErINzYw==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.29.13.tgz",
"integrity": "sha512-fWegr+WsEsiMVW4KUNdIIzzMTZSzNiqhy0//6+2MRi2d5NFNsL/AYwBQW8eBnRbTBVRnDVzjaJEUqKaCWjMMdQ==",
"dev": true,
"license": "MIT",
"engines": {
@@ -1081,27 +1074,27 @@
}
},
"node_modules/@babel/helpers": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/helpers/-/helpers-7.29.7.tgz",
"integrity": "sha512-1k2lAGRMfHTcwuNYcCNUmaUffmQv8KWMfh2iJUUeRlwlwH4FdNG7mfPI10NPfLHJFThE4Tyr4mv7kTNZOiPuBg==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/helpers/-/helpers-7.29.13.tgz",
"integrity": "sha512-laoRpD/kO/iA4CDlgJbixHekg7oPvWALK0PQyK3Ob2I2njZRvVAD77RfWCIImtqWCq6KijoBoZK7bp/HlTWASQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/template": "^7.29.7",
"@babel/types": "^7.29.7"
"@babel/types": "^7.29.11"
},
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@babel/parser": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.29.7.tgz",
"integrity": "sha512-hnORnjP/1P/zFEndoeX+n+t1RwWRJiJpM/jO7FW32Kn9r5+sJB2JWOdYo4L6k78j15eCwY3Gm/7364B1EMwtNg==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.29.13.tgz",
"integrity": "sha512-fJM50YYjlTiBhgdkGmC1tBpv+FMAyo4annf2X6Aks+0yRlaiVkYW/3f07xAfVVkYIoZ9fFZp+mEThUFnuybhkQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/types": "^7.29.7"
"@babel/types": "^7.29.11"
},
"bin": {
"parser": "bin/babel-parser.js"
@@ -1135,18 +1128,18 @@
}
},
"node_modules/@babel/traverse": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.29.7.tgz",
"integrity": "sha512-EhlfNQtZ+NK22w5BM61ciuiq1m58ed33Wr1Xan//ZRTy6hgjnwyCffRYwzsGXdASJSUJ1guZILsErh1eQcl+zw==",
"version": "7.29.13",
"resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.29.13.tgz",
"integrity": "sha512-0ve0xMlFA73dgY1oyJRY6RoW+/bncdcOQqg6jjYcVfXBK90OdLXjm9R6lmRetdZVrPjXkKIoMVBqYFPjcAlZ4Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/code-frame": "^7.29.7",
"@babel/generator": "^7.29.7",
"@babel/generator": "^7.29.13",
"@babel/helper-globals": "^7.29.7",
"@babel/parser": "^7.29.7",
"@babel/parser": "^7.29.13",
"@babel/template": "^7.29.7",
"@babel/types": "^7.29.7",
"@babel/types": "^7.29.11",
"debug": "^4.3.1"
},
"engines": {
@@ -1154,13 +1147,13 @@
}
},
"node_modules/@babel/types": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/types/-/types-7.29.7.tgz",
"integrity": "sha512-4zBIxpPzowiZpusoFkyGVwakdRJUyuH5PxQ/PrqghfdFWWasvnCdPfQXHrenDai+gyLARulZjZowCOj6fjT4pA==",
"version": "7.29.11",
"resolved": "https://registry.npmjs.org/@babel/types/-/types-7.29.11.tgz",
"integrity": "sha512-WCsemW40Cpc6gouC1SrY6PBaZ+oLRlEVY9XBvZBI7il/HyUrb2nJ505wE1MHEFyDZNnrWDEuy4jL50WVHfTTSQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/helper-string-parser": "^7.29.7",
"@babel/helper-string-parser": "^7.29.11",
"@babel/helper-validator-identifier": "^7.29.7"
},
"engines": {
@@ -7460,18 +7453,18 @@
"license": "MIT"
},
"node_modules/body-parser": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.3.0.tgz",
"integrity": "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==",
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.4.0.tgz",
"integrity": "sha512-Ifk1KcLrxaGt1occQagQNFNNOqRt1LFuehEnHHzB8Z6cRgIxRU1TBaiIsUiCdA+gy5PL7Y1IX31DNKkRmxtXjQ==",
"license": "MIT",
"dependencies": {
"bytes": "^3.1.2",
"content-type": "^2.0.0",
"content-type": "^2.1.0",
"debug": "^4.4.3",
"http-errors": "^2.0.1",
"iconv-lite": "^0.7.2",
"iconv-lite": "^0.7.3",
"on-finished": "^2.4.1",
"qs": "^6.15.2",
"qs": "^6.16.1",
"raw-body": "^3.0.2",
"type-is": "^2.1.0"
},
@@ -7484,9 +7477,9 @@
}
},
"node_modules/body-parser/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.1.0.tgz",
"integrity": "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag==",
"license": "MIT",
"engines": {
"node": ">=18"
@@ -9581,9 +9574,9 @@
}
},
"node_modules/fastify": {
"version": "5.12.5",
"resolved": "https://registry.npmjs.org/fastify/-/fastify-5.12.5.tgz",
"integrity": "sha512-OB2k1dlxs5/NAABqeKV2FUHkSD2BbENsCak8yULVcymn3fHIPDVa9TI3SDnJSWYSllZmSYuZXy2gTnsT+Sut1A==",
"version": "5.12.7",
"resolved": "https://registry.npmjs.org/fastify/-/fastify-5.12.7.tgz",
"integrity": "sha512-v4pEPsTRPuqlDIYiQFB+BHjVN4SdHX9u0ABk0ZDUKbbdY1vsNryUd1j4Z+jQEZajvYIVk047Kimva9/AJ79GYw==",
"funding": [
{
"type": "github",
@@ -16587,9 +16580,9 @@
"license": "MIT"
},
"node_modules/undici": {
"version": "7.30.1",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.30.1.tgz",
"integrity": "sha512-03KrMbjIFDWQR5RNPrSw3bd1r12JpyFQl9/yxjUbJtqkjmllqC5lFjZh6ujeuJCKr8EOQvRW872Oy/az+nLYxg==",
"version": "7.30.2",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.30.2.tgz",
"integrity": "sha512-mLyPQnHwUxaM/rideoK/+KL8DaAJLRl4aBCfmX444aCSLnIaomqPLtNRe2jdCYcsk5wZzXrDIQC8ZeYNRAYyIA==",
"license": "MIT",
"engines": {
"node": ">=20.18.1"
@@ -18141,6 +18134,13 @@
"node": ">=10"
}
},
"node_modules/yallist": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz",
"integrity": "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==",
"dev": true,
"license": "ISC"
},
"node_modules/yaml": {
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
@@ -18233,9 +18233,9 @@
}
},
"node_modules/zod": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz",
"integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==",
"version": "4.6.7",
"resolved": "https://registry.npmjs.org/zod/-/zod-4.6.7.tgz",
"integrity": "sha512-g8W2D2kU6a8untc053fX1uVBKzM7yqAxW/3zwk83+6RyF2+SKOuV+c0wBDhUwzV1PaGapmZH+rmcUzPf+3CGwg==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/colinhacks"
@@ -18300,7 +18300,7 @@
"mime-types": "^2.1.35",
"murmurhash": "^2.0.1",
"mysql2": "^3.24.6",
"nodemailer": "^10.1.4",
"nodemailer": "^10.1.7",
"openai": "^6.34.0",
"otpauth": "^9.5.2",
"pg": "^8.21.0",
@@ -18312,10 +18312,10 @@
"svg-captcha": "^1.4.0",
"together-ai": "^0.33.0",
"ua-parser-js": "^1.0.41",
"undici": "^7.30.1",
"undici": "^7.30.2",
"uuid": "^14.0.3",
"validator": "^13.15.35",
"ws": "^8.22.0"
"ws": "^8.22.2"
},
"devDependencies": {
"@types/bcrypt": "^6.0.0",
@@ -18343,10 +18343,19 @@
"dev": true,
"license": "MIT"
},
"src/backend/node_modules/nodemailer": {
"version": "10.1.7",
"resolved": "https://registry.npmjs.org/nodemailer/-/nodemailer-10.1.7.tgz",
"integrity": "sha512-OQa6FDln5fLttmqSbjDiRYvFDOFNrBqfiOwHPlEbURil3b+vPB1v8xpi+Sp46drzNDlsTnNsS502vthmz56r4w==",
"license": "MIT-0",
"engines": {
"node": ">=20.0.0"
}
},
"src/backend/node_modules/ws": {
"version": "8.22.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.22.0.tgz",
"integrity": "sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg==",
"version": "8.22.2",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.22.2.tgz",
"integrity": "sha512-/rEZ2ejown2N5gxVTHUu9bVVC7ttu0/Tgzf5hbTOixR3Fs7mbbXgn2DbdbcKgJOmpQMtrIUWivtYuVMnp/W/yA==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
-1
View File
@@ -125,7 +125,6 @@ describe('core without a teams registration', () => {
actor: { user: row },
} as never,
{ title: 'Puter' } as never,
{} as never,
);
const params = JSON.parse(
/gui\((\{.*?\})\);/s.exec(html)![1].replaceAll('\\u003c', '<'),
@@ -167,6 +167,38 @@ describe('AbstractDatabaseClient — replica-aware reads', () => {
expect(client.reads).toEqual(['SELECT 1']);
expect(client.preads).toEqual([]);
});
describe('tryHardReadAll', () => {
const reads = [{ query: 'SELECT 1' }, { query: 'SELECT 2' }];
it('keeps replica rows without reading the primary', async () => {
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
expect(await client.tryHardReadAll(reads)).toEqual([
[{ id: 1 }],
[{ id: 1 }],
]);
expect(client.preads).toEqual([]);
});
it.each([[], new Error('replica unavailable')])(
'reads every query on the primary when the replica has none',
async (replica) => {
const client = new FixtureDatabaseClient(replica, [{ id: 2 }]);
expect(await client.tryHardReadAll(reads)).toEqual([
[{ id: 2 }],
[{ id: 2 }],
]);
expect(client.preads).toEqual(['SELECT 1', 'SELECT 2']);
},
);
it('issues one read per query when there is no replica', async () => {
const client = new FixtureDatabaseClient([], [{ id: 2 }], false);
expect(await client.tryHardReadAll(reads)).toEqual([[], []]);
expect(client.reads).toEqual(['SELECT 1', 'SELECT 2']);
expect(client.preads).toEqual([]);
});
});
});
describe('AbstractDatabaseClient — SQL generation', () => {
@@ -172,6 +172,34 @@ export class AbstractDatabaseClient extends PuterClient {
return result.value;
}
/** Reads the primary only when every replica read is empty or fails. */
@Span('db.tryHardReadAll', (reads: unknown[]) => ({
'db.batch_size': reads.length,
}))
async tryHardReadAll(
reads: { query: string; params?: unknown[] }[],
): Promise<Record<string, unknown>[][]> {
if (!this.hasReadReplica()) {
return Promise.all(
reads.map(({ query, params = [] }) => this.read(query, params)),
);
}
const onReplica = await Promise.all(
reads.map(async ({ query, params = [] }) => {
try {
return await this.replicaRead(query, params);
} catch {
// A failed replica read counts as a miss.
return [];
}
}),
);
if (onReplica.some((rows) => rows.length > 0)) return onReplica;
return Promise.all(
reads.map(({ query, params = [] }) => this.pread(query, params)),
);
}
/**
* Apply the `.sql` files named `<engineName>*` in each configured migration
* directory, in filename order. Files must be idempotent: nothing records
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 98;
const CURRENT_SCHEMA_VERSION = 99;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -705,6 +705,60 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
}
});
it('fills a share row missing its entry owner from 0103, invites left alone', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-share-owner-'));
const path = join(dir, 'puter.sqlite');
try {
const before = new SqliteDatabaseClient(
sqliteConfig({ inMemory: false, path }),
);
await before.onServerStart();
const [user] = (await before.read(
'SELECT MIN(`id`) AS id FROM `user`',
)) as { id: number }[];
await before.write(
'INSERT INTO `fsentries` (`uuid`, `name`, `user_id`, `modified`) ' +
'VALUES (?, ?, ?, ?)',
['22222222-2222-4222-8222-222222222222', 'owned.txt', user.id, 0],
);
const [entry] = (await before.read(
'SELECT `id` FROM `fsentries` WHERE `uuid` = ?',
['22222222-2222-4222-8222-222222222222'],
)) as { id: number }[];
await before.write(
'INSERT INTO `share` (`uid`, `issuer_user_id`, ' +
'`recipient_email`, `fsentry_id`) VALUES (?, ?, ?, ?)',
['share-no-owner', user.id, '[email protected]', entry.id],
);
await before.write(
'INSERT INTO `share` (`uid`, `issuer_user_id`, ' +
'`recipient_email`) VALUES (?, ?, ?)',
['share-invite', user.id, '[email protected]'],
);
// The full schema as it stood before 0103, so only 0103 reruns.
await before.write('PRAGMA user_version = 98');
before.onServerShutdown();
const after = new SqliteDatabaseClient(
sqliteConfig({ inMemory: false, path }),
);
await after.onServerStart();
await expect(
after.read(
'SELECT `uid`, `entry_owner_user_id` FROM `share` ' +
'WHERE `uid` IN (?, ?) ORDER BY `uid`',
['share-invite', 'share-no-owner'],
),
).resolves.toEqual([
{ uid: 'share-invite', entry_owner_user_id: null },
{ uid: 'share-no-owner', entry_owner_user_id: user.id },
]);
after.onServerShutdown();
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('indexes notification.created_at for the retention sweep', async () => {
const rows = await client.read('PRAGMA index_list(`notification`)');
expect(rows.map((r) => String(r.name))).toContain(
@@ -137,6 +137,7 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[95, ['0100_notification-send-key.sql']],
[96, ['0101_user-scoped-handlers.sql']],
[97, ['0102_notification-scope-id-index.sql']],
[98, ['0103_share-entry-owner-backfill.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Fills `share`.`entry_owner_user_id` on any row that names an entry but was
-- written without the owner. Mirrors SQLite migration 0103. No per-file
-- applied-state tracking; a replay finds nothing to fill.
UPDATE `share` JOIN `fsentries` ON `fsentries`.`id` = `share`.`fsentry_id`
SET `share`.`entry_owner_user_id` = `fsentries`.`user_id`
WHERE `share`.`fsentry_id` IS NOT NULL
AND `share`.`entry_owner_user_id` IS NULL;
@@ -0,0 +1,27 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Fills "share".entry_owner_user_id on any row that names an entry but was
-- written without the owner. Mirrors SQLite migration 0103. No per-file
-- applied-state tracking; a replay finds nothing to fill.
UPDATE "share"
SET entry_owner_user_id = f.user_id
FROM fsentries f
WHERE f.id = "share".fsentry_id
AND "share".fsentry_id IS NOT NULL
AND "share".entry_owner_user_id IS NULL;
@@ -0,0 +1,27 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Fills `share`.`entry_owner_user_id` on any row that names an entry but was
-- written without the owner, so reads can rely on the column alone. Rows that
-- name no entry (invites) keep a NULL owner. A replay finds nothing to fill.
UPDATE `share`
SET `entry_owner_user_id` = (
SELECT `user_id` FROM `fsentries` WHERE `fsentries`.`id` = `share`.`fsentry_id`
)
WHERE `fsentry_id` IS NOT NULL
AND `entry_owner_user_id` IS NULL;
@@ -168,6 +168,204 @@ describe('EmailClient — sending', () => {
});
});
describe('EmailClient — streams', () => {
const STREAMS = {
notify: {
from: '"Notices" <[email protected]>',
headers: { 'X-Example-Stream': 'notify', 'X-Example-Kind': 'n' },
tagsHeader: 'X-Example-Tags',
},
plain: { headers: { 'X-Example-Stream': 'plain' } },
};
const streamClient = () =>
startClient({
email: { jsonTransport: true, from: FROM, streams: STREAMS },
} as unknown as Partial<IConfig>);
const base = { to: '[email protected]', subject: 's', text: 't' };
it('adds the stream headers and from address to the message', async () => {
const info = await streamClient().sendRaw({
...base,
stream: 'notify',
});
const message = sentMessage(info);
expect(message.headers).toMatchObject({
'X-Example-Stream': 'notify',
'X-Example-Kind': 'n',
});
expect(message.from.address).toBe('[email protected]');
});
it('lets caller headers and from win over the stream', async () => {
const info = await streamClient().sendRaw({
...base,
stream: 'notify',
from: '[email protected]',
headers: { 'x-example-stream': 'mine' },
});
const message = sentMessage(info);
expect(message.from.address).toBe('[email protected]');
expect(message.headers?.['x-example-stream']).toBe('mine');
expect(message.headers).not.toHaveProperty('X-Example-Stream');
expect(message.headers?.['X-Example-Kind']).toBe('n');
});
it('keeps the default from address for a stream without one', async () => {
const info = await streamClient().sendRaw({ ...base, stream: 'plain' });
expect(sentMessage(info).from.address).toBe('[email protected]');
});
it('leaves the message unchanged for an unknown or missing stream', async () => {
const client = streamClient();
for (const stream of ['nope', undefined]) {
const message = sentMessage(
await client.sendRaw({ ...base, stream, tags: { a: 'b' } }),
);
expect(message.from.address).toBe('[email protected]');
expect(message.headers ?? {}).toEqual({});
}
});
it('renders valid tags into the stream tags header and drops the rest', async () => {
const info = await streamClient().sendRaw({
...base,
stream: 'notify',
tags: {
kind: 'share',
'app.id': 'a-1:x',
'bad key': 'v',
ok: 'bad value',
long: 'x'.repeat(129),
},
});
expect(sentMessage(info).headers?.['X-Example-Tags']).toBe(
'kind=share, app.id=a-1:x',
);
});
it('keeps a tags header the caller set', async () => {
const info = await streamClient().sendRaw({
...base,
stream: 'notify',
headers: { 'X-Example-Tags': 'own=1' },
tags: { kind: 'share' },
});
expect(sentMessage(info).headers?.['X-Example-Tags']).toBe('own=1');
});
it('sends no tags header when the stream names none or no tag survives', async () => {
const client = streamClient();
const plain = sentMessage(
await client.sendRaw({
...base,
stream: 'plain',
tags: { kind: 'share' },
}),
);
expect(plain.headers).toEqual({ 'X-Example-Stream': 'plain' });
const dropped = sentMessage(
await client.sendRaw({
...base,
stream: 'notify',
tags: { 'bad key': 'v' },
}),
);
expect(dropped.headers).not.toHaveProperty('X-Example-Tags');
});
it('does not hand stream or tags to the transport', async () => {
const info = await streamClient().sendRaw({
...base,
stream: 'notify',
tags: { kind: 'share' },
});
const raw = JSON.parse((info as { message: string }).message);
expect(raw).not.toHaveProperty('stream');
expect(raw).not.toHaveProperty('tags');
});
it('keeps streams out of the transport options', async () => {
const nodemailer = (await import('nodemailer')).default;
const create = vi.spyOn(nodemailer, 'createTransport');
try {
streamClient();
expect(create).toHaveBeenCalledTimes(1);
const [options] = create.mock.calls[0] as [Record<string, unknown>];
expect(options).not.toHaveProperty('streams');
expect(options).not.toHaveProperty('from');
expect(options).toMatchObject({ jsonTransport: true });
} finally {
create.mockRestore();
}
});
it('forwards the template send options to sendRaw', async () => {
const client = streamClient();
const sendRaw = vi.spyOn(client, 'sendRaw').mockResolvedValue(null);
await client.send(
'[email protected]',
'email_verification_code',
{ code: '1' },
{ stream: 'notify', replyTo: '[email protected]' },
);
expect(sendRaw).toHaveBeenCalledWith(
expect.objectContaining({
stream: 'notify',
replyTo: '[email protected]',
}),
);
});
it('applies the stream from address to a template send', async () => {
const info = await streamClient().send(
'[email protected]',
'email_verification_code',
{ code: '1' },
{ stream: 'notify' },
);
expect(sentMessage(info).from.address).toBe('[email protected]');
});
it('reads a stream only from its own config entry', async () => {
const inherited = Object.create({
base: { from: '[email protected]', headers: { 'X-Example-B': '1' } },
});
const client = startClient({
email: { jsonTransport: true, from: FROM, streams: inherited },
} as unknown as Partial<IConfig>);
const message = sentMessage(
await client.sendRaw({ ...base, stream: 'base' }),
);
expect(message.from.address).toBe('[email protected]');
expect(message.headers ?? {}).toEqual({});
});
it('prints the stream name and no tags when no transport is configured', async () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
try {
const client = new EmailClient({
port: 0,
extensions: [],
} as unknown as IConfig);
await client.sendRaw({
...base,
stream: 'notify',
tags: { kind: 'share' },
});
const printed = warn.mock.calls[0][1] as Record<string, unknown>;
expect(printed).not.toHaveProperty('tags');
expect(printed.stream).toBe('notify');
} finally {
warn.mockRestore();
}
});
});
describe('EmailClient — template rendering', () => {
it('renders the code into both subject and body', async () => {
const client = startClient();
+65 -8
View File
@@ -69,14 +69,50 @@ export interface SendMailOptions {
attachments?: EmailAttachment[];
/** Extra message headers (e.g. List-Unsubscribe), passed to the transport. */
headers?: Record<string, string>;
/** Names the `email.streams` entry whose from address and headers apply. */
stream?: string;
/** Rendered into the stream's `tagsHeader`; ignored when it has none. */
tags?: Record<string, string>;
}
/** Options a templated {@link EmailClient.send} forwards to `sendRaw`. */
export type SendTemplateOptions = Pick<SendMailOptions, 'replyTo' | 'stream'>;
const TAG_PART = /^[A-Za-z0-9_.:-]{1,128}$/;
interface CompiledTemplate {
subject: ReturnType<typeof template>;
html: ReturnType<typeof template>;
text?: ReturnType<typeof template>;
}
/** Pairs whose key or value falls outside `TAG_PART` are dropped. */
const renderTags = (tags: Record<string, string> | undefined): string =>
Object.entries(tags ?? {})
.filter(([name, value]) => TAG_PART.test(name) && TAG_PART.test(value))
.map(([name, value]) => `${name}=${value}`)
.join(', ');
const hasHeader = (
headers: Record<string, string> | undefined,
name: string,
): boolean =>
Object.keys(headers ?? {}).some(
(key) => key.toLowerCase() === name.toLowerCase(),
);
// Header names are case-insensitive: `x-a` from `over` replaces `X-A` in `base`.
const mergeHeaders = (
base: Record<string, string> | undefined,
over: Record<string, string> | undefined,
): Record<string, string> => {
const merged: Record<string, string> = {};
for (const [key, value] of Object.entries(base ?? {})) {
if (!hasHeader(over, key)) merged[key] = value;
}
return { ...merged, ...over };
};
// -- EmailClient ------------------------------------------------------
/**
@@ -111,7 +147,8 @@ export class EmailClient extends PuterClient {
return;
}
this.transport = nodemailer.createTransport(emailConf);
const { streams: _streams, from: _from, ...transportConf } = emailConf;
this.transport = nodemailer.createTransport(transportConf);
console.log('[email] transport configured');
}
@@ -125,7 +162,8 @@ export class EmailClient extends PuterClient {
/**
* Render a template and send it to `to`. `options.replyTo` sets the
* Reply-To header (e.g. so a recipient can respond to the originator of the
* message rather than the no-reply From address).
* message rather than the no-reply From address); `options.stream` is
* passed to {@link sendRaw}.
*
* Returns the transport's send result, or `null` when none is configured —
* {@link sendRaw} drops the message instead of throwing. A caller that
@@ -135,7 +173,7 @@ export class EmailClient extends PuterClient {
to: string,
template: T,
values: Record<string, unknown> = {},
options: { replyTo?: string } = {},
options: SendTemplateOptions = {},
) {
const compiled = this.compiledTemplates[template];
if (!compiled) {
@@ -143,12 +181,12 @@ export class EmailClient extends PuterClient {
}
return await this.sendRaw({
from: this.defaultFrom(),
to,
subject: compiled.subject(values),
html: compiled.html(values),
...(compiled.text ? { text: compiled.text(values) } : {}),
...(options.replyTo ? { replyTo: options.replyTo } : {}),
...(options.stream ? { stream: options.stream } : {}),
});
}
@@ -159,19 +197,38 @@ export class EmailClient extends PuterClient {
* Returns the transport's send result, or `null` when no transport is
* configured (the send is a no-op in that case — callers that must not
* silently drop mail should check `config.email` before composing, or the
* `null` return after).
* `null` return after). A configured `stream` supplies the from address
* when the caller gives none, and headers the caller's own override.
*/
async sendRaw(options: SendMailOptions) {
const { stream: streamName, tags, ...message } = options;
if (!this.transport) {
console.warn(
'[email] attempted to send email without transport. If you need to send email, configure an SMTP transport in your config file (see docs for details). Email content:',
options,
{ ...message, ...(streamName ? { stream: streamName } : {}) },
);
return null;
}
const streams = this.config.email?.streams;
const stream =
streamName && streams && Object.hasOwn(streams, streamName)
? streams[streamName]
: undefined;
let headers = mergeHeaders(stream?.headers, message.headers);
const tagsValue = stream?.tagsHeader ? renderTags(tags) : '';
if (
stream?.tagsHeader &&
tagsValue &&
!hasHeader(message.headers, stream.tagsHeader)
) {
headers = mergeHeaders(headers, {
[stream.tagsHeader]: tagsValue,
});
}
return await this.transport.sendMail({
...options,
from: options.from ?? this.defaultFrom(),
...message,
...(Object.keys(headers).length > 0 ? { headers } : {}),
from: message.from ?? stream?.from ?? this.defaultFrom(),
});
}
+41 -12
View File
@@ -20,6 +20,7 @@ import type {
Response as ExpressResponse,
} from 'express';
import { Actor } from '../../core';
import type { EmailTemplateName } from '../email/templates';
import type { UsageInput } from '../../services/metering/types';
import { FSEntry } from '../../stores/fs/FSEntry';
@@ -247,6 +248,12 @@ export type EventMap = {
requires_phone_verification?: boolean;
/** Set by the abuse harness — require card verification post-signup. */
requires_card_verification?: boolean;
/**
* The email stream the confirmation message is sent on: lowercase
* letters, digits, `:`, `_` and `-`, up to 64 characters, starting with
* a letter or digit. Any other value is ignored.
*/
email_stream?: string | null;
[key: string]: unknown;
};
'puter.signup.success': {
@@ -298,6 +305,8 @@ export type EventMap = {
message: string | null;
[key: string]: unknown;
};
/** Before notification emails; a listener clears `allow` per recipient. */
'email.notify.validate': EmailNotifyValidateEvent;
// Inbound mail to a feedback address (`abuse@`, `fbl@`, ...), emitted via
// `emitAndWait`. A listener that takes it sets `handled` and it reaches no
// mailbox; otherwise it is routed like any other message.
@@ -420,7 +429,11 @@ export type EventMap = {
old_username: string;
new_username: string;
};
'user.email-changed': { user_id: number; new_email: string };
'user.email-changed': {
user_id: number;
old_email: string | null;
new_email: string;
};
/** An account was suspended, or its suspension lifted. Local events. */
'user.suspended': { userId: number };
'user.unsuspended': { userId: number };
@@ -732,6 +745,8 @@ export type EventMap = {
'outer.permission.flatInvalidated': {
entries: Array<{ holderUserId: number; permission: string }>;
};
/** Links on these nodes were removed; read them from the primary. */
'outer.share.linksRevoked': { fsentryIds: number[]; uuids: string[] };
/**
* A user's subscription set changed, so every process must drop its cached
* "does this user have any subscriptions" answer. Carries the counter the
@@ -870,8 +885,11 @@ export type EventMap = {
session_uids: string[];
};
/** One access token was revoked; only its own connections should drop. */
'auth.access-token.revoked': { token_uid: string };
/**
* One access token was revoked; only its own connections should drop.
* `outer.*` for the same reason as `outer.auth.sessions.revoked`.
*/
'outer.auth.access-token.revoked': { token_uid: string };
/** Same-cluster eviction for raw realtime WebSockets. */
'pubsub.realtime.auth-revoked': {
@@ -970,6 +988,21 @@ export type EventMap = {
'outer.pubsub.metering.credits-changed': { userUuid: string };
} & IExtensionEventMap;
/** A union of `Names`, each of which has to be an email template. */
type TemplateNames<Names extends EmailTemplateName> = Names;
/** The emails that ask `email.notify.validate`, by template name. */
export type EmailNotifyKind = TemplateNames<
'file_shared_with_you' | 'file_shared_invite' | 'app-user-feedback'
>;
/** Payload for `email.notify.validate`: one kind of mail, all its recipients. */
export type EmailNotifyValidateEvent = {
kind: EmailNotifyKind;
/** `to` is the address as it will be sent; `allow` starts true. */
recipients: Array<{ to: string; allow: boolean }>;
};
/** Payload for `subscription.gate.<surface>` events. */
export type SubscriptionGateEvent = {
actor: Actor;
@@ -1079,11 +1112,10 @@ export type EventKey = keyof EventMap & string;
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
? | `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
@@ -1094,10 +1126,7 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
/** Acked, not acked, no socket anywhere, or sockets only in a peer region. */
export type NotifDeliveryOutcome =
| 'received'
| 'unreceived'
| 'no-socket'
| 'remote-only';
'received' | 'unreceived' | 'no-socket' | 'remote-only';
/** One socket's receipt for a live notification. */
export interface NotifDeliveryResponder {
+73
View File
@@ -21,6 +21,8 @@
import { GetObjectCommand } from '@aws-sdk/client-s3';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { existsSync } from 'node:fs';
import { createServer, type Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
@@ -200,6 +202,77 @@ describe('S3Client — remote endpoint configuration', () => {
});
});
describe('S3Client — bucket in another region', () => {
let server: Server;
let client: S3Client;
let signedRegions: string[];
// Answers like a bucket that lives in us-west-2: a request signed for any
// other region gets S3's 301 naming the bucket's region.
beforeEach(async () => {
signedRegions = [];
server = createServer((req, res) => {
const region =
/Credential=[^/]+\/\d{8}\/([^/]+)\//.exec(
req.headers.authorization ?? '',
)?.[1] ?? '';
signedRegions.push(region);
if (region !== 'us-west-2') {
res.writeHead(301, {
'content-type': 'application/xml',
'x-amz-bucket-region': 'us-west-2',
});
res.end(
'<?xml version="1.0" encoding="UTF-8"?><Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint.</Message></Error>',
);
return;
}
res.writeHead(200, {
'content-type': 'text/plain',
'content-length': '5',
});
res.end('hello');
});
await new Promise<void>((resolve) =>
server.listen(0, '127.0.0.1', resolve),
);
const { port } = server.address() as AddressInfo;
client = new S3Client({
port: 0,
extensions: [],
s3: {
s3Config: {
endpoint: `http://127.0.0.1:${port}`,
accessKeyId: 'fakeAccessKeyId',
secretAccessKey: 'fakeSecretAccessKey',
forcePathStyle: true,
},
},
} as unknown as IConfig);
await client.onServerStart();
});
afterEach(async () => {
await client.onServerShutdown();
server.closeAllConnections();
await new Promise<void>((resolve) => server.close(() => resolve()));
});
it("retries a read in the bucket's region when S3 redirects it", async () => {
const result = await client
.get('ap-northeast-2')
.send(
new GetObjectCommand({
Bucket: 'other-bucket',
Key: 'index.html',
}),
);
expect(await result.Body?.transformToString()).toBe('hello');
expect(signedRegions).toEqual(['ap-northeast-2', 'us-west-2']);
});
});
describe('S3Client — legacy storage migration', () => {
let client: S3Client;
let dir: string;
+3
View File
@@ -235,6 +235,9 @@ export class S3Client extends PuterClient {
const client = new AwsS3Client({
region,
// Rows without a stored region are read with the server's own, so
// retry where S3's 301 says the bucket actually lives.
followRegionRedirects: true,
requestStreamBufferSize: 32 * 1024,
requestHandler: new NodeHttpHandler({
socketTimeout: 5000,
@@ -52,8 +52,8 @@ const APP_READ_LIMIT = {
/**
* Unauthenticated icon serving; no actor to key on, so the bucket is the
* address — and an address is a NAT, a campus or a carrier gateway that can
* hold hundreds of desktops. Each desktop boot pulls the taskbar's icons at
* once, so a single burst from one network is already thousands of requests.
* hold hundreds of desktops. Each dashboard load pulls its app icons at once,
* so a single burst from one network is already thousands of requests.
* Responses are publicly cacheable and usually a redirect, so the ceiling is
* not protecting bandwidth; it is there so a client looping on a broken icon
* can't spin unbounded.
@@ -523,7 +523,7 @@ export class AppController extends PuterController {
};
// Icons are <img src> targets from the GUI (root) AND resolved via
// api_base_url in taskbar payloads. Register on both so either origin
// api_base_url in app summaries. Register on both so either origin
// works without a cross-subdomain redirect.
router.get(
'/app-icon/:app_uid',
@@ -1271,7 +1271,7 @@ describe('AppController app-icon rate limit', () => {
it('sizes the icon bucket for a whole network rather than one desktop', () => {
// Icons are unauthenticated `<img src>` targets, so the only key is
// the address — which one NAT shares across every desktop behind it,
// and each boot pulls the taskbar's icons in a burst.
// and each dashboard load pulls its app icons in a burst.
for (const path of ['/app-icon/:app_uid', '/app-icon/:app_uid/:size']) {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === path,
@@ -23,8 +23,10 @@ import { makeActor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { createSecret as otpCreateSecret } from '../../services/auth/OTPUtil.js';
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
import { tokenClaimsDigest } from '../../../gui/src/helpers/guiToken.js';
import {
createTestUser,
lagReadReplica,
setupPuterTestEnv,
type PuterTestEnv,
} from '../../testUtil.js';
@@ -350,6 +352,98 @@ describe('revoke-own-access-token over HTTP', () => {
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
});
const ACCESS_TOKEN_QUERIES =
/FROM `access_token_permissions`|`access_token_uid` = \?/;
const tokenUidOf = (token: string) =>
(
env.server.services.token.verify('auth', token) as {
token_uid: string;
}
).token_uid;
it('serves a read URL fetched right after minting while the replica lags', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const replica = await lagReadReplica(env.server, ACCESS_TOKEN_QUERIES);
try {
const readToken = await mintReadToken(owner.token, file.uid);
expect(await tokenReadStatus(file.uid, readToken)).toBe(200);
const cached = await env.server.clients.redis.get(
`perms:token:${tokenUidOf(readToken)}`,
);
expect(cached).toBe(JSON.stringify([`fs:${file.uid}:read`]));
} finally {
replica.catchUp();
}
});
it('mints from a just-created personal API token while the replica lags', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const replica = await lagReadReplica(env.server, ACCESS_TOKEN_QUERIES);
try {
const pat = await mintPersonalToken(owner);
const readToken = await mintReadToken(pat, file.uid);
expect(await tokenReadStatus(file.uid, readToken)).toBe(200);
} finally {
replica.catchUp();
}
});
it('revokes a just-created token by its uuid while the replica lags', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const { actor } = await env.server.services.auth.authenticate(
owner.token,
);
const replica = await lagReadReplica(env.server, ACCESS_TOKEN_QUERIES);
let readToken: string;
try {
readToken = await mintReadToken(owner.token, file.uid);
await env.server.services.auth.revokeAccessToken(
actor!,
tokenUidOf(readToken),
);
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
} finally {
replica.catchUp();
}
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
expect(await whoamiStatus(readToken)).toBe(401);
});
it('keeps a revoked token dead while the replica still shows it live', async ({
skip,
}) => {
// The frozen replica is a sqlite copy.
skip(process.env.PUTER_TEST_DB_ENGINE?.toLowerCase() === 'postgres');
const owner = env.users.user;
const file = await makeFile(owner);
const { actor } = await env.server.services.auth.authenticate(
owner.token,
);
const readToken = await mintReadToken(owner.token, file.uid);
const tokenUid = tokenUidOf(readToken);
const replica = await lagReadReplica(env.server, ACCESS_TOKEN_QUERIES, {
mode: 'frozen',
});
try {
await env.server.services.auth.revokeAccessToken(actor!, tokenUid);
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
expect(
await env.server.stores.permission.listAccessTokenPerms(
tokenUid,
),
).toEqual([]);
expect(
await env.server.clients.redis.get(`perms:token:${tokenUid}`),
).toBeNull();
} finally {
replica.catchUp();
}
expect(await whoamiStatus(readToken)).toBe(401);
});
// These three run last in the file's shared `env.users.user`: each one
// revokes a session belonging to `owner.token` / `owner.workerToken`, so
// later tests can't rely on those credentials still being live.
@@ -393,7 +487,10 @@ describe('revoke-own-access-token over HTTP', () => {
it('a revoked GUI session still gets a reauth_token', async () => {
const owner = env.users.user;
const decoded = env.server.services.token.verify('auth', owner.token) as {
const decoded = env.server.services.token.verify(
'auth',
owner.token,
) as {
session_uid: string;
};
await env.server.stores.session.removeByUuid(decoded.session_uid);
@@ -559,7 +656,6 @@ describe('2FA enrollment over HTTP', () => {
expect(enabled.status).toBe(200);
expect((await otpState(account.userId)).enabled).toBe(true);
});
});
/**
@@ -714,4 +810,67 @@ describe('session token helpers over HTTP', () => {
'puter_auth_token=',
);
});
describe("the shell's guiTokenDigest", () => {
/** The session cookie `/session/sync-cookie` sets for `token`. */
const sessionCookie = async (token: string) => {
const sync = await get('/session/sync-cookie', token);
expect(sync.status).toBe(204);
const value = /puter_auth_token=([^;]+)/.exec(
sync.headers.get('set-cookie') ?? '',
)?.[1];
expect(value).toBeTruthy();
return `puter_auth_token=${value}`;
};
/** What the shell at `path` renders for `guiTokenDigest`. */
const shellDigest = async (path: string, cookie?: string) => {
const res = await fetch(new URL(path, env.origin), {
headers: cookie ? { cookie } : {},
});
expect(res.status).toBe(200);
const html = await res.text();
const match = /"guiTokenDigest":(null|"[0-9a-f]{32}")/.exec(html);
if (!match) {
expect(html).not.toContain('guiTokenDigest');
return undefined;
}
return JSON.parse(match[1]) as string | null;
};
it("names the token /get-gui-token hands over for the session cookie", async () => {
const { token } = await signupTemp();
const cookie = await sessionCookie(token);
const gui = await fetch(new URL('/get-gui-token', env.origin), {
headers: { cookie },
});
expect(gui.status).toBe(200);
const { token: guiToken } = (await gui.json()) as { token: string };
const digest = await shellDigest('/', cookie);
expect(digest).toBe(await tokenClaimsDigest(guiToken));
// The token the browser stored at sign-up is for the same session.
expect(await tokenClaimsDigest(token)).toBe(digest);
});
it('is null without a session cookie, even with a token in the URL', async () => {
const { token } = await signupTemp();
expect(await shellDigest('/')).toBe(null);
expect(
await shellDigest(`/?auth_token=${encodeURIComponent(token)}`),
).toBe(null);
});
it('is left out when the session cookie does not resolve to a session', async () => {
const { token } = await signupTemp();
const cookie = await sessionCookie(token);
await env.server.services.auth.revokeSession(claims(token).session_uid);
expect(await shellDigest('/', cookie)).toBe(undefined);
expect(await shellDigest('/', 'puter_auth_token=garbage')).toBe(
undefined,
);
});
});
});
@@ -30,6 +30,7 @@
import { installSeatStub } from '../../testFixtures/seatStub.js';
import bcrypt from 'bcrypt';
import { inspect } from 'node:util';
import jwt from 'jsonwebtoken';
import { TOTP } from 'otpauth';
import { v4 as uuidv4, v5 as uuidv5 } from 'uuid';
@@ -1416,7 +1417,7 @@ describe('AuthController.handleLogin', () => {
is_user_token: true,
});
expect(body.user.directories).toBeTypeOf('object');
expect(Array.isArray(body.user.taskbar_items)).toBe(true);
expect(body.user).not.toHaveProperty('taskbar_items');
});
// A deleted row still answering from cache reaches the session INSERT.
@@ -4765,14 +4766,24 @@ describe('AuthController.handleSendConfirmEmail', () => {
).rejects.toMatchObject({ statusCode: 403 });
});
it('rotates the email_confirm_code and returns {} on success', async () => {
it('rotates the email_confirm_code and reports the send', async () => {
const { user, actor } = await makeUserAndActor();
const before = await server.stores.user.getById(user.id, {
force: true,
});
const send = vi
.spyOn(server.clients.email, 'send')
.mockResolvedValue({ messageId: 'm' } as never);
const res = makeRes();
await controller.handleSendConfirmEmail(makeReq({}, { actor }), res);
expect(res.body).toEqual({});
try {
await controller.handleSendConfirmEmail(
makeReq({}, { actor }),
res,
);
} finally {
send.mockRestore();
}
expect(res.body).toEqual({ sent: true });
const after = await server.stores.user.getById(user.id, {
force: true,
});
@@ -4782,7 +4793,7 @@ describe('AuthController.handleSendConfirmEmail', () => {
// The resend is the recovery path for a lost code, so it must not fail
// silently either.
it('alarms — and still returns {} — when the resend send throws', async () => {
it('alarms, and still answers 200 with sent: false, when the resend send throws', async () => {
const { actor } = await makeUserAndActor();
const alarm = vi
.spyOn(server.clients.alarm, 'create')
@@ -4797,7 +4808,8 @@ describe('AuthController.handleSendConfirmEmail', () => {
res,
);
// Not the caller's fault; still a 200.
expect(res.body).toEqual({});
expect(res.statusCode).toBe(200);
expect(res.body).toEqual({ sent: false });
const raised = alarm.mock.calls.find(
(c) => c[0] === 'auth:confirmation-email-send-failed:resend',
@@ -7468,6 +7480,11 @@ describe('AuthController user-protected mutations (validation paths)', () => {
{ expiresIn: '1h' },
);
const heard: Array<Record<string, unknown>> = [];
eventClient.on('user.email-changed', (_k, data) => {
heard.push(data as Record<string, unknown>);
});
const req = makeReq({});
(req as unknown as { query: Record<string, string> }).query = {
token: linkJwt,
@@ -7475,6 +7492,11 @@ describe('AuthController user-protected mutations (validation paths)', () => {
const res = makeRes();
await controller.handleChangeEmailConfirm(req, res);
expect(res.sent).toMatch(/Email changed successfully/);
expect(heard).toContainEqual({
user_id: user.id,
old_email: user.email ?? null,
new_email: newEmail,
});
const after = await server.stores.user.getById(user.id, {
force: true,
@@ -7508,6 +7530,46 @@ describe('AuthController.handleSaveAccount', () => {
};
};
it('sends the confirmation code on the account confirmation stream', async () => {
const { user, actor } = await makeTempActor();
await server.stores.user.updateMetadata(user.id, {
emailStream: 'review',
});
const send = vi
.spyOn(server.clients.email, 'send')
.mockResolvedValue(null);
const alarm = vi
.spyOn(server.clients.alarm, 'create')
.mockImplementation(() => undefined);
try {
const username = `s_${uniq()}`;
await controller.handleSaveAccount(
makeReq(
{
username,
email: `${username}@test.local`,
password: 'correct-horse-battery',
},
{ actor },
),
makeRes(),
);
expect(send).toHaveBeenCalledWith(
`${username}@test.local`,
'email_verification_code',
expect.anything(),
{ stream: 'review' },
);
// The dropped send is reported under its own stage.
expect(alarm.mock.calls.map((c) => c[0])).toEqual([
'auth:confirmation-email-send-failed:save-account',
]);
} finally {
send.mockRestore();
alarm.mockRestore();
}
});
it('rejects non-temp accounts with 400', async () => {
const { actor } = await makeUserAndActor();
await expect(
@@ -8283,11 +8345,10 @@ describe('AuthController session endpoints', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('revoke-session: 403 when revoking someone else’s session', async () => {
it('revoke-session: 404 when revoking someone else’s session', async () => {
const { user: u1 } = await makeUserAndActor();
const { actor: a2 } = await makeUserAndActor();
// Create a real session for u1 so the lookup succeeds, then attempt
// to revoke it as a2 — must 403.
// A real session of u1, revoked as a2: answers as a missing one.
const sessionRes = await server.services.auth.createSessionToken(
u1,
{},
@@ -8300,7 +8361,7 @@ describe('AuthController session endpoints', () => {
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 403 });
).rejects.toMatchObject({ statusCode: 404 });
});
it('rename-session: 400 when uuid param is missing', async () => {
@@ -9500,14 +9561,18 @@ describe('AuthController.handleSignup additional branches', () => {
},
async () => {
const username = `cefail_${uniq()}`;
const res = makeRes();
await controller.handleSignup(
makeReq({
username,
email: `${username}@test.local`,
password: 'correct-horse-battery',
}),
makeRes(),
res,
);
expect(res.body).toMatchObject({
confirmation_email_sent: false,
});
},
);
@@ -9528,6 +9593,93 @@ describe('AuthController.handleSignup additional branches', () => {
});
});
describe('AuthController.handleSignup confirmation email outcome', () => {
const signupWithSend = async (send: () => Promise<unknown>) => {
const spy = vi
.spyOn(server.clients.email, 'send')
.mockImplementation(send as never);
const alarm = vi
.spyOn(server.clients.alarm, 'create')
.mockImplementation(() => undefined);
const res = makeRes();
const username = `ceo_${uniq()}`;
let alarms: string[] = [];
try {
await withSignupValidateOverride(
(event) => {
event.requires_email_confirmation = true;
},
() =>
controller.handleSignup(
makeReq({
username,
email: `${username}@test.local`,
password: 'correct-horse-battery',
}),
res,
),
);
alarms = alarm.mock.calls.map((c) => String(c[0]));
} finally {
spy.mockRestore();
alarm.mockRestore();
}
return {
body: res.body as {
proceed: boolean;
confirmation_email_sent?: boolean;
user: { requires_email_confirmation: boolean };
},
alarms,
};
};
it('reports a send that throws and still completes the signup', async () => {
const { body, alarms } = await signupWithSend(async () => {
throw new Error('refused');
});
expect(body.proceed).toBe(true);
expect(body.confirmation_email_sent).toBe(false);
expect(body.user.requires_email_confirmation).toBeTruthy();
expect(alarms).toContain('auth:confirmation-email-send-failed:signup');
});
it('reports a send the transport accepted', async () => {
const { body } = await signupWithSend(async () => ({ messageId: 'm' }));
expect(body.confirmation_email_sent).toBe(true);
});
it('keeps the confirmation stream out of the returned user details', async () => {
const res = makeRes();
const username = `ces_${uniq()}`;
await withSignupValidateOverride(
(event) => {
(event as Record<string, unknown>).email_stream = 'review';
},
() =>
controller.handleSignup(
makeReq({
username,
email: `${username}@test.local`,
password: 'correct-horse-battery',
}),
res,
),
);
const stored = await server.stores.user.getByUsername(username);
expect(stored!.metadata?.emailStream).toBe('review');
const details = (res.body as { user: { metadata?: object } }).user;
expect(details.metadata ?? {}).not.toHaveProperty('emailStream');
});
it('leaves the field out when no confirmation email is due', async () => {
const res = makeRes();
await controller.handleSignup(makeReq({ is_temp: true }), res);
expect(res.body).toMatchObject({ proceed: true });
expect(res.body).not.toHaveProperty('confirmation_email_sent');
});
});
describe('AuthController.handleSendPassRecoveryEmail additional branches', () => {
it('rejects an invalid email format with 400 (when no username supplied)', async () => {
await expect(
@@ -10072,6 +10224,77 @@ describe('AuthController.handleRevokeSession additional branches', () => {
expect(Array.isArray(body.sessions)).toBe(true);
});
const expiredSessionOf = async (user: { id: number }) => {
const row = await server.stores.session.create(user.id, {
expires_at: Math.floor(Date.now() / 1000) - 1,
});
return row.uuid as string;
};
it('revokes an expired session the caller owns', async () => {
const { user, actor } = await makeUserAndActor();
const uuid = await expiredSessionOf(user);
const res = makeRes();
await controller.handleRevokeSession(makeReq({ uuid }, { actor }), res);
const body = res.body as { sessions: Array<{ uuid: string }> };
expect(body.sessions.map((s) => s.uuid)).not.toContain(uuid);
const rows = await server.clients.db.read(
'SELECT `revoked_at` FROM `sessions` WHERE `uuid` = ?',
[uuid],
);
expect(rows[0].revoked_at).not.toBeNull();
});
it('revoking an expired session ends its live child token', async () => {
const { user, actor } = await makeUserAndActor();
const parent = await expiredSessionOf(user);
const child = await server.stores.session.getOrCreateGodmodeToken(
user.id,
{
parentSessionUid: parent,
appUid: `app-${uuidv4()}`,
expiresAt: Math.floor(Date.now() / 1000) + 3600,
maxAgeSeconds: 3600,
label: 'Godmode app',
auth_id: user.uuid,
},
);
const childUuid = child!.row.uuid as string;
expect(await server.stores.session.getByUuid(childUuid)).toBeTruthy();
await controller.handleRevokeSession(
makeReq({ uuid: parent }, { actor }),
makeRes(),
);
expect(await server.stores.session.getByUuid(childUuid)).toBeNull();
});
it('answers 404 for an expired session of another user', async () => {
const { user: owner } = await makeUserAndActor();
const { actor } = await makeUserAndActor();
const uuid = await expiredSessionOf(owner);
await expect(
controller.handleRevokeSession(
makeReq({ uuid }, { actor }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('answers 404 for a session that is already revoked', async () => {
const { user, actor } = await makeUserAndActor();
const row = await server.stores.session.create(user.id);
await server.stores.session.removeByUuid(row.uuid);
await expect(
controller.handleRevokeSession(
makeReq({ uuid: row.uuid }, { actor }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('refuses to revoke the caller’s OWN current session row (400)', async () => {
// Invariant: a self-revoke leaves the client in an ambiguous
// identity state because the response can't write fresh auth
@@ -10552,10 +10775,10 @@ describe('AuthController.loginWait audience binding', () => {
// party holding the session id is somewhere else entirely.
const token = await mintAppToken(actor, OPENER);
const { res, outcome } = await waitWithRelay(
uuidv4(),
{ origin: 'https://evil.test' },
token,
const session = uuidv4();
const [{ res, outcome }] = await runWaits(
[{ session, origin: 'https://evil.test' }],
{ during: () => setToken(session, token) },
);
// Same 408 the empty path returns — a mismatched caller must not be
// able to tell "nothing arrived" from "something arrived for someone
@@ -10631,10 +10854,10 @@ describe('AuthController.loginWait audience binding', () => {
);
const guiToken = (loginRes.body as { token: string }).token;
const { res, outcome } = await waitWithRelay(
uuidv4(),
{ origin: OPENER },
guiToken,
const session = uuidv4();
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{ during: () => setToken(session, guiToken) },
);
expect(outcome).toMatchObject({ statusCode: 408 });
expect(res.body).toBeUndefined();
@@ -10649,14 +10872,406 @@ describe('AuthController.loginWait audience binding', () => {
{ keyid: 'v2' },
);
const { res, outcome } = await waitWithRelay(
uuidv4(),
{ origin: OPENER },
forged,
const session = uuidv4();
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{ during: () => setToken(session, forged) },
);
expect(outcome).toMatchObject({ statusCode: 408 });
expect(res.body).toBeUndefined();
});
/** Post a token to `/login/set` with no wait running. */
const setToken = (session: string, token: string) =>
controller.loginSet(makeReq({ session, auth_token: token }), makeRes());
/**
* Run waits on fake timers so the 10 s window passes without real waiting.
* `during` runs once every wait has subscribed; `now` pins the clock.
*/
const runWaits = async (
calls: Array<{ session: string; origin: string; clientGone?: boolean }>,
opts: { now?: number; during?: () => Promise<unknown> } = {},
) => {
const subscribed = vi.spyOn(eventClient, 'on');
vi.useFakeTimers({
toFake:
opts.now === undefined
? ['setTimeout']
: ['setTimeout', 'Date'],
});
if (opts.now !== undefined) vi.setSystemTime(opts.now);
const tick = () => new Promise((r) => setImmediate(r));
try {
let pending = calls.length;
const runs = calls.map(({ session, origin, clientGone }) => {
const res = makeRes();
const req = makeReq({ session }, { headers: { origin } });
if (clientGone) {
Object.assign(req.socket, { destroyed: true });
}
const outcome = controller
.loginWait(req, res)
.then(
() => 'ok' as const,
(e: unknown) => e,
)
.finally(() => {
pending--;
});
return { res, outcome };
});
const allSubscribed = () =>
calls.every(({ session }) =>
subscribed.mock.calls.some(
([key]) => key === `pubsub.login.${session}`,
),
);
for (let i = 0; i < 200 && pending > 0 && !allSubscribed(); i++) {
await tick();
}
for (let i = 0; i < 20; i++) await tick();
await opts.during?.();
for (let i = 0; i < 200 && pending > 0; i++) {
await tick();
await vi.advanceTimersByTimeAsync(1000);
}
return await Promise.all(
runs.map(async ({ res, outcome }) => ({
res,
outcome: await outcome,
})),
);
} finally {
vi.useRealTimers();
subscribed.mockRestore();
}
};
const waitAlone = async (session: string, origin: string, now?: number) =>
(await runWaits([{ session, origin }], { now }))[0];
it('delivers a token set before the wait subscribes', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
await setToken(session, token);
const { res, outcome } = await waitAlone(session, OPENER);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it('delivers a token set between one wait timing out and the next', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const first = await waitAlone(session, OPENER);
expect(first.outcome).toMatchObject({ statusCode: 408 });
await setToken(session, token);
const { res, outcome } = await waitAlone(session, OPENER);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it('hands a token out only once', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
await setToken(session, token);
expect((await waitAlone(session, OPENER)).outcome).toBe('ok');
const again = await waitAlone(session, OPENER);
expect(again.outcome).toMatchObject({ statusCode: 408 });
expect(again.res.body).toBeUndefined();
});
it('withholds a stored token from a mismatched Origin and keeps it for the matching one', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
await setToken(session, token);
const other = await waitAlone(session, 'https://other.test');
expect(other.outcome).toMatchObject({ statusCode: 408 });
expect(other.res.body).toBeUndefined();
const { res, outcome } = await waitAlone(session, OPENER);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it('drops a stored token after a couple of minutes', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const early = uuidv4();
const late = uuidv4();
await setToken(early, token);
await setToken(late, token);
const start = Date.now();
expect((await waitAlone(early, OPENER, start + 110_000)).outcome).toBe(
'ok',
);
const expired = await waitAlone(late, OPENER, start + 121_000);
expect(expired.outcome).toMatchObject({ statusCode: 408 });
});
it('keeps waiting past a stored token for another app and delivers the real one', async () => {
const { actor } = await makeUserAndActor();
const otherToken = await mintAppToken(actor, 'https://other.test');
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
await setToken(session, otherToken);
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{
during: () => setToken(session, token),
},
);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it('still relays to a subscribed wait when storing the token fails', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const failingSet = vi
.spyOn(server.clients.redis, 'set')
.mockRejectedValueOnce(new Error('unavailable'));
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
try {
let setRes: MockRes | undefined;
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{
during: async () => {
setRes = makeRes();
await controller.loginSet(
makeReq({ session, auth_token: token }),
setRes,
);
},
},
);
expect(setRes?.body).toEqual({ success: true });
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
} finally {
failingSet.mockRestore();
warn.mockRestore();
}
});
it('removes the stored copy once a subscribed wait takes the token', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const { outcome } = await waitWithRelay(
session,
{ origin: OPENER },
token,
);
expect(outcome).toBe('ok');
const later = await waitAlone(session, OPENER);
expect(later.outcome).toMatchObject({ statusCode: 408 });
});
it('hands a stored token to only one of two concurrent waits', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
await setToken(session, token);
const results = await runWaits([
{ session, origin: OPENER },
{ session, origin: OPENER },
]);
const outcomes = results.map((r) => r.outcome);
expect(outcomes.filter((o) => o === 'ok')).toHaveLength(1);
expect(outcomes.filter((o) => o !== 'ok')).toEqual([
expect.objectContaining({ statusCode: 408 }),
]);
});
it('relays a token to every wait already subscribed', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const results = await runWaits(
[
{ session, origin: OPENER },
{ session, origin: OPENER },
],
{ during: () => setToken(session, token) },
);
expect(results.map((r) => r.outcome)).toEqual(['ok', 'ok']);
for (const { res } of results) {
expect((res.body as { auth_token: string }).auth_token).toBe(token);
}
});
it('keeps the stored copy when the subscribed wait has lost its client', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
await runWaits([{ session, origin: OPENER, clientGone: true }], {
during: () => setToken(session, token),
});
const { res, outcome } = await waitAlone(session, OPENER);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it('still relays to a subscribed wait when the stored copy cannot be read or removed', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const failingGet = vi
.spyOn(server.clients.redis, 'get')
.mockRejectedValue(new Error('unavailable'));
const failingEval = vi
.spyOn(server.clients.redis, 'eval')
.mockRejectedValue(new Error('unavailable'));
try {
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{ during: () => setToken(session, token) },
);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
} finally {
failingGet.mockRestore();
failingEval.mockRestore();
}
});
it('answers a subscribed wait when a second token is set right after the first', async () => {
const { actor } = await makeUserAndActor();
const first = await mintAppToken(actor, OPENER);
// A later `iat` makes the second token differ from the first.
await new Promise((r) => setTimeout(r, 1100));
const second = await mintAppToken(actor, OPENER);
expect(second).not.toBe(first);
const session = uuidv4();
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{
during: async () => {
await setToken(session, first);
await setToken(session, second);
},
},
);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(first);
});
it('keeps a subscribed wait listening past a token for another app', async () => {
const { actor } = await makeUserAndActor();
const otherToken = await mintAppToken(actor, 'https://other.test');
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const [{ res, outcome }] = await runWaits(
[{ session, origin: OPENER }],
{
during: async () => {
await setToken(session, otherToken);
await setToken(session, token);
},
},
);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it("keeps a stored token when a later set carries another app's token or an unsigned one", async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const otherToken = await mintAppToken(actor, 'https://other.test');
const session = uuidv4();
await setToken(session, token);
await setToken(session, otherToken);
await setToken(session, 'x'.repeat(100_000));
const { res, outcome } = await waitAlone(session, OPENER);
expect(outcome).toBe('ok');
expect((res.body as { auth_token: string }).auth_token).toBe(token);
});
it('stores only a token this server signed for an app', async () => {
const { actor } = await makeUserAndActor();
const real = await mintAppToken(actor, OPENER);
const forged = jwt.sign(
jwt.decode(real) as object,
'not-the-server-secret',
{ keyid: 'v2' },
);
const set = vi.spyOn(server.clients.redis, 'set');
try {
const session = uuidv4();
await setToken(session, forged);
await setToken(session, 'x'.repeat(100_000));
expect(set).not.toHaveBeenCalled();
await setToken(session, real);
expect(set).toHaveBeenCalledTimes(1);
} finally {
set.mockRestore();
}
});
it('leaves the token out of the warning when storing it fails', async () => {
const { actor } = await makeUserAndActor();
const token = await mintAppToken(actor, OPENER);
const session = uuidv4();
const failure = Object.assign(new Error('ERR unavailable'), {
command: { name: 'set', args: [session, token, 'EX', '120'] },
});
const failingSet = vi
.spyOn(server.clients.redis, 'set')
.mockRejectedValueOnce(failure);
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
try {
const res = makeRes();
await controller.loginSet(
makeReq({ session, auth_token: token }),
res,
);
expect(res.body).toEqual({ success: true });
expect(warn).toHaveBeenCalled();
const logged = warn.mock.calls
.flat()
.map((arg) => inspect(arg, { depth: 10 }))
.join(' ');
expect(logged).not.toContain(token);
} finally {
failingSet.mockRestore();
warn.mockRestore();
}
});
it('answers 400 when auth_token is not a string', async () => {
await expect(
controller.loginSet(
makeReq({ session: uuidv4(), auth_token: { nested: true } }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
});
});
// -- Batch grant / revoke + cross-app data grants -----------------------
@@ -11013,3 +11628,100 @@ describe('AuthController — a seat address nobody confirmed', () => {
expect(await emailOf(seat.id)).toBeNull();
});
});
describe('AuthController account mail streams', () => {
const spySend = () =>
vi.spyOn(server.clients.email, 'send').mockResolvedValue(null);
it('sends password recovery on the auth stream', async () => {
const { user } = await makeUserAndActor();
const send = spySend();
try {
await controller.handleSendPassRecoveryEmail(
makeReq({ email: user.email! }),
makeRes(),
);
expect(send).toHaveBeenCalledWith(
user.email,
'email_password_recovery',
expect.anything(),
{ stream: 'auth' },
);
} finally {
send.mockRestore();
}
});
it('sends the password change notice on the auth stream', async () => {
const { user, actor } = await makeUserAndActor();
const req = makeReq({ new_pass: 'a-fresh-password-456' }, { actor });
(req as unknown as { userProtected: unknown }).userProtected = { user };
const send = spySend();
try {
await controller.handleChangePassword(req, makeRes());
expect(send).toHaveBeenCalledWith(
user.email,
'password_change_notification',
expect.anything(),
{ stream: 'auth' },
);
} finally {
send.mockRestore();
}
});
it('sends both change-email messages on the auth stream', async () => {
const { user, actor } = await makeUserAndActor();
const newEmail = `cs_${uniq()}@test.local`;
const send = spySend();
const sendRaw = vi
.spyOn(server.clients.email, 'sendRaw')
.mockResolvedValue(null);
try {
await controller.handleChangeEmail(
makeReq({ new_email: newEmail }, { actor }),
makeRes(),
);
expect(send).toHaveBeenCalledWith(
newEmail,
'email_verification_link',
expect.anything(),
{ stream: 'auth' },
);
expect(sendRaw).toHaveBeenCalledWith(
expect.objectContaining({ to: user.email, stream: 'auth' }),
);
} finally {
send.mockRestore();
sendRaw.mockRestore();
}
});
it('sends the 2FA enable and disable notices on the auth stream', async () => {
const { user, actor } = await makeUserAndActor({ email_confirmed: 1 });
const send = spySend();
try {
const { secret } = await setup2fa(actor);
await enable2fa(actor, { code: liveTotp(user.username, secret) });
expect(send).toHaveBeenCalledWith(
user.email,
'enabled_2fa',
{ username: user.username },
{ stream: 'auth' },
);
await controller.handleDisable2fa(
makeReq({ code: liveTotp(user.username, secret) }, { actor }),
makeRes(),
);
expect(send).toHaveBeenCalledWith(
user.email,
'disabled_2fa',
{ username: user.username },
{ stream: 'auth' },
);
} finally {
send.mockRestore();
}
});
});
+146 -86
View File
@@ -71,6 +71,7 @@ import { sessionCookieFlags } from '../../util/cookieFlags.js';
import { cleanEmail, isStorableEmail } from '../../util/email.js';
import { isGodmodeApp } from '../../util/godmodeApps.js';
import { parsePhone } from '../../util/phone.js';
import { deleteIfEqual } from '../../util/redisLock.js';
import {
bonusCodeInvalidError,
checkSignupBonus,
@@ -143,6 +144,12 @@ const DISPATCH_ID_MAX_LENGTH = 128;
// One name for the flag, so the write and the read cannot drift apart.
const APP_AUTHENTICATED_FLAG = 'flag:app-is-authenticated';
// Long enough to span the gap between one `/login/wait` and the client's next.
const LOGIN_RELAY_TTL_SECONDS = 120;
// Keyed by app too, so a token for one app never replaces another app's.
const loginRelayKey = (session: string, appUid: string) =>
`login:relay:${session}:${appUid}`;
// -- Post-login route limits -----------------------------------------
//
// The credential legs above (login, signup, recovery, confirmation) each
@@ -345,36 +352,43 @@ export class AuthController extends PuterController {
const expectedAppUid =
await this.services.auth.appUidFromOrigin(reqOrigin);
const { resolve, promise } = Promise.withResolvers<void>();
let token: string | null = null;
const listener = (_key: string, value: { authtoken: string }) => {
token = value.authtoken;
resolve();
};
this.clients.event.on(`pubsub.login.${session}`, listener);
const timeout = new Promise<void>((resolve) =>
setTimeout(resolve, 10000),
);
await Promise.race([promise, timeout]);
this.clients.event.off(`pubsub.login.${session}`, listener);
if (!token) {
throw new HttpError(408, 'Request timeout.', {
legacyCode: 'request_timeout',
});
}
const relayKey = loginRelayKey(session, expectedAppUid);
const { resolve, promise } = Promise.withResolvers<string>();
// Audience check. The postMessage hand-off this relay stands in for
// is origin-bound for free — it posts with `targetOrigin`, so a page
// can only ever receive a token minted for *itself*. Delivering
// server-side dropped that binding; this restores it. Without it a
// popup talked into minting for app X (see `trustsOpenerOriginParam`
// in the GUI) hands X's token to whoever holds the session id.
if (!this.#tokenIsForApp(token, expectedAppUid)) {
// Deliberately the same 408 the no-token path returns: a caller
// learns only that nothing arrived for them, not that a token
// for a different app went past.
// server-side dropped that binding; this restores it. Any other token
// is passed over and the wait runs on, so a caller learns only that
// nothing arrived for them.
const listener = (_key: string, value: { authtoken: string }) => {
if (this.#relayedTokenAppUid(value.authtoken) === expectedAppUid) {
resolve(value.authtoken);
}
};
this.clients.event.on(`pubsub.login.${session}`, listener);
let token: string | null;
try {
// Read after subscribing, so a token set before this wait is still delivered.
token = await this.#takeStoredLoginToken(relayKey, expectedAppUid);
if (!token) {
const timeout = new Promise<null>((resolve) =>
setTimeout(resolve, 10000, null),
);
token = await Promise.race([promise, timeout]);
// A wait whose client went away leaves the copy for the next one.
if (token && !req.socket?.destroyed) {
await deleteIfEqual(
this.clients.redis,
relayKey,
token,
).catch(() => false);
}
}
} finally {
this.clients.event.off(`pubsub.login.${session}`, listener);
}
if (!token) {
throw new HttpError(408, 'Request timeout.', {
legacyCode: 'request_timeout',
});
@@ -386,24 +400,46 @@ export class AuthController extends PuterController {
}
/**
* Whether a relayed token is an app-under-user token minted for
* `expectedAppUid`. Verifies the signature — an unverified decode would let
* a caller relay a token whose claims it wrote itself.
* Take the copy `/login/set` kept for this app, once. Null when there is
* none, another wait took it, or Redis fails; the wait then just listens.
*/
#tokenIsForApp(token: string, expectedAppUid: string): boolean {
async #takeStoredLoginToken(
relayKey: string,
expectedAppUid: string,
): Promise<string | null> {
try {
const kept = await this.clients.redis.get(relayKey);
if (!kept || this.#relayedTokenAppUid(kept) !== expectedAppUid) {
return null;
}
const took = await deleteIfEqual(
this.clients.redis,
relayKey,
kept,
);
return took ? kept : null;
} catch {
return null;
}
}
/**
* The app a relayed app-under-user token was minted for, or null. Verifies
* the signature — an unverified decode would let a caller relay a token
* whose claims it wrote itself.
*/
#relayedTokenAppUid(token: string): string | null {
try {
const payload = this.services.token.verify<{
type?: string;
app_uid?: string;
}>('auth', token);
return (
payload.type === 'app-under-user' &&
!!payload.app_uid &&
payload.app_uid === expectedAppUid
);
return payload.type === 'app-under-user' && payload.app_uid
? payload.app_uid
: null;
} catch {
// Malformed, expired, or signed with a key we don't hold.
return false;
return null;
}
}
@Post('/login/set', {
@@ -418,12 +454,34 @@ export class AuthController extends PuterController {
})
async loginSet(req: Request, res: Response) {
const { session, auth_token } = req.body ?? {};
if (!session || !auth_token || !validateUuid(session)) {
if (
!session ||
typeof auth_token !== 'string' ||
!auth_token ||
!validateUuid(session)
) {
throw new HttpError(400, 'session and auth_token are required.', {
legacyCode: 'bad_request',
});
}
// Only a token a wait could deliver is kept: one this server signed for an app.
const appUid = this.#relayedTokenAppUid(auth_token);
if (appUid) {
try {
await this.clients.redis.set(
loginRelayKey(session, appUid),
auth_token,
'EX',
LOGIN_RELAY_TTL_SECONDS,
);
} catch (e) {
console.warn(
'[login-set] storing the relayed token failed:',
e instanceof Error ? e.message : String(e),
);
}
}
this.clients.event.emit(
`pubsub.login.${session}`,
{
@@ -882,14 +940,19 @@ export class AuthController extends PuterController {
if (!bonusCode) throw bonusCodeInvalidError();
}
const user = await this.services.signup.signup({
req,
body,
isTemp: is_temp,
fingerprint,
bonusCode,
const { user, confirmationEmailSent } =
await this.services.signup.signup({
req,
body,
isTemp: is_temp,
fingerprint,
bonusCode,
});
await this.#completeLogin(req, res, user, undefined, {
...(confirmationEmailSent !== undefined
? { confirmation_email_sent: confirmationEmailSent }
: {}),
});
await this.#completeLogin(req, res, user);
}
/**
@@ -1028,10 +1091,12 @@ export class AuthController extends PuterController {
email_confirm_code: code,
});
await this.services.signup.sendConfirmationEmail(user, 'resend', {
code,
});
res.json({});
const sent = await this.services.signup.sendConfirmationEmail(
user,
'resend',
{ code },
);
res.json({ sent });
}
@Post('/confirm-email', {
@@ -2019,6 +2084,7 @@ export class AuthController extends PuterController {
user.email,
'email_password_recovery',
{ link },
{ stream: 'auth' },
);
} catch (e) {
console.warn('[send-pass-recovery-email] send failed:', e);
@@ -2282,6 +2348,7 @@ export class AuthController extends PuterController {
{
username: user.username,
},
{ stream: 'auth' },
);
} catch (e) {
console.warn('[change-password] notification send failed:', e);
@@ -2507,6 +2574,7 @@ export class AuthController extends PuterController {
new_email,
'email_verification_link',
{ link },
{ stream: 'auth' },
);
} catch (e) {
console.warn('[change-email] new-address email failed:', e);
@@ -2514,15 +2582,8 @@ export class AuthController extends PuterController {
// Notify the old address too
if (user.email) {
try {
await (
this.clients.email as unknown as {
sendRaw: (opts: {
to: string;
subject: string;
text: string;
}) => Promise<unknown>;
}
).sendRaw({
await this.clients.email.sendRaw({
stream: 'auth',
to: user.email,
subject: 'Your Puter email change was requested',
text: `A change to ${new_email} was requested on your account. If this wasn't you, please contact support.`,
@@ -2647,6 +2708,7 @@ export class AuthController extends PuterController {
'user.email-changed' as never,
{
user_id: user.id,
old_email: user.email ?? null,
new_email: newEmail,
} as never,
{},
@@ -2810,18 +2872,11 @@ export class AuthController extends PuterController {
username,
});
// Send confirmation email
if (this.clients.email) {
try {
await this.clients.email.send(
email,
'email_verification_code',
{ code: email_confirm_code },
);
} catch (e) {
console.warn('[save-account] confirmation email failed:', e);
}
}
await this.services.signup.sendConfirmationEmail(
{ ...user, username, email },
'save-account',
{ code: email_confirm_code },
);
try {
this.clients.event?.emit(
@@ -3584,21 +3639,18 @@ export class AuthController extends PuterController {
{ legacyCode: 'bad_request' },
);
}
// `getByUuid` returns null when the row is missing, already
// soft-revoked, or past `expires_at` — surface as 404 so a stale
// manage-sessions UI doesn't 500 when it clicks revoke on a row
// that already went away.
const session = await this.stores.session.getByUuid(uuid);
if (!session) {
// Missing, already revoked and other users' rows all answer 404; an
// expired row is still revoked so its children go with it.
const session = await this.stores.session.getByUuidAny(uuid);
if (
!session ||
session.revoked_at != null ||
session.user_id !== req.actor!.user.id
) {
throw new HttpError(404, 'Session not found', {
legacyCode: 'not_found',
});
}
if (session.user_id !== req.actor!.user.id) {
throw new HttpError(403, 'Can only revoke your own sessions', {
legacyCode: 'unauthorized',
});
}
await this.services.auth.revokeSession(uuid);
const sessions = await this.services.auth.listSessions(req.actor!);
res.json({ sessions });
@@ -4274,9 +4326,12 @@ export class AuthController extends PuterController {
if (this.clients.email && user.email) {
try {
await this.clients.email.send(user.email, 'enabled_2fa', {
username: user.username,
});
await this.clients.email.send(
user.email,
'enabled_2fa',
{ username: user.username },
{ stream: 'auth' },
);
} catch (e) {
console.warn('[configure-2fa] email send failed:', e);
}
@@ -4350,9 +4405,12 @@ export class AuthController extends PuterController {
if (this.clients.email && user.email) {
try {
await this.clients.email.send(user.email, 'disabled_2fa', {
username: user.username,
});
await this.clients.email.send(
user.email,
'disabled_2fa',
{ username: user.username },
{ stream: 'auth' },
);
} catch (e) {
console.warn('[disable-2fa] email send failed:', e);
}
@@ -4768,6 +4826,7 @@ export class AuthController extends PuterController {
res: Response,
user: UserRow,
oidcContinuation?: unknown,
extra: { confirmation_email_sent?: boolean } = {},
): Promise<void> {
const redirectUrl =
oidcContinuation === undefined
@@ -4802,6 +4861,7 @@ export class AuthController extends PuterController {
token: gui_token,
user: details,
...(redirectUrl ? { redirectUrl } : {}),
...extra,
});
}
}
@@ -15,7 +15,6 @@ export interface OidcLoginContinuation {
embeddedInPopup: boolean;
openerOrigin: string | null;
messageId: string | null;
switchedToLogin: boolean;
}
interface OidcReturnDeps {
@@ -56,7 +55,6 @@ export function completeOidcLoginReturn(
data.userUuid !== user.uuid ||
typeof data.redirectUri !== 'string' ||
typeof data.embeddedInPopup !== 'boolean' ||
typeof data.switchedToLogin !== 'boolean' ||
(data.openerOrigin !== null && typeof data.openerOrigin !== 'string') ||
(data.messageId !== null && typeof data.messageId !== 'string')
) {
@@ -91,6 +89,5 @@ export function completeOidcLoginReturn(
),
);
}
if (data.switchedToLogin) append('oidc_switched', 'login');
return target;
}
@@ -25,18 +25,13 @@ const ALLOWED_SORT_BY = ['name', 'size', 'modified', 'type'];
const ALLOWED_SORT_ORDER = ['asc', 'desc'];
/**
* Desktop/UI preference routes.
*
* Two categories:
*
* - User-level: desktop background, taskbar items (UserStore)
* - Folder-level: layout, sort_by/sort_order (fsentries table)
* Folder view preference routes: layout and sort_by/sort_order, stored on the
* folder's fsentries row.
*/
/**
* Desktop preference writes — background, taskbar, layout, sort order. All four
* persist to the user row on every call, and the GUI fires them on direct user
* action, so a per-minute ceiling well above human speed is enough to catch a
* stuck client.
* Desktop preference writes — layout, sort order. Each persists on every call,
* and the GUI fires them on direct user action, so a per-minute ceiling well
* above human speed is enough to catch a stuck client.
*/
const PREFERENCE_WRITE_LIMIT = {
scope: 'desktop-preference',
@@ -47,89 +42,6 @@ const PREFERENCE_WRITE_LIMIT = {
export class DesktopController extends PuterController {
registerRoutes(router) {
// -- Desktop background --------------------------------------
router.post(
'/set-desktop-bg',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { url, color, fit } = req.body ?? {};
const patch = {};
if (url !== undefined) {
if (url !== null && typeof url !== 'string') {
throw new HttpError(
400,
'`url` must be a string or null',
{ legacyCode: 'bad_request' },
);
}
patch.desktop_bg_url = url;
}
if (color !== undefined) {
if (color !== null && typeof color !== 'string') {
throw new HttpError(
400,
'`color` must be a string or null',
{ legacyCode: 'bad_request' },
);
}
patch.desktop_bg_color = color;
}
if (fit !== undefined) {
if (fit !== null && typeof fit !== 'string') {
throw new HttpError(
400,
'`fit` must be a string or null',
{ legacyCode: 'bad_request' },
);
}
patch.desktop_bg_fit = fit;
}
if (Object.keys(patch).length === 0) {
throw new HttpError(400, 'No fields provided', {
legacyCode: 'bad_request',
});
}
await this.stores.user.update(req.actor.user.id, patch);
res.json({});
},
);
// -- Taskbar items -------------------------------------------
router.post(
'/update-taskbar-items',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { items } = req.body ?? {};
if (!Array.isArray(items)) {
throw new HttpError(
400,
'Missing or invalid `items` array',
{ legacyCode: 'bad_request' },
);
}
await this.stores.user.update(req.actor.user.id, {
taskbar_items: JSON.stringify(items),
});
res.json({});
},
);
// -- Folder layout -------------------------------------------
router.post(
@@ -130,147 +130,17 @@ const callRoute = async (
});
};
// ── /set-desktop-bg ─────────────────────────────────────────────────
describe('DesktopController POST /set-desktop-bg', () => {
it('persists url/color/fit on the user row', async () => {
const { actor, userId } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({
body: {
url: 'https://cdn.test/wallpaper.png',
color: '#000000',
fit: 'cover',
},
actor,
}),
res,
);
expect(captured.body).toEqual({});
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.desktop_bg_url).toBe(
'https://cdn.test/wallpaper.png',
);
expect(refreshed?.desktop_bg_color).toBe('#000000');
expect(refreshed?.desktop_bg_fit).toBe('cover');
describe('DesktopController routes', () => {
it('has no desktop background route', () => {
expect(
router.routes.some((r) => r.path === '/set-desktop-bg'),
).toBe(false);
});
it('persists only the supplied fields (partial update)', async () => {
const { actor, userId } = await makeUser();
const { res } = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { color: '#ffffff' }, actor }),
res,
);
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.desktop_bg_color).toBe('#ffffff');
// Untouched fields remain at their column default.
expect(refreshed?.desktop_bg_url).toBeFalsy();
expect(refreshed?.desktop_bg_fit).toBeFalsy();
});
it('passes through `null` to clear a field', async () => {
const { actor, userId } = await makeUser();
// First populate, then clear.
const populate = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({
body: { url: 'https://cdn.test/x.png' },
actor,
}),
populate.res,
);
const before = await server.stores.user.getById(userId);
expect(before?.desktop_bg_url).toBe('https://cdn.test/x.png');
const clear = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { url: null }, actor }),
clear.res,
);
const after = await server.stores.user.getById(userId);
expect(after?.desktop_bg_url).toBeNull();
});
it('throws 400 when url is not a string or null', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { url: 123 }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when no fields are provided', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── /update-taskbar-items ───────────────────────────────────────────
describe('DesktopController POST /update-taskbar-items', () => {
it('persists items as a JSON-encoded string', async () => {
const { actor, userId } = await makeUser();
const items = [{ name: 'editor' }, { name: 'browser' }];
const { res } = makeRes();
await callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: { items }, actor }),
res,
);
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.taskbar_items).toBe(JSON.stringify(items));
});
it('throws 400 when items is missing', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when items is not an array', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: { items: 'oops' }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
it('has no taskbar items route', () => {
expect(
router.routes.some((r) => r.path === '/update-taskbar-items'),
).toBe(false);
});
});
@@ -324,7 +324,7 @@ describe('DriverController — concurrent acquire/release', () => {
const driver = {
...makeSyntheticDriver(),
ping: async () => {
Context.get('keepDriverSlot')?.(
Context.get('keepConcurrencySlot')?.(
new Promise<void>((resolve) => {
settleWork = resolve;
}),
@@ -501,6 +501,29 @@ describe('DriverController.#handleCall (via captured router)', () => {
expect(body.service.name).toBe('puter-kvstore');
});
it('treats null `args` as no arguments', async () => {
const actor = await makeUserActor();
await server.stores.permission.setFlatUserPerm(
actor.user!.id!,
'service:puter-subdomains:ii:puter-subdomains',
{
permission: 'service:puter-subdomains:ii:puter-subdomains',
deleted: false,
issuer_user_id: actor.user!.id!,
} as never,
);
const res = makeRes();
const req = makeReq(
{ interface: 'puter-subdomains', method: 'select', args: null },
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](req, res as unknown as Response, () => {}),
);
expect((res.body as { success: boolean }).success).toBe(true);
});
it('pipes a stream-shaped result instead of JSON', async () => {
// Use an ephemeral driver registered onto the controller's bag.
const streamPayload = Buffer.from('hello-world');
@@ -51,7 +51,7 @@ import { PermissionUtil } from '../../services/permission/permissionUtil.js';
import type { WithLifecycle } from '../../types';
import { withSpan } from '../../util/span.js';
import { PuterController } from '../types.js';
import { CallSlot } from './callSlot.js';
import { CallSlot } from '../../core/http/callSlot.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
@@ -276,8 +276,10 @@ export class DriverController extends PuterController {
interface: ifaceName,
method,
driver: driverName,
args = {},
args: rawArgs,
} = (req.body ?? {}) as Record<string, unknown>;
// Drivers read fields off `args`, so a null one counts as absent.
const args = rawArgs ?? {};
if (!ifaceName || typeof ifaceName !== 'string') {
throw new HttpError(400, 'Missing or invalid `interface`', {
@@ -444,7 +446,7 @@ export class DriverController extends PuterController {
});
}
slot.attach(handle, res);
Context.set('keepDriverSlot', (work) => slot.keep(work));
Context.set('keepConcurrencySlot', (work) => slot.keep(work));
}
// Stash the requested driver name in Context so multi-provider
@@ -154,6 +154,26 @@ describe('GET /fs/readdir over HTTP', () => {
).toEqual(['a', 'a/b']);
});
it('answers 404 for a directory whose parent chain is broken', async () => {
const { username, token } = env.users.user;
const path = `/${username}/Documents/http-orphan-${Date.now()}`;
const created = await fetch(new URL('/fs/mkdir', env.apiOrigin), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ path, auth_token: token }),
});
expect(created.status).toBe(200);
const { uid } = (await created.json()) as { uid: string };
await env.server.clients.db.write(
'UPDATE fsentries SET path = NULL, parent_uid = ? WHERE uuid = ?',
['does-not-exist', uid],
);
await env.server.stores.fsEntry.invalidateEntryCacheByUuid(uid);
const response = await fetch(readdirUrl({ uid, auth_token: token }));
expect(response.status).toBe(404);
});
it('mkdir does not return internal ids, storage columns or tokens', async () => {
const { username, token } = env.users.user;
const response = await fetch(new URL('/fs/mkdir', env.apiOrigin), {
+3 -1
View File
@@ -1850,7 +1850,9 @@ export class FSController extends PuterController {
}
async #assertCanDedupeCreate(actor: Actor, targetPath: string) {
const existing = await this.stores.fsEntry.getEntryByPath(targetPath);
const existing = await this.stores.fsEntry.getEntryByPath(targetPath, {
foldedNames: true,
});
if (!existing) return;
const parent = pathPosix.dirname(targetPath);
await this.#assertAccess(
@@ -168,6 +168,27 @@ describe('legacy file downloads over HTTP', () => {
},
);
it('GET /token-read types a file by its name, and one with no known extension as a byte stream', async () => {
const typed = await requestFor(
'/token-read',
await writeFile('notes.txt', Buffer.from('hello')),
);
expect(typed.headers.get('content-type')).toBe(
'text/plain; charset=utf-8',
);
for (const name of ['README', 'archive.unknownext']) {
const response = await requestFor(
'/token-read',
await writeFile(name, Buffer.from('hello')),
);
expect(response.status).toBe(200);
expect(response.headers.get('content-type')).toBe(
'application/octet-stream',
);
}
});
it('GET /read answers byte_count on an empty file with an empty 200', async () => {
const file = await writeFile('empty-byte-count.bin', Buffer.alloc(0));
for (const query of ['byte_count=512', 'offset=0']) {
@@ -5111,9 +5111,8 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
// ── GET /get-launch-apps ────────────────────────────────────────────
//
// `recent` is a launch-metadata producer: it returns `index_url` for
// apps the user has opened before. The taskbar happens to launch these
// by name (so AppDriver's hosted-backing guard applies), but the field
// is served regardless, so the guard is applied here too.
// apps the user has opened before. The field is served whoever reads it,
// so AppDriver's hosted-backing guard is applied here too.
describe('LegacyFSController GET /get-launch-apps', () => {
const hostedUrl = (sub: string) => `https://${sub}.site.puter.localhost/`;
@@ -401,11 +401,9 @@ export class LegacyFSController extends PuterController {
.filter(Boolean);
// Don't hand out an index_url whose puter-hosted backing is
// gone or reclaimed. The taskbar launches recents by name (so
// AppDriver's guard applies), but this list is a
// launch-metadata producer like any other — a future consumer
// reading index_url straight off it shouldn't inherit a stale
// origin.
// gone or reclaimed. This list is a launch-metadata producer
// like any other — a consumer reading index_url straight off
// it shouldn't inherit a stale origin.
let backingGoneFlags: boolean[];
try {
backingGoneFlags =
@@ -767,8 +765,10 @@ export class LegacyFSController extends PuterController {
normalizedTarget,
);
if (dedupeName) {
const existing =
await this.stores.fsEntry.getEntryByPath(normalizedTarget);
const existing = await this.stores.fsEntry.getEntryByPath(
normalizedTarget,
{ foldedNames: true },
);
if (existing) {
const parent = pathPosix.dirname(normalizedTarget);
await assertAccess(
@@ -950,9 +950,7 @@ export class LegacyFSController extends PuterController {
// Trash, and `null`/`{}` when restoring. See
// `src/gui/src/helpers.js` → `window.move_items`.
newMetadata: (body.new_metadata ?? undefined) as
| Record<string, unknown>
| null
| undefined,
Record<string, unknown> | null | undefined,
});
const overwrittenEntry = replacedByMove(overwriteTarget, moved, name);
const oldPath = source.path;
@@ -1324,7 +1322,7 @@ export class LegacyFSController extends PuterController {
if (options.realMime) {
res.setHeader(
'Content-Type',
contentTypeFromMime(entry.name) as string,
contentTypeFromMime(entry.name) || 'application/octet-stream',
);
} else {
res.setHeader('Content-Type', 'application/octet-stream');
@@ -1426,8 +1424,7 @@ export class LegacyFSController extends PuterController {
}
type SignedOrEmpty =
| (SignedFile & { path?: string })
| Record<string, never>;
(SignedFile & { path?: string }) | Record<string, never>;
const result: { signatures: SignedOrEmpty[]; token?: string } = {
signatures: [],
};
@@ -2054,10 +2051,7 @@ export class LegacyFSController extends PuterController {
const subjectRef = body.subject;
const appRef = body.app;
const mode = (getString(body, 'mode') ?? 'read') as
| 'see'
| 'list'
| 'read'
| 'write';
'see' | 'list' | 'read' | 'write';
if (!subjectRef || !appRef)
throw new HttpError(400, '`subject` and `app` are required', {
legacyCode: 'bad_request',
@@ -277,4 +277,124 @@ describe('writes inside a folder shared with write', () => {
expect(JSON.stringify(res.json)).not.toContain(s.realDir);
}
}, 180_000);
describe('a file the recipient opened with an app', () => {
const makeApp = (ownerUserId: number) =>
(
env.server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `sfw-app-${Math.random().toString(36).slice(2, 9)}`,
title: 'Shared folder app',
index_url: `https://sfw-${Math.random().toString(36).slice(2, 9)}.test/`,
},
{ ownerUserId },
);
/** Opens `note.txt` with a new app, as a launch does. */
const openWithApp = async (
s: Awaited<ReturnType<typeof shareAFolder>>,
) => {
const recipientId = (await env.server.stores.user.getByUsername(
s.recipient.username,
))!.id;
const app = await makeApp(recipientId);
const signed = await post('/sign', s.recipient.token, {
items: [{ uid: s.noteUid, action: 'write' }],
app_uid: app.uid,
});
expect(signed.status).toBe(200);
return { app, appToken: signed.json!.token as string };
};
const read = async (token: string, query: Record<string, string>) => {
const res = await fetch(
new URL(`/read?${new URLSearchParams(query)}`, env.apiOrigin),
{ headers: { authorization: `Bearer ${token}` } },
);
return { status: res.status, text: await res.text() };
};
const write = (token: string, path: string, content: string) =>
post('/fs/write', token, {
fileMetadata: {
path,
size: content.length,
contentType: 'text/plain',
overwrite: true,
},
fileContent: content,
});
it('reads and saves it by path and uid', async () => {
const s = await shareAFolder();
const { appToken } = await openWithApp(s);
const masked = `${s.maskedDir}/note.txt`;
const real = `${s.realDir}/note.txt`;
expect((await write(appToken, masked, 'one')).status).toBe(200);
expect(await read(appToken, { uid: s.noteUid })).toEqual({
status: 200,
text: 'one',
});
expect((await write(appToken, real, 'two')).status).toBe(200);
expect(await read(appToken, { file: real })).toEqual({
status: 200,
text: 'two',
});
expect(
(await post('/stat', appToken, { uid: s.noteUid })).status,
).toBe(200);
}, 180_000);
it('only reads it once the share drops to read', async () => {
const s = await shareAFolder();
const { appToken } = await openWithApp(s);
const dir = await post('/stat', s.owner.token, { path: s.realDir });
const reshared = await post('/share', s.owner.token, {
recipients: [s.recipient.username],
items: [{ uid: dir.json!.uid }],
mode: 'read',
});
expect(reshared.status).toBe(200);
expect((await read(appToken, { uid: s.noteUid })).status).toBe(200);
expect(
(await write(appToken, `${s.maskedDir}/note.txt`, 'x')).status,
).toBe(403);
}, 180_000);
it('gives nothing to another app of the recipient', async () => {
const s = await shareAFolder();
const { appToken } = await openWithApp(s);
const recipientId = (await env.server.stores.user.getByUsername(
s.recipient.username,
))!.id;
const other = await makeApp(recipientId);
const minted = await post(
'/auth/get-user-app-token',
s.recipient.token,
{ app_uid: other.uid },
);
expect(minted.status).toBe(200);
const otherToken = minted.json!.token as string;
// The same requests the opened app gets through.
const real = `${s.realDir}/note.txt`;
expect((await read(appToken, { uid: s.noteUid })).status).toBe(200);
expect((await write(appToken, real, 'one')).status).toBe(200);
expect((await read(otherToken, { uid: s.noteUid })).status).toBe(
404,
);
expect((await write(otherToken, real, 'x')).status).toBe(404);
expect(await read(appToken, { uid: s.noteUid })).toEqual({
status: 200,
text: 'one',
});
}, 180_000);
});
});
@@ -27,7 +27,7 @@ import type { Actor } from '../../core/actor.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { embedFrameAncestors } from './HomepageController.js';
import { embedFrameAncestors, embedHostOrigin } from './HomepageController.js';
// ── Test harness ────────────────────────────────────────────────────
//
@@ -79,6 +79,7 @@ interface CapturedResponse {
statusCode: number;
body: unknown;
contentType?: string;
location?: string;
}
const makeReq = (init: {
@@ -87,6 +88,7 @@ const makeReq = (init: {
actor?: Actor;
hostname?: string;
protocol?: string;
originalUrl?: string;
}): Request => {
return {
body: {},
@@ -94,6 +96,7 @@ const makeReq = (init: {
headers: {},
params: init.params ?? {},
path: init.path ?? '/',
originalUrl: init.originalUrl ?? init.path ?? '/',
hostname: init.hostname ?? 'test.local',
protocol: init.protocol ?? 'http',
actor: init.actor,
@@ -131,6 +134,11 @@ const makeRes = () => {
captured.contentType = value;
return res;
}),
redirect: vi.fn((location: string) => {
captured.statusCode = 302;
captured.location = location;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
@@ -214,23 +222,72 @@ describe('HomepageController shell routes', () => {
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
it('serves the shell on /settings, /dashboard, /desktop, /action, /@:username', async () => {
for (const path of [
'/settings',
'/settings/*splat',
'/dashboard',
'/dashboard/',
'/desktop',
'/desktop/',
'/action/*splat',
'/@:username',
]) {
it('leaves guiTokenDigest out for a session cookie that is not a user session', async () => {
const { actor } = await makeUser();
const appActor = {
...actor,
app: { id: 1, uid: 'app-test' },
session: { uid: uuidv4() },
} as unknown as Actor;
const req = Object.assign(makeReq({ path: '/', actor: appActor }), {
tokenSource: 'cookie',
cookies: { puter_token_v2: 'legacy' },
});
const { res, captured } = makeRes();
await callRoute('get', '/', req, res);
expect(String(captured.body)).not.toContain('guiTokenDigest');
});
it('serves the shell on /dashboard and /action', async () => {
for (const path of ['/dashboard', '/dashboard/', '/action/*splat']) {
const { res, captured } = makeRes();
await callRoute('get', path, makeReq({ path }), res);
expect(typeof captured.body).toBe('string');
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
}
});
it('redirects /desktop to the root, keeping the query', async () => {
for (const path of ['/desktop', '/desktop/']) {
const { res, captured } = makeRes();
await callRoute(
'get',
path,
makeReq({ path, originalUrl: `${path}?ref=abc&x=1` }),
res,
);
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe('/?ref=abc&x=1');
expect(captured.body).toBeUndefined();
}
const { res, captured } = makeRes();
await callRoute('get', '/desktop', makeReq({ path: '/desktop' }), res);
expect(captured.location).toBe('/');
});
it('redirects /settings to the dashboard Home tab', async () => {
for (const path of ['/settings', '/settings/*splat']) {
const { res, captured } = makeRes();
await callRoute('get', path, makeReq({ path }), res);
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe('/dashboard#home');
expect(captured.body).toBeUndefined();
}
});
it('redirects /@:username to the dashboard', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/@:username',
makeReq({ path: '/@alice', params: { username: 'alice' } }),
res,
);
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe('/');
expect(captured.body).toBeUndefined();
});
});
// ── /app/:name ──────────────────────────────────────────────────────
@@ -360,47 +417,21 @@ describe('HomepageController GET /app/:name', () => {
expect(html).not.toContain('owner_user_id');
});
it('serves the same app shell under /desktop/app/:name', async () => {
const { userId } = await makeUser();
const name = `desk-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.app.create(
{
name,
title: 'Desktop App',
description: 'opens on the desktop instead of the dashboard',
index_url: `https://example.com/${name}/`,
approved_for_listing: 1,
},
{ ownerUserId: userId },
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/desktop/app/:name',
makeReq({ params: { name }, path: `/desktop/app/${name}` }),
res,
);
expect(captured.statusCode).toBe(200);
const html = String(captured.body);
expect(html).toMatch(/<!DOCTYPE html>/i);
expect(html).toContain('Desktop App');
});
it('returns 404 under /desktop/app/:name when the app is unknown', async () => {
it('redirects /desktop/app/:name to /app/:name, keeping the query', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/desktop/app/:name',
makeReq({
params: { name: 'no-such-app' },
path: '/desktop/app/no-such-app',
params: { name: 'editor' },
path: '/desktop/app/editor',
originalUrl: '/desktop/app/editor?file=%2Falice%2Fa.txt&x=1',
}),
res,
);
expect(captured.statusCode).toBe(404);
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe('/app/editor?file=%2Falice%2Fa.txt&x=1');
expect(captured.body).toBeUndefined();
});
it('omits index_url even for a public app — the shell is not the launch authority', async () => {
@@ -434,40 +465,39 @@ describe('HomepageController GET /app/:name', () => {
// -- app landing rate limit --
describe('HomepageController app landing rate limit', () => {
it('rate-limits /app/:name and /desktop/app/:name per network', () => {
for (const path of ['/app/:name', '/desktop/app/:name']) {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === path,
);
if (!route) throw new Error(`No GET ${path} route`);
expect(route.options.rateLimit).toEqual({
scope: 'app-landing',
limit: 600,
window: 60_000,
key: 'ip',
});
}
it('rate-limits /app/:name per network', () => {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === '/app/:name',
);
if (!route) throw new Error('No GET /app/:name route');
expect(route.options.rateLimit).toEqual({
scope: 'app-landing',
limit: 600,
window: 60_000,
key: 'ip',
});
});
});
// ── /show/* ─────────────────────────────────────────────────────────
describe('HomepageController GET /show/*splat', () => {
it('emits a launch_app explorer call with the post-/show path', async () => {
it('redirects to the path in the dashboard Files tab, keeping the query', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/show/*splat',
makeReq({ path: '/show/alice/Documents' }),
makeReq({
path: '/show/alice/My%20Documents',
originalUrl: '/show/alice/My%20Documents?ref=abc',
}),
res,
);
// The launch payload is JSON-serialized into the rendered HTML.
// We assert the rendered shell included the explorer launch hint
// pointing at the expected (slashed) path.
const html = String(captured.body);
expect(html).toContain('launch_app');
expect(html).toContain('explorer');
expect(html).toContain('/alice/Documents');
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe(
'/dashboard?ref=abc#files/alice/My%20Documents',
);
expect(captured.body).toBeUndefined();
});
});
@@ -550,3 +580,26 @@ describe('embedFrameAncestors', () => {
).toBe("'self'");
});
});
describe('embedHostOrigin', () => {
const config = {
origin: 'https://example.test',
domain: 'example.test',
api_base_url: 'https://backend.example.test',
static_hosting_domain: 'site.example.test',
};
it.each([
['https://sheets.example.test/doc/1', 'https://sheets.example.test'],
['https://backend.example.test/x', null],
['https://api.example.test/', null],
['https://someone.site.example.test/', null],
['https://a.sheets.example.test/', null],
['http://sheets.example.test/', null],
['https://example.test/', null],
['not a url', null],
[undefined, null],
])('%s frames as %s', (referer, expected) => {
expect(embedHostOrigin(config, referer)).toBe(expected);
});
});
@@ -20,6 +20,7 @@
import express from 'express';
import path from 'node:path';
import { PuterController } from '../types.js';
import { isPlainUserActor } from '../../core/actor.js';
import { toAppShellView } from '../../util/appShellView.js';
import type { PuterRouter } from '../../core/http/PuterRouter';
import {
@@ -27,12 +28,31 @@ import {
subdomainLabelOf,
} from '../../core/http/middleware/hostRedirects.js';
import type { IConfig } from '../../types';
import type { UserRow } from '../../stores/user/UserStore';
import type {
PuterHomepageService,
PageMeta,
LaunchOptions,
PopupFrame,
ShellContext,
} from '../../services/homepage/PuterHomepageService';
/**
* GUI popups a first-party page can show in a frame at `/embed/<action>`,
* instead of opening `/action/<action>` in a window, plus the upgrade and
* top-up dialogs, which it would otherwise open in a dashboard tab, and the
* email confirmation, which it otherwise can't show.
*/
export const FRAMED_POPUP_ACTIONS = [
'sign-in',
'show-open-file-picker',
'show-save-file-picker',
'show-directory-picker',
'upgrade',
'top-up',
'request-email-confirmation',
'attach-custom-domain',
] as const;
/** Longest description/og:description/twitter:description on `/app/<name>`. */
export const APP_META_DESCRIPTION_MAX = 150;
@@ -61,32 +81,34 @@ export function appMetaDescription(text: string): string {
return `${cut.replace(/[\s,;:.!?…—-]+$/, '')}…`;
}
type EmbedFramingConfig = Pick<
IConfig,
| 'origin'
| 'domain'
| 'api_base_url'
| 'static_hosting_domain'
| 'static_hosting_domain_alt'
| 'private_app_hosting_domain'
| 'private_app_hosting_domain_alt'
>;
/**
* `frame-ancestors` for the app browser embed. A wildcard over `*.<domain>`
* The origin of a first-party page that may frame an `/embed/*` page, named
* exactly from the request's referrer, or null. A wildcard over `*.<domain>`
* would also admit user-hosted sites whenever the hosting domains sit under it,
* so the embedding page is named exactly, from the request's referrer. Only a
* single label under `<domain>` qualifies, and not one this server answers
* itself (the API, WebDAV, SDK and native-app hosts).
* so only a single label under `<domain>` qualifies, and not one this server
* answers itself (the API, WebDAV, SDK and native-app hosts).
*/
export function embedFrameAncestors(
config: Pick<
IConfig,
| 'origin'
| 'domain'
| 'api_base_url'
| 'static_hosting_domain'
| 'static_hosting_domain_alt'
| 'private_app_hosting_domain'
| 'private_app_hosting_domain_alt'
>,
export function embedHostOrigin(
config: EmbedFramingConfig,
referer: string | undefined,
): string {
): string | null {
const gui = new URL(config.origin ?? `https://${config.domain}`);
let parent: URL;
try {
parent = new URL(referer ?? '');
} catch {
return "'self'";
return null;
}
const host = parent.hostname.toLowerCase();
const label = subdomainLabelOf(host, config.domain);
@@ -105,7 +127,16 @@ export function embedFrameAncestors(
!isReservedSubdomain(label) &&
host !== new URL(config.api_base_url).hostname &&
!hostingDomains.some((d) => host === d || host.endsWith(`.${d}`));
return allowed ? `'self' ${parent.origin}` : "'self'";
return allowed ? parent.origin : null;
}
/** `frame-ancestors` for an `/embed/*` page: the GUI and {@link embedHostOrigin}. */
export function embedFrameAncestors(
config: EmbedFramingConfig,
referer: string | undefined,
): string {
const host = embedHostOrigin(config, referer);
return host ? `'self' ${host}` : "'self'";
}
/**
@@ -141,7 +172,7 @@ export class HomepageController extends PuterController {
req: express.Request,
res: express.Response,
metaOverrides: Partial<PageMeta> = {},
launch: LaunchOptions = {},
popupFrame?: PopupFrame,
) => {
const meta = { ...defaultMeta(req), ...metaOverrides };
const actor =
@@ -150,19 +181,36 @@ export class HomepageController extends PuterController {
actor?: Parameters<typeof homepage.send>[0]['actor'];
}
).actor ?? null;
await homepage.send({ req, res, actor }, meta, launch);
await homepage.send(
{
req,
res,
actor,
guiTokenDigest: this.#guiTokenDigest(req, actor),
},
meta,
popupFrame,
);
};
// -- Root + path-aliased shell routes ------------------------
router.get('/', {}, (req, res) => sendShell(req, res));
router.get('/embed/apps', {}, async (req, res) => {
// SDK bootstrap parameters must not change this page's account or API origin.
// Checked on the raw URL; `req.query` drops pairs past the parser's limit.
/**
* Headers shared by the `/embed/*` pages; false when the request was
* redirected instead. SDK bootstrap parameters must not change an
* embed's account or API origin, so a query string is dropped. Checked
* on the raw URL; `req.query` drops pairs past the parser's limit.
*/
const prepareEmbed = (
req: express.Request,
res: express.Response,
path: string,
): boolean => {
if (req.originalUrl.includes('?')) {
res.redirect('/embed/apps');
return;
res.redirect(path);
return false;
}
res.removeHeader('X-Frame-Options');
res.setHeader(
@@ -170,6 +218,11 @@ export class HomepageController extends PuterController {
`frame-ancestors ${embedFrameAncestors(this.config, req.get('referer'))}`,
);
res.setHeader('Cache-Control', 'no-store');
return true;
};
router.get('/embed/apps', {}, async (req, res) => {
if (!prepareEmbed(req, res, '/embed/apps')) return;
// Shown to visitors without a session; a signed-in embed fetches
// its own list, so a failure here only costs signed-out visitors.
const guestApps = await this.services.recommendedApps
@@ -178,23 +231,83 @@ export class HomepageController extends PuterController {
res.send(homepage.renderAppBrowserEmbed({ guestApps }));
});
router.get('/settings', {}, (req, res) => sendShell(req, res));
router.get('/settings/*splat', {}, (req, res) => sendShell(req, res));
router.get('/embed/feedback', {}, (req, res) => {
if (!prepareEmbed(req, res, '/embed/feedback')) return;
// The feedback is recorded against this origin, which
// frame-ancestors has just pinned to the actual framing page.
res.send(
homepage.renderFeedbackEmbed({
hostOrigin: embedHostOrigin(
this.config,
req.get('referer'),
),
}),
);
});
// The GUI runs the popup's own boot, answering the framing page
// instead of an opener.
for (const action of FRAMED_POPUP_ACTIONS) {
const embedPath = `/embed/${action}`;
router.get(embedPath, {}, async (req, res) => {
// Routes match case-insensitively and with a trailing slash,
// and the shell boots the desktop for any other spelling, so
// only the exact one may be framed.
if (req.originalUrl !== embedPath) {
res.redirect(embedPath);
return;
}
if (!prepareEmbed(req, res, embedPath)) return;
await sendShell(
req,
res,
{},
{
hostOrigin: embedHostOrigin(
this.config,
req.get('referer'),
),
},
);
});
}
// Old settings links land on the dashboard's Home tab.
router.get('/settings', {}, (_req, res) =>
res.redirect('/dashboard#home'),
);
router.get('/settings/*splat', {}, (_req, res) =>
res.redirect('/dashboard#home'),
);
router.get('/dashboard', {}, (req, res) => sendShell(req, res));
router.get('/dashboard/', {}, (req, res) => sendShell(req, res));
router.get('/desktop', {}, (req, res) => sendShell(req, res));
router.get('/desktop/', {}, (req, res) => sendShell(req, res));
const queryOf = (req: express.Request) => {
const separator = req.originalUrl.indexOf('?');
return separator === -1 ? '' : req.originalUrl.slice(separator);
};
// Old desktop links land on the dashboard, query intact.
router.get('/desktop', {}, (req, res) =>
res.redirect(`/${queryOf(req)}`),
);
router.get('/desktop/', {}, (req, res) =>
res.redirect(`/${queryOf(req)}`),
);
router.get('/action/*splat', {}, (req, res) => sendShell(req, res));
router.get('/@:username', {}, (req, res) => sendShell(req, res));
// Old links to a user's public folder land on the dashboard.
router.get('/@:username', {}, (_req, res) => res.redirect('/'));
// Old desktop app links open the app in the dashboard, query intact.
router.get('/desktop/app/:name', {}, (req, res) => {
const name = encodeURIComponent(String(req.params.name ?? ''));
res.redirect(`/app/${name}${queryOf(req)}`);
});
// -- /app/:name - app metadata baked into the shell ----------
// `/desktop/app/:name` is the same landing booted on the desktop
// instead of the dashboard; the GUI strips the prefix and treats the
// rest of the path identically, so both render the same shell.
const sendAppShell = async (
req: express.Request,
@@ -249,26 +362,11 @@ export class HomepageController extends PuterController {
{ rateLimit: APP_LANDING_LIMIT },
sendAppShell,
);
router.get(
'/desktop/app/:name',
{ rateLimit: APP_LANDING_LIMIT },
sendAppShell,
);
// -- /show/* - launch explorer with the requested file path --
// Old /show links open the path in the dashboard's Files tab, query intact.
router.get('/show/*splat', {}, (req, res) => {
const filePath = req.path.slice('/show'.length);
const launch: LaunchOptions = {
on_initialized: [
{
$: 'window-call',
fn_name: 'launch_app',
args: [{ name: 'explorer', path: filePath }],
},
],
};
return sendShell(req, res, {}, launch);
res.redirect(`/dashboard${queryOf(req)}#files${filePath}`);
});
// -- /sw.js - the service worker, scoped to the whole origin --
@@ -297,6 +395,38 @@ export class HomepageController extends PuterController {
);
}
}
/**
* Lets the GUI skip `/get-gui-token`, which reads the session cookie: the
* digest of the token it would hand over, null when there is no session
* cookie (either name the auth probe reads), and undefined (the GUI asks)
* when a cookie didn't resolve to a session.
*/
#guiTokenDigest(
req: express.Request,
actor: ShellContext['actor'],
): string | null | undefined {
if (req.tokenSource === 'cookie') {
if (!actor || !isPlainUserActor(actor) || !actor.session?.uid) {
return undefined;
}
return this.services.auth.guiTokenDigest(
actor.user as UserRow,
actor.session.uid,
);
}
const cookies = (req.cookies ?? {}) as Record<string, unknown>;
const hasSessionCookie = [
this.config.cookie_name,
'puter_token_v2',
].some(
(name) =>
!!name &&
typeof cookies[name] === 'string' &&
cookies[name] !== '',
);
return hasSessionCookie ? undefined : null;
}
}
const safeJsonParse = (s: string): Record<string, unknown> | null => {
@@ -372,26 +372,6 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
);
});
it('bakes a whitelisted desktop app landing (/desktop/app/<name>) into the state redirect_uri', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/:provider/start',
makeReq({
params: { provider: 'custom' },
query: { return_to: '/desktop/app/my-App_2' },
}),
res,
);
const state = new URL(captured.redirectUrl ?? '').searchParams.get(
'state',
);
const decoded = oidc().verifyState(state!);
expect(String(decoded?.redirect_uri)).toBe(
`${TEST_ORIGIN}/desktop/app/my-App_2`,
);
});
it('ignores a non-whitelisted return_to', async () => {
const bad_values = [
'/app/evil/extra',
@@ -400,6 +380,8 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
'//evil.test',
'/settings',
`/app/${'a'.repeat(101)}`,
'/desktop',
'/desktop/app/my-App_2',
'/desktop/app/evil/extra',
'/desktop/app/',
'/dashboard/app/name',
@@ -476,11 +458,11 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
sharedPath('b.txt'),
sharedPath('a.txt'),
],
'/desktop',
'/dashboard',
),
);
const url = new URL(uri);
expect(url.origin + url.pathname).toBe(`${TEST_ORIGIN}/desktop`);
expect(url.origin + url.pathname).toBe(`${TEST_ORIGIN}/dashboard`);
expect(url.searchParams.getAll('shared')).toEqual([
sharedPath('a.txt'),
sharedPath('b.txt'),
@@ -1620,7 +1602,7 @@ describe('OIDCController signup callback', () => {
expect(captured.redirectUrl).toContain('action=signup');
});
it('creates a fresh user without oidc_switched on first signup', async () => {
it('creates a fresh user on first signup', async () => {
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
@@ -1646,11 +1628,9 @@ describe('OIDCController signup callback', () => {
);
expect(captured.cookies).toHaveLength(1);
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
// Fresh creation — must NOT advertise "you've actually been signed in".
expect(captured.redirectUrl).not.toContain('oidc_switched=login');
});
it('adds oidc_switched=login when signup hits an already-linked account', async () => {
it('signs in to an already-linked account on signup, returning as a login would', async () => {
// Seed an account linked on (provider, sub) so the signup callback
// resolves to `linked-sub` rather than creating a new user.
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
@@ -1686,7 +1666,7 @@ describe('OIDCController signup callback', () => {
res,
);
expect(captured.cookies).toHaveLength(1);
expect(captured.redirectUrl).toContain('oidc_switched=login');
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
});
it('redirects to action=signup with auth_error when user resolution fails (unconfirmed email)', async () => {
+11 -26
View File
@@ -102,16 +102,14 @@ function signupOptionsFromState(
}
// GUI pages an OIDC flow may return to: the root (where a share email lands),
// /desktop, /dashboard, and direct app landings (/app/<name> and its
// desktop-booted twin /desktop/app/<name>, mirroring APP_NAME_REGEX in
// AppDriver). Strict whitelist — never a client-supplied URL (no open
// redirect).
// /dashboard, and direct app landings (/app/<name>, mirroring
// APP_NAME_REGEX in AppDriver). Strict whitelist — never a client-supplied URL
// (no open redirect).
function isWhitelistedReturnPath(path: string): boolean {
return (
path === '/' ||
path === '/desktop' ||
path === '/dashboard' ||
/^(\/desktop)?\/app\/[a-zA-Z0-9_-]{1,100}$/.test(path)
/^\/app\/[a-zA-Z0-9_-]{1,100}$/.test(path)
);
}
@@ -438,9 +436,8 @@ export class OIDCController extends PuterController {
let appRedirectUri = flowRedirects[flow] ?? (origin || '/');
// Optional GUI return path so login started from /desktop,
// /dashboard, an /app/<name> landing, or a share link lands
// back there.
// Optional GUI return path so login started from /dashboard,
// an /app/<name> landing, or a share link lands back there.
const rawReturnTo = Array.isArray(req.query.return_to)
? req.query.return_to[0]
: req.query.return_to;
@@ -688,14 +685,7 @@ export class OIDCController extends PuterController {
);
}
// If we landed on an existing account (either via the
// provider_sub or via email match), signal the GUI so it can
// render a "signed in" flow rather than "account created".
const extra =
resolved.origin === 'created'
? undefined
: { oidc_switched: 'login' };
await this.#finishLogin(req, res, user, stateDecoded, extra);
await this.#finishLogin(req, res, user, stateDecoded);
};
router.get('/auth/oidc/callback/signup', cbOpts, signupCb);
router.post('/auth/oidc/callback/signup', cbOpts, signupCb);
@@ -834,17 +824,14 @@ if (window.opener) {
attempt = 0,
): Promise<
| { error: string; code?: string; requestCode?: string }
| {
user: import('../../stores/user/UserStore.js').UserRow;
origin: 'linked-sub' | 'linked-email' | 'created';
}
| { user: import('../../stores/user/UserStore.js').UserRow }
> {
// 1. Existing provider/sub link.
const linked = await this.services.oidc.findUserByProviderSub(
provider,
userinfo.sub,
);
if (linked) return { user: linked, origin: 'linked-sub' };
if (linked) return { user: linked };
// 2/3. Email match branch.
const claimedEmail =
@@ -881,7 +868,7 @@ if (window.opener) {
error: outcome.error ?? 'Failed to link provider.',
};
}
return { user: byEmail, origin: 'linked-email' };
return { user: byEmail };
}
}
@@ -911,7 +898,7 @@ if (window.opener) {
requestCode: outcome.requestCode,
};
}
return { user: outcome.user, origin: 'created' };
return { user: outcome.user };
}
async #processCallback(
@@ -1000,7 +987,6 @@ if (window.opener) {
res: Response,
user: UserRow,
stateDecoded: Record<string, unknown>,
extraQueryParams?: Record<string, string>,
): Promise<void> {
const origin = (this.config.origin ?? '').replace(/\/$/, '');
const continuation: OidcLoginContinuation = {
@@ -1018,7 +1004,6 @@ if (window.opener) {
stateDecoded.msg_id == null
? null
: String(stateDecoded.msg_id),
switchedToLogin: extraQueryParams?.oidc_switched === 'login',
};
if (user.otp_enabled) {
const challenge = this.services.token.sign(
@@ -139,7 +139,7 @@ describe('OIDC account second-factor login', () => {
expect(await response.json()).toMatchObject({
proceed: true,
next_step: 'complete',
redirectUrl: `${env.origin}/dashboard${flow === 'signup' ? '?oidc_switched=login' : ''}`,
redirectUrl: `${env.origin}/dashboard`,
});
expect(
response.headers
@@ -314,8 +314,8 @@ describe('OIDC account second-factor login', () => {
it('preserves ordinary OIDC login for accounts without 2FA', async () => {
await account(false);
const response = await callback('login', { return_to: '/desktop' });
expect(response.headers.get('location')).toBe(`${env.origin}/desktop`);
const response = await callback('login', { return_to: '/dashboard' });
expect(response.headers.get('location')).toBe(`${env.origin}/dashboard`);
expect(
response.headers.get('set-cookie')?.includes('puter_auth_token='),
).toBe(true);
+20 -13
View File
@@ -24,11 +24,7 @@ import { computeNetworkFingerprint } from '../../core/http/middleware/rateLimit.
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterController } from '../types.js';
import { PEER_COSTS } from './costs.js';
import {
readClaimedGrantIdentifier,
signGuestGrant,
verifyGuestGrant,
} from './guestGrant.js';
import { signGuestGrant, verifyGuestGrant } from './guestGrant.js';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
@@ -191,19 +187,30 @@ export class PeerController extends PuterController {
// guests share one bucket and no host can be relayed for by
// more guests per minute than this — the only ceiling on guest
// spend we can apply before the bytes are already spent.
// A grant that doesn't parse can't name a bucket, so those
// requests fall back to the caller's own network.
// The limiter counts a request when it admits it, so only a
// grant that would redeem names a host; otherwise anyone
// holding the host's identifier could drain the bucket and
// lock out every real guest. Anything else - forged, expired,
// malformed, or no secret configured - is charged to the
// caller's own network.
rateLimit: {
scope: 'peer-guest-turn',
limit: 60,
window: 60_000,
key: (req: Request) => {
const claimed = readClaimedGrantIdentifier(
(req.body as { grant?: unknown } | undefined)
?.grant,
);
return claimed
? `host:${claimed}`
const secret =
this.config.peers?.guest_turn?.grant_secret;
const verified = secret
? verifyGuestGrant({
grant: (
req.body as
{ grant?: unknown } | undefined
)?.grant,
secret,
})
: null;
return verified?.status === 'ok'
? `host:${verified.customIdentifier}`
: `net:${computeNetworkFingerprint(req)}`;
},
},
+1 -1
View File
@@ -19,7 +19,7 @@
import { toMicroCents } from '../../services/metering/utils';
// Microcents per byte of TURN egress ($0.05/GB).
// Microcents per byte of TURN egress ($0.10/GB).
export const PEER_COSTS = {
'turn:egress-bytes': toMicroCents(0.1 / 1000 ** 3),
} as const;
@@ -1,7 +1,6 @@
import { createHmac } from 'node:crypto';
import { describe, expect, it } from 'vitest';
import {
readClaimedGrantIdentifier,
signGuestGrant,
verifyGuestGrant,
} from './guestGrant.js';
@@ -243,59 +242,3 @@ describe('signGuestGrant / verifyGuestGrant', () => {
});
});
});
describe('readClaimedGrantIdentifier', () => {
it('reads the identifier from a valid grant', () => {
const { grant } = signGuestGrant({
customIdentifier: USER_ID,
ttlSeconds: 60,
secret: SECRET,
});
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
});
it('reads the claimed identifier without checking the signature', () => {
// Bucketing runs before verification, so this is expected: a forged
// grant still names a bucket, and the handler still rejects it.
const grant = forgeGrant(
{ id: USER_ID, exp: 9_999_999_999, n: 'x' },
{ secret: 'wrong-secret' },
);
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
});
it('reads an expired grant, which the handler then rejects', () => {
const { grant } = signGuestGrant({
customIdentifier: USER_ID,
ttlSeconds: 60,
secret: SECRET,
now: 1_000_000_000_000,
});
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
expect(verifyGuestGrant({ grant, secret: SECRET })).toEqual({
status: 'expired',
});
});
it.each([
['a non-string', 42],
['undefined', undefined],
['a wrong-shaped string', 'not-a-grant'],
['an unknown version', 'pg2.abc.def'],
['an over-long string', `pg1.${'a'.repeat(600)}.sig`],
[
'a non-JSON payload',
`pg1.${Buffer.from('x').toString('base64url')}.sig`,
],
])('returns null for %s', (_label, grant) => {
expect(readClaimedGrantIdentifier(grant)).toBeNull();
});
it('returns null for an identifier of the wrong shape', () => {
const grant = forgeGrant({ id: 'nope', exp: 9_999_999_999 });
expect(readClaimedGrantIdentifier(grant)).toBeNull();
});
});
@@ -171,37 +171,3 @@ export const verifyGuestGrant = ({
expiresAt: payload.exp,
};
};
/**
* Read the claimed identifier out of a grant _without_ verifying it, for
* rate-limit bucketing only.
*
* Bucketing has to happen before the handler runs, and a forged grant is
* rejected there without reaching the upstream API — so an unverified read is
* enough to put a host's guests in one bucket, and the worst a forger achieves
* is choosing which bucket their own rejections land in. Never use this to
* decide attribution.
*
* @returns The claimed identifier, or null if the grant doesn't parse.
*/
export const readClaimedGrantIdentifier = (grant: unknown): string | null => {
if (typeof grant !== 'string' || grant.length > MAX_GRANT_LENGTH) {
return null;
}
const parts = grant.split('.');
if (parts.length !== 3 || parts[0] !== GRANT_VERSION) return null;
try {
const payload = JSON.parse(
Buffer.from(parts[1]!, 'base64url').toString('utf8'),
) as { id?: unknown };
if (
typeof payload?.id !== 'string' ||
!IDENTIFIER_RE.test(payload.id)
) {
return null;
}
return payload.id;
} catch {
return null;
}
};
@@ -44,7 +44,7 @@ import {
} from 'vitest';
import type { Actor } from '../../core/actor.js';
import type { ChatCompletionDriver } from '../../drivers/ai-chat/ChatCompletionDriver.js';
import type { ChatCompletionDriver } from '../../drivers/ai/interfaces/chat/ChatCompletionDriver.js';
import { PuterServer } from '../../server.js';
import { inRequestScope, setupTestServer } from '../../testUtil.js';
import { PuterAIController } from './PuterAIController.js';
@@ -49,8 +49,8 @@ import type { RouteOptions } from '../../core/http/index.js';
import {
COUNT_TOKENS,
type ChatCompletionDriver,
} from '../../drivers/ai-chat/ChatCompletionDriver.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../../drivers/util/aiLimits.js';
} from '../../drivers/ai/interfaces/chat/ChatCompletionDriver.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../../drivers/ai/shared/aiLimits.js';
import { PuterServer } from '../../server.js';
import { inRequestScope, setupTestServer } from '../../testUtil.js';
import { PuterAIController } from './PuterAIController.js';
@@ -30,25 +30,28 @@ import type { PuterRouter } from '../../core/http/PuterRouter.js';
import {
COUNT_TOKENS,
type ChatCompletionDriver,
} from '../../drivers/ai-chat/ChatCompletionDriver.js';
} from '../../drivers/ai/interfaces/chat/ChatCompletionDriver.js';
import type {
IChatCompleteResult,
IChatMessageResult,
IChatModel,
ICompleteArguments,
UsageDetails,
} from '../../drivers/ai-chat/types.js';
} from '../../drivers/ai/interfaces/chat/types.js';
import {
outputFormatFromResponseFormat,
rejectStatefulResponsesFields,
toolChoiceFromWire,
} from '../../drivers/ai-chat/utils/openaiParams.js';
} from '../../drivers/ai/protocols/openaiCompatible/openaiParams.js';
import {
promoteStopForToolCalls,
toFinishReason,
} from '../../drivers/ai-chat/utils/stopReason.js';
} from '../../drivers/ai/interfaces/chat/utils/stopReason.js';
import { isDriverStreamResult } from '../../drivers/meta.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../../drivers/util/aiLimits.js';
import {
AI_CONCURRENT,
AI_RATE_LIMIT,
} from '../../drivers/ai/shared/aiLimits.js';
import { PuterController } from '../types.js';
import { parseAnthropicRequest, toAnthropicMessage } from './anthropicWire.js';
import { pipeNdjsonStream } from './ndjsonStream.js';
@@ -1301,13 +1304,18 @@ export class PuterAIController extends PuterController {
return driver;
}
/** A completion that stops, and stops billing, when the caller hangs up. */
/**
* A completion whose caller may hang up. It runs on to its usage report
* either way, so it holds the route's concurrency slot until it settles.
*/
#complete(
res: Response,
args: ICompleteArguments,
): Promise<IChatCompleteResult> {
Context.set('abortSignal', abortOnDisconnect(res));
return this.#driver().complete(args);
const completion = this.#driver().complete(args);
Context.get('keepConcurrencySlot')?.(completion);
return completion;
}
}
@@ -18,7 +18,7 @@
*/
import { describe, expect, it } from 'vitest';
import type { IChatMessageResult } from '../../drivers/ai-chat/types.js';
import type { IChatMessageResult } from '../../drivers/ai/interfaces/chat/types.js';
import {
anthropicUsage,
parseAnthropicRequest,
@@ -32,7 +32,7 @@
import type { IncomingHttpHeaders } from 'node:http';
import { HttpError } from '../../core/http/HttpError.js';
import { sanitizeCacheControl } from '../../drivers/ai-chat/providers/claude/anthropicPolicy.js';
import { sanitizeCacheControl } from '../../drivers/ai/providers/claude/chat/anthropicPolicy.js';
import type {
ICompleteArguments,
IChatMessageResult,
@@ -42,9 +42,9 @@ import type {
ThinkingConfig,
ToolChoice,
UsageDetails,
} from '../../drivers/ai-chat/types.js';
import { needsOpenAICoercion } from '../../drivers/ai-chat/utils/normalizeToOpenAI.js';
import { fromFinishReason } from '../../drivers/ai-chat/utils/stopReason.js';
} from '../../drivers/ai/interfaces/chat/types.js';
import { needsOpenAICoercion } from '../../drivers/ai/interfaces/chat/utils/normalizeToOpenAI.js';
import { fromFinishReason } from '../../drivers/ai/interfaces/chat/utils/stopReason.js';
const toStringOrEmpty = (v: unknown): string =>
typeof v === 'string' ? v : '';
+4 -4
View File
@@ -20,14 +20,14 @@
/**
* SSE transport (`startSse`) plus the Anthropic `/v1/messages` streaming writer
* (`AnthropicSseWriter`), which turns the driver's NDJSON chunk protocol
* (`drivers/ai-chat/utils/Streaming.js`) into the exact Anthropic event
* sequence: `message_start` → `content_block_start/delta/stop`* →
* (`drivers/ai/interfaces/chat/utils/Streaming.js`) into the exact Anthropic
* event sequence: `message_start` → `content_block_start/delta/stop`* →
* `message_delta` → `message_stop`.
*/
import type { Response } from 'express';
import type { UsageDetails } from '../../drivers/ai-chat/types.js';
import { promoteStopForToolCalls } from '../../drivers/ai-chat/utils/stopReason.js';
import type { UsageDetails } from '../../drivers/ai/interfaces/chat/types.js';
import { promoteStopForToolCalls } from '../../drivers/ai/interfaces/chat/utils/stopReason.js';
import { anthropicUsage } from './anthropicWire.js';
// -- SSE transport ------------------------------------------------------------
@@ -301,6 +301,7 @@ export class SystemController extends PuterController {
if (this.clients.email && req.actor.user?.email) {
try {
await this.clients.email.sendRaw({
stream: 'notify',
to: supportEmail,
replyTo: req.actor.user.email,
subject: `Contact from ${req.actor.user.username}`,
@@ -547,6 +547,7 @@ describe('SystemController POST /contactUs — attachments', () => {
expect(sendRaw).toHaveBeenCalledTimes(1);
const sent = sendRaw.mock.calls[0][0] as {
stream: string;
text: string;
attachments: Array<{
filename: string;
@@ -555,6 +556,7 @@ describe('SystemController POST /contactUs — attachments', () => {
contentDisposition: string;
}>;
};
expect(sent.stream).toBe('notify');
expect(sent.attachments).toHaveLength(1);
expect(sent.attachments[0].filename).toBe('payload.png');
expect(sent.attachments[0].contentType).toBe('image/png');
@@ -813,7 +813,10 @@ export class WebDAVController extends PuterController {
const userId = actor.user!.id as number;
await this.#assertCreate(actor, davPath);
const existing = await this.stores.fsEntry.getEntryByPath(davPath);
// A name only the collation matches is taken too.
const existing = await this.stores.fsEntry.getEntryByPath(davPath, {
foldedNames: true,
});
if (existing)
throw new HttpError(405, 'Already exists', {
legacyCode: 'bad_request',
@@ -937,7 +940,9 @@ export class WebDAVController extends PuterController {
await this.#assertCreate(actor, destPath);
const overwrite = req.headers.overwrite !== 'F';
const destExists = await this.stores.fsEntry.getEntryByPath(destPath);
const destExists = await this.stores.fsEntry.getEntryByPath(destPath, {
foldedNames: true,
});
if (destExists && !overwrite)
throw new HttpError(412, 'Destination exists and Overwrite=F', {
legacyCode: 'conflict',
@@ -985,7 +990,9 @@ export class WebDAVController extends PuterController {
await this.#assertCreate(actor, destPath);
const overwrite = req.headers.overwrite !== 'F';
const destExists = await this.stores.fsEntry.getEntryByPath(destPath);
const destExists = await this.stores.fsEntry.getEntryByPath(destPath, {
foldedNames: true,
});
if (destExists && !overwrite)
throw new HttpError(412, 'Destination exists and Overwrite=F', {
legacyCode: 'conflict',
@@ -0,0 +1,89 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import {
setupPuterTestEnv,
TEST_USER_CREDENTIALS,
type PuterTestEnv,
} from '../../testUtil.js';
/** A PUT body is the file, whatever content type the client labels it with. */
describe('PUT stores the body as sent', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 60_000);
afterAll(async () => {
await env?.shutdown();
});
const dav = (path: string, init: RequestInit) => {
const port = new URL(env.apiOrigin).port;
const auth = Buffer.from(
`${TEST_USER_CREDENTIALS.username}:${TEST_USER_CREDENTIALS.password}`,
).toString('base64');
return fetch(`http://dav.puter.localhost:${port}${encodeURI(path)}`, {
...init,
headers: { Authorization: `Basic ${auth}`, ...init.headers },
});
};
it.each([
['application/json', '{"name":"report","rows":[1,2,3]}'],
// Not JSON at all: the label alone must not get the body rejected.
['application/json', 'not json'],
['text/plain;actually=json', '{"a":1}'],
['application/x-www-form-urlencoded', 'a=1&b=two'],
['application/octet-stream', 'plain bytes'],
])(
'with Content-Type %s',
async (contentType, content) => {
const path = `/${env.users.user.username}/Documents/put-${randomUUID().slice(0, 8)}.bin`;
const put = await dav(path, {
method: 'PUT',
headers: { 'Content-Type': contentType },
body: content,
});
expect(put.status).toBe(201);
const get = await dav(path, { method: 'GET' });
expect(get.status).toBe(200);
expect(await get.text()).toBe(content);
},
30_000,
);
it('with a form body over the form parser limit', async () => {
const path = `/${env.users.user.username}/Documents/put-${randomUUID().slice(0, 8)}.bin`;
const content = `a=${'x'.repeat(200 * 1024)}`;
const put = await dav(path, {
method: 'PUT',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: content,
});
expect(put.status).toBe(201);
const get = await dav(path, { method: 'GET' });
expect(await get.text()).toBe(content);
}, 30_000);
});
@@ -243,6 +243,33 @@ describe('WispController', () => {
expect(await verifyStatus(token)).toBe(403);
});
it('binds a token minted by an access token to that token while the replica lags', async () => {
const { sessionActor, apiTokenActor, apiToken } = await freshUser();
const db = server.clients.db;
const original = db.read.bind(db);
const spy = vi
.spyOn(db, 'read')
.mockImplementation(async (sql: string, params?: unknown[]) =>
sql.includes("FROM `sessions` WHERE `kind` = 'access_token'")
? []
: original(sql, params),
);
let token: string;
try {
token = await relayTokenFor(apiTokenActor);
} finally {
spy.mockRestore();
}
expect(await verifyStatus(token)).toBe(200);
await server.services.auth.revokeAccessToken(
sessionActor,
apiToken,
);
expect(await verifyStatus(token)).toBe(403);
});
it('applies the account checks to a token with no session claim', async () => {
const { row } = await freshUser();
const token = server.services.token.sign(
@@ -151,11 +151,12 @@ export class WispController extends PuterController {
/**
* The session a relay token lives no longer than. An access token's is its
* own session row; the actor doesn't carry it.
* own session row.
*/
async #sessionUidFor(actor: Actor): Promise<string | undefined> {
if (actor.session?.uid) return actor.session.uid;
if (!actor.accessToken) return undefined;
if (actor.accessToken.sessionUid) return actor.accessToken.sessionUid;
const row = await this.stores.session.findActiveByAccessTokenUid(
actor.accessToken.uid,
);
+5
View File
@@ -47,6 +47,11 @@ export interface ActorAccessToken {
* reach while revokes, audits and gate hooks can still name the app.
*/
godmodeApp?: ActorApp | null;
/**
* The token's own session row, checked live when it authenticated. Unset
* when the token carries no `session_uid` naming its row.
*/
sessionUid?: string;
}
export interface Actor {
+5 -5
View File
@@ -69,11 +69,11 @@ export interface KnownContextFields {
*/
abortSignal: AbortSignal;
/**
* Keeps the call's concurrency slot until `work` settles (set by
* DriverController when the method is capped), for work that outlives the
* response.
* Keeps the request's concurrency slot until `work` settles, for work that
* outlives the response. Set by DriverController for a capped method and by
* the route concurrency gate.
*/
keepDriverSlot: (work: Promise<unknown>) => void;
keepConcurrencySlot: (work: Promise<unknown>) => void;
/**
* Aborts the chat driver's upstream request. A client disconnect doesn't
* fire it: the upstream bills a request it has accepted either way, so the
@@ -102,7 +102,7 @@ const KNOWN_KEY_FLAGS: Record<keyof KnownContextFields, true> = {
requestId: true,
driverName: true,
abortSignal: true,
keepDriverSlot: true,
keepConcurrencySlot: true,
upstreamSignal: true,
strictUpstreamErrors: true,
platformWrite: true,
@@ -28,9 +28,9 @@ interface SlotHandle {
}
/**
* A driver call's concurrency slot, held until the response is done and the
* call's work has settled, including work a driver carries on after the caller
* hung up or was answered.
* A request's concurrency slot, held until the response is done and the work
* registered with `keep` has settled, including work that carries on after the
* caller hung up or was answered.
*/
export class CallSlot {
#handle?: SlotHandle;
@@ -50,6 +50,13 @@ declare global {
tokenAuthFailed?: boolean;
/**
* What `authenticate` threw while checking the token (a database
* failure, say). The token may be fine; gates rethrow this instead
* of answering 401.
*/
tokenAuthError?: unknown;
/**
* Set when a token authenticated but its app is on the origin
* blocklist. The auth probe leaves `actor` unset; gates translate
@@ -0,0 +1,110 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../../testUtil.js';
const loadShed = () =>
Object.assign(new Error('Database operation failed'), {
code: 'dbBatchFailed',
});
/**
* A token whose lookup fails is not a dead token: clients discard a token on
* 401 `token_auth_failed`, so that answer is kept for tokens that checked out
* invalid.
*/
describe('auth probe when the token lookup fails', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const get = (path: string, token: string) =>
fetch(new URL(path, env.apiOrigin), {
headers: { authorization: `Bearer ${token}` },
});
/** Run `fn` while the next user lookup rejects with `error`. */
const withFailedUserLookup = async <T>(
error: Error,
fn: () => Promise<T>,
): Promise<T> => {
const spy = vi
.spyOn(env.server.stores.user, 'getByUuid')
.mockRejectedValueOnce(error);
try {
return await fn();
} finally {
spy.mockRestore();
}
};
it.each([
['a session token', () => env.users.user.token],
['an access token', () => env.users.user.apiToken],
])(
'answers %s with 503 db_unavailable during a database outage',
async (_kind, tokenOf) => {
const token = tokenOf();
const res = await withFailedUserLookup(loadShed(), () =>
get('/whoami', token),
);
expect(res.status).toBe(503);
expect(res.headers.get('retry-after')).toBe('5');
expect(await res.json()).toEqual({
error: 'Service temporarily unavailable',
message: 'Service temporarily unavailable',
code: 'db_unavailable',
});
expect((await get('/whoami', token)).status).toBe(200);
},
);
it('answers an unexpected lookup failure with a generic 500, not a 401', async () => {
const res = await withFailedUserLookup(new Error('boom'), () =>
get('/whoami', env.users.user.token),
);
expect(res.status).toBe(500);
expect((await res.json()).code).toBe('internal_error');
});
it('serves a route that does not require auth anonymously', async () => {
const res = await withFailedUserLookup(loadShed(), () =>
get('/version', env.users.user.token),
);
expect(res.status).toBe(200);
});
it('still answers an invalid token with 401 token_auth_failed', async () => {
const res = await get('/whoami', 'not-a-real-jwt');
expect(res.status).toBe(401);
expect(await res.json()).toEqual({
error: 'Authentication failed',
message: 'Authentication failed',
code: 'token_auth_failed',
});
});
});
@@ -555,17 +555,18 @@ describe('createAuthProbe — actor attachment + failure tracking', () => {
expect(req.tokenAuthFailed).toBe(true);
});
it('never rejects — sets tokenAuthFailed even when AuthService throws', async () => {
it('never rejects — records the error, not tokenAuthFailed, when AuthService throws', async () => {
const stub = makeStubAuth();
stub.setNext('throw');
const probe = createAuthProbe({ authService: stub.service });
const { req, next } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer bad-tok' } }),
makeReq({ headers: { authorization: 'Bearer some-tok' } }),
);
// Critical invariant: the probe NEVER rejects, no matter what.
expect(next).toHaveBeenCalledWith();
expect(req.tokenAuthFailed).toBe(true);
expect(req.tokenAuthFailed).toBeUndefined();
expect(req.tokenAuthError).toBeInstanceOf(Error);
expect(req.actor).toBeUndefined();
});
@@ -880,4 +881,27 @@ describe('createAuthProbe (integration) — real session token → real actor',
expect(req.actor).toBeUndefined();
expect(req.tokenAuthFailed).toBe(true);
});
it('records a failed user lookup on a valid token without marking it invalid', async () => {
const user = await makeUser();
const { token } = await authService.createSessionToken(user);
const lookupError = Object.assign(new Error('db down'), {
code: 'dbBatchFailed',
});
const spy = vi
.spyOn(server.stores.user, 'getByUuid')
.mockRejectedValueOnce(lookupError);
try {
const probe = createAuthProbe({ authService });
const { req } = await runProbe(
probe,
makeReq({ headers: { authorization: `Bearer ${token}` } }),
);
expect(req.actor).toBeUndefined();
expect(req.tokenAuthFailed).toBeUndefined();
expect(req.tokenAuthError).toBe(lookupError);
} finally {
spy.mockRestore();
}
});
});
@@ -45,9 +45,9 @@ interface AuthProbeOptions {
* and valid — attaches an `Actor` to `req.actor`.
*
* Key property: this middleware **never rejects**. Missing tokens, malformed
* tokens, expired tokens, tokens pointing at deleted users — all result in
* `req.actor` being left undefined. Per-route gates decide whether absence is
* acceptable.
* tokens, expired tokens, tokens pointing at deleted users, and lookups that
* fail outright all result in `req.actor` being left undefined. Per-route gates
* decide whether absence is acceptable.
*
* Token lookup order:
*
@@ -153,8 +153,10 @@ export const createAuthProbe = (opts: AuthProbeOptions): RequestHandler => {
} else if (result.invalid) {
req.tokenAuthFailed = true;
}
} catch {
req.tokenAuthFailed = true;
} catch (err) {
// Not a verdict on the token: routes that require auth answer
// with this error, routes that don't serve the request anonymously.
req.tokenAuthError = err;
}
next();
};
@@ -224,7 +226,7 @@ const extractToken = (
}
}
// 5. Query string (used by e.g. QR login, asset URLs).
// 5. Query string (used by e.g. asset URLs).
const queryToken = (req.query as { auth_token?: unknown } | undefined)
?.auth_token;
if (typeof queryToken === 'string' && queryToken.length > 0) {
@@ -288,6 +288,22 @@ describe('requireAuthGate', () => {
});
expectHttpError(got, 401, 'reauth_required');
});
it('passes on the error when the token could not be checked', () => {
const lookupError = Object.assign(new Error('down'), {
code: 'dbBatchFailed',
});
const got = runGate(requireAuthGate(), { tokenAuthError: lookupError });
expect(got).toBe(lookupError);
});
it('still answers token_auth_failed when a check found the token invalid', () => {
const got = runGate(requireAuthGate(), {
tokenAuthFailed: true,
tokenAuthError: new Error('down'),
});
expectHttpError(got, 401, 'token_auth_failed');
});
});
// ── requireUserActorGate ────────────────────────────────────────────
@@ -330,6 +346,14 @@ describe('requireUserActorGate', () => {
expectHttpError(got, 401, 'token_missing');
});
it('passes on the error when the token could not be checked', () => {
const lookupError = new Error('down');
const got = runGate(requireUserActorGate(), {
tokenAuthError: lookupError,
});
expect(got).toBe(lookupError);
});
it('rejects a FULL-ACCESS access token too — account routes stay closed', () => {
// The account wall is actor-type based: even a full-access PAT (which
// the resource wall lets through) is rejected here, so it can never
+4 -1
View File
@@ -50,7 +50,7 @@ import { assertVerifiedEmail } from '../verifiedEmail';
// Make sure the `Express.Request.actor` augmentation is in scope.
import '../expressAugmentation';
const rejectAuth = (req: Request): HttpError => {
const rejectAuth = (req: Request): unknown => {
if (req.requiresReauth) {
return new HttpError(401, 'Re-authentication required', {
legacyCode: 'reauth_required',
@@ -71,6 +71,9 @@ const rejectAuth = (req: Request): HttpError => {
legacyCode: 'token_auth_failed',
});
}
// The token couldn't be checked, so a 401 would wrongly tell the client
// to discard it; the error handler answers (503 for a database outage).
if (req.tokenAuthError !== undefined) return req.tokenAuthError;
return new HttpError(401, 'Missing authentication token', {
legacyCode: 'token_missing',
});
@@ -187,9 +187,9 @@ describe('createUserSubdomainNotFound', () => {
});
it('passes through reserved subdomains (api, js, native apps, etc.)', () => {
// `api`, `js`, `dav`, `docs`, `developer`, `editor`, `pdf`,
// `api`, `js`, `dav`, `docs`, `developer`, `about`, `apps`,
// `puter-app-icons`, etc. all pass through.
for (const sub of ['api', 'js', 'docs', 'editor', 'puter-app-icons']) {
for (const sub of ['api', 'js', 'docs', 'about', 'puter-app-icons']) {
const { out, next } = run(
createUserSubdomainNotFound(config),
makeReq({
@@ -865,13 +865,13 @@ describe('createNativeAppStatic', () => {
beforeAll(() => {
root = mkdtempSync(nodePath.join(tmpdir(), 'native-apps-'));
mkdirSync(nodePath.join(root, 'editor', 'assets'), { recursive: true });
mkdirSync(nodePath.join(root, 'about', 'assets'), { recursive: true });
writeFileSync(
nodePath.join(root, 'editor', 'index.html'),
'<h1>editor</h1>',
nodePath.join(root, 'about', 'index.html'),
'<h1>about</h1>',
);
writeFileSync(
nodePath.join(root, 'editor', 'assets', 'app.js'),
nodePath.join(root, 'about', 'assets', 'app.js'),
'console.log(1);',
);
mkdirSync(nodePath.join(root, 'docs', 'dist'), { recursive: true });
@@ -893,7 +893,7 @@ describe('createNativeAppStatic', () => {
it('leaves a native app name alone on a host outside our domains', async () => {
const req = staticReq({
subdomains: ['com', 'example', 'editor'],
subdomains: ['com', 'example', 'about'],
path: '/index.html',
});
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
@@ -907,7 +907,7 @@ describe('createNativeAppStatic', () => {
const { out, next } = await runStatic(
createNativeAppStatic(config),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/index.html',
}),
);
@@ -915,17 +915,24 @@ describe('createNativeAppStatic', () => {
expect(next).toHaveBeenCalledTimes(1);
});
it('passes through subdomains that are not native apps', async () => {
const { out, next } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'api'],
path: '/index.html',
}),
);
expect(out.sent).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
});
// `editor`, `markus` and `pdf` are web apps on standard host routing,
// even with a folder of their own under the native apps root.
it.each(['api', 'editor', 'markus', 'pdf'])(
'passes %s through, as it is not a native app',
async (sub) => {
mkdirSync(nodePath.join(root, sub), { recursive: true });
writeFileSync(nodePath.join(root, sub, 'index.html'), sub);
const { out, next } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', sub],
path: '/index.html',
}),
);
expect(out.sent).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
},
);
it('passes through when there is no subdomain at all', async () => {
const { next } = await runStatic(
@@ -939,13 +946,13 @@ describe('createNativeAppStatic', () => {
const { out, next } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/index.html',
}),
);
expect(out.sent).toEqual({
path: '/index.html',
root: nodePath.join(root, 'editor'),
root: nodePath.join(root, 'about'),
});
expect(next).not.toHaveBeenCalled();
});
@@ -954,11 +961,11 @@ describe('createNativeAppStatic', () => {
const { out } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'EDITOR'],
subdomains: ['localhost', 'puter', 'ABOUT'],
path: '/index.html',
}),
);
expect(out.sent?.root).toBe(nodePath.join(root, 'editor'));
expect(out.sent?.root).toBe(nodePath.join(root, 'about'));
});
it('serves docs out of its dist/ subdirectory, not the app root', async () => {
@@ -979,7 +986,7 @@ describe('createNativeAppStatic', () => {
const { out, next } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/assets',
originalUrl: '/assets?v=2',
}),
@@ -995,7 +1002,7 @@ describe('createNativeAppStatic', () => {
const { out } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/assets/',
}),
);
@@ -1007,7 +1014,7 @@ describe('createNativeAppStatic', () => {
const { out, next } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/nope.html',
}),
);
@@ -1019,7 +1026,7 @@ describe('createNativeAppStatic', () => {
const { next } = await runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/index.html',
}),
new Error('send failed'),
@@ -1032,7 +1039,7 @@ describe('createNativeAppStatic', () => {
runStatic(
withRoot(),
staticReq({
subdomains: ['localhost', 'puter', 'editor'],
subdomains: ['localhost', 'puter', 'about'],
path: '/../../etc/passwd',
}),
),
@@ -0,0 +1,263 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { brotliCompressSync, deflateSync, gzipSync } from 'node:zlib';
import express, {
type ErrorRequestHandler,
type RequestHandler,
} from 'express';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { SLICE_BYTES } from '../../../util/slicedJson';
import { createJsonBodyParser } from './jsonBody';
const LIMIT = 4 * 1024 * 1024;
const TEXT_AS_JSON = 'text/plain;actually=json';
type Verify = NonNullable<Parameters<typeof express.json>[0]>['verify'];
const captureRawBody: Verify = (req, _res, buf) => {
(req as { rawBody?: Buffer }).rawBody = buf;
};
/** An app that answers with what its parsers left on the request. */
const listen = (parsers: RequestHandler[]): Promise<Server> => {
const app = express();
app.use(...parsers);
app.post('/', (req, res) => {
const rawBody = (req as { rawBody?: Buffer }).rawBody;
res.json({
body: req.body,
hasBody: req.body !== undefined,
rawBody: rawBody?.toString('base64'),
});
});
const onError: ErrorRequestHandler = (err, _req, res, _next) => {
res.status(err.statusCode ?? 500).json({ type: err.type });
};
app.use(onError);
return new Promise((resolve) => {
const server = app.listen(0, '127.0.0.1', () => resolve(server));
});
};
describe('createJsonBodyParser', () => {
let sliced: Server;
let framework: Server;
beforeAll(async () => {
sliced = await listen([
createJsonBodyParser({ limit: LIMIT, verify: captureRawBody }),
]);
// What the server ran before: one parser per content type.
framework = await listen([
express.json({ limit: LIMIT, verify: captureRawBody }),
express.json({
limit: LIMIT,
type: (req) => req.headers['content-type'] === TEXT_AS_JSON,
verify: captureRawBody,
}),
]);
});
afterAll(() => {
sliced?.close();
framework?.close();
});
const post = async (
server: Server,
body: string | Buffer | undefined,
headers: Record<string, string>,
) => {
const { port } = server.address() as AddressInfo;
const response = await fetch(`http://127.0.0.1:${port}/`, {
method: 'POST',
headers,
body,
});
return { status: response.status, answer: await response.json() };
};
const large = `${JSON.stringify({
text: 'é😀 "quoted" \\ line\n'.repeat(SLICE_BYTES / 8),
list: Array.from({ length: 20_000 }, (_, i) => ({
i,
on: i % 2 === 0,
})),
}).slice(0, -1)},"__proto__":{"nested":true}}`;
it.each<[string, string | Buffer | undefined, Record<string, string>]>([
['an object', '{"a":1}', { 'Content-Type': 'application/json' }],
['an array', '[1,2]', { 'Content-Type': 'application/json' }],
[
'a body sent as text',
'{"interface":"x","auth_token":"t"}',
{ 'Content-Type': TEXT_AS_JSON },
],
['a large body', large, { 'Content-Type': 'application/json' }],
['a large body sent as text', large, { 'Content-Type': TEXT_AS_JSON }],
[
'a charset parameter',
large,
{ 'Content-Type': 'application/json; charset=UTF-8' },
],
[
'a quoted charset parameter',
'{"a":"é"}',
{ 'Content-Type': 'application/json;charset="utf-8"' },
],
[
'another parameter',
'{"a":1}',
{ 'Content-Type': 'application/json; profile=x' },
],
['an empty body', '', { 'Content-Type': 'application/json' }],
['no body', undefined, { 'Content-Type': 'application/json' }],
[
'a byte order mark',
`\ufeff${large}`,
{ 'Content-Type': 'application/json' },
],
[
'only a byte order mark',
'\ufeff',
{ 'Content-Type': 'application/json' },
],
['only whitespace', ' \n', { 'Content-Type': 'application/json' }],
['a bare string', '"text"', { 'Content-Type': 'application/json' }],
['a bare number', ' 12', { 'Content-Type': 'application/json' }],
['null', 'null', { 'Content-Type': 'application/json' }],
['malformed JSON', '{"a":', { 'Content-Type': 'application/json' }],
[
'malformed large JSON',
`${large.slice(0, -1)},]`,
{ 'Content-Type': 'application/json' },
],
[
'a large bare string',
JSON.stringify('x'.repeat(4 * SLICE_BYTES)),
{ 'Content-Type': 'application/json' },
],
[
'invalid UTF-8',
Buffer.concat([
Buffer.from('{"a":"'),
Buffer.from([0xff, 0xe2, 0x82]),
Buffer.from('"}'),
]),
{ 'Content-Type': 'application/json' },
],
[
'a body over the limit',
`["${'x'.repeat(LIMIT)}"]`,
{ 'Content-Type': 'application/json' },
],
[
'a gzipped body',
gzipSync(large),
{ 'Content-Type': 'application/json', 'Content-Encoding': 'gzip' },
],
[
'a deflated body',
deflateSync(large),
{
'Content-Type': 'application/json',
'Content-Encoding': 'deflate',
},
],
[
'a brotli body',
brotliCompressSync(large),
{ 'Content-Type': 'application/json', 'Content-Encoding': 'br' },
],
[
'a body that inflates past the limit',
gzipSync(`["${'x'.repeat(LIMIT)}"]`),
{ 'Content-Type': 'application/json', 'Content-Encoding': 'gzip' },
],
[
'an unknown content encoding',
'{"a":1}',
{ 'Content-Type': 'application/json', 'Content-Encoding': 'lzw' },
],
['plain text', '{"a":1}', { 'Content-Type': 'text/plain' }],
[
'text with the marker and a charset',
'{"a":1}',
{ 'Content-Type': `${TEXT_AS_JSON}; charset=utf-8` },
],
[
'a JSON-suffixed type',
'{"a":1}',
{ 'Content-Type': 'application/vnd.api+json' },
],
[
'form data',
'a=1',
{ 'Content-Type': 'application/x-www-form-urlencoded' },
],
['no content type', '{"a":1}', { 'Content-Type': '' }],
])('answers %s the way express.json does', async (_name, body, headers) => {
const got = await post(sliced, body, headers);
const want = await post(framework, body, headers);
expect(got).toEqual(want);
});
it('leaves a body in another charset unread', async () => {
for (const charset of ['utf-16', 'utf8', 'latin1']) {
const { status, answer } = await post(
sliced,
Buffer.from('{"a":1}', 'utf16le'),
{ 'Content-Type': `application/json; charset=${charset}` },
);
expect(status).toBe(200);
expect(answer.hasBody).toBe(false);
}
// The last charset named is the one that counts.
const { answer } = await post(sliced, '{"a":1}', {
'Content-Type': 'application/json; charset=utf-8; charset=utf-16',
});
expect(answer.hasBody).toBe(false);
});
it('parses bodies that arrive together, each to its own result', async () => {
const bodies = Array.from({ length: 6 }, (_, n) =>
JSON.stringify({ n, pad: 'x'.repeat((n + 1) * SLICE_BYTES) }),
);
const answers = await Promise.all(
bodies.map((body) =>
post(sliced, body, { 'Content-Type': 'application/json' }),
),
);
answers.forEach(({ status, answer }, n) => {
expect(status).toBe(200);
expect(answer.body).toEqual(JSON.parse(bodies[n]!));
});
// A failed parse hands its turn on like a finished one.
const mixed = await Promise.all([
post(sliced, `${bodies[5]!},`, {
'Content-Type': 'application/json',
}),
post(sliced, bodies[4], { 'Content-Type': 'application/json' }),
post(sliced, bodies[3], { 'Content-Type': 'application/json' }),
]);
expect(mixed.map(({ status }) => status)).toEqual([400, 200, 200]);
});
});
@@ -0,0 +1,127 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import express, { type Request, type RequestHandler } from 'express';
import { parseJsonInSlices, SLICE_BYTES } from '../../../util/slicedJson';
/** What puter.js labels a JSON driver call, to keep it a simple CORS request. */
const TEXT_AS_JSON = 'text/plain;actually=json';
const UTF8_CHARSET = /;\s*charset=("?)utf-8\1\s*(?:;|$)/i;
/** No charset, or exactly UTF-8. Anything else is left to `express.json`. */
const declaresUtf8 = (contentType: string): boolean => {
const mentions = contentType.match(/charset/gi)?.length ?? 0;
return mentions === 0 || (mentions === 1 && UTF8_CHARSET.test(contentType));
};
const isUtf8Json = (req: Request): boolean => {
const contentType = req.headers['content-type'];
if (contentType === TEXT_AS_JSON) return true;
return (
typeof contentType === 'string' &&
Boolean(req.is('application/json')) &&
declaresUtf8(contentType)
);
};
/** The error `express.json` raises, which the error handler answers with a 400. */
const parseFailed = (message: string): Error =>
Object.assign(new SyntaxError(message), {
status: 400,
statusCode: 400,
expose: true,
type: 'entity.parse.failed',
});
const waiting: Array<() => void> = [];
let parsing = false;
/**
* Bodies too large for one `JSON.parse` call parse one at a time, so a single
* tree is under construction however many arrive together.
*/
const parseInTurn = async (body: Buffer): Promise<unknown> => {
if (body.length <= SLICE_BYTES) return parseJsonInSlices(body);
if (parsing) await new Promise<void>((resolve) => waiting.push(resolve));
parsing = true;
try {
return await parseJsonInSlices(body);
} finally {
// Handing over keeps `parsing` set, so a new arrival can't cut in.
const next = waiting.shift();
if (next) next();
else parsing = false;
}
};
interface JsonBodyOptions {
limit: number;
verify?: NonNullable<Parameters<typeof express.raw>[0]>['verify'];
}
/**
* `express.json` for UTF-8 bodies, with the parse done in slices so a large
* body can't hold the event loop. Reading, limits, decompression and `verify`
* are the framework's; the result and the errors match `express.json`.
*/
export const createJsonBodyParser = ({
limit,
verify,
}: JsonBodyOptions): RequestHandler => {
const read = express.raw({
limit,
verify,
type: (req) => isUtf8Json(req as Request),
});
return (req, res, next) => {
read(req, res, async (err?: unknown) => {
// Not a Buffer: the reader skipped this request.
if (err || !Buffer.isBuffer(req.body)) {
next(err);
return;
}
let body: Buffer = req.body;
req.body = undefined;
// The framework's decoder drops a byte order mark.
if (body[0] === 0xef && body[1] === 0xbb && body[2] === 0xbf) {
body = body.subarray(3);
}
if (body.length === 0) {
req.body = {};
next();
return;
}
let parsed: unknown;
try {
parsed = await parseInTurn(body);
} catch (e) {
next(parseFailed((e as Error).message));
return;
}
if (typeof parsed !== 'object' || parsed === null) {
next(parseFailed('JSON body must be an object or an array'));
return;
}
req.body = parsed;
next();
});
};
};
@@ -0,0 +1,184 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { request, type Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { setImmediate as nextTurn } from 'node:timers/promises';
import express, {
type ErrorRequestHandler,
type RequestHandler,
} from 'express';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createJsonBodyParser } from './jsonBody';
import {
createLargeJsonBodyParser,
DEFAULT_JSON_LIMIT_BYTES,
} from './largeJsonBody';
const MiB = 1024 * 1024;
const TOKEN = 'valid-token';
const PAD = 'x'.repeat(DEFAULT_JSON_LIMIT_BYTES);
/** Stands in for the auth probe: one token is valid, wherever it is sent. */
const authProbe: RequestHandler = (req, _res, next) => {
const token =
(req.body as { auth_token?: string } | undefined)?.auth_token ??
req.headers.authorization;
if (!req.actor && token === TOKEN) {
req.actor = { user: { uuid: 'u1' } } as typeof req.actor;
}
next();
};
describe('a JSON body over the default limit', () => {
let server: Server;
beforeAll(async () => {
const app = express();
app.post(
'/call',
createLargeJsonBodyParser({
authProbe,
limitBytes: 128 * MiB,
budgetBytes: 256 * MiB,
perUserBytes: 128 * MiB,
}),
);
app.use(createJsonBodyParser({ limit: DEFAULT_JSON_LIMIT_BYTES }));
app.use(authProbe);
app.post('/call', (req, res) => {
const body = req.body as { auth_token?: string; pad: string };
res.json({
user: req.actor?.user.uuid ?? null,
token: body.auth_token ?? null,
padLength: body.pad.length,
});
});
const onError: ErrorRequestHandler = (err, _req, res, _next) => {
res.status(err.statusCode ?? 500).json({});
};
app.use(onError);
await new Promise<void>((resolve) => {
server = app.listen(0, '127.0.0.1', resolve);
});
});
afterAll(() => {
server?.close();
});
/** Sends `chunks` one write at a time, so the server sees them apart. */
const post = (chunks: string[], headers: Record<string, string> = {}) =>
new Promise<{ status: number; answer: Record<string, unknown> }>(
(resolve, reject) => {
const { port } = server.address() as AddressInfo;
const req = request(
{
host: '127.0.0.1',
port,
path: '/call',
method: 'POST',
headers: {
'Content-Type': 'text/plain;actually=json',
'Content-Length': chunks.reduce(
(total, chunk) =>
total + Buffer.byteLength(chunk),
0,
),
...headers,
},
},
(res) => {
const parts: Buffer[] = [];
res.on('data', (part: Buffer) => parts.push(part));
res.on('end', () =>
resolve({
status: res.statusCode!,
answer: JSON.parse(
Buffer.concat(parts).toString(),
),
}),
);
},
);
req.on('error', reject);
void (async () => {
for (const chunk of chunks) {
req.write(chunk);
await nextTurn();
}
req.end();
})();
},
);
it('is accepted when it opens with a valid auth_token', async () => {
const { status, answer } = await post([
`{"auth_token":"${TOKEN}","pad":"${PAD}"}`,
]);
expect(status).toBe(200);
expect(answer).toEqual({
user: 'u1',
token: TOKEN,
padLength: PAD.length,
});
}, 60_000);
it('is accepted when the token arrives in pieces, with spacing', async () => {
const opening = ` {\n "auth_token" : "${TOKEN}" ,`;
const { status, answer } = await post([...opening, `"pad":"${PAD}"}`]);
expect(status).toBe(200);
expect(answer.user).toBe('u1');
}, 60_000);
it('is accepted with a header credential, as before', async () => {
const { status, answer } = await post([`{"pad":"${PAD}"}`], {
Authorization: TOKEN,
});
expect(status).toBe(200);
expect(answer).toEqual({
user: 'u1',
token: null,
padLength: PAD.length,
});
}, 60_000);
it.each([
[
'its token comes after other fields',
`{"pad":"${PAD}","auth_token":"${TOKEN}"}`,
],
['its token is not valid', `{"auth_token":"expired","pad":"${PAD}"}`],
['it has no token', `{"pad":"${PAD}"}`],
[
'a later field replaces the token it opens with',
`{"auth_token":"${TOKEN}","pad":"${PAD}","auth_token":"other"}`,
],
[
'the token it opens with is nested',
`{"args":{"auth_token":"${TOKEN}"},"pad":"${PAD}"}`,
],
])(
'is refused with 413 when %s',
async (_name, body) => {
expect((await post([body])).status).toBe(413);
},
60_000,
);
});
@@ -18,6 +18,7 @@
*/
import { EventEmitter } from 'node:events';
import { Readable } from 'node:stream';
import type { NextFunction, Request, RequestHandler, Response } from 'express';
import { describe, expect, it, vi } from 'vitest';
import { isHttpError } from '../HttpError';
@@ -35,7 +36,7 @@ const makeReq = (
user: string | null = 'u1',
body?: unknown,
) =>
({
Object.assign(Readable.from([]), {
complete: true,
body,
headers: {
+115 -65
View File
@@ -17,13 +17,54 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import express, { type Request, type RequestHandler } from 'express';
import type { Request, RequestHandler, Response } from 'express';
import { actorUid } from '../../actor';
import { HttpError } from '../HttpError';
import { createJsonBodyParser } from './jsonBody';
/** The global JSON parser's limit, which applies before auth. */
export const DEFAULT_JSON_LIMIT_BYTES = 50 * 1024 * 1024;
/** How much of a body is read to find a token it opens with. */
const LEADING_TOKEN_BYTES = 8 * 1024;
const LEADING_TOKEN =
/^[ \t\n\r]*\{[ \t\n\r]*"auth_token"[ \t\n\r]*:[ \t\n\r]*"([^"\\]+)"/;
/**
* The first `bytes` of the body, put back for the parser to read. Empty when
* the body ends or fails before that many arrive.
*/
const peekBody = (req: Request, bytes: number): Promise<Buffer> =>
new Promise((resolve) => {
const settle = (head: Buffer | null) => {
req.off('readable', onReadable);
req.off('end', onGone);
req.off('close', onGone);
req.off('error', onGone);
if (head) req.unshift(head);
resolve(head ?? Buffer.alloc(0));
};
const onReadable = () => {
// Null until `bytes` are buffered.
const head = req.read(bytes) as Buffer | null;
if (head) settle(head);
};
const onGone = () => settle(null);
req.on('readable', onReadable);
req.on('end', onGone);
req.on('close', onGone);
req.on('error', onGone);
});
/**
* The `auth_token` a JSON body opens with, which puter.js sends first so it can
* be read ahead of the body.
*/
const leadingBodyToken = async (req: Request): Promise<string | undefined> => {
const head = await peekBody(req, LEADING_TOKEN_BYTES);
return LEADING_TOKEN.exec(head.toString('latin1'))?.[1];
};
interface LargeJsonBodyOptions {
/** The global auth probe; it skips requests that already carry an actor. */
authProbe: RequestHandler;
@@ -38,9 +79,9 @@ interface LargeJsonBodyOptions {
/**
* Parses JSON bodies over the default limit, up to `limitBytes`, once the
* caller's header or cookie credential resolves. Unauthenticated requests and
* requests without a Content-Length fall through to the global parser and its
* limit.
* caller's credential resolves: a header, a cookie, or an `auth_token` the body
* opens with. Unauthenticated requests and requests without a Content-Length
* fall through to the global parser and its limit.
*
* The parsed body lives as long as the request, so a body's bytes count against
* both budgets until its response closes. Over the user's budget gets a 429,
@@ -53,82 +94,91 @@ export const createLargeJsonBodyParser = ({
perUserBytes,
accepts,
}: LargeJsonBodyOptions): RequestHandler => {
const json = express.json({ limit: limitBytes });
const textAsJson = express.json({
limit: limitBytes,
type: (req) =>
req.headers['content-type'] === 'text/plain;actually=json',
});
const json = createJsonBodyParser({ limit: limitBytes });
let inFlightBytes = 0;
const inFlightByUser = new Map<string, number>();
return (req, res, next) => {
const probe = (req: Request, res: Response) =>
new Promise<void>((resolve) => {
void authProbe(req, res, () => resolve());
});
return async (req, res, next) => {
const declared = Number(req.headers['content-length']);
if (!(declared > DEFAULT_JSON_LIMIT_BYTES && declared <= limitBytes)) {
next();
return;
}
void authProbe(req, res, () => {
// `res.closed`: the client left during the probe, so `close` has
// already fired and would never release a charge.
if (!req.actor || res.closed) {
next();
await probe(req, res);
let bodyToken: string | undefined;
if (!req.actor && !res.closed) {
bodyToken = await leadingBodyToken(req);
if (bodyToken) {
// Where the probe looks for a body token.
req.body = { auth_token: bodyToken };
await probe(req, res);
req.body = undefined;
}
}
// `res.closed`: the client left during the probe, so `close` has
// already fired and would never release a charge.
if (!req.actor || res.closed) {
next();
return;
}
const user = req.actor.user.uuid ?? actorUid(req.actor);
const userBytes = inFlightByUser.get(user) ?? 0;
if (userBytes + declared > perUserBytes) {
res.setHeader('Retry-After', '5');
next(
new HttpError(
429,
'Too many large requests in progress for this account',
{ legacyCode: 'too_many_requests' },
),
);
return;
}
if (inFlightBytes + declared > budgetBytes) {
res.setHeader('Retry-After', '5');
next(
new HttpError(503, 'Too many large requests in progress', {
code: 'large_body_capacity',
// Load shedding, not a fault: record it, don't page.
alarmSeverity: 'info',
}),
);
return;
}
inFlightBytes += declared;
inFlightByUser.set(user, userBytes + declared);
res.once('close', () => {
inFlightBytes -= declared;
const left = (inFlightByUser.get(user) ?? 0) - declared;
if (left > 0) inFlightByUser.set(user, left);
else inFlightByUser.delete(user);
});
json(req, res, (err?: unknown) => {
if (err) {
next(err);
return;
}
const user = req.actor.user.uuid ?? actorUid(req.actor);
const userBytes = inFlightByUser.get(user) ?? 0;
if (userBytes + declared > perUserBytes) {
res.setHeader('Retry-After', '5');
// The caller was resolved from the token the body opens with,
// so the parsed body has to name the same one.
const tokenMoved =
bodyToken !== undefined &&
(req.body as { auth_token?: unknown } | undefined)
?.auth_token !== bodyToken;
if (tokenMoved || (accepts && !accepts(req))) {
// The same answer the default limit gives.
next(
new HttpError(
429,
'Too many large requests in progress for this account',
{ legacyCode: 'too_many_requests' },
),
);
return;
}
if (inFlightBytes + declared > budgetBytes) {
res.setHeader('Retry-After', '5');
next(
new HttpError(503, 'Too many large requests in progress', {
code: 'large_body_capacity',
// Load shedding, not a fault: record it, don't page.
alarmSeverity: 'info',
new HttpError(413, 'request entity too large', {
legacyCode: 'client_bad_request',
}),
);
return;
}
inFlightBytes += declared;
inFlightByUser.set(user, userBytes + declared);
res.once('close', () => {
inFlightBytes -= declared;
const left = (inFlightByUser.get(user) ?? 0) - declared;
if (left > 0) inFlightByUser.set(user, left);
else inFlightByUser.delete(user);
});
json(req, res, (err?: unknown) => {
if (err) {
next(err);
return;
}
textAsJson(req, res, (err?: unknown) => {
if (err) {
next(err);
return;
}
if (accepts && !accepts(req)) {
// The same answer the default limit gives.
next(
new HttpError(413, 'request entity too large', {
legacyCode: 'client_bad_request',
}),
);
return;
}
next();
});
});
next();
});
};
};
@@ -18,7 +18,7 @@
*/
import type { Request } from 'express';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { makeActor } from '../../actor';
import {
@@ -1132,14 +1132,150 @@ describe('resolvePublicHostedIdentity', () => {
const reqOf = (init: {
cookies?: Record<string, string>;
actor?: unknown;
headers?: Record<string, string>;
query?: Record<string, string>;
}): Request =>
({
cookies: init.cookies ?? {},
actor: init.actor,
headers: {},
query: {},
headers: init.headers ?? {},
query: init.query ?? {},
}) as unknown as Request;
describe('with the launch token in the Referer', () => {
const host = 'launch.site.puter.localhost';
const cookieName = HOSTED_ASSET_COOKIES.public.name;
const launch = async () => {
const user = await makeUser();
const sub = `launch-${uuidv4().slice(0, 8)}`;
const app = await server.stores.app.create(
{ name: sub, title: sub, index_url: `http://${host}/` },
{ ownerUserId: user.id },
);
const token = await authService.getUserAppToken(
{ user } as unknown as Parameters<
typeof authService.getUserAppToken
>[0],
app.uid,
);
const actor = (await authService.authenticateFromToken(token))!;
const cookie = await authService.createHostedAssetToken('public', {
appUid: app.uid,
userUid: user.uuid,
sessionUuid: actor.session?.uid,
subdomain: 'launch',
host,
});
return { user, appUid: app.uid, token, cookie, actor };
};
const resolve = (
appUid: string,
init: Parameters<typeof reqOf>[0],
expectedHost = host,
) =>
resolvePublicHostedIdentity({
req: reqOf(init),
authService,
sessionCookieName: 'puter_auth_token',
expectedAppUid: appUid,
expectedSubdomain: 'launch',
expectedHost,
});
const refererFor = (token: string) =>
`http://${host}/?puter.auth.token=${encodeURIComponent(token)}`;
it('takes the cookie when the token names its user and session', async () => {
const { user, appUid, token, cookie } = await launch();
const authenticate = vi.spyOn(authService, 'authenticate');
try {
const out = await resolve(appUid, {
cookies: { [cookieName]: cookie },
headers: { referer: refererFor(token) },
});
expect(out).toMatchObject({
source: 'public-cookie',
userUid: user.uuid,
hasValidPublicCookie: true,
});
expect(authenticate).not.toHaveBeenCalled();
} finally {
authenticate.mockRestore();
}
});
it('authenticates a query token even when it names the cookie identity', async () => {
const { user, appUid, token, cookie } = await launch();
const out = await resolve(appUid, {
cookies: { [cookieName]: cookie },
query: { 'puter.auth.token': token },
});
expect(out.source).toBe('query');
expect(out.userUid).toBe(user.uuid);
expect(
(out as { hasValidPublicCookie?: boolean })
.hasValidPublicCookie,
).toBeUndefined();
});
it('authenticates the token when the cookie was minted for another host', async () => {
const { user, appUid, token, cookie } = await launch();
const out = await resolve(
appUid,
{
cookies: { [cookieName]: cookie },
headers: { referer: refererFor(token) },
},
'other.site.puter.localhost',
);
expect(out.source).toBe('referrer');
expect(out.userUid).toBe(user.uuid);
});
it('authenticates a token that names another app under the same session', async () => {
const { user, appUid, cookie, actor } = await launch();
const otherAppToken = server.services.token.sign('auth', {
type: 'app-under-user',
version: '2',
user_uid: user.uuid,
app_uid: `app-${uuidv4()}`,
session_uid: actor.session!.uid,
});
const authenticate = vi.spyOn(authService, 'authenticate');
try {
const out = await resolve(appUid, {
cookies: { [cookieName]: cookie },
headers: { referer: refererFor(otherAppToken) },
});
expect(authenticate).toHaveBeenCalledTimes(1);
expect(out.source).toBe('public-cookie');
} finally {
authenticate.mockRestore();
}
});
it('never resolves a scoped access token, with or without the cookie', async () => {
const { user, appUid, cookie } = await launch();
const scoped = await authService.createAccessToken(
makeActor({
user: { id: user.id, uuid: user.uuid, username: user.username },
}),
[[`user:${user.uuid}:email:read`]],
);
const withCookie = await resolve(appUid, {
cookies: { [cookieName]: cookie },
headers: { referer: refererFor(scoped) },
});
expect(withCookie.source).toBe('public-cookie');
const withoutCookie = await resolve(appUid, {
headers: { referer: refererFor(scoped) },
});
expect(withoutCookie.source).toBe('none');
});
});
it('returns the cookie identity when present and valid', async () => {
const user = await makeUser();
const appUid = `app-${uuidv4()}`;
@@ -86,6 +86,10 @@ export interface PrivateIdentity {
hasValidPrivateCookie?: boolean;
}
type PublicCookieClaims = Awaited<
ReturnType<AuthService['verifyHostedAssetToken']>
>;
interface SubdomainLike {
user_id?: number | null;
}
@@ -533,8 +537,63 @@ export async function resolvePublicHostedIdentity(opts: {
}
}
const { name: publicCookieNameV2, legacyName: publicCookieNameLegacy } =
HOSTED_ASSET_COOKIES.public;
const publicCookieToken =
(typeof cookies?.[publicCookieNameV2] === 'string'
? cookies[publicCookieNameV2]
: null) ??
(typeof cookies?.[publicCookieNameLegacy] === 'string'
? cookies[publicCookieNameLegacy]
: null);
let publicCookieClaims: Promise<PublicCookieClaims | null> | undefined;
const verifyPublicCookie = () =>
(publicCookieClaims ??= (async () => {
if (!publicCookieToken) return null;
try {
return await authService.verifyHostedAssetToken(
'public',
publicCookieToken,
{
appUid: expectedAppUid,
subdomain: expectedSubdomain,
host: expectedHost,
},
);
} catch {
// v1-signed cookies land here and get re-minted.
return null;
}
})());
const publicCookieIdentity = (claims: PublicCookieClaims) => ({
source: 'public-cookie' as const,
userUid: claims.userUid,
sessionUuid: claims.sessionUuid,
hasValidPublicCookie: true,
});
const bootstrap = getBootstrapToken(req);
if (bootstrap) {
// A launched app's own assets carry its launch token in the Referer.
// When that token names the cookie's user and session, the cookie
// already stands for it and nothing needs re-authenticating or minting.
if (
bootstrap.source === 'referrer' &&
publicCookieToken &&
expectedAppUid
) {
const named = authService.signedSessionClaims(bootstrap.token);
if (named && (!named.appUid || named.appUid === expectedAppUid)) {
const claims = await verifyPublicCookie();
if (
claims &&
claims.userUid === named.userUid &&
claims.sessionUuid === named.sessionUuid
) {
return publicCookieIdentity(claims);
}
}
}
try {
const actor = await authService.authenticateFromToken(
bootstrap.token,
@@ -554,36 +613,8 @@ export async function resolvePublicHostedIdentity(opts: {
}
}
const { name: publicCookieNameV2, legacyName: publicCookieNameLegacy } =
HOSTED_ASSET_COOKIES.public;
const publicCookieToken =
(typeof cookies?.[publicCookieNameV2] === 'string'
? cookies[publicCookieNameV2]
: null) ??
(typeof cookies?.[publicCookieNameLegacy] === 'string'
? cookies[publicCookieNameLegacy]
: null);
if (publicCookieToken) {
try {
const claims = await authService.verifyHostedAssetToken(
'public',
publicCookieToken,
{
appUid: expectedAppUid,
subdomain: expectedSubdomain,
host: expectedHost,
},
);
return {
source: 'public-cookie',
userUid: claims.userUid,
sessionUuid: claims.sessionUuid,
hasValidPublicCookie: true,
};
} catch {
/* fall through — v1-signed cookies land here and get re-minted */
}
}
const claims = await verifyPublicCookie();
if (claims) return publicCookieIdentity(claims);
return { source: 'none' };
}
@@ -2763,6 +2763,321 @@ describe('public hosted asset billing identity', () => {
});
});
describe('public hosted assets requested with the launch token in the Referer', () => {
const cookieName = HOSTED_ASSET_COOKIES.public.name;
const makeLaunchedApp = async () => {
const owner = await makeUserWithHome();
const home = (await server.stores.fsEntry.getEntryByPath(
`/${owner.username}`,
))!;
const sub = `ref-${uuidv4().slice(0, 8)}`;
const host = `${sub}.site.puter.localhost`;
await server.stores.subdomain.create({
userId: owner.id,
subdomain: sub,
rootDirId: home.id,
});
await writeFile(
owner.id,
`/${owner.username}/index.html`,
Buffer.from('<html></html>'),
);
await writeFile(
owner.id,
`/${owner.username}/app.css`,
Buffer.from('body{}'),
);
const app = await server.stores.app.create(
{ name: sub, title: sub, index_url: `http://${host}/` },
{ ownerUserId: owner.id },
);
return { owner, sub, host, appUid: app.uid };
};
const cookieFor = async (
token: string,
site: { sub: string; host: string; appUid: string },
) => {
const actor = (await server.services.auth.authenticateFromToken(
token,
))!;
return server.services.auth.createHostedAssetToken('public', {
appUid: site.appUid,
userUid: actor.user.uuid,
sessionUuid: actor.session?.uid,
subdomain: site.sub,
host: site.host,
});
};
const refererFor = (host: string, token: string) =>
`http://${host}/?puter.app_instance_id=x&puter.auth.token=${encodeURIComponent(token)}`;
const serve = async (
host: string,
init: {
path?: string;
cookies?: Record<string, string>;
headers?: Record<string, string>;
query?: Record<string, string>;
} = {},
) => {
const req = makeReq({ hostname: host, path: '/app.css', ...init });
const { res, out } = makeRes();
await buildMiddleware()(req, res, vi.fn());
await new Promise<void>((resolve) => setImmediate(resolve));
expect(out.statusCode).toBe(200);
return { req, out };
};
const claimsOf = (cookie: string, appUid: string, host: string) =>
server.services.auth.verifyHostedAssetToken('public', cookie, {
appUid,
host,
});
const REDIS_METHODS = [
'get',
'mget',
'exists',
'hget',
'hgetall',
'smembers',
'set',
'del',
'expire',
'incr',
'incrby',
'hset',
'sadd',
'pipeline',
'multi',
];
/** Redis calls, JWT signs and full authentications made by `fn`. */
const measure = async <T>(fn: () => Promise<T>) => {
const redis = server.clients.redis as unknown as Record<
string,
unknown
>;
const redisSpies = REDIS_METHODS.filter(
(m) => typeof redis[m] === 'function',
).map((m) => [m, vi.spyOn(redis as never, m as never)] as const);
const sign = vi.spyOn(server.services.token, 'sign');
const authenticate = vi.spyOn(server.services.auth, 'authenticate');
try {
const result = await fn();
const redisCalls: Record<string, number> = {};
for (const [m, spy] of redisSpies) {
const n = (spy as unknown as { mock: { calls: unknown[] } })
.mock.calls.length;
if (n) redisCalls[m] = n;
}
return {
result,
redisCalls,
signs: sign.mock.calls.length,
authentications: authenticate.mock.calls.length,
};
} finally {
for (const [, spy] of redisSpies) spy.mockRestore();
sign.mockRestore();
authenticate.mockRestore();
}
};
it('costs the same as the cookie alone when the token names the cookie user and session', async () => {
const site = await makeLaunchedApp();
const visitor = await makeUser();
const token = await server.services.auth.getUserAppToken(
actorFor(visitor),
site.appUid,
);
const cookies = { [cookieName]: await cookieFor(token, site) };
const headers = { referer: refererFor(site.host, token) };
// Warm the caches both paths read.
await serve(site.host, { cookies });
await serve(site.host, { cookies, headers });
const cookieOnly = await measure(() => serve(site.host, { cookies }));
const withReferer = await measure(() =>
serve(site.host, { cookies, headers }),
);
expect(withReferer.result.req.egressActor?.user?.uuid).toBe(
visitor.uuid,
);
expect(withReferer.result.out.cookies).toEqual({});
expect(withReferer.signs).toBe(0);
expect(withReferer.authentications).toBe(0);
expect(withReferer.redisCalls).toEqual(cookieOnly.redisCalls);
expect(cookieOnly.signs).toBe(0);
});
it('bills the token account when the launch could not replace an older cookie', async () => {
const site = await makeLaunchedApp();
const previous = await makeUser();
const current = await makeUser();
const previousToken = await server.services.auth.getUserAppToken(
actorFor(previous),
site.appUid,
);
const currentToken = await server.services.auth.getUserAppToken(
actorFor(current),
site.appUid,
);
const oldCookies = {
[cookieName]: await cookieFor(previousToken, site),
};
const mint = vi
.spyOn(server.services.auth, 'createHostedAssetToken')
.mockRejectedValueOnce(new Error('mint failed'));
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
let launch;
try {
launch = await serve(site.host, {
path: '/index.html',
cookies: oldCookies,
query: { 'puter.auth.token': currentToken },
});
} finally {
mint.mockRestore();
warn.mockRestore();
}
expect(launch.out.cookies).toEqual({});
expect(launch.req.egressActor?.user?.uuid).toBe(current.uuid);
const asset = await serve(site.host, {
cookies: oldCookies,
headers: { referer: refererFor(site.host, currentToken) },
});
expect(asset.req.egressActor?.user?.uuid).toBe(current.uuid);
const replaced = asset.out.cookies[cookieName];
expect(replaced).toBeTruthy();
expect((await claimsOf(replaced, site.appUid, site.host)).userUid).toBe(
current.uuid,
);
});
it('bills the token account when a cached page of another account loads assets', async () => {
// The document never reaches the server (served from an app's own
// cache), so only the assets show which account launched it.
const site = await makeLaunchedApp();
const previous = await makeUser();
const current = await makeUser();
const previousToken = await server.services.auth.getUserAppToken(
actorFor(previous),
site.appUid,
);
const currentToken = await server.services.auth.getUserAppToken(
actorFor(current),
site.appUid,
);
const oldCookies = {
[cookieName]: await cookieFor(previousToken, site),
};
for (const path of ['/app.css', '/index.html']) {
const asset = await serve(site.host, {
path,
cookies: oldCookies,
headers: { referer: refererFor(site.host, currentToken) },
});
expect(asset.req.egressActor?.user?.uuid).toBe(current.uuid);
const replaced = asset.out.cookies[cookieName];
expect(
(await claimsOf(replaced, site.appUid, site.host)).userUid,
).toBe(current.uuid);
}
});
it('re-mints the cookie when the token names the same user under another session', async () => {
const site = await makeLaunchedApp();
const visitor = await makeUser();
const appToken = await server.services.auth.getUserAppToken(
actorFor(visitor),
site.appUid,
);
const cookies = { [cookieName]: await cookieFor(appToken, site) };
const { session, token: sessionToken } =
await server.services.auth.createSessionToken(visitor as never);
const asset = await serve(site.host, {
cookies,
headers: { referer: refererFor(site.host, sessionToken) },
});
expect(asset.req.egressActor?.user?.uuid).toBe(visitor.uuid);
const replaced = asset.out.cookies[cookieName];
expect(replaced).toBeTruthy();
expect(
(await claimsOf(replaced, site.appUid, site.host)).sessionUuid,
).toBe(session.uuid);
});
it('falls back to the cookie when the Referer token does not verify', async () => {
const site = await makeLaunchedApp();
const visitor = await makeUser();
const token = await server.services.auth.getUserAppToken(
actorFor(visitor),
site.appUid,
);
const cookies = { [cookieName]: await cookieFor(token, site) };
const tampered = `${token.slice(0, -4)}AAAA`;
const asset = await serve(site.host, {
cookies,
headers: { referer: refererFor(site.host, tampered) },
});
expect(asset.req.egressActor?.user?.uuid).toBe(visitor.uuid);
expect(asset.out.cookies).toEqual({});
});
it('bills the site owner once the shared session is revoked', async () => {
const site = await makeLaunchedApp();
const visitor = await makeUser();
const token = await server.services.auth.getUserAppToken(
actorFor(visitor),
site.appUid,
);
const cookies = { [cookieName]: await cookieFor(token, site) };
await server.services.auth.revokeAllSessionsForUserId(visitor.id);
const asset = await serve(site.host, {
cookies,
headers: { referer: refererFor(site.host, token) },
});
expect(asset.req.egressActor?.user?.uuid).toBe(site.owner.uuid);
expect(asset.out.cookies).toEqual({});
});
it('keeps responses that set the cookie out of shared caches', async () => {
const site = await makeLaunchedApp();
const visitor = await makeUser();
const token = await server.services.auth.getUserAppToken(
actorFor(visitor),
site.appUid,
);
const launch = await serve(site.host, {
path: '/index.html',
query: { 'puter.auth.token': token },
});
const cookie = launch.out.cookies[cookieName];
expect(cookie).toBeTruthy();
expect(launch.out.headers['Cache-Control']).toBe('private');
const asset = await serve(site.host, {
cookies: { [cookieName]: cookie },
headers: { referer: refererFor(site.host, token) },
});
expect(asset.out.cookies).toEqual({});
expect(asset.out.headers['Cache-Control']).toBeUndefined();
});
});
describe('site rooted in the owner tree, root moved to Trash', () => {
it('stops serving a trashed folder and resumes when restored', async () => {
const owner = await makeUserWithHome();
@@ -548,6 +548,8 @@ export const createPuterSiteMiddleware = (
requestHostname: host,
}),
);
// A shared cache must not hand this visitor's cookie to others.
res.set('Cache-Control', 'private');
}
} catch (e) {
// Best-effort — don't block the public file serve.
+12 -11
View File
@@ -20,6 +20,8 @@
import crypto from 'node:crypto';
import { withSpan } from '../../../util/span.js';
import { Context } from '../../context.js';
import { CallSlot } from '../callSlot.js';
import { HttpError } from '../HttpError.js';
import { subscriptionOverride } from '../../../services/metering/consts.js';
@@ -412,8 +414,8 @@ async function peekKv(kv, key, limit) {
//
// Lifecycle is the wedge between rate and concurrent limits: rate just
// records a tick, concurrent has to track "still in flight" → "done"
// across a request boundary. The route middleware hooks `res.finish` /
// `res.close`; the driver helper wraps the invocation in `try/finally`.
// across a request boundary. The route middleware and the driver controller
// both hold the slot in a `CallSlot`.
// Orphan TTL safety nets — used when the process dies between acquire
// and release. Memory backend doesn't need one (the process is also
@@ -1137,8 +1139,9 @@ async function resolveSubscriptionLimit(req, opts) {
* bySubscription: { user_free: 2, unlimited: 50 } } } { concurrent: { limit:
* 10, backend: 'redis', scope: 'expensive-op' } }
*
* On accept, schedules release on `res.finish` / `res.close` so even aborted
* requests give their slot back. On reject, 429 with the same
* On accept, holds the slot until the response is done and any work the handler
* registered through `keepConcurrencySlot` has settled, so work that outlives a
* caller who hung up still counts. On reject, 429 with the same
* `too_many_requests` legacyCode as the rate gate (clients already handle that
* branch). Fails open on backend error.
*/
@@ -1164,13 +1167,11 @@ export function concurrencyGate(opts) {
return;
}
// `finish` (response sent) and `close` (connection closed, possibly
// aborted before finish) can both fire; the guard releases once.
const { release } = guardSlot(slot, '[concurrent]');
const onDone = () => void release();
res.once('finish', onDone);
res.once('close', onDone);
if (res.destroyed || res.writableFinished) onDone();
const held = new CallSlot();
held.attach(guardSlot(slot, '[concurrent]'), res);
if (Context.current()) {
Context.set('keepConcurrencySlot', (work) => held.keep(work));
}
next();
};
}
@@ -29,6 +29,7 @@ import {
vi,
} from 'vitest';
import { EventEmitter } from 'node:events';
import { Context, runWithContext } from '../../context.js';
import { isHttpError } from '../HttpError.js';
import { setupTestServer } from '../../../testUtil.ts';
import {
@@ -977,6 +978,27 @@ describe('concurrencyGate — memory backend', () => {
expect(isHttpError(third.err)).toBe(true);
});
it('holds the slot past the response for work the handler keeps', async () => {
const opts = { limit: 1, key: 'ip', scope: 'cg-kept-work' };
let finishWork;
const work = new Promise((resolve) => {
finishWork = resolve;
});
const res = makeRes();
await runWithContext({}, async () => {
await concurrencyGate(opts)(baseReq(), res, () => {
Context.get('keepConcurrencySlot')(work);
});
});
res.emit('close');
await new Promise((r) => setImmediate(r));
expect((await runGate(opts, baseReq())).err?.statusCode).toBe(429);
finishWork();
await new Promise((r) => setImmediate(r));
expect((await runGate(opts, baseReq())).err).toBeUndefined();
});
it('different keys do not share slots', async () => {
const opts = { limit: 1, key: 'ip', scope: 'cg-keyed' };
const a = await runGate(opts, baseReq({ ip: '1.1.1.1' }));
+4 -4
View File
@@ -263,10 +263,10 @@ export interface RouteOptions {
rateLimit?: RouteRateLimit | RouteRateLimit[];
/**
* Cap simultaneous in-flight requests per key; slots are released on
* response finish or close. `bySubscription` overrides `limit` per
* `SubscriptionPolicy.id` when metering is wired in. `key`, `scope`,
* `backend` as in `rateLimit`.
* Cap simultaneous in-flight requests per key; a slot is released once the
* response is done and any work the handler kept it for has settled.
* `bySubscription` overrides `limit` per `SubscriptionPolicy.id` when
* metering is wired in. `key`, `scope`, `backend` as in `rateLimit`.
*/
concurrent?: {
limit: number;
@@ -1,132 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import {
messagesHaveCompaction,
toAnthropicContextManagement,
toOpenAiContextManagement,
wantsCompaction,
} from './compaction.js';
describe('toOpenAiContextManagement', () => {
it('returns undefined when compaction is off', () => {
expect(toOpenAiContextManagement({})).toBeUndefined();
expect(
toOpenAiContextManagement({ compaction: false }),
).toBeUndefined();
});
it('builds a compaction entry from `true`', () => {
expect(toOpenAiContextManagement({ compaction: true })).toEqual([
{ type: 'compaction' },
]);
});
it('maps trigger_tokens to compact_threshold', () => {
expect(
toOpenAiContextManagement({ compaction: { trigger_tokens: 1000 } }),
).toEqual([{ type: 'compaction', compact_threshold: 1000 }]);
});
it('passes a raw context_management payload through verbatim', () => {
const raw = [{ type: 'compaction', compact_threshold: 5 }];
expect(
toOpenAiContextManagement({
compaction: true,
context_management: raw,
}),
).toBe(raw);
});
});
describe('toAnthropicContextManagement', () => {
it('returns undefined when compaction is off', () => {
expect(toAnthropicContextManagement({})).toBeUndefined();
});
it('builds a compact_20260112 edit from `true`', () => {
expect(toAnthropicContextManagement({ compaction: true })).toEqual({
edits: [{ type: 'compact_20260112' }],
});
});
it('maps trigger_tokens to an input_tokens trigger', () => {
expect(
toAnthropicContextManagement({
compaction: { trigger_tokens: 2000 },
}),
).toEqual({
edits: [
{
type: 'compact_20260112',
trigger: { type: 'input_tokens', value: 2000 },
},
],
});
});
it('passes a raw context_management payload through verbatim', () => {
const raw = { edits: [{ type: 'compact_20260112' }] };
expect(
toAnthropicContextManagement({ context_management: raw }),
).toBe(raw);
});
});
describe('wantsCompaction', () => {
it('is false without opt-in', () => {
expect(wantsCompaction({})).toBe(false);
expect(wantsCompaction({ compaction: false })).toBe(false);
});
it('is true for the neutral opt-in or a raw passthrough', () => {
expect(wantsCompaction({ compaction: true })).toBe(true);
expect(wantsCompaction({ compaction: { trigger_tokens: 1 } })).toBe(
true,
);
expect(wantsCompaction({ context_management: [] })).toBe(true);
});
});
describe('messagesHaveCompaction', () => {
it('detects a round-tripped compaction content block', () => {
expect(
messagesHaveCompaction([
{ role: 'user', content: [{ type: 'text', text: 'hi' }] },
{
role: 'assistant',
content: [
{ type: 'compaction', encrypted_content: 'ENC' },
],
},
]),
).toBe(true);
});
it('is false for ordinary messages or non-arrays', () => {
expect(
messagesHaveCompaction([
{ role: 'user', content: [{ type: 'text', text: 'hi' }] },
]),
).toBe(false);
expect(messagesHaveCompaction(undefined)).toBe(false);
expect(messagesHaveCompaction('nope')).toBe(false);
});
});
Loaded 100 of 908 files, more files were not shown because too many files have changed in this diff. Show more