Compare commits

...
Author SHA1 Message Date
404oops 8c31670453 Align ZAI modality metadata with source conventions 2026-10-13 20:43:58 +02:00
404oops 16d0ce8b3e Advertise supported image input for ZAI Flash models 2026-10-13 20:43:58 +02:00
404oops 8c64f54a87 Remove ignored-option paragraph from speech reference 2026-10-13 20:43:54 +02:00
404oops 7abd7b4417 Remove unsupported speaker reference option 2026-10-13 20:43:54 +02:00
404oops 29e0e1e2aa Clarify speech catalog units and cover ignored transcription options 2026-10-13 20:43:54 +02:00
404oops 4857898af4 Assert finite ElevenLabs catalog prices 2026-10-13 20:43:54 +02:00
404oops 526a250c95 Correct speech catalog pricing and legacy option declarations 2026-10-13 20:43:54 +02:00
404oops 8c18c45e63 feat(ai): refresh verified provider pricing catalogs 2026-10-13 20:43:43 +02:00
404oops 124f95a5f9 Label provider catalog fixtures as test data 2026-10-13 20:43:43 +02:00
404oops ddafc2d04c fix(ai-chat): normalize remote catalogs and conditional pricing 2026-10-13 18:25:36 +02:00
Daniel Salazar b52fc01712 perf: cut godmode app launch latency (cached token renewal, launch reuse) (#4221)
* perf(auth): renew godmode app tokens from cache instead of the primary

Every godmode launch read its token row from the primary and rewrote its
expiry, a few cross-region round trips per launch far from the primary.

- A cache pointer names the row per (user, web session, app). The row is
  read through the uuid cache token auth uses and re-checked (owner,
  session, app, revoked, expired, max age); anything else falls back to the
  primary read. Revoking the row drops the pointer.
- The expiry write is skipped when it would move by under an hour. A
  renewal that hits a revoked or lapsed row mints a new one instead.
- The response carries the row's actual expires_at.

* perf(gui): reuse a just-minted godmode token for repeat launches

A launch of a godmode app reuses the token this page minted for it within
the last ten minutes under the same desktop session, instead of asking
again. The cache is dropped when an app reports its token dead, after a
session revoke from the session manager or an uninstall, and across tabs
via a BroadcastChannel that carries token row ids, never tokens. A mint in
flight during a drop is not remembered.
2026-10-13 08:53:55 -07:00
Himanchal Kaushale f37cb22d8a fix(gui): resize app content above mobile keyboard (#4165)
Add interactive-widget=resizes-content to the mobile viewport meta tag
so the layout viewport shrinks when the soft keyboard opens. The app
window's height chain resolves against the layout viewport, so the
iframe folds up above the keyboard instead of clipping behind it.
2026-10-13 08:53:44 -07:00
Reynaldi Chernando 948772165d Various docs fixes and AGENTS.md update (#4211) 2026-10-13 17:29:46 +07:00
Neal Shah b8bb6292d6 Fix webdav locks issue and tighten GUI ipc rules for setWindow* (#4207) 2026-10-12 17:39:09 -04:00
Filip Kujundžić 542ca9ee1c Merge pull request #4202 from HeyPuter/filipkujundzic/speech-contract-fixes
fix(ai-speech): honor ElevenLabs output formats, voice IDs and URL audio
2026-10-12 16:21:33 +02:00
404oopsandClaude Opus 5.5 8e92bf6803 fix(ai-speech): honor ElevenLabs output formats, voice IDs and URL audio
- TTS sends output_format as the query parameter ElevenLabs reads; the body
  field was ignored, so every request came back as MP3. Unknown formats are
  rejected before credits are reserved, and the OpenAI-style names mp3, opus,
  pcm and wav map to native formats.
- Voice IDs go straight to synthesis. The library listing omits voices that
  still synthesize, the default 21m00Tcm4TlvDq8ikWAM among them, and runs to
  9 MB per call. Names resolve through /v2/voices search, and premade voices
  ("Bella - Professional, Bright, Warm") also match their short name.
- speech2speech accepts HTTPS audio URLs through secureFetch, as documented.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 12:40:43 +02:00
404oopsandClaude Opus 5.5 b61448c8b7 feat(ai-image): add Gemini Nano Banana 2.1
Catalog gemini-nano-banana-2.1 (aliases google/gemini-nano-banana-2.1 and
nano-banana-2.1) at Google's standard rates: $1.50/1M input, $7.50/1M text
and thinking output, $30/1M image output, with 1K/2K/4K tiers at
1120/1680/3780 image tokens. 3.1 Flash Image stays the provider default.

Pre-flight credit checks assumed 50 text/thinking output tokens per image
for every Gemini model; live samples run 270-1122 depending on the model, so
each model now carries its own estimate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 12:39:49 +02:00
404oops f9cde23937 Merge branch 'filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so' into filipkujundzic/put-1907-ai-streaming-buffering-and-error-shape-cleanup
# Conflicts:
#	src/backend/drivers/ai-chat/utils/inlineImages.test.ts
#	src/backend/drivers/ai-chat/utils/inlineImages.ts
#	src/backend/drivers/util/fileInput.ts
2026-10-12 12:39:35 +02:00
404oops 94e84a2819 Merge branch 'main' into filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so 2026-10-12 12:36:26 +02:00
404oopsandClaude Opus 5.5 f0b55e7605 docs(ai): trim listModels return value and AI overview per review
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 12:33:05 +02:00
Daniel Salazar d8f9aff3dd fix: follow-ups to #4161–#4203 (rate limits, broadcast, AI routing and billing, fs subtree walks) (#4205)
* fix(rate-limit): count the newest KV rate rows first

The KV backend's rate check, peek and concurrent acquire list the key's
rows with a limit + 1 cap that the store applies before dropping expired
rows, so unswept rows from past windows (oldest keys) could fill the fetch
and the live count read as zero. They now list newest first.

* fix(broadcast): split webhook flushes under a byte budget, per-peer backlogs

Flushes go out in bodies of at most `broadcast.outboundMaxBodyBytes` (default
4 MiB) instead of one POST, and each peer keeps its own backlog of what it
missed, capped by count and four bodies' worth of bytes, oldest dropped first,
so one failing peer can no longer grow every peer's body past their 50 MB limit.

* fix(ai-chat): classify dropped upstream connections as provider failures

A connection reset or refusal (SDK APIConnectionError, Mistral's
ConnectionError, an undici reset on the cause) carries no status, so a chain
of them fell through to 500 internal_error. Attempts now record
connectionFailed; a chain of only resets and 5xx answers 400
upstream_provider_unavailable, and a reset counts as an upstream signal in a
mixed chain.

* fix(ai-chat): keep SDK retries on Anthropic count_tokens

The Claude client is built with SDK retries off so the driver can fall back,
but count_tokens has no fallback route. It now passes sdkRetryOptions, like
the OpenAI moderation call, so a transient 429 or reset is retried.

* fix(ai-chat): mark a route unhealthy for 60 s after a transient failure

With SDK retries off, one 5xx, 429, timeout or dropped connection kept a
route marked for 10 minutes, sending that model's traffic to resellers. Those
failures now mark the route for 60 s; bad credentials, an exhausted account
and other route-level failures keep 10 minutes.

* fix(metering): don't memoise spent addons inside the force-exact window

The spent-addons memo could cache a pre-grant record read from a lagging
replica right after a credits-changed announcement, so overage skipped the
new credit and requireCredits kept answering 402 for up to 15 s. A spent
record is now only cached once the actor's force-exact window has ended.

* fix(share): send the digest entries a flush claimed when another take fails

Digest entries are taken concurrently, and a single rejected take threw
away every entry the other takes had already removed from KV. The flush
now logs the failed takes and sends what it claimed; an entry whose take
failed stays queued for a later flush.

* fix(fs): announce an overwriting move's replaced entry only when it took its name

/move and v2 /fs/move looked up the entry at the destination name before
moving and always announced its removal. A move into someone else's Trash
renames instead of replacing, so the owner's desktop hid a live entry and
legacy /move reported it as `overwritten`. Both routes now pass the target
on only when the moved entry kept the name it was looked up by.

* fix(ai): name the vendor in upstream error fields from media providers

upstreamFetch now takes the provider id plus a display label, and
UpstreamResponseError carries the id. translateProviderError copies a
string `provider` off the error into `fields`, and the BytePlus video and
Cloudflare image errors set one, so upstream_auth_failed and
upstream_credits_exhausted alarms say which vendor failed again.

* fix(teams): delete a provisioned seat whose home folders could not be created

provisionUser logs a failed default-folder creation and still returns the
account, so provisionAccount went on to add the membership, issue a password,
mail the seat and emit team.account.created for an account with no home. It
now cascade-deletes the new user and returns 500 internal_error before any of
that.

* fix(ai-chat): load model catalogs without holding up server start

onServerStart awaited every gateway catalog (up to its 10s fetch timeout)
while HTTP was already listening, so drivers started after it had no
providers yet and chat requests found an empty model map. The first build
now runs in the background; complete(), models() and token counting wait
on it instead of failing.

* fix(ai-chat): serve direct models while gateway catalogs load at boot

The first model-map build now publishes a fresh map as each provider's
catalog lands, so static catalogs are routable at once. complete() and
token counting wait for the full first build only when the model, or the
provider asked for, isn't in the map yet. Refreshes still swap once.

* fix(ai-chat): split gateway-billed cost across input and output

A completion billed at the cost OpenRouter, Infron or Neuralwatt reports
was quoted to `ai.prompt.cost-calculated` as all input and no output. The
ledger still records one `billedUsage` line; the reported per-key costs now
spread that bill over the usage keys at the catalog's relative rates, in
whole µ¢ that sum to it. Neuralwatt energy units take no share.

* fix(ai-image): hold Replicate credits for the whole prediction

The output and input-surcharge holds took the default 10-minute TTL, the
same as the prediction window alone, so a slow prediction could outlive
them once creation and input measurement were added. They now last
15 minutes, like the video job holds.

* feat(ai-chat): report what a completion was charged as usage.cost

Chat usage (the non-stream result and the stream's final usage line) now
carries `cost`: the µ¢ the ledger records for the request after the cost
factor, from the request's recorded charges and the gate's factor lookup.
`usd_cents` is unchanged. A hooked recording now copies the usage object
first, so keys the driver adds later never land in the ledger.

* fix(ai): charge streamed speech once its first byte arrives

Per-character TTS and the voice changer recorded their charge when the
upstream's response headers arrived, so a body that failed before any audio
was billed in full. A shared relay now charges on the first byte, or when
the caller hangs up, and charges nothing for an upstream that fails or ends
before sending audio.

* fix(ai): release credit holds only after the request's usage lands

A hold was released as soon as its operation returned, while the usage it
stood for was still being written (behind the cost-factor lookup), so a
concurrent request briefly saw neither. AI metering now tracks the
request's usage writes and its holds (chat included) release once those
settle, without delaying the response; a streamed charge chains its own.
A failed write still releases the hold, and the hold's TTL backstops it.

* test(ai-chat): wait for a stream's credit hold to come back after its usage lands

* perf(fs): page subtree delete and copy by parent link, not by depth over the path range

Each page of a recursive delete or copy sorted the whole remaining subtree by a
computed depth, so a walk cost grew with the square of its size. It now pages
the children of up to 100 sibling directories at a time by parent_uid, depth
first, keyset on (parent_uid, id), so a page reads only its own rows; rows with
no stored path are reached too. SQLite gets the parent_uid index the others have.

* fix(fs): find children by parent_uid in the empty check and supplementary-name lookup

Rows the old backend wrote carry parent_uid and no parent_id, so a non-recursive
delete of a directory holding only such rows passed the empty check and orphaned
them, and on MySQL the supplementary-character child lookup missed them, leaving
their subtrees out of listings, counts and renames. Both now key on parent_uid;
the empty check still probes parent_id too, which the delete cascade follows.

* docs(fs): drop the single-POST note from the entry broadcast cap
2026-10-12 01:10:54 -07:00
Daniel Salazar b88aa55b24 fix: drop moved descendants from the entry cache; read cursor=null as the first page (#4204)
Renaming or moving a directory rewrote its descendants' paths but left their
cached copies (by id, uuid and old path) live for up to 60 s. A lookup of an
old path kept resolving, and a write beneath one landed at a path its parent
no longer had. updatePathPrefixForUser and renameUserHome now drop those keys,
here and, within the broadcast budget, in peer regions.

puter.js sent `cursor=null` for `listShared`/`listSharedByMe({ cursor: null })`,
which decoded to null and failed every keyset listing with a 500. decodeCursor
reads it as no cursor, and the SDK no longer sends it.
2026-10-12 00:19:11 -07:00
Daniel Salazar ff13edc19a Merge branch 'main' into filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so 2026-10-12 02:21:00 -04:00
Daniel Salazar f07a786b41 fix: reserve storage before creating directories on signed write starts (#4161)
startBatchWrite and startWrite created directory items and missing
parents before (or alongside) the storage reservation, so an over-quota
start paid for every directory write, left the directories behind, and
waited on them before returning 413. File targets are now resolved and
storage reserved first; directories are created only after, and the
reservation is released if that fails.
2026-10-11 22:14:57 -07:00
Daniel Salazar 21131ad3d6 fix(kv): spread upload sessions across system KV partitions (#4162)
Upload sessions shared the system namespace's single partition key. They
now go to one of 16 system partitions hashed from the session id, via a
new system-actor-only `systemPartition` KV option; partitions are never
cached and emit no mutation events. Reads fall back to the unpartitioned
key for sessions written before this (max session lifetime is one hour).
2026-10-11 22:14:11 -07:00
Daniel Salazar 38c986927c fix(fs): page subtree walks over supplementary-character names on MySQL (#4203)
listDescendantsInOrder, which drives paged recursive delete and copy, still
used a bare `path LIKE` range, so on MySQL a child whose name starts with a
supplementary character (and its subtree) was skipped: delete left it to the
parent_id cascade and copy dropped it. It now ORs the subtree patterns like the
other subtree queries, paging each pattern group with the same seek and
merging the pages in order.
2026-10-11 22:11:39 -07:00
Daniel Salazar 590c87c00a refactor(stores): one row cache and keyset page helper, batched cache round trips, dead store methods removed (#4191)
* refactor(stores): one read-through RowCache for UserStore and AppStore

UserStore and AppStore each carried the same multi-key row cache: key
builder, cache read, tombstone-guarded backfill, broadcast refresh,
invalidate and tombstone-on-delete. They now share RowCache on PuterStore,
which also single-flights concurrent misses for one key and batches
getByIds/getByUids: one pipelined tombstone check for the loaded rows and
one pipelined backfill instead of per-row awaits. Cache keys and TTLs are
unchanged.

* perf(apps): invalidate cached app lists by generation, not by tracker scan

Every app write SMEMBERS'd a global set of every cached list key, parsed
each key back into its filters and deleted the matches. List pages are now
keyed by their owner's generation token (or a shared one for unscoped
lists), and a write replaces the tokens it affects: O(1) per write, and a
page loaded before a write can no longer be cached after it.

* refactor(stores): one keyset page helper for store listings

Thirteen store listings each cut a LIMIT n+1 read to a page and sealed the
last row's position, and each opened an id cursor its own way. They now use
keysetPage() and openIdCursor() from util/pagination. Cursor payloads and
sealing are unchanged; a cursor without a usable id still fails with 400 in
ShareStore and still restarts the listing elsewhere (strict option).

* refactor(stores): delete store methods nothing calls

AppStore.existsByIndexUrl, OIDCStore.unlinkByUserId,
TeamStore.listByOwner/listActiveSeatUuids, SubdomainStore.getByDomain/
listByDomain, FSEntryStore.ensureDirectoriesForUser,
ShareStore.listByIssuer/listByRecipientEmail/deleteByRecipientEmail/
getDailyShareCount/listByHolderAndFsentries and
PermissionStore.invalidateScanCache/deleteUserUserPermsByPermissionPrefix
had no production callers. ShareStore.listByFsentries was a copy of
listReaching; its callers use the latter.

SubdomainStore SMEMBERS'd a per-user prefix-list tracker on every write and
broadcast its deletion, but nothing ever wrote that set or cached a prefix
listing. The reads are gone.

* refactor(stores): shared escapeLike and bumpGeneration helpers

escapeLike replaces the LIKE escaping copied into PermissionStore (twice),
ShareStore and FSEntryStore, and now also guards SubdomainStore's prefix
filters, which passed caller prefixes into LIKE unescaped.

bumpGeneration sends INCR and EXPIRE in one pipeline instead of two awaited
commands in PermissionStore, EventSubscriptionStore and PresenceStore.

* refactor(stores): read affected rows off WriteResult

Every database client returns affectedRows and anyRowsAffected, so the
`affectedRows ?? changes ?? 0` fallbacks in ShareStore, NotificationStore,
UserStore and UserBlockStore read those directly.

* perf(stores): batch flat-permission KV deletes and durable cache drops

PermissionStore deleted each flat KV entry with its own kv.del; one
kv.batchDel now covers the set. DurableSubscriptionStore's suspend and reap
dropped each row from the cache separately; EventSubscriptionStore gains
dropDurables, which drops them per owner in one pass, and dropDurable calls
it. suspend keeps one conditional UPDATE per row, which is how concurrent
settles learn which rows each of them suspended.

* refactor(fs): fold FSEntryStore's duplicate getters, sort maps and cache keys

- getEntryByUuid and getEntryById share one by-column getter.
- listChildren and listChildrenPage read one sort table; the offset listing
  now coalesces NULL modified/size like the cursor listing, so both order
  NULLs the same way on every engine.
- Entry cache keys come from one builder instead of hand-built strings.
- getEntriesByIds, getEntriesByPaths and the per-user path reader fetch
  their cache hits in one pipelined round trip instead of a GET per key,
  through readCacheValues, which RowCache.readMany now uses too.

* fix(apps): cache the row a lost createFromOrigin race reads from the primary

Keeps AppStore's private #writeCache as a RowCache write, so code that
still calls it (#4166's read-back) loads in either merge order.
2026-10-11 22:07:53 -07:00
Daniel Salazar ed22e2b285 docs: tighten contributor and architecture guides (#4194) 2026-10-11 22:07:02 -07:00
Daniel Salazar 9030b45459 refactor(ai-chat): one OpenAI-compatible provider, one pricing path, cached catalogs (#4192)
* perf(ai-chat): cache gateway catalogs with one remote-catalog helper

cachedRemoteCatalog serves a fetched catalog for 15 minutes, shares one
in-flight fetch between callers, abandons a fetch after 10 s, remembers a
failure for 30 s and keeps the last good value through a failed refresh.

Together, OpenRouter, Infron, Neuralwatt (models and quota) and Ollama use it
in place of their kv-singleton caches. The cached value is the converted
catalog, so a completion no longer re-coerces the whole gateway catalog
(OpenRouter did it twice per request). Ollama now refreshes instead of caching
its first non-empty list forever, and Together's catalog failure no longer
rejects models().

* refactor(ai-chat): build every chat SDK client with retries off

sdkClientOptions() gives each OpenAI, Groq, Together and Anthropic client
maxRetries: 0 and an explicit timeout. The SDK default of two retries, each up
to the full timeout, stacked under the driver's three-route fallback chain; a
failed attempt now goes straight back to the driver, which marks the route
and moves to the next provider for the model.

Timeouts keep what each provider already had: ten minutes, Groq's SDK default
of one minute, and Claude's 10 m 0.6 s. Non-streaming Mistral calls, which had
no timeout once a cancellation signal was passed, are bounded at ten minutes.
The Azure providers are left for after the open Azure fixes land.

* refactor(ai-chat): meter every OpenAI-family completion through one pricing path

meterChatUsage records a completion's usage at buildCostsOverride's rates, or
as one billedUsage line when the gateway reports what it billed, and returns
the per-key costs it recorded. Providers hand those to the driver through the
usageCosts channel, now carried by the Chat Completions handlers too, so the
usd_cents and cost-calculated figures equal the ledger. The driver's own
#computeCost is replaced by pricing.ts usageCost, which shares the rate
resolution with buildCostsOverride.

Billing changes, all toward what the upstream charges:
- DeepSeek, Moonshot, Alibaba, Groq, MiniMax, Z.AI, Hoonify, BytePlus and the
  Neuralwatt catalog fallback took cached reads out of prompt_tokens nowhere,
  so a cached token was billed at input + cache rate. splitCachedPrompt bills
  it once, at the cache rate; usage.prompt_tokens on those routes now excludes
  cached reads, as it already did for OpenAI, Gemini and Meta.
- Catalog entries that listed cached_tokens: 0 (38 Alibaba, 5 Groq) relied on
  that double count to bill cached reads at the input rate. The zero is
  dropped, so they keep exactly that price through the input-rate fallback.
  Neuralwatt models with no published cache price likewise.
- An unpriced key falls back to the input/output rate instead of NaN (0), e.g.
  OpenRouter cached reads on its no-cost fallback path.
- A gateway-reported cost of 0 records 0 µ¢ rather than 1 µ¢.

Claude, Azure and xAI keep their metering; xAI's cached-token handling waits
on the open Grok decision.

* refactor(ai-chat): one OpenAICompatProvider for the OpenAI-compatible vendors

DeepSeek, Moonshot, Alibaba, Groq, MiniMax, Z.AI, Hoonify, BytePlus, Together,
Gemini, Meta, Ollama and the complete() paths of OpenRouter, Infron and
Neuralwatt were copies of one Chat Completions call. Each is now an
OpenAICompatOptions entry (client, catalog, metering prefix, wire-id prefix,
max-token param and default, passthrough fields, openAICompatParams scope,
message cleanup, image inlining, stream usage, context-length retry) plus the
hooks its API needs: prepareMessages (DeepSeek, Groq, Gemini), vendorParams
(MiniMax, Hoonify, Z.AI, BytePlus, Meta, OpenRouter, Infron, Neuralwatt),
meteredUsage (Gemini and the gateways) and reportedUsage (Neuralwatt). xAI is
left until the open xAI fix lands.

Deliberate changes:
- Tools go through make_openai_tools for every vendor (six did, ten sent them
  raw): Anthropic-only tool extras and server tools no longer reach these
  upstreams, and an empty tool list is omitted.
- Model lookup is one indexed, case-insensitive pass per catalog for every
  vendor (Together and Neuralwatt already ignored case).
- Per-provider console logging of upstream errors is gone; the driver logs
  every failed attempt.
- Every vendor honours a configured apiBaseUrl.

tool_choice forwarding stays as each vendor had it (MiniMax, Hoonify, Z.AI
auto-only, BytePlus, Meta auto-only, OpenRouter); widening it needs each
vendor's support confirmed.

* perf(ai-chat): resolve the model once in the driver and keep its map fresh

The driver hands each provider its own catalog entry for the route it chose,
so providers stop re-resolving the id per request; the OpenAI-compatible
providers only look an id up (indexed, case-insensitively) for a direct
caller, and an unknown id is a 400 instead of silently becoming the default
model. Gateways with mixed-case ids (Infron, Neuralwatt) used to miss the
driver's lowercased id and fall through to that default.

The model map is now built from every provider's catalog in parallel, with a
failing catalog left out rather than aborting the build, swapped in whole,
awaited in onServerStart, and rebuilt every 15 minutes so gateway catalog
changes reach routing without a restart.

* refactor(ai-chat): drop provider surface and OpenAIUtil code nothing uses

- list() leaves IChatProvider and every provider that isn't Azure or xAI
  (both wait on the open fix for those files); only tests called it.
- OpenAI Chat, OpenAI Responses and Meta no longer inline puter_path parts:
  the driver does it before every attempt, so theirs found nothing to do. The
  stores and FS service they took for it go too.
- providers/ChatProvider.ts had no subclasses.
- OpenAIUtil: one hoistToolUseBlocks for the Chat and Responses message
  passes (Responses keeps canonical_id), and the dead create_usage_calculator,
  unused Responses deviations and constant mode/toolblock are gone.

checkModeration stays on IChatProvider: the open Azure moderation fix calls it
through that interface.

* refactor(ai-chat): register keyed providers from one table

Sixteen near-identical blocks in #registerProviders, with three constructor
argument orders, become a { name, configKey, Class } table read by one loop;
every entry takes (config, metering) and gets apiBaseUrl when configured.
Claude, Azure, OpenAI (sibling wiring), Ollama (enabled unless turned off) and
the fake provider stay explicit. Gemini's constructor takes config first like
the rest.

* fix(ai-chat): send Ollama the model name it listed

Ollama catalog ids are ollama:ollama/<name>, built from /api/tags. Only the
ollama: part was stripped, so the local server was asked for ollama/<name>, a
model it doesn't have. Strip the whole ollama:ollama/ namespace.

* fix(ai-chat): keep SDK retries on the OpenAI moderation call

The completion clients run with SDK retries off so the driver can fall back.
A moderation check has no fallback, so it passes the SDK's default two
retries per request.
2026-10-11 22:06:18 -07:00
Daniel Salazar f3134b2530 refactor(ai-media): shared upstream fetch, provider registry and bounded inputs for AI media drivers (#4188)
* refactor(ai): route TTS, STT and voice-changer REST calls through upstreamFetch

upstreamFetch bounds a provider call with AbortSignal.timeout and throws an
UpstreamResponseError carrying the upstream status and code, which the
/drivers/call boundary maps like any SDK error. Replaces five hand-written
status translators (ElevenLabs TTS, xAI TTS, Speechify, xAI STT, voice
changer); caller-facing messages are now sanitized by the shared mapper.

* perf(ai): bound input image fetches and share one data-URI parser

fetchImageBytes now streams under a byte cap (30 MB by default) with a
30-second timeout, via a readBoundedBody helper shared with loadFileInput's web
branch and Replicate's input measuring. assertInputImagesShape caps
input_images at 16. Chat image inlining uses fetchImageBytes instead of
buffering with no timeout. fileInput, imageInput and mediaParts parse data URIs
through one parseDataUri.

* refactor(ai-tts): one per-character metering path for TTS providers

meterPerCharacter holds text length x price, runs the synthesis and meters the
same amount once it succeeds; characterCostReport builds the per-character
cost lines. Both sit on a new withAiCreditHold (reserve, run, release) that
Gemini TTS uses directly. SAMPLE_AUDIO_URL is defined once. The TTSProvider
base class is gone: its methods were stubs every provider overrode, and only
Polly read its untyped providerConfig.

* refactor(ai): one ProviderRegistry and key reader for the AI media drivers

ProviderRegistry holds a driver's provider catalog (ids, default, aliases) and
its configured instances: alias resolution, the not-found 400, default
fallback and parallel 'all' listing. TTS, speech-to-text, OCR and the voice
changer route through it. readProviderKey reads apiKey, secret_key, api_key and
key in that order for every AI media driver, so providers.xai.secret_key now
enables xAI TTS as it already did xAI image and speech-to-text. ElevenLabs
caches its voice list for 10 minutes, as Polly does.

* perf(ai): stream TTS and voice-changer audio instead of buffering it

OpenAI, xAI and ElevenLabs TTS and the voice changer hand the upstream body
straight to the caller through upstreamBodyStream; their costs are known
before the call, so nothing waits on the download. A body that fails
mid-stream ends the response early instead of raising an unhandled stream
error. xAI speech-to-text keeps one copy of the audio (the upload Blob)
instead of up to three. The voice changer and xAI STT hold credits through
withAiCreditHold.

* perf(ai-ocr): read Puter files only when the provider needs the bytes

resolveFileInput resolves and access-checks a Puter file and defers the read
to load(); loadFileInput is that plus the read. OCR resolves first: Textract
reads the object straight from S3 and downloads it only if Textract can't
reach the bucket, and only Mistral loads the bytes. A file over the size cap
is refused from its recorded size before any read. countPdfPages inflates
object streams on the libuv pool instead of the request thread.

* perf(ai): build image and video model catalogs once and share output saving

ModelCatalog holds what the public model routes serve (models, sorted names,
cost lines), computed once after boot instead of on every listing. The video
driver now awaits its model map before serving. resolveOutputPath and
saveGeneratedMediaToFS replace the copies of puter_output_path handling in
both drivers, using expandTildePath and normalizeAbsolutePath. The video
driver normalizes a copy of the caller's args instead of mutating them, and
its never-taken stream branch is gone.

* fix(drivers): abort the response when a streamed driver result fails mid-body

upstreamBodyStream now errors when the provider body fails instead of ending
cleanly, and DriverController pipes stream results through pipeStreamResult
(stream.pipeline): a failed source destroys the response so the client sees
an aborted transfer, the failure is logged once, and nothing is left to raise
an unhandled stream error. A client hanging up stops the source.

* fix(drivers): drop a streamed driver result when the caller already hung up

pipeline() throws synchronously on a closed destination, which surfaced as
a 500 alarm on every early disconnect and left the upstream body open.

* fix(ai): bound streamed upstream calls by time to first byte, not download time

upstreamFetch takes a `stream` option: the timeout covers the request until
the response (or a non-2xx error body) is read, then is cleared so a body
piped to the client drains at the client's pace. ElevenLabs TTS, xAI TTS and
the voice changer pass it; non-streamed calls keep the full-response bound.
A stalled upstream body is still cut by undici's body idle timeout.

* fix(drivers): type pipeStreamResult's response by the members it uses
2026-10-11 22:05:37 -07:00
Daniel Salazar 45f6e9d8ee refactor(drivers-wiring): AppService views and launch gates, batched app lookups, shared driver policy and guards (#4186)
* perf(workers): skip hot-reload lookups for files no worker is bound to

The fs.write.file / fs.remove.node / fs.move.node listeners listed every
workers.puter.* row of the file's owner on every event, for every user.
The event's FSEntry already carries the worker rows bound to it, so a file
with none returns before any I/O; a bound file reads just its own rows,
batched, from the primary.

* perf(apps): read the uid alias only after a direct lookup misses

Every AppDriver uid lookup fired an uncached system-KV GetItem for the
merge alias in parallel with the direct read, even when the direct read
hit. The alias is now read only on a miss.

* refactor(apps): one app view with the launch gates, built by AppService

AppDriver's serializer and its private-app / hosted-backing gates move to
AppService.views, built on a pure toAppView that is the one place
index_url is withheld. AppDriver, the subdomains driver's embedded apps and
the GUI shell view all use it. Visibility of protected apps
(filterReadable) moves with it.

Subdomain embeds (associated_app, app_owner) used to return index_url with
no gate; they now withhold it from a viewer who may not open a private app
or whose hosted backing is gone, and carry privateAccess, godmode and
feedback_enabled like a direct app read.

* refactor(apps): one toAppSummary; batch the recommended and taskbar lookups

SuggestedApps, RecommendedApps, /get-launch-apps and the taskbar each built
their own launch summary; they now share toAppSummary (the extra
feedback_enabled / external flags appear on the lists that lacked them).

getRecommendedApps awaited getByName once per recommended app (29) and
getTaskbarItems did the same per taskbar entry on every whoami; both now
make one batched call per key through the new AppStore.getByNames, which
keeps getByName's case-insensitive and old-name matching. The taskbar's
raw UPDATE of the user row goes through UserStore.update, and
/get-launch-apps checks hosted backings in one batch and reaches the
stores without casts.

* refactor(apps): app routes read through AppService; /delete-site calls the subdomains driver

AppController used the global driversContainers through a getter whose
boot-order comment no longer held; it uses the injected drivers and
AppService. GET /apps/:name looked each name up, read it again through the
driver and resolved private access a second time, emitting
app.privateAccess.resolveLaunch twice per private app; it now batches the
names and resolves once. POST /query/app did a lookup, a driver read and a
raw app_filetype_association SELECT per selector (up to 200, serially); it
now makes one batched lookup per key, one grant check and one filetype read.

The driver's read shape is a pure toAppReadView both use.
HostingController's /delete-site re-implemented the subdomains driver's
delete (protected check, delete, subdomain.delete event); it keeps its
owner-scoped 404 and calls the driver. Hosting and Desktop controllers drop
constructors that discarded the drivers argument, and trivial getters.

* refactor(drivers): one per-method policy validator and resolver; drop @Driver

Four policy blocks (rateLimit, concurrent, requireSubscription,
requireReputation) share the { default, methods } shape but each had its
own validator and resolver. They become validatePerMethod(value, label,
policy) and resolvePerMethod(cfg, method); error messages are unchanged.

The @Driver class decorator and the prototype keys it wrote had no users
outside its own tests; drivers declare their fields, and resolveDriverMeta
reads only those. DriverController takes the driver name from the meta it
already resolved instead of probing the prototype.
resolveDriverMethodRateLimit/Concurrent stay as deprecated aliases for the
email extension.

* refactor(drivers): shared actor guards and crud-q concurrency policy

The apps, subdomains, workers and notifications drivers each carried a
private #requireActor, two carried the same #requireVerified, and apps and
subdomains the same owner/app_owner write predicate. These become
requireContextActor, requireContextUserActor, assertActorEmailVerified and
actorOwnsRow in core/actor.ts; each driver keeps its own error for the
cases that differed (system actors still read apps; notifications still
answer 403 for an actor without a user). The identical crud-q concurrency
block is one CRUD_CONCURRENT constant.

The metering, installedApps and appTelemetry extensions re-checked an actor
their routes already require; those checks and the tests for them go.

* refactor(extensions): store methods instead of raw SQL in OSS extensions

installedApps and appTelemetry queried user_to_app_permissions directly
and metering read apps by name with its own SELECT. PermissionStore gains
listAppsGrantedByUser, listAppAuthenticatedUsers (with the email-share
check folded in) and countAppUsers; metering uses AppStore.getByName.

* refactor(server): one layer builder and one lifecycle hook runner

PuterServer built each of its five layers with its own copy of the same
loop (built-ins, then extension additions, into the layer and its export
container), and ran each lifecycle hook with five near-identical loops per
hook. Both are now one helper each: #instantiateLayer and
#runHook(order, hook). Construction order, the args each layer receives,
and hook order (start and prepare bottom-up, shutdown top-down) are
unchanged.

* fix(workers): tear down a deleted source's worker after its FK nulls root_dir_id
2026-10-11 22:04:23 -07:00
Daniel Salazar d162549c1d refactor(controllers): one signup, login, confirm and window.user path; userProtected/internalAuth route options (#4189)
* refactor(auth): one signup path for form, temp, OIDC and team accounts

Signup steps (validate hook, phone/card forcing, bonus verdict, provisioning,
signup events) move from AuthController and OIDCService into SignupService.
Username rules, availability and generation move to util/username.ts, and
user-row + group + default-folder creation to provisionUser, which
TeamService and DefaultUserService now share.

Behaviour fixes: a provisioned team seat joins default_user_group like every
other account, and generated temp usernames skip names whose home path is
occupied.

* refactor(auth): one web-session start for password and OIDC sign-in

startWebSession creates the session with the request's IP, user agent, origin
and host and sets the session cookie. OIDC sign-ins used to create their
session without that metadata, so they showed up blank in the session list.

* refactor(auth): one confirmEmail for the code and link confirmation routes

Both routes now re-run the email checks, refuse an address another account
confirmed, strip it from other rows, promote the group and push
user.email_confirmed to the account's sockets (with the originating socket id
on the code route). The link route's full-table UPDATE clearing other
accounts' pending change-email requests is dropped: /change_email/confirm
already refuses an address a confirmed account owns.

* refactor(auth): build window.user once for whoami and sign-in

buildUserDetails and scrubSensitive move from the whoami extension into
util/userDetails.ts. /whoami calls them; sign-in responses now return the same
user-actor payload, scrubbed, so the login copy no longer ships the phone
number and a no-reload login gets oidc_only/otp/directories up front. Taskbar
resolution is best-effort on both.

* refactor(http): userProtected route option replaces AuthController's route walker copy

The user-protected gate is materialized from a userProtected RouteOption
(implies requireUserActor), so AuthController no longer re-implements the
decorator walker to wire it per instance; those routes and the session
routes are plain decorators now. AuthRequired also counts requireVerified
and userProtected.

* refactor(http): internalAuth route option and a shared bodyRecord

EventsController and PeerController each checked x-puter-internal-auth by
hand with different compares; an internalAuth RouteOption now picks the
secret from config and one constant-time gate checks it. ShareController and
EventsController share bodyRecord instead of private copies.

* fix(fs): cap legacy /delete at 500 paths per request

The bulk form resolved, ACL-checked and removed an unbounded paths[] list one
by one. It now refuses more than FS_BATCH_WRITE_MAX_ITEMS (500), the cap the
batch write routes already use, before touching any entry.

* refactor(controllers): user and subdomain writes and reads through their stores

AuthController's 2FA setup/enable/disable and recovery-code writes go through
UserStore.update instead of raw UPDATEs plus a manual invalidate, and
/readdir-subdomains pages SubdomainStore.listByUserId instead of an unbounded
SELECT; the response shape is unchanged.

* refactor(auth): note SignupService's call-time peers

* fix(auth): type userDetails' taskbar deps independently of TaskbarDeps.clients
2026-10-11 22:00:53 -07:00
Daniel Salazar 576e893a5b refactor(services-core): read-only socket reauth, one token-grant read per check, shared grant/revoke and hosted-asset paths (#4187)
* perf(socket): re-check live sockets without touching their sessions

The 5-minute sweep reused the request-path authenticate(), which slid
every connected session's expiry and wrote user activity each tick, and
could fill SessionStore's touch-throttle map. authenticate() now takes
touch: false for re-checks; the sweep uses it, checks credentials
through runWithConcurrencyLimitSettled, and a call while a sweep runs
joins it instead of starting another.

* perf(acl): read a scoped token's grants once per check

ACLService.check did a Redis GET + JSON.parse of the token's grant list
for every ancestor x mode, and the permission scan did the same per
option. Both now read the list once (PermissionStore.listAccessTokenPerms,
which replaces the per-permission hasAccessTokenPerm) and test it in
memory.

* refactor(metering): incrementUsage goes through batchIncrementUsages

incrementUsage was a copy of the batch path that let an infinite amount
through and skipped NaN, where the batch path counts either as one unit.
It now calls batchIncrementUsages with a one-entry batch, so both apply
the batch clamping (documented on batchIncrementUsages). The overuse
alarm loses the single-path fields nothing sets any more.

* perf(metering): memoize an actor's spent addons on the increment path

Every increment and credit refresh read the actor's addons record from
the KV store. A record with no purchased credit left can only change
through a grant, and every grant announces credits-changed, so it is now
kept per actor beside the credit memo and dropped with it. Records with
credit left are still read fresh, since increments on other nodes draw
them down without announcing it. A read that straddles a drop isn't
kept.

* refactor(services): route raw table writes through their stores

- access_token_permissions: AuthService inserted grants one row at a
  time through a cast to the store's db client, read the authorizer, and
  deleted the manifest with raw SQL, then asked PermissionStore to drop
  its cache. PermissionStore now owns insertAccessTokenPerms (chunked
  multi-row insert), deleteAccessTokenPerms and
  getAccessTokenAuthorizerId, and invalidates inside the writes.
- TeamService's 2FA reset clears the OTP columns through UserStore.update.
- AppIconService's data-URL icon swap is AppStore.replaceDataUrlIcon.

* fix(user): correct cascadeDelete's FK note; seat cleanup is best effort

The header said most user_id foreign keys are ON DELETE SET NULL; the
migrations cascade nearly all of them and keep SET NULL for audit and
analytics history. forgetSeat ran unguarded after the row was already
gone, so a cache failure there failed the whole delete; it is now
logged and stepped over like the other teardown steps.

* refactor(acl): one grant/revoke path per holder kind

ACLService's user and group stat/set pairs were copies differing only in
which PermissionService methods they called. They now share #stat and
#set over a holder adapter ({ lockKey, query, grant, revoke }); lock keys
and errors are unchanged.

PermissionService's eleven grant/revoke methods repeated the same guard
blocks. Each keeps its own order of checks, now through shared helpers
(#requireUserId, #refuseAppActor, #assertCanManage, #requireUser,
#requireApp, #assertFitsColumn, #audit), so every refusal keeps its
status, legacyCode and wording. Audit writes stay fire-and-forget.

* refactor(auth): one hosted-asset token API keyed by kind

The private-asset and public hosted-actor cookies had separate create/
verify methods, four cookie-name getters and two identical options
getters, differing only in the kind claim and what the host field was
called. They are now createHostedAssetToken(kind, claims),
verifyHostedAssetToken(kind, token, expected),
getHostedAssetCookieOptions(), and the exported HOSTED_ASSET_COOKIES
names. Token claims, cookie names and every error message are unchanged;
puterSite and privateAppGate call the new methods.

* refactor(auth): one revoked/expired check for every actor builder

The session, app-under-user and access-token builders (and the access
token's parent-session check) each spelled out the same revoked/expired
tests. They now share #endedSession, which returns the same reauth
results; only a session token passes its auth_id. Adds coverage for an
access token whose parent session expires, is revoked, or is gone.
2026-10-11 21:55:20 -07:00
Daniel Salazar ecef154630 refactor(services): page fs subtree walks, gate share fan-out, one fs ACL helper (#4190)
* refactor(acl): read ancestor chains from FSEntryStore and share one fs ACL helper

getAncestorChain (split a path, batch-read the entries) moves to
FSEntryStore, with getAncestors for callers that want the rows. Services
read it from the store instead of calling FSService sideways;
FSService.getAncestorChain stays for drivers and extensions.

ACLService gains fsDescriptor (memoized chain), assertFsAccess,
assertFsCreate and refusal, plus the pure aclRefusal mapping. The eight
hand-rolled descriptor and safe-error copies in FSService, ShareService,
events/authorization, FSController, legacyFsHelpers, LegacyFSController,
AuthController and WebDAVController use them. Each route keeps its
status, legacyCode and message: fs routes answer access_denied on a 403,
the legacy routes still hide refusals from apps as 404, share and events
keep forbidden and their own 404 text.

* perf(fs): page recursive deletes and batch their event dispatch

A recursive delete read every descendant into memory, and an empty-check
for a non-recursive one did the same. It now walks the subtree a page at a
time in descending path order (children before the directory holding
them, so the parent_id cascade never takes a row before it is handled),
deleting objects and rows and publishing each page as it goes. A
non-recursive delete probes with one LIMIT 1 query.

Removals publish once per entry, after the row is gone (the same change
as the open fix for the double emit), with one surviving-ancestor chain
read at most once for the whole subtree. Events-API dispatch for a batch
runs as one background task at bounded concurrency instead of a
fire-and-forget promise per entry.

FSEntryStore gains hasDescendants and listDescendantsInOrder (keyset
paged by path, then id).

* perf(fs): copy directories a level at a time

A directory copy walked every descendant in memory and copied them one
by one: an insert per directory and, per file, an S3 CopyObject, a
batchCreateEntries of one and its read-back. It now pages the source in
ascending path order (parents before children) and takes each page a
depth at a time: rows for directories, links and empty files at bounded
concurrency, then the depth's S3 copies at bounded concurrency and one
batchCreateEntries for all of them. A single-file copy goes through the
same path.

A failed S3 copy still fails the call (a ghost file still answers 404),
but only after the objects that did copy have their rows, so nothing is
left in the bucket unreferenced. Copied entries publish per depth in one
dispatch.

* refactor(share): build masked share paths in one place

sharePathMask exports maskedRoot(owner, uuid, name) and
maskUnder(root, realPath). ShareService's maskedPathVia goes, its
maskedSelfPath and shareDeepLink's maskedSharePath build on them, and the
masker, maskUnderAnchor and learnShareRoots use maskedRoot. Output is
byte-for-byte the same; tests pin it.

* fix(share): send old_path on a recipient's item.moved

The desktop's item.moved handler reads old_path; ShareService sent the
former masked path only as from_path. Both are sent now.

* perf(share): skip the audience lookup for owners with nothing shared

Every fs write, create, rename, move and delete looked for share
recipients: an ancestor walk plus two share queries, for an owner who in
nearly every case has shared nothing. #sharesReaching now reads the
chain through FSEntryStore.getAncestors (no third copy of the walk) and
asks a per-owner "has anything shared" flag before the share queries.

The flag is the per-user cache events already keeps for subscriptions
(SubscriptionCache): one ShareStore.ownerHasShares read per owner per
TTL, coalesced across a burst. Creating a share (user, team, claimed
invite) or moving a subtree to a new owner bumps it locally and over
outer.pubsub.share.ownerShared; the TTL covers a process that misses
the bump. A failed lookup reads as shared, so the full lookup decides.

* perf(share): batch the per-row reads behind share listings

Share listings looked things up one row at a time:

- teams by id: TeamStore.getByIdsIncludingDeleted (the single-id read
  calls it) for listSharedByMe, listSharesOf and #revokeGroupSharesBy
- team shares per node and ancestor: ShareStore.listGroupOnFsentries,
  one query for the chain, in the same node-then-ancestor order
- #liveGroupGrants: PermissionStore.queryIssuerGroupPermsByPrefixes
  reads every issuer/group pair under every prefix in one query
- #revokeGroupSharesBy: the five canManagePermission checks per row run
  together instead of one after another
- ShareNotificationService #unblocked: users via getByIds and blocks via
  UserBlockStore.blockersAmong, two queries instead of two per member
- digest flush: entries are taken at bounded concurrency rather than one
  at a time

* perf(events): ask background consent once per holder and app on a revoke settle

Withdrawing events:background re-checked consent row by row, resolving the holder's actor and asking the permission service for every one of their background rows. Rows of one holder and app stand on one consent, so it is asked once and shared.

* refactor(events): drop the registry's unused push fields

notify, defaultDelivery, PushMessage, NotifyProjection and pushProjection were read only by registry.test. Every subject was unpushable and broadcast by default; the subscription's delivery class stays in DeliveryClass, which the stores use.

* refactor(share): retire trashed shares through onEntryOwnerChanged

onEntryTrashed was a copy of onEntryOwnerChanged: the same subtree share sweep, grant retire and row delete.

* fix(fs): page subtree walks by depth, not path

MySQL sorts a PAD SPACE string on its first max_sort_length (1024) bytes,
so under utf8mb4_unicode_ci paths past about 512 characters tie and fall
back to id. A file moved under a newer directory then sorted after it: the
delete walk reached the directory first and the parent_id cascade took the
file without its object delete or event, and the keyset skipped or
repeated rows. Pages now order by segment count, then id, which no
collation can reorder.

* fix(share): cache an owner as shared when the bump arrives

A bump only dropped the cached answer, so the next lookup could read a
replica that hadn't applied the new share yet and cache "nothing shared"
for another TTL. The bump now records "shared" outright.

* fix(fs): probe a directory for children by parent_id, from the primary

* fix(fs): keep the copied-size check in the batched directory copy

A copied file records the size CopyObject actually duplicated; growth past
the source row is charged against the allowance, and a copy that no longer
fits is deleted and refused, as a single-file copy already does.
2026-10-11 21:54:13 -07:00
Daniel Salazar c815dc82a1 refactor(core-util): shared bounded map, cheaper route lifecycle, DDB/DB/rate-limit/gate dedup (#4185)
* refactor(util): add BoundedTtlMap for core caches and log throttles

One size-capped, LRU-evicting map with an optional TTL and a once-per-window
shouldEmit helper. Replaces the hand-rolled maps in the DynamoDB throttle and
repair loggers (the repair one was unbounded), the Slack repeat throttle, the
auth probe's reauth log throttle, and the card-verification status memo.

* perf(http): skip route lifecycle work when nothing listens

The per-route lifecycle middleware built a payload, opened an
event.emitAndWait span and hooked finish/close on every request, listeners
or not. Each phase now checks hasListeners first. EventClient's three copies
of the key-prefix walk become one memoized #matchKeys.

* perf(ddb): write batch chunks concurrently and retry the TTL sweep

batchPut/batchDel sent their 25-item chunks one after another; they now run
up to four at a time through runWithConcurrencyLimit, and a failed chunk
stops the ones not yet started. batchGet and the batch write share one
#sendWithUnprocessedRetry instead of two copies of the backoff loop, and the
TTL sweep goes through #batchWrite, so the unprocessed deletes it used to
drop are retried. The AWS and local clients are built by one #bindClient.

* refactor(db): share migrations and tryHardRead in the base client

MySQL and Postgres each carried a copy of the migration-path runner and of
the replica-racing tryHardRead; both now live on AbstractDatabaseClient
(applyMigrationPaths, and tryHardRead gated on hasReadReplica), and SQLite
drops its own tryHardRead override. Removes MySQL reinitPrimary,
reinitReplica, isRetriableError and readWithRetry and the base requireRead,
which had no callers outside tests. SQLite prepared statements and the
Postgres SQL rewrite are memoized per query text in a bounded map.

* refactor(rate-limit): route every entry point through one charge()

The nine rate/peek/acquire entry points each repeated key and limit
resolution, the backend call and the fail-open catch. They now share
charge(), routeKey() and guardSlot(); log lines are unchanged. Driver
concurrency slots release at most once like the other slot handles, and
checkRateLimit takes { failOpen: false } for limits that must hold when the
backend is down.

* refactor(util): add secretsEqual for constant-time secret comparison

One HMAC-then-timingSafeEqual helper, safe for inputs of any length, so
callers stop hand-rolling the length guard. File signature verification
uses it.

* refactor(fs): infer signed-file MIME types with mime-types

mimeFromName looks extensions up in mime-types instead of a 35-entry inline
table, keeping the two types the table reported differently (ico, wav). The
purpose-labelled signing key is derived once per secret instead of on every
signature, where signFile derived it twice.

* refactor(http): build route gates from one gate() helper

Each gate repeated a try/catch -> next(err) wrapper or a .then(next, next)
chain around an assertion. gate(check) does that once: a throw or rejection
goes to next(err), and a synchronous check still calls next synchronously.
The credits, subscription and reputation gate files held nothing but such a
wrapper; their factories move into gates.ts and the files go.

* perf(hosting): build the puter-site hosting config once

createPuterSiteMiddleware rebuilt the normalized hosting-domain config on
every hosted-site request; it is derived from static config, like the domain
lists the middleware already builds at construction.

* fix(db): tryHardRead's replica leg doesn't fail over to the primary

MySQL's read() retries on the primary after a load-shed or transient
replica error, so routing tryHardRead through it read the primary twice
on that path. The replica leg now reads the replica alone.

* fix(ddb): settle in-flight batch chunks before rethrowing

A failed chunk stopped unstarted chunks but rejected while others were
still retrying, so writes could land after the caller had handled the
error and invalidated its cache.
2026-10-11 21:53:33 -07:00
Daniel Salazar b5eefaa9cc fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135) (#4198)
* fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135)

MySQL ends the idx_fsentries_path range for `path LIKE 'p/%'` at 'p/' +
U+FFFF, and utf8mb4_unicode_ci sorts supplementary characters above it, so
a child whose name starts with one (most emoji) was skipped with its whole
subtree by listing, delete, copy, rename/move rewrites, counts and sizes.
On MySQL, FSEntryStore now looks up those children by parent_id and ORs a
LIKE pattern for each onto the subtree match, recursing into directories.
Every extra pattern starts with the prefix, so only the scanned ranges
widen; rewrites past 500 patterns run in several statements.

The prefix rewrite also passed a UTF-16 length to SUBSTR, which counts
characters, so a rename under a path holding an emoji dropped characters
from every descendant path on all engines. It now counts code points.

* test(fs): seed the supplementary-name subtree with createNonFileEntry
2026-10-11 21:21:18 -07:00
Daniel Salazar af9ebb0ef1 fix(ai): map BytePlus video and Cloudflare image upstream errors at the driver boundary (#4184) 2026-10-11 21:20:27 -07:00
Daniel Salazar 2131b980ce fix(redis): release locks with an atomic compare-and-delete (#4183)
* fix(redis): release locks with an atomic compare-and-delete

ACLService, TeamService and the share digest flush each hand-rolled a SET NX
lock whose release was a GET-then-DEL (or a blind DEL), so a holder whose TTL
lapsed could delete the next holder's lock. They now share withRedisLock,
which claims with a random token and releases via a Lua compare-and-delete.

* chore(team): place the redis lock import with the other runtime imports
2026-10-11 21:19:37 -07:00
Daniel Salazar 96909b0800 fix(fs): send the same outer.gui.item.* payload from every fs controller (#4179)
* fix(fs): send the same outer.gui.item.* payload from every fs controller

The v2 /fs/move and /fs/delete routes spread the raw camelCase FSEntry into
item.moved / item.removed and named the old path `from_path`, while the
desktop reads the legacy snake_case shape and `old_path`. One helper,
emitGuiItemEvent, now builds the payload for LegacyFSController,
FSController and WebDAVController.

v2 and WebDAV copy/move with overwrite now also emit item.removed for the
entry they replace, as the legacy routes do, and v2 /fs/move forwards
new_metadata to the service like the legacy /move.

* fix(fs): announce a replaced entry before the one replacing it

The desktop hides rows by path on item.removed, so a removal sent after the
overwriting copy's added (or move's moved) hid the new row too. The shared
emitter now takes the replaced entry and sends its removal first, awaited,
on the v2, legacy and WebDAV copy/move paths.
2026-10-11 21:18:47 -07:00
Daniel Salazar 906d6b7886 fix(email): one canonicalization and blocklist for every address path (#4176)
Share invites, share notifications and app feedback checked addresses through
EmailClient.clean/validate, a second copy of the rules in util/email.ts that
stripped dots on iCloud, stripped `+tags` on every domain, and matched the
blocklist case-sensitively. They now use util/email.ts: isBlockedEmail for the
blocklist, abuseKey for the per-address notification budget, and cleanEmail
for the invite digest key. EmailClient keeps transport and templates only.

isBlockedEmail also lowercases the configured suffixes, so a mixed-case
blocklist entry still matches.
2026-10-11 18:24:28 -07:00
Daniel Salazar 6a938bd8e8 fix(db): detect duplicate keys with isUniqueViolation everywhere (#4175)
FSService, OIDCStore, UserStore and TeamService each carried their own
duplicate-key check. FSService only matched mysql and the bare
SQLITE_CONSTRAINT code, so the mkdir/move race recovery never ran for a
SQLite or Postgres unique violation. OIDCStore matched any message
containing "UNIQUE" or the bare SQLITE_CONSTRAINT code, UserStore any
SQLITE_CONSTRAINT_* code, and TeamService any message mentioning unique or
duplicate. All four now use util/dbError's isUniqueViolation; UserStore
still checks the index name on top.
2026-10-11 18:24:13 -07:00
Daniel Salazar 847a4fc031 fix(fs): route thumbnail writes through FSEntryStore (#4174)
* fix(fs): route thumbnail writes through FSEntryStore

Add FSEntryStore.updateEntryThumbnailByUuid, which sets a thumbnail with
no owner check (callers authorize) and refreshes the entry cache, with an
optional expected-value guard.

The thumbnails extension's inline data: URL migration, its copy
duplication, and the legacy /update-fsentry-thumbnail endpoint now write
through it instead of raw UPDATEs, so cached entries stop serving the old
thumbnail. The migration validates the image like the upload path, awaits
the write, and only repoints a row that still holds the data URL; a read
that loses that race deletes its upload.

* fix(fs): report a thumbnail update by whether the row changed

updateEntryThumbnailByUuid returned the refreshed entry, which is null
when the cache misses and the replica hasn't seen the row yet. The
read-time migration took that null for a lost race and deleted the object
the row had just been pointed at. It now returns whether the UPDATE
changed a row; the cache refresh still runs after it.
2026-10-11 18:23:58 -07:00
Daniel Salazar 3296526e8e fix(ai-chat): derive Azure model pricing from the OpenAI and xAI catalogs (#4173)
Azure entries that front an OpenAI or xAI model now come from a `mirror`
helper that copies every non-identity field from the source catalog entry.
The hand-copied entries had drifted: none carried `long_context_pricing`, so
long prompts on Azure-served gpt-5.4 and Grok 4.3/4.20 were billed at 1x;
grok-4.3 had lost `pdf` input and the Grok 4.20 deployments still listed the
old 2M context. Models no longer in the direct catalogs stay inline.
2026-10-11 17:57:14 -07:00
Daniel Salazar c553a380ae fix(ai-chat): forward max_tokens to xAI; moderate Azure through OpenAI (#4172)
xAI sent a fixed max_tokens of 1000, ignoring both the caller's value and
the credit gate's cap. It now forwards max_tokens like the other
OpenAI-compatible providers and omits it when unset.

Azure chat's checkModeration threw "Method not implemented", so
`moderation: true` on an Azure route metered the completion, then failed it
as a route error: Azure was marked unhealthy and the fallback billed a
second completion. Azure's v1 API has no /moderations endpoint, so both
Azure providers now delegate moderation to the OpenAI chat provider
(Azure chat via its Responses sibling, as it does for web_search).

A moderation call that throws is now a 400 `moderation_unavailable` that
the driver treats like `moderation_flagged`: final, no fallback, and the
route stays healthy.
2026-10-11 17:55:38 -07:00
Daniel Salazar cf9df8719a fix(alarm): forget idle alarms and cap how many the client holds (#4171)
AlarmClient kept every alarm id it had seen for the life of the process,
and ids that embed a user, key or invoice grew it without bound. The
registry is now ordered by last occurrence: alarms idle for 24 hours are
dropped on the next create, and past 1000 the least recently raised goes
first. Clearing or evicting an alarm only removes its short-id alias when
the alias still points at it.
2026-10-11 17:54:26 -07:00
Daniel Salazar 7ad963bacf fix(cache): broadcast fs entry and team invalidations to peer regions (#4169)
* fix(cache): broadcast fs entry and team invalidations to peer regions

FSEntryStore's invalidations called publishCacheKeys without broadcast,
so a delete, move, rename or overwrite only cleared the writer's
cluster; other regions served the old row until the 60 s TTL. They now
broadcast. Batch paths (deleteEntries per chunk, the batch overwrite)
send one event for the whole batch. Read-path backfills stay local.

TeamStore's busts were a local redis.del, so a removed member kept
passing getMembership in other regions until the TTL. They now go
through publishCacheKeys with broadcast, and softDelete busts every key
in one call.

Remove FSEntryCacheInvalidationEventHandler. It re-derived the same
invalidations from outer.gui.item.* with a replica SELECT per holder,
and its fs.remove.node branch never ran (FSEntry has no get()). The
store's own broadcast covers what it did.

* fix(fs): cap how many entry invalidations a node broadcasts per window

FSEntryStore now broadcasts at most 10,000 entries' cache invalidations
per broadcast flush window (outbound_flush_ms, 2 s by default). Past
that, keys are dropped locally only and peers fall back to the 60 s
entry-cache TTL; the store logs once per window when this happens.

Each flush goes to a peer as one POST that is re-sent whole on failure,
so a large recursive delete could otherwise build a batch the peer
rejects on every retry. The budget is per store instance, not per call,
so a delete issued page by page is bounded too, and single-entry writes
count against the same budget.
2026-10-11 17:53:26 -07:00
Daniel Salazar 0066d49f74 fix(fs): emit fs.remove.node once per entry on recursive delete (#4167)
#removeDescendantsStorage emitted fs.remove.node for every file
descendant while grouping S3 keys, and remove() emitted it again for
every descendant after deleting the rows, so file listeners ran twice.
Drop the storage-pass emit; remove() now emits once per entry after its
row is deleted.
2026-10-11 17:52:27 -07:00
Daniel Salazar b67acdbf6f fix(stores): return written rows without reading them from a lagging replica (#4166)
* fix(stores): read rows back from the primary after writing them

ShareStore (create, upsertPending, upsertActive, upsertActiveGroup,
applyPending, upsertAnyone), AppStore (create, createFromOrigin, update,
claimOwnership), NotificationStore.create, SubdomainStore.update and
NotificationDriver.create returned the row they had just written by
reading it back through `db.read`, which goes to the replica. Under
replica lag that came back null or stale. Those read-backs now go to the
primary, through a `primary` option on the getters. AppStore caches the
primary copy on create, and claimOwnership no longer reads through
getById, which cached the replica's pre-claim row on a miss.

* fix(stores): skip the notify read-back; read OIDC link conflicts from the primary

NotificationStore gains `insert`, which writes and returns the uid;
`create` is now `insert` plus the primary read-back. NotificationService
.notify ignores the row, so it uses `insert` and no longer pays a primary
round trip per recipient. NotificationDriver.create still reads the row
back from the primary.

OIDCStore.link reads the conflicting row from the primary after a unique
violation, so a replica that has not seen another account's link can no
longer make the call report success.

* perf(stores): return written rows instead of reading them back

ShareStore.create, NotificationStore.create, AppStore.create and
AppStore.createFromOrigin build the row they return from the values they
inserted and the insert id, instead of reading it back from the primary.
The DB-defaulted creation timestamp is now written explicitly from unix
seconds (`timestampFromUnixSql`), so the returned value matches what a
read returns (`timestampValue`); postgres INSERTs add RETURNING id. The
AppStore rows fill every column the INSERT leaves to the schema and are
cached as before.

ShareStore.upsertPending's re-invite branch selects the full pending row
in its dedupe read and returns it with the new mode and data; it falls
back to the primary only when the UPDATE matched nothing.

The remaining primary read-backs stay and say why: the share upserts and
the new-invite insert (an upsert's surviving uid/created_at and the
entry-owner subquery are only known to the primary), applyPending and
claimOwnership (only an id or uid in hand), AppStore.update and
SubdomainStore.update (the base row is a replica copy that would be
cached), NotificationDriver.create (notify returns only the uid) and the
OIDC link conflict.
2026-10-11 17:51:38 -07:00
404oops 64db150a90 fix(ai): bound streaming buffers and normalize input errors 2026-10-11 21:14:16 +02:00
Daniel Salazar f88cadb4b9 fix: a copy records the size of the bytes it duplicated (#4197)
A copied file's row took its size from the source row the request had
resolved, which can predate an overwrite whose bytes CopyObject then
copied. The copy now reads the new object's size and records that. When
it is larger than the source row said, the allowance is checked again
against the real size; a copy that no longer fits is refused with 413
and its object deleted.

If the size check fails, the copy falls back to the source row's size,
as completing a signed upload already does.
2026-10-11 10:48:52 -07:00
Daniel Salazar aef0a32d72 fix(ai): hold credits for image and video generation (#4164)
Image and video providers checked the balance with hasEnoughCredits /
getRemainingUsage and took no hold, so concurrent requests each saw the
full balance until the first one recorded usage. They now reserve the
estimate through the cost-factored reserveAiCredits and release it in a
finally block once usage is recorded, like chat, speech and OCR.

The video duration clamp now prices seconds at the model's cost factor
and holds the clamped clip's cost for the job (poll window plus create
time). Replicate holds the output-only estimate before fetching inputs
and the input surcharge on top once it is measured.
2026-10-11 09:18:13 -07:00
404oops 800e300a8c fix(ai-tts): resolve ElevenLabs voice names before synthesis 2026-10-11 16:33:05 +02:00
Daniel Salazar 491729e060 fix: backend test flakes and a socket shutdown hang (#4170)
- SocketService's prepare-shutdown hook awaited io.close(), which also
  closes the http server and waits for every HTTP connection to end. A
  connection mid-request when close() runs stays open for the keep-alive
  timeout, so shutdown stalled before PuterServer severs connections. The
  hook no longer waits on the http close.
- Test servers disable Ollama unless a test configures it. Its boot probe
  logged after files had ended, which failed shared-worker shards with
  EnvironmentTeardownError.
- vitest ignores that teardown error for a console call still in flight.
- setupPuterTestEnv boots again on a fresh port after EADDRINUSE, and a
  failed setupTestServer start shuts the server down.
- UploadReservationStore expiry tests pin the clock instead of racing a
  50 ms TTL.
2026-10-10 19:34:32 -04:00
Daniel Salazar d4589caefe fix: session store revokes (#4168) 2026-10-10 07:55:43 -07:00
Daniel Salazar 4fca4809b9 fix(health): discount total event-loop stall time from db liveness latency (#4160)
The stall watcher kept only the longest gap, so a saturated loop with many sub-second stalls still failed database-liveness. It now sums blocked time across the probe (ignoring timer jitter) and reports total and longest.
2026-10-09 18:14:48 -07:00
Daniel Salazar cc8b9858a2 feat: let extensions claim a subdomain (PUT-2100) (#4155)
`extension.claimSubdomain(name, { authorize, checkRoute? })` lets an
extension serve `<name>.<domain>` without a hard-coded reserved name.

- A claimed host is dispatched where the unknown-subdomain 404 used to be,
  before CORS, the OPTIONS responder, body parsing, auth and every route
  (including `subdomain: '*'` ones).
- `authorize` returning anything but true, or throwing a non-HttpError,
  yields exactly the unknown-subdomain 404. A thrown HttpError goes to the
  error handler.
- Routes whose `subdomain` is the claimed name (decorated controllers and
  extension routes alike) are materialized onto the claim's own router, so
  they only run after `authorize` and never on the main app. Their options
  are limited to ones that don't rely on the main pipeline's actor; an
  optional `checkRoute` lets the claim refuse routes at boot. Case-variant
  labels and claimed names mixed into subdomain arrays fail the boot.
- The unknown-subdomain check now compares the port-stripped hostname and
  requires a dot boundary, so a host with an explicit port behaves like one
  without it. Hosts served by the local worker proxy still pass through.
2026-10-09 18:14:04 -07:00
Daniel Salazar 6060de8fbd fix(alerts): overspend alarm in credits at warning; no-choices completions stop paging (#4154)
- The per-account overspend alarm now reads in credits and raises at
  warning so it reaches the pager instead of only chat.
- A non-stream completion with no choices (an upstream answering 200 with
  an error body) threw a TypeError that classified as internal_error and
  paged critical. It now throws bad_response, which falls back like any
  other empty response and surfaces as a 400.
2026-10-09 18:13:44 -07:00
Daniel Salazar 0e0cedb02b fix(auth): no email password recovery for built-in admin accounts (#4153)
- /send-pass-recovery-email for `admin`/`system` (by username or email)
  returns the usual generic message, writes no token, sends no mail, and
  raises a critical alarm with the requester's IP, user agent and, when
  signed in, their user.
- /verify-pass-recovery-token and /set-pass-using-token reject recovery
  tokens for those accounts as no longer valid.
2026-10-09 18:13:22 -07:00
Neal Shah 43094a84a0 PUT-1901 (#4159) 2026-10-09 20:12:45 -04:00
Neal Shah 5fb025c109 Misc subdomain and worker fixes (#4157)
* PUT 1902

* LocalWorker fetch policy fixes for 127.0.0.1
2026-10-09 19:59:52 -04:00
Juan Fernando Castro 50bcdc9c6c fix: let a revocation be seen by the reads and the nodes that outlive it (#4151)
* fix: let a revocation be seen by the reads and the nodes that outlive it

Four places where a revoke lands on the primary and the next reader can
still be told the old answer.

The u2a cache key was dropped but not re-filled, so the next miss read a
replica that may still carry the row and cached it for the TTL -- and a
revoke's own settling pass is usually that miss. It re-fills from the
primary, as the u2u side already does.

A revoked session dropped its keys the same way. It now leaves the
revoked row in the cache, read from the primary, so there is nothing for
a lagging replica to re-introduce.

A link share is its own authority rather than an index over a
permission, so a replica still carrying a withdrawn row answers with it.
For a short window after one is withdrawn, those reads go to the
primary.

Socket eviction rode a local event, so it only reached the node that did
the revoking. It is announced on the fan-out channel too, which is where
the node actually holding the connection hears it.

* refactor: carry the revocation on outer.* instead of outer.pubsub.*

Eviction runs through `io.in(room).disconnectSockets()`, which goes via the
socket.io Redis adapter a cluster shares, so one node handling the event
disconnects that user on every node in it. Fanning out to siblings had each
of them repeat the same cluster-wide disconnect.

The adapter's Redis is per-cluster, so peer clusters still need the hop --
`outer.*` reaches them and skips the sibling fan-out. The local
`auth.sessions.revoked` goes with it: `outer.*` fires on the emitting node
too, and nothing else subscribed.
2026-10-09 16:36:26 -04:00
Juan Fernando Castro ff35fbd66e Share attribution and cascade: finish PUT-1856 and PUT-1857 (#4119)
Replayed onto main after #4118 squash-merged. Same tree as the reviewed
head (8d8bd871f); only the redundant merges from the old base are gone.
2026-10-09 16:23:18 -04:00
Juan Fernando Castro 1ff5d93244 fix: a signed overwrite lands on the live key only once its completion is accepted (#4136)
* fix: a signed overwrite lands on the live key only once its completion is accepted

A presigned overwrite URL was signed against the live object key, so the
browser's PUT replaced the owner's bytes directly. That PUT never passes
through Puter, which leaves the access check on `completeWrite` with
nothing left to protect: by the time it runs, what /read serves has
already changed.

The upload now goes to a key of its own. `completeWrite` is already the
authorized step, so that is where the staged object is copied onto the
live key, server-side, and the staged one dropped. The entry keeps its
uuid, so everything written against it still holds. Aborting deletes the
staged object, which the old shared key made unsafe to do.

`copyObjectOfAnySize` carries the copy past the single-request ceiling
by copying part by part, so a large file is no different.

Two behaviours follow from the key no longer being shared: completing an
overwrite that uploaded nothing is refused rather than reported as an
overwrite of unchanged bytes, and abort always drops the session's
object. Only the two presigned paths changed; `prepareBatchWrites`
uploads server-side within one authorized request and still writes in
place.

* fix: the review's findings on the staged overwrite

The batch signed path still signed against the live key, so the route
the SDK's signed upload actually takes was unchanged. It stages now,
which also makes the abort simplification safe: the guard that had
protected a shared key was removed while that path still shared one, so
aborting a batch overwrite would have deleted the owner's object.

The completion is allowed against a path but lands on the uuid captured
at start, and a move between the two puts the bytes somewhere the caller
was never authorized for. The promote refuses unless the entry is still
where the check looked.

The batch promote failing left every session pending with its lease
held; it now fails them as the rest of the function does.

The pending GUI tile and the thumbnail key were keyed on the staged
uuid, which no entry ever carries, so neither correlated with what the
completion emitted. Responses carry `entryUid` for that.

* fix: do not treat a pre-staging session's key as this upload's own

A session signed by the previous release points straight at the live key,
since `objectKey` was `existingEntry?.uuid ?? uuidv4()`. Completing one is
already handled -- the promote returns early when the two keys match -- but
abort deleted `objectKey` unconditionally, which during a rolling deploy
would take the owner's file rather than the upload's litter.

Abort now deletes only a key the upload owns. The multipart branch was
always safe: discarding parts never touches an object already in place.
2026-10-09 16:22:36 -04:00
Juan Fernando Castro 86cc3490f7 fix: settle which plan an org seat resolves to, and confirm a seat's address (#4139)
* fix: settle which default plan an org seat resolves to, and what counts as paid

Default subscription resolvers ran in registration order, which is boot
order: whichever layer registered first answered first. A seat is
provisioned by a team and its address is never confirmed, so the seat
resolver now states a priority and wins over anything a deployment
infers from an address.

The seat cap asked whether the owner's plan was outside the free set,
which a default plan can be without anyone having bought it. Metering
now reports whether an explicit subscription answered, and the cap rests
on that.

* fix: the review's findings on plan resolution

`fromDefault` was read off which resolver answered, not off which policy
applied. An explicit id naming a policy nobody registered falls through
to the default one, and that read as bought. It follows the applied
policy now.

A deployment with unlimited metering has no explicit resolver, so every
owner on it became unpaid and its teams dropped from the paid seat cap
to the free one. Unlimited answers for what a paid plan would, so it
counts as one.

Also: the new constants had been placed between `FREE_SUBSCRIPTION_IDS`
and its docblock, the register docstring still described first-wins, and
prettier had reflowed an untouched licence header.

* feat: a seat confirms the address its team typed in

An address on a seat is the team's claim about someone else's inbox, and
nothing asked the inbox. It is now demanded and sent a code, so the
address counts for nothing until the seat hands it back. Resolvers that
read a domain are the reason this matters.

No address is still fine and demands nothing: the team is the trust
anchor for the account, not for a mailbox it does not hold.

`/confirm-email` allows an unconfirmed actor, so a seat can sign in with
its temporary credential and clear the demand; nothing locks it out.

* fix: clear the confirmation demand with the address it names

`clearUnconfirmedEmail` releases a seat's address so the inbox's owner can
sign up with it. The demand to confirm that address has to go at the same
write: the gate reads `requires_email_confirmation && !email_confirmed`, and
once the address is gone there is nothing left to send a code to.

* fix: keep a seat's address correctable, and only ask what can be answered

Review of the confirmation demand turned up two ways a seat could be left
holding one it had no way to meet:

- With no mail transport configured, the code is never sent and a resend
  cannot reach the seat either, so the demand stood with nothing able to
  lift it. The seat is now gated only when a code can actually travel.
- An address the owner mistyped gated the seat on an inbox nobody could
  read, and the gate itself closes `/user-protected/change-email`, the
  route the seat would have fixed it by. `updateSeatEmail` lets the owner
  who typed the address retarget it, and refuses once it is confirmed,
  when it belongs to whoever proved it.

Also resolves the owner's plan sequentially, so the second read takes the
cache the first wrote and the pair cannot come from two resolutions, and
restores the licence header a jsdoc reflow had rewrapped.

* fix: let the confirm-email window open before the profile has loaded

Booted from a stored token, the window renders before `window.user` is
populated, and reading `.email` off null threw inside the window itself --
so the one screen that could have asked for the code never drew, leaving a
blank page. Reproducible without a team: sign up, then load the GUI again
before confirming.

The address is now encoded rather than interpolated raw, and both forms of
the line are i18n keys; it was hardcoded English before.
2026-10-09 16:17:51 -04:00
Juan Fernando Castro 7de8360a6e Sharing cleanup: bounded outbound listings, and what see does not reach (#4118)
* fix: keep a file name to one line in a subject, and the inbound total to the account

**A name reaches the digest subject as the user typed it.** Subjects compile
with `noEscape` on purpose -- a header is not HTML, and escaping would show
entities to the reader -- and the comment there says free-form values collapse
whitespace upstream. A file name does not: `#assertUsableName` refuses only an
empty name, a slash, `.` and `..`, so a name may carry newlines. It is
collapsed to one line and bounded before it goes in.

**`includeTotal` answered an app with the account's figure.** The page is
filtered by what the caller can reach -- an app sees only what its user handed
it -- while `countByHolder` counts every inbound share. The docs already call
`total` an approximation for the filtering; they did not cover an app reading
the size of everything else. It is answered only for a caller acting as the
account now, and the page documents that.

The team-share fixture built its actor as `{ user }` with no `effectiveApp`,
which reads as an app rather than a session. A session acts as the account, so
it says so.

* fix: stop `see` reaching inside a folder, and bound mode changes

**A `see`-only folder recipient was pushed every child's name, size and type.**
The fan-out picked holders by reach -- any live grant on a reaching root -- and
`see` is a grant. But `see` says a node is there; `list` is what says what is
inside it, and the payload carries exactly what `list` would: name, path,
`is_dir`, type, size, modified. Descendants now go only to a holder whose mode
reaches inside. A change to the shared node itself still goes to everyone who
holds it, which is what `see` covers.

**Mode changes skipped the daily quota entirely.** Not spending share budget is
right -- changing a mode is not new reach, and the existing test says so -- but
unbounded it lets one account write a pair as fast as it can ask. They have
their own budget now, ten times the share limit by default and configurable,
so the exemption stays an exemption from *that* budget rather than from any.

* fix: let a half-finished downgrade be finished, and stop Trash answering

**A mode downgrade that failed between its two halves could not be repaired.**
`setUserUser` grants the new mode and then revokes the superseded ones. If the
revokes did not all land, the higher grant stood -- and the retry returned
early, because the mode it was asked for was now present. "Nothing to do" now
means this mode is the only one standing, so asking again is what finishes it.
The same two halves, and the same early return, were in the group variant.

**Trashing someone else's item answered for what is in their Trash.** Moving
into the owner's Trash is deliberately authorized without write on it -- that
is what lets a recipient delete from a shared folder -- but a name collision
there came back as `An entry already exists at /owner/Trash/<name>`. The mover
picks the name, so that is a question they can ask about a folder they cannot
list. It dedupes instead, which is also what the GUI wants: trashing should
not fail over a name.

* perf: record the entry's owner on the share row

The outbound listings answer "shares on things I own that someone else issued"
by joining `share` to `fsentries` and filtering on the owner there. Neither side
of that join bounds the work: driven from `fsentries` it walks every file the
user owns, driven from `share` it walks every share until it has a page. For a
user who owns a lot and delegates nothing, that is the whole table for an empty
result. `#outboundAppsSql` is worse than the ticket says -- it has no LIMIT at
all, so every call aggregates the join across both tables.

The owner now rides on the row, indexed with `id`, so all three read it
directly. An entry cannot change owner -- a move out of the owner's tree is
refused -- so it is written once, with the row.

Set in SQL from `fsentry_id` rather than threaded through callers: four insert
sites, each of which already has the id, and none of which can now forget.
The risk in denormalising is a write path that does: a row with no owner
recorded is a share that silently stops appearing in its owner's listing. The
new test covers exactly that -- it fails when one insert is made to write NULL.

Verified against a real boot: a pre-existing share row, on a database that
genuinely predates the column, comes out with the owner its entry has.

* fix: the review's findings on this branch

**The Trash collision still had both halves open.** `overwrite` was checked
before the trash case, so a caller setting it -- it is wire-supplied -- could
still delete whatever it collided with in the owner's Trash, and the 409
survived for anyone who did not. The trash case is decided first now.

Dedupe alone did not close it either: a name that changes only when there is a
collision answers the same question the refusal did, and the mover sees that
name even though the path is masked. Trashing someone else's item renames it
every time, so the result carries no signal. `original_name` already records
the real name, which is what a restore reads. Two tests pinned the old name;
they say what it is now.

**The owner column had no fallback.** Reads went only through it, so a row
written by anything that predates it -- a pod mid-roll -- would never appear in
its owner's listing again. The three reads now also cover `IS NULL` rows
through the old join, indexed on the null and empty once none remain.

**The grant became conditional on a replica read.** `if (!already)` skipped it
on a stale row while the revokes below stayed unconditional, which could leave
the holder with nothing where the old early return left their access alone. It
grants unconditionally again; only the early return changed.

**The re-issue budget covered one path of four.** Team, link and invite
re-issues reserved nothing, so three ways round it were left open. They share
one `#reserveRemodeQuota` now.

Two docblocks had been pushed off the functions they describe, and two comments
still described the join that is gone.

* chore: one-line the comments added here

* test: measure the prefix walk by what it keys on, not by the clock

`checks many long paths without re-reading every prefix of each` gave
the walk 1500ms of wall clock. It measured nothing useful: the shape it
guards against — keying each step on the accumulated prefix text rather
than on the parent id — costs 29ms against the current 8ms on that
input, so both ends sit far under the bound. Under coverage the same
test ran 1511ms and failed, which is the only thing the number could
ever report on.

`PathPrefixIds` now counts the characters it keys on, and the walk is
asserted on that: proportional to the input, and twice the depth costs
twice as much rather than four times. Keying on the accumulated text
fails both. The original case keeps its real assertion, that a request
this size is refused.

* fix: keep the recorded entry owner current, and the trash guard on the retry

A move into someone else's tree re-owns the whole subtree, which the
migration comment claimed could not happen. The shares on it are
re-pointed at the new owner, or they drop out of the listing the column
keys and stay in the old owner's.

The unique-violation retry re-entered the collision resolver without the
trash flag, which is the one call site the guard was not threaded
through.
2026-10-09 16:14:23 -04:00
Daniel Salazar be78281d9d ci: shard backend tests and share workers across server-booting files (#4149)
The suite was CPU-bound on one runner, and about half of that CPU went to
re-importing the backend for every test file.

- Run the suite as 4 shards and merge their blob reports for coverage.
  `test (pr)` stays as the required check and fails if any shard fails.
- Drop the per-PR base leg; pushes to main publish the coverage PRs
  compare against.
- Files that boot a server without module mocks or extensions run in a
  `shared-worker` vitest project (`isolate: false`), so the backend is
  imported once per worker. testSharedWorkerSetup.ts resets the redis
  mock, configContainer and extensionStore between files.
2026-10-09 01:50:33 -07:00
Daniel Salazar 8a6929847d Auth limits, reborn-app tokens, seat addresses, drop-area encode (#4148)
* auth: per-account 2FA code limit, single-use otp-login tokens, recovery mail limits

- /login/otp and /login/recovery-code share a per-account bucket charged on
  wrong codes; an otp-login token completes one sign-in.
- /send-pass-recovery-email adds an IP bucket, a per-account cap and captcha,
  and reuses the outstanding recovery token instead of rotating it.
- Reauth tokens carry the rejection reason; /signup re-attaches a temp
  account only from an expired session.

* auth: app-issued access tokens older than their app stop authenticating

An access token whose row or parent session predates the app now holding its
uid is refused, the same check app and worker sessions already get.

* teams: signup can take an address a seat never confirmed

Password signup, save-account and a provider-verified OIDC sign-up take an
address a team seat holds unconfirmed; the seat's copy is cleared at the
write. A seat address that was confirmed still blocks.

* dev-center: encode dropped directory child names
2026-10-09 00:12:59 -07:00
Daniel Salazar 360f6b1079 fix(events): bill reached deliveries, bound dispatch reads, seal cursors (PUT-1900) (#4146)
* fix(events): bill reached deliveries only, bound dispatch reads, seal cursors

- A broadcast socket copy is billed only when a connection for its holder
  exists on this node, anywhere in this region (connection count) or, for a
  durable row, in a region presence names. Without peers it bills as before.
- Compiled match filters are held in a bounded LRU with a TTL, and dropped
  when another process reports a subscription ended.
- GET /events/fetch reads the continuation from `cursor`, falling back to
  `after`.
- Cursors from /events/fetch, /events/subscriptions and /events/kv-handles
  are encrypted with sealCursor/openCursor; plain cursors still read.
- A durable subscribe counts again after its insert and removes its own row
  when over the cap; the generation bump for that removal is published.
- Subscribe checks access before reading the anchor's owner, and an anchor
  gone in between answers with the subject the caller sent.
- Dispatch reads at most the rows per token routing can use, scanning large
  token hashes; a removal still settles every row on the removed anchor.
- puter.events.unsubscribe() stops routing after the server ends the
  subscription or reports it gone; fetch() accepts `cursor`.

* fix(events): forward kv values only to regions with rows that ask

- A region counts its session rows asking for kv values per token and
  announces the token's value variant (`v#<token>`) over the existing
  `watch` item on the first such row, withdrawing it with the last. Add,
  remove, reap, reanchor and refresh all keep it, reading `includeValue` off
  the row as it is dropped.
- A kv write reads the value variants alongside the watch index and attaches
  the value only to forwards bound for a region that announced one.

* fix(pagination): seal id-keyed cursors on every list endpoint

Apps, subdomains, workers, team (members, directory, teams, audit, own
audit), shares (inbound, outbound), the user-to-user permission audit and
readdir/descendant listings now hand out cursors sealed with sealCursor and
read them with openCursor, so a cursor no longer carries a row id. Plain
cursors issued before still read.

* test(events): pass socket ids to noteConnect in the billing tests
2026-10-09 00:08:13 -07:00
Daniel Salazar 8ecfeec8c3 fix(puter-js): request idle timeout, callback/listener leaks, guarded message handlers (PUT-1908) (#4143)
* fix(puter-js): free one-shot callbacks and listeners, guard message handlers

- setAuthToken starts the GUI cache refresh loop once instead of per call
- IPC reply callbacks are dropped after their reply; dehydrated callbacks stay
- AppConnection removes its window listener when the target app closes
- reply branches for app data and instance count drop their callback
- UI, Debug, tool and GUI-token message listeners no longer reject; an
  unknown or failing tool answers toolResponse with an error
- parseResponse handles a blob response with no content-type
- retry waits remove their abort listener when the wait ends
- EmailConfirmationDialog escapes its message
- EventListener uses a real Map and snapshots listeners per emit
- Context get/set/runWithContext route every known key the same way

* fix(puter-js): time out requests that make no progress

- buildXhr starts an idle clock on send, reset by state changes and body
  progress (upload progress only on requests already carrying a bearer)
- read-safe requests get 60 s while progress is observable, everything
  else 15 min; timeout: 0 on fetchUrl/driverCall/sendWithRetry turns it off
- a timed-out read replays once; writes and AI calls never replay
- timeouts reject with code request_timeout on fetchUrl (TypeError),
  driverCall, streams and the legacy XHR path
- document the limits in rate-limits-and-quotas and the chat stream note
2026-10-09 00:07:20 -07:00
Daniel Salazar ae234ffcc2 fix: only sign-in and picker popups hand the opener a token (#4147)
* fix: only sign-in and picker popups hand the opener a token

Popup actions are an allowlist now: sign-in popups (no action, sign-in,
login, signup) deliver at boot, pickers deliver with the pick, and
request-permission keeps its exchange. Any other action runs no exchange
and posts no token. The consent check in postAuthActions applies to every
action that delivers at boot instead of only no-action and sign-in.

* fix: a dismissed account picker hands the opener no token
2026-10-09 00:05:44 -07:00
Daniel Salazar 2ae2e093e7 keep archive entries inside the extract folder, encode template labels (#4145) 2026-10-08 23:59:33 -07:00
Daniel Salazar dcfb1263de fix(events): single-writer presence items, live-socket counts and region-wide leave grace (PUT-1759) (#4142)
* fix(events): keep each region's presence item written only by that region

- A region that receives a forward for a pair it holds no socket for
  retires its own presence item through the pin-ordered leave path,
  gated by a region-shared claim. The sender no longer retires another
  region's item, and the forward reply no longer carries noSocket.
- Presence reads skip items for regions outside the peer list instead
  of retiring them.
- removeConnection decrements and deletes the count in one script.
- A failed join releases the join pin, and any connect that finds the
  pin free writes the join.
- A single delivery treats a socket on any node of the region as local,
  using the region-wide connection count, and picks the next remote
  candidate by the regions already tried instead of by index.
- Correct the connection-count TTL comment.

* fix(events): count presence by live socket and honor the leave window region-wide

- A region's presence for a pair is a sorted set of socket ids scored by
  last renewal under a new key, replacing the integer count. A socket
  counts while renewed within the concurrency-slot window; add, renew and
  remove each prune lapsed members and report the live count in one
  script, so a node that dies stops holding the pair.
- The last socket going marks the pair as leaving region-wide for the
  grace window, in the same script; a connect clears it. A forward that
  lands on another node inside the window does not retire the item.
- Add the events.presence.write counter, labelled by op (join, leave,
  refresh, retire).
2026-10-08 23:55:55 -07:00
velzie 50e2af34bf feat(puter-js): Add custom streams and perfect negotiation to Peer (#4058)
* add perfect renegotiation

* more advanced peer logic

* add video tracks to peer api

* several fixes

* small fixes, revert e006b1977

* proper reconnect

* fixes

* fixes 2

* fixes 3

* fixes 4

* remove uneeded tests
2026-10-08 23:34:55 -07:00
Daniel Salazar 7adc03cb75 fix: team seats caps (#4141) 2026-10-08 17:50:45 -07:00
dependabot[bot] 807368f7b8 chore(deps): bump handlebars from 4.7.9 to 4.7.10 (#4137)
Bumps [handlebars](https://github.com/handlebars-lang/handlebars.js) from 4.7.9 to 4.7.10.
- [Release notes](https://github.com/handlebars-lang/handlebars.js/releases)
- [Changelog](https://github.com/handlebars-lang/handlebars.js/blob/v4.7.10/release-notes.md)
- [Commits](https://github.com/handlebars-lang/handlebars.js/compare/v4.7.9...v4.7.10)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-version: 4.7.10
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-08 17:20:25 -07:00
Daniel Salazar 4f3a524ace fix: credit usage gui (#4140) 2026-10-08 17:20:00 -07:00
Krish KumarandDaniel Salazar 95b6f86fcc fix(events): resolve relative fs: paths like fs operations do (#4085)
* fix(events): resolve relative fs: paths like fs operations do

* docs(events): use bare relative paths in examples, test both relative forms

* docs(events): clarify that relative fs: subjects come back resolved

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-08 15:23:53 -07:00
Juan Fernando Castro bb829885ef fix: a picker popup hands its opener a token once the user has picked (#4133)
* fix: a picker popup hands its opener a token once the user has picked

A picker opens because the site asked, not because the user agreed to
anything, so the hand-off now waits for the answer the popup was opened
to get. The exchange still runs at boot — the app row and host_app_uid
are what the dialog is opened against — but the token travels with
fileOpenPicked, directoryPicked or fileSaved. Cancelling sends nothing.

deliversTokenAtBoot is the one gate every boot-time hand-off reads, so
the plain exchange, the cross-origin-isolated /login/set path, temp-user
creation and manual signup all follow the same rule. A failed exchange
still reports and closes, having nothing to defer.

* fix: the review's findings on the picker hand-off

Double-clicking a file answers the opener from openItem.js, not from the
UIWindow button, so that pick returned the item and no token. The
deliverer is a global now and every path that answers an opener calls
it: both UIWindow handlers, openItem, and the save flow.

A failed exchange posted `{success: false, token: null}`, and the SDK's
handler sets that as readily as a real token, so a picker that failed
wiped whatever the site already held. The failure now goes only where a
boot-time token would have.
2026-10-08 15:11:47 -04:00
Juan Fernando Castro c55437245b fix: settle the url file-access gate from a grant held on a folder above (#4132)
* fix: settle the url file-access gate from a grant held on a folder above

The gate asks `/auth/check-permissions` with `app_uid`, which resolved
through the permission scan. `fs:` access inherits down a tree and only
ACL walks that, so a grant on a folder never answered for a file inside
it and the gate asked again, writing another per-file row on each Allow.

The `app_uid` branch now routes `fs:<uid>:<mode>` through
`ACLService.check` as the app-under-user actor, which is the same
authority the operation itself will face. Everything else keeps the
scan, including `puter.perms.holds`, which sends no `app_uid`.

* fix: the review's findings on the consent gate

A permission that is not a string reached `startsWith` and faulted the
whole request, where it used to answer false inside the catch; both
parsers now refuse a non-string, which also covers the other callers
that read a request's list.

The path spelling of a permission took the scan while the uid spelling
took ACL, so one file could answer two ways. Both spellings resolve to
their entry and take the same route.

The checks run together rather than one after another: the caller gives
up after five seconds and prompts, and a deep path is several reads.
2026-10-08 15:05:01 -04:00
Juan Fernando Castro d1d55a6bc3 SDK auth plumbing: cache scope, dedupe keys and the token listener (#4112)
* fix: pin the token sender, scope the grant flag, and stop two stragglers

Five of the SDK auth-plumbing items.

**The global `puter.token` handler pinned origin but not source.** Its three
siblings all pin both, and each says why: the GUI origin is shared by every
frame on that domain, so origin alone identifies a domain, not a sender.
`signIn` pins its popup and binds `msg_id`, the consent dialog pins its popup,
and the reauth wait pins the embedder. This one pinned the embedder only when
`env === 'app'`. Since `authenticateWithPuter` now hands off to `signIn`, which
adopts the token itself, the embedder is the only sender this handler is left
serving, so it pins in every environment.

**`is_grant_user_app_permission` was a request-wide flag held across an await.**
It tells the `app-root-dir` rewriter that a row is being written, and while it
is set that rewriter resolves to a real `fs:` permission instead of
`PERMISSION_FOR_NOTHING_IN_PARTICULAR`. Anything evaluating an `app-root-dir:`
permission beside the grant -- the grant route runs one under `Promise.all`
with fs work -- read the same flag. It now lives in a scope derived for that
call, so what the write needs to say about itself is not also said to its
neighbours. `runInDerivedContext` is new: `runWithContext` starts an empty
store, which would drop the `actor` the rewriter reads.

**`fs.stat` and `fs.readdir` deduped on parameters alone**, so two identities
in one realm could share a result. They now key on origin and token as
`os/user.js` does, which documents the reason.

**`signIn`'s relay poll never stopped.** Closing the popup settles the promise;
the loop kept asking `/login/wait` every second for the life of the page.

**`triggerReauth`'s five-minute timeout was never cleared**, so a successful
reauth left a timer to fire into a settled promise.

The concurrency test parks the rewrite and reads the flag while it is held --
my first attempt read before the flag was even set and passed either way.

* fix: scope the fs cache per identity, and retarget two paths that never ran

The remaining SDK auth-plumbing items.

**`puter._cache` outlived the identity that filled it.** `_clearAuthToken`
dropped the token and the stored session but left the cache and `whoami` as
they were, and its keys were raw paths -- `item:/alice/Documents` and a
`~`-relative key alike, identical whoever asked. A GUI user switch without a
reload could be served the previous user's stat or readdir. Keys now carry the
origin and token they were read as, built in one place so the writers, the
readers, the invalidations and the warm-up probes cannot drift apart, and
signing out drops the cache and `whoami` with the token.

**The driver permission prompt could not fire.** Both paths waited for a 200
carrying `{success:false, error:{code:'permission_denied'}}`. The driver route
answers `success: true` or throws, so a refusal arrives as a 403 -- the shape
the retry now reads. Its two tests synthesised the old envelope, so they passed
against a response the API does not send; they use the real one now, and the
prompt goes from never called to called once.

This is a behaviour change: a driver call refused for permissions will now
prompt, which is what the path was written to do. Happy to delete it instead if
the prompt is not wanted.

The half-written copy of the same idea in `utils.js` is gone -- it prompted for
one hard-coded interface and its retry was a `// todo`.

**The verification-gate single-flight was keyed on nothing**, so a phone-gated
request in flight beside an email-gated one took the email answer, replayed,
failed again and surfaced raw without ever asking. Keyed per gate: same gate
still shares one dialog, a different gate gets its own.

* test: ask the cache the same way the code does

The API suites built `item:<path>` keys by hand, so they stopped finding
anything once the real keys carried the identity they were read as. They go
through `fsCacheKey` now, which is the point: a test that spells the key itself
passes until the moment the spelling changes, and then fails for a reason that
has nothing to do with the behaviour it is guarding.

These run against a live server, so a plain `vitest run` does not reach them --
they were green locally and red in CI.

* fix: the review's findings on this branch

Eight, and three of them were regressions this branch introduced.

**Pinning the token sender to `globalThis.parent` in every environment broke
the case it was meant to protect.** On a top-level page `parent` is the page
itself, and the token arrives from a popup, so the handler stopped accepting
anything. The GUI posts `puter.token` to `window.opener` for the file pickers
too, not only for sign-in, and this handler is their only consumer -- a site
calling `showOpenFilePicker()` would have signed the user in, dropped the token
and run the following read with none. It now pins the embedder when framed and
a window this SDK opened when not; `UI.js` already tracked its picker popups
for the same reason, so that set is shared rather than duplicated.

**The GUI still spelled a cache key by hand.** `updateSubdomainsForItems` built
`item:<path>`, which no longer matches what the SDK writes, so its `get` could
not hit and its `set` wrote an orphan nothing reads and nothing evicts -- a
published website's badge quietly stopped appearing on a cached render. It goes
through `fsCacheKey` now, and that is the last hand-spelled key in the tree.

**Retargeting the driver permission prompt to 403 was wrong and is reverted to
the deletion the ticket asked for.** The route refuses with `forbidden`, not
`permission_denied`, so the retarget still missed it; meanwhile a driver that
raises `permission_denied` for its own reasons would have opened a prompt that
had nothing to do with the refusal. And the prompt asks for
`driver:<iface>:<method>` while the route checks `service:<driver>:ii:<iface>`,
so granting it changes nothing. The path cannot work as written.

The rest were pre-existing or latent:

`cacheWhoami_` wrote its answer without checking the token it was issued under
was still current, so a reauth landing mid-flight could repopulate `whoami`
with the previous identity. `_dropIdentityCaches` also left `whoamiCache_`,
which other call sites await.

`setAuthToken` now drops the identity caches when the token actually changes:
keys carry the token and the store is persisted with no TTL, so a rotation that
did not go through `_clearAuthToken` orphaned a whole key space with nothing to
reclaim it.

The verification-gate entry was registered after the body could already have
deleted it -- a synchronous throw stranded a resolved "not verified" in the map
for the life of the page. It registers first and clears on a microtask.

`signIn`'s poll is bounded as well as settle-checked: on a cross-origin
isolated page opened from a gesture, nothing watches the popup, so closing it
never settled the promise and the loop ran on exactly as before.

`runInDerivedContext` says in its docblock that callee writes do not survive;
only the one flag needs the scope, and a future caller should not expect to
hand a value back through it.

* fix: the review's findings on the SDK auth plumbing

**The relay bound stopped asking without saying so.** The deadline left the
`signIn` promise pending and its listener attached. On an isolated page opened
from a gesture the relay is the only thing that can settle it -- nothing watches
the popup there -- so a user who took longer than the deadline was left with a
promise that never resolved, and `authenticateWithPuter` queues every later
implicit-auth call behind it. It rejects now.

**Pinning the token source left `signIn`'s own popup untracked.** Only the
picker popups registered, so on a top-level page the global handler bailed on a
sign-in it had opened itself: `puter.onAuth` never fired, and
`puterAuthState.authGranted` was never set. `openAuthPopup` is the one place
auth popups are opened, so they register there.

**A closed popup is ours too.** Rejecting one that had already closed loses a
message from a popup that posts and then closes itself, which is why `UI.js`
tolerates the same case. Kept, with the set bounded instead.

**The gate key carried the factors**, so one route naming them and another not
opened two dialogs for the same gate. It keys on the gate, as `ctx.done` does.

**`fsCacheKey` left `~` unresolved**, so a writer and an invalidation spelling
the same place differently missed each other -- the drift the one key builder
was meant to end.

The dead `opts.permission` went with the path that used to read it.

Four workerd-runner failures are left: `ai > chat stream adapter supports
cancellation`, `kv > a GUI boot key written by another client reads fresh` and
two cache-warmer ones. All four fail the same way on `origin/main`, and the node
and browser runners pass.

* chore: one-line the comments added here

* chore: reflow the comments the removed retry cause left ragged
2026-10-08 15:04:49 -04:00
Michael Lip 4ea93e6a73 fix(i18n): add GUI keys that rendered as raw key names (#4074)
The 2FA spinner showed "verifying", the files selection bar showed a
"done" button, and two login and signup errors showed their key names,
because i18n() returns the key itself when en.js has no entry for it.
The login error now uses the existing login_email_username_required key,
which already has translations. A test checks that every literal i18n()
key in the GUI exists in en.js.
2026-10-08 11:42:07 -07:00
Daniel Salazar 80736de0f1 test(events): wait for the recorded failure, not any future lease score (#4127)
The in-flight claim's lease (2x the invoke timeout) is also a future score, so
polling heldForMs > 0 could pass before the failed attempt's hold was written:
the backoff test read 600 instead of 2000, and the 404/hang/429/suspended-owner
tests ended while their attempt was still in flight, letting it land in the
next test. Wait on the subscription's failure counter instead, which only moves
after the hold or gap marker is written.
2026-10-08 11:39:06 -07:00
dependabot[bot] 085ab1774f chore(deps): bump shell-quote from 1.10.0 to 1.12.0 in /src/puter-js (#4120)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.10.0 to 1.12.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.10.0...v1.12.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.12.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-08 11:32:57 -07:00
dependabot[bot] 505fd13d97 chore(deps): bump shell-quote from 1.9.0 to 1.12.0 (#4121)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.9.0 to 1.12.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.9.0...v1.12.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.12.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-08 11:30:52 -07:00
Reynaldi Chernando 47a1be7f70 Rename "Add Existing Account" menu item to "Add Another Account" (#4124) 2026-10-08 11:25:52 -07:00
Filip Kujundžić 38c7191009 Merge pull request #4135 from HeyPuter/feat/claude-haiku-5-5 2026-10-08 19:56:01 +02:00
404oops b6f31e3b02 feat(ai): add Claude Haiku 5.5 2026-10-08 19:44:17 +02:00
Filip Kujundžić 3c852cf9cf Merge pull request #4134 from HeyPuter/feat/mistral-large-4-sonnet-pricing
feat(ai): add Mistral Large 4 and update model pricing
2026-10-08 19:37:53 +02:00
404oops 0afe8ccfdd docs(ai): keep model update documentation focused 2026-10-08 19:29:36 +02:00
404oops 266390d23d feat(ai): add Mistral Large 4 and update model pricing 2026-10-08 19:26:06 +02:00
Filip Kujundžić b6150d63f3 Merge pull request #4090 from HeyPuter/fix/ai-stream-cancellation
fix(ai): propagate cancellation and stream termination failures
2026-10-08 13:33:50 +02:00
404oops 4e8277ea4b docs(ai): clarify streaming errors and stopping streams 2026-10-08 13:21:38 +02:00
Daniel Salazar b6dfc6f903 fix: harden card verification (#4126) 2026-10-07 23:11:49 -07:00
dependabot[bot] 3d7deaa403 chore(deps): bump smtp-server from 3.19.13 to 3.19.17 (#4083)
Bumps [smtp-server](https://github.com/nodemailer/smtp-server) from 3.19.13 to 3.19.17.
- [Release notes](https://github.com/nodemailer/smtp-server/releases)
- [Changelog](https://github.com/nodemailer/smtp-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/smtp-server/compare/v3.19.13...v3.19.17)

---
updated-dependencies:
- dependency-name: smtp-server
  dependency-version: 3.19.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 22:36:57 -07:00
Daniel Salazar 199d72cbdf fix(events): refuse scoped access tokens on durable events routes (PUT-2048) (#4105)
Durable subscribe, list, fetch and unsubscribe now apply the same scoped
token check as the handler, worker and kv-handle routes:
`isAccessTokenActor(actor) && !isAccountContext(actor)`. An access token
an app issued (effectiveApp = the app) is treated the same as one the
account issued: subscribe answers 403 `events_durable_requires_account`,
list and fetch return an empty page, unsubscribe answers 404.

`rowInActorScope` returns false for these tokens, which covers durable
unsubscribe and ack. The session verbs that also use it already refuse
access tokens at the socket handshake.

Full-access tokens, app tokens and sessions are unchanged.
2026-10-07 22:35:09 -07:00
Daniel Salazar f234bd477c fix(auth): answer a username lost to a concurrent write with the taken-name 400 (#4115)
The duplicate-username check in the signup placeholder claim, change-username
and save-account runs before the user-row write, so a concurrent write can
take the name in between and the unique index rejects ours. Each path now
re-reads the holder from the primary on a unique violation and returns the
400 its pre-check returns for a taken name, instead of a 500. In all three,
the home-directory rename, rename budget charge and username-changed event
run after the user-row write, so nothing has happened yet when it fails.
2026-10-07 22:34:31 -07:00
Daniel Salazar 2638e9ce93 fix(kv): batch-read GUI boot keys only when the SDK runs as the GUI (PUT-1899) (#4117)
The boot-key batch now starts only when puter.env is 'gui'. In every other
environment a get() of one of those names reads that key on its own, like
any other key, instead of fetching all 14 and serving them for 4 s.
2026-10-07 22:34:22 -07:00
dependabot[bot] accf33a50e chore(deps-dev): bump source-map-js from 1.2.1 to 1.2.2 (#4123)
Bumps [source-map-js](https://github.com/7rulnik/source-map-js) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/7rulnik/source-map-js/releases)
- [Changelog](https://github.com/7rulnik/source-map-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/7rulnik/source-map-js/compare/v1.2.1...v1.2.2)

---
updated-dependencies:
- dependency-name: source-map-js
  dependency-version: 1.2.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 17:14:30 -07:00
dependabot[bot] ab175b0fb4 chore(deps): bump proxy-addr from 2.0.7 to 2.0.8 (#4122)
Bumps [proxy-addr](https://github.com/jshttp/proxy-addr) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/jshttp/proxy-addr/releases)
- [Changelog](https://github.com/jshttp/proxy-addr/blob/master/HISTORY.md)
- [Commits](https://github.com/jshttp/proxy-addr/compare/v2.0.7...v2.0.8)

---
updated-dependencies:
- dependency-name: proxy-addr
  dependency-version: 2.0.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 17:13:23 -07:00
dependabot[bot] 3fcbd895a7 chore(deps): bump sharp from 0.35.4 to 0.35.5 (#4081)
Bumps [sharp](https://github.com/lovell/sharp) from 0.35.4 to 0.35.5.
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.35.4...v0.35.5)

---
updated-dependencies:
- dependency-name: sharp
  dependency-version: 0.35.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 16:12:01 -07:00
dependabot[bot] 4c9e4d8e79 chore(deps): bump mysql2 from 3.24.4 to 3.24.5 (#4082)
Bumps [mysql2](https://github.com/sidorares/node-mysql2) from 3.24.4 to 3.24.5.
- [Release notes](https://github.com/sidorares/node-mysql2/releases)
- [Changelog](https://github.com/sidorares/node-mysql2/blob/master/Changelog.md)
- [Commits](https://github.com/sidorares/node-mysql2/compare/v3.24.4...v3.24.5)

---
updated-dependencies:
- dependency-name: mysql2
  dependency-version: 3.24.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 16:11:46 -07:00
Juan Fernando Castro cf574e4b34 Merge pull request #4110 from HeyPuter/juancastro/icloud-dot-significance
Treat dots as significant on Apple's mail domains
2026-10-07 17:11:59 -04:00
Daniel Salazar 2e275a8f6b fix: godmode installation (#4116)
* fix: godmode installation

* fix: godmode classifier

* fix: vitest
2026-10-07 13:16:10 -07:00
Nariman Jelveh b5bd917119 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-10-07 12:39:47 -07:00
Nariman Jelveh 3999dd062f Update RecommendedAppsService.ts 2026-10-07 12:39:34 -07:00
Daniel Salazar 3e234d7329 fix: 402 error messaging (#4114) 2026-10-07 12:34:45 -07:00
Daniel Salazar f674e51b4d fix: answer lost write races with 4xx, and time health-check timeouts against loop stalls (#4107)
- signup: a username taken by a concurrent signup between the duplicate
  check and the insert now returns the pre-check's 400 instead of
  surfacing the unique-key error as a 500. The holder is read from the
  primary.
- fs move: when the UPDATE trips the (parent_id, name) unique key after
  the collision probe missed the occupant, re-read it from the primary and
  resolve it the same way the probe would have (overwrite, dedupe, or the
  409 item_with_same_name_exists), then retry the move once.
- server health: a check that hits the per-check timeout reports how long
  the event loop was blocked during it, like the database-liveness latency
  failure already does.
2026-10-07 12:02:05 -07:00
Daniel Salazar d395a3aa94 fix(kv): page through flush, cap per-call batch sizes, validate decr and del (#4106)
- flush follows LastEvaluatedKey until the namespace is empty and rejects
  when a batch delete fails; cached reads for every key a page returned are
  still invalidated and the flushed event still fires.
- decr validates the amount map before negating it, so it accepts exactly
  what incr accepts.
- del checks the key like every other store method (oversized key is a 400).
- list offset emulation treats a missing continuation key as end of data
  instead of restarting from the first key.
- Array get maps results through a Map instead of a linear find per key.
- The driver gates includeTotal on credit like value listings.
- New driver caps in drivers/kv/limits.ts: 1,000 keys per array get, 1,000
  items per batchPut (both rejected past the cap), list limit lowered to
  1,000. Published in rate-limits-and-quotas.md and the set/list pages.
2026-10-07 12:00:50 -07:00
Daniel Salazar fde323842e fix(kv): SDK list rejections, boot-cache read-after-write, byte-accurate size checks (PUT-1899) (#4108)
- puter.kv.list() no longer throws synchronously for invalid options; it
  returns a rejected promise, or for stream: true an iterator whose first
  next() rejects. Error codes unchanged.
- Writes through puter.kv (set, del, incr, decr, add, remove, update,
  expire, expireAt, flush) stop the boot-key batch from serving the keys
  they touch, so the next get() reads the store. Untouched boot keys are
  still served from the batch.
- Client-side key/value size checks measure UTF-8 bytes, and values are
  measured as their JSON encoding, matching the store. Same codes and
  messages.
2026-10-07 11:48:45 -07:00
Daniel Salazar 4a54beac6b fix(fs): SDK uid stat cache, readdir page types, move lookup errors, UID-shaped paths (#4109)
- stat: only path-addressed results are cached; uid-addressed stats no
  longer share one cache key.
- readdir: overloads declare the page result for recursive + limit/offset
  and for the positional page/stream forms.
- move: only a not-found destination lookup means "new path"; other
  lookup failures reject. A uid destination skips the lookup.
- getAbsolutePathForApp always resolves paths. read/copy/move/delete use
  getAbsolutePathOrUidForApp, which keeps sending UID-shaped strings as uids.
- Docs for read, copy, move, delete and readdir.
2026-10-07 11:46:22 -07:00
Juan Castro 0555ea6479 Merge branch 'main' into juancastro/icloud-dot-significance 2026-10-07 11:01:12 -04:00
Juan Fernando Castro 27eebc7300 Merge pull request #4086 from HeyPuter/juancastro/permissions-access-control-cleanup
Permissions and access-control cleanup
2026-10-07 10:57:41 -04:00
Juan Castro 4b5f509066 fix: only rewrite rows the old rule actually wrote, and fix the invites too
Two findings from review, both of which contradict what the last commit
claimed.

**The backfill could merge keys, not only split them.** I asserted it "can only
split rows apart, so it cannot collide". Not true, and reproducible: a row
predating the `clean_email` column has a NULL there, so its index key is
`lower(email)` -- filling it in folds `+tag`, and `[email protected]` lands on a
key `[email protected]` already owns. A legacy row holding an un-lowercased
`clean_email` collides the same way once lowercased. Either throws, and the two
engines that re-run every file on every boot would then refuse to start until
someone operated on the database by hand.

Both updates now fire only where the stored value is exactly what the old rule
produced and differs from the correct one. That makes "splits only" true: the
sole change is dots coming back. NULLs and legacy values are left alone.

The same predicate makes them write-idempotent. Postgres has no no-op-update
suppression, so without it every boot rewrote a tuple per Apple row forever.

**The pending-invite residue misdelivers rather than just going missing.**
`claimPendingShares` looks the row up by the canonical address, so whoever
confirms `[email protected]` claims the invites addressed to
`[email protected]` -- someone else's file, handed over on confirmation. I had
written this off in the PR as unfindable invites that "expire on their own";
`share` has no expiry column and the lookup has no time predicate, so they are
claimable indefinitely.

Fixed here rather than later: the typed address is kept as `invitedAddress`
whenever it differed from the canonical, which is exactly the collapsed rows,
so the invites recompute from it under the same guard.

Verified by seeding a database at the old version with the two rows that used
to abort the migration, and booting: it reaches version 87, repairs the
collapsed row, and leaves the NULL and legacy rows untouched. Replaying the
statements changes zero rows.
2026-10-07 10:23:30 -04:00
Juan Castro ece8ad5f1d fix: register the Apple-domain backfill with the sqlite runner
The sqlite migration list is hand-maintained, so the file alone did nothing —
a fresh boot still reported 87 files and stopped at version 86. MySQL and
Postgres read their directories, so only this one needed the entry.

Caught by booting a local instance rather than by the suite: the client tests
pin `CURRENT_SCHEMA_VERSION`, which fails once the entry exists but says
nothing while the file is simply unregistered.

Verified against a seeded database at the old version. `[email protected]`
went from `[email protected]` to `[email protected]`, `[email protected]`
from `[email protected]` to `[email protected]` — subaddressing still folded — and a gmail row
was left alone.
2026-10-07 09:45:26 -04:00
Juan Castro cf817b0f0a fix: treat dots as significant on Apple's mail domains
`cleanEmail()` applied gmail's `dots_dont_matter` rule to icloud.com, me.com
and mac.com. Apple allocates the exact string: `[email protected]` and
`[email protected]` are two mailboxes with two owners. Collapsing them made one
canonical address stand for both.

What that canonical address feeds:

- `findEmailOwner` falls back to it, so an OIDC sign-in whose verified claim is
  the dotted address resolves to the account holding the undotted one, and
  `linkProviderToUser` has nothing left to object to -- the provider verified a
  real mailbox and the account's address is confirmed.
- The unique index on COALESCE(clean_email, lower(email)) reads it, so the
  rightful owner of a dotted Apple address is refused signup because the
  undotted account already "owns" it.
- Pending share invites are stored and claimed on it.

Subaddressing is untouched: Apple does fold `+tag`, so that rule stays. Gmail
keeps both rules -- there the fold is correct.

A migration recomputes `clean_email` for those domains on all three engines,
since rows written before this still hold the collapsed value and the index
reads the stored one. Verified the SQL against the function rather than by
inspection: same five inputs through `cleanEmail()` and through sqlite give
byte-identical output. It can only split rows apart, never merge them, so it
cannot collide with the unique index.

Not changed: `findUserByEmail` still matches on the canonical address. The
cross-match it documents is correct where the provider really does fold --
`[email protected]` is the same inbox as `[email protected]`. Requiring an
exact match would break that for no benefit here; the fix is for the data to
say what the provider does.
2026-10-07 09:17:03 -04:00
Juan Castro 6efb96d560 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-07 09:00:40 -04:00
Juan Castro 5596e70f25 Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-07 09:00:18 -04:00
Daniel Salazar e477bca3eb test(events): wait for the failure hold, not any future lease score
The in-flight claim's lease (2x the invoke timeout) is also a future score,
so polling for "held" could read it before the failed attempt's backoff
hold was written. Wait for deferAfterFailure to settle instead.
2026-10-06 22:32:02 -07:00
Daniel Salazar 11e48c68a9 fix: verification for sms (#4101) 2026-10-06 22:03:07 -07:00
Daniel Salazar 464f758a97 fix: verification for sms (#4100) 2026-10-06 22:00:14 -07:00
Daniel Salazar fadee6fac5 fix: verification for sms (#4099) 2026-10-06 21:59:58 -07:00
Daniel Salazar 6bce256589 refactor(ai): trim redundant cancellation checks
- Chat driver runs each attempt through one helper and checks the abort
  signal only on entry and in the attempt catch blocks, where it keeps a
  cancelled request from marking the route unhealthy.
- A stream ended after cancellation still emits ai.prompt.cost-calculated;
  only ai.prompt.complete is skipped.
- pipeNdjsonStream owns its line buffer and onEnd is optional; drop the
  errored guards that can no longer fire.
- Drop the Claude post-loop abort check (finalMessage already throws).
- Skip provider error logs when the caller cancelled.
- Simplify the usage_limited cap condition.
2026-10-06 19:49:25 -07:00
Daniel Salazar d2cdde5147 fix: verification for sms (#4098) 2026-10-06 19:36:18 -07:00
Daniel Salazar 50ee0818ea feat: trial cleanup (#4097) 2026-10-06 18:20:18 -07:00
Juan Castro ad1be70d06 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-06 19:44:37 -04:00
Juan Castro 6277f106c4 Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-06 19:44:19 -04:00
Juan Castro 738a8058ef fix: close the path oracle where it actually was
Local testing against a running instance found the service-side fix was not
enough. Both answers were 403, but they did not read the same:

  exists  -> permission_denied: fs:/other/rd:read
  missing -> Cannot create outside your home directory   (forbidden)

The second comes from `#planFsPathCreate` in the controller, which probes the
path itself and only calls the create guard once it knows the path is absent.
Its comment said so outright -- checked "only once the path is confirmed
missing, so an existing shared path outside the caller's home is not a new
rejection" -- which is the oracle stated as an intention.

A foreign path now refuses word for word whichever it is. Responses are
byte-identical apart from the caller's own input.

`rewritePermissionForActor` collapses the create guard's `forbidden` the same
way it already collapsed the 404, for the paths that reach it through the
service rather than the controller.

The existing test pinned `forbidden` here; its subject is that nothing is
created in someone else's tree, which still holds. Alongside it is the
invariant itself: refuse a present and an absent foreign path, mask the path,
compare the whole error. Without the controller check that fails.
2026-10-06 19:41:55 -04:00
Juan Fernando Castro 0e2f9663b2 Merge pull request #4091 from HeyPuter/juancastro/teams-cleanup
Teams cleanup: pagination, resumable delete, and credentials that stop leaving by mail
2026-10-06 19:27:09 -04:00
Juan Castro b8fcbffe3f fix: the five remaining review findings
**The path oracle was not closed.** The rewrite only remapped the *error*, so
success-vs-403 still answered whether a foreign path exists. Narrower than
reported: every other grant path already gates on `canManagePermission`, and
only `grantUserAppPermission` has no authority check -- deliberately, since
granting your own app something you don't hold is inert. It now refuses a
foreign path the actor cannot see, with the same error a missing one gives.

My first test for this passed without the fix: `makeUserActor` creates no
fsentries, so both paths 404'd and nothing distinguished them. It inserts a
real entry now, and the control fails with "expected null to match 403" --
null being the grant that succeeded, which is the oracle.

**`#validateEmail` still failed open** on the main signup / change-email /
confirm path while its OIDC twin was hardened. Same shape as OIDC now: a gate
that did not run is not a pass.

**The 16-permission cap broke `puter.perms.check()`** for longer lists. The
SDK's own docblock promised "one round trip however many are asked about",
which the cap made untrue, so it batches instead. The reported second half --
the UI grant poller burning its timeout -- is not reachable:
`MAX_REQUESTED_PERMISSIONS` already caps that path at the same 16.

**`listener_failed` was set by wildcard listeners.** An unrelated observer on
`email.*` throwing would refuse every signup. Only an exact-key listener
implements the event, so only that one speaks for the gate.

**`whoami()` labelled everything `Unauthorized`.** A 502 with an HTML body is
not a 401; the server's `code` now rides along and the fallback names the
status.
2026-10-06 19:23:40 -04:00
Juan Castro 755b58fd49 fix: don't hand the owner's subtree size to a see-only caller
Switching `getSubtreeSize` to `entry.userId` fixed a recipient reading 0 for a
shared folder, but `/stat` is gated on `see` and `return_size` is caller-
supplied. `see` is the weakest level -- the folder is visible, what is under it
is not -- so the sum now covered entries the caller had no way to enumerate.

It is answered only for a caller who passes a `list` check, which is where the
contents become enumerable anyway. Everyone below that gets the directory row's
own size, which is what they got before this branch.

`#assertAccess` grew a non-throwing sibling for this; the descriptor both need
moves into `#aclDescriptor`.

Test grants `see`, asserts 0, grants `list`, asserts the real size. Without the
gate the `see` case reads the file's bytes.
2026-10-06 19:08:46 -04:00
Juan Castro bd5aa901ab perf: batch the per-page reads in a team delete
Measured first, against a real server, counting DB round trips: a 40-seat
delete was 284 queries, 7.1 per seat, scaling linearly. Four of those seven
answered for the whole page and had no reason to be asked per account.

  120x SELECT * FROM `user` WHERE `id` = ?
   41x INSERT INTO `audit_team_membership`
   40x SELECT COALESCE(SUM(size), 0) FROM fsentries
   40x UPDATE `user` SET `suspended` = ?
   40x SELECT `uuid` FROM `sessions` WHERE `user_id` = ?

The page now reads who is already suspended, what each holds, and the rows the
notices need in one query each, and writes the audit as one statement. What is
genuinely per account -- the suspension, its billing event, the mail -- stays
per account under the same concurrency bound.

Same 40-seat delete is now 170 queries, 4.3 per seat.

Three new store methods rather than raw SQL at the call site:

`getSuspendedByIds` reads the primary, deliberately not `getByIds`, which is
cache-first. A stale `suspended` is exactly what re-charges a seat on a retry,
which is the thing this path was just fixed to stop doing.

`getHeldBytesForUsers` is `getHeldBytes` with a GROUP BY.

`appendAuditMany` is `appendAudit` as one multi-row insert, still written
before any suspension, so a failed append cannot leave an unlogged one.

What is left is concentrated in `#suspend`: the UPDATE, the forced re-read the
user store does afterwards to refresh its cache, and the session drop. Those
are shared store paths and batching them means owning their cache invalidation,
so they are not in scope here.
2026-10-06 18:55:53 -04:00
Juan Castro b84fc6c158 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-06 18:26:31 -04:00
Juan Castro edb0c2a712 Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-06 18:13:27 -04:00
Juan Castro 55e29666d0 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-06 18:13:02 -04:00
Juan Castro 66774ff90c Merge branch 'main' into juancastro/permissions-access-control-cleanup 2026-10-06 18:12:39 -04:00
Juan Castro 0fbb6def5a Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-06 18:12:25 -04:00
Juan Fernando Castro d9ee207ed0 Merge pull request #4092 from HeyPuter/juancastro/storable-email-gate
Gate an email address once, before it is stored
2026-10-06 18:10:19 -04:00
Juan Castro 0495ea0f9d fix: close the holes the review found in the storage gate
Four, all real.

The gate threw on a non-string. `validator.isEmail` asserts its argument, and
two call sites hand it an unvalidated body value -- `send-pass-recovery-email`
is public, so `{"email": 1}` was a 500 rather than a 400. The check belongs in
the gate, not in each caller, which is the whole premise of having one.

OIDC signup never shape-checked the claim at all. `claims.email` went from a
truthiness check straight into `user.create`, so a provider we do not run could
store exactly the string this branch exists to keep out. `email_verified`
attests the mailbox, not the characters.

The change-email confirm handler promoted the staged column without re-reading
it through anything. A row staged before this deploys still lands on `email`;
one check at promotion closes that window.

The invite path kept its own regex, and it was never the same rule. It admitted
`[email protected]`, `.[email protected]`, `[email protected]`, `a;[email protected]` and `a,[email protected]` -- the last of
which nodemailer reads as two recipients -- each one a permanent pending row
that spent daily quota, which is the failure the comment above the constant
said it existed to prevent. It uses the gate now, and the second regex is gone.

A regression test each, all four confirmed to fail without their fix.
2026-10-06 17:20:35 -04:00
Juan Castro 73698be964 fix: one gate for an address we are about to store
`validator.isEmail` accepts a quoted local part, so an address whose local part
is an arbitrary string passes it. Five call sites relied on that check alone
before writing the address to `user.email`, and the invite path's own shape
regex was equally permissive.

`isStorableEmail` is now the single gate: the library check, plus a refusal of
`<`, `>` and `"`, which are legal in an address only inside the quoted form.
Nothing deliverable uses that form, so the rule costs no real address --
apostrophes and ampersands in a local part still pass, and the suite covers
them.

One function rather than a check each caller has to remember, because the next
place that stores an address will not remember.
2026-10-06 16:47:15 -04:00
Juan Castro 25f132b28d fix: address code review on the teams cleanup
Six findings, all real.

`total` described rows the page withheld. The count ran over every live team the
user belongs to, while the app-actor filter ran afterwards on the items only, so
`GET /teams?includeTotal=true` from an app returned `items: []` with a non-zero
`total` -- the exact thing the filter exists to prevent, and newly reachable
because `includeTotal` was inert before this branch. The directory-open
restriction moves into the query, so items, cursor and total all describe the
same set.

The GUI told owners the opposite of what the server now does: "we email the
username and temporary password to it." An owner who believed it would dismiss
the one copy of the password and strand the seat. Rewritten.

`runWithConcurrencyLimit` rejects on the first error but cannot cancel the
workers beside it, so a failed seat let up to seven others finish suspending and
emailing "your team has been closed" for a team that was still live. The page
now settles before the rejection is rethrown, which leaves a state the retry can
resume from instead of an arbitrary one. My earlier claim that failure
semantics were unchanged was wrong.

`#disableSeatForDelete` returned `true` on both paths, so the boolean read as if
it distinguished them. It returns nothing; the counter uses the settled results.

`listTeamsForUser` had picked up a second stacked docblock -- the same defect
this branch removes from `#cached` one file over.

`reissueCredential` sent the provisioning mail, which after the credential
change was a byte-identical duplicate saying nothing about the password it had
just invalidated, while the GUI promises the member hears about it. It has its
own template now.

Regression tests for the count leak and the reissue notice; both confirmed to
fail without their fix.
2026-10-06 16:22:48 -04:00
Juan Castro 36a22b05a2 docs: stop claiming the directory discloses what /members does
Both docblocks on the directory route said it carries nothing a colleague
cannot already read through `/members`. It carries `uuid`, and `/members`
withholds `uuid` from everyone but the owner, with its own comment pointing
apps at the directory to get one.

The disclosure itself is the opt-in working as designed: `directory_enabled` is
the owner's consent to expose a roster, every member reaches the route the same
way whether they call it from a session or through an app, and the SDK and the
teams recipes already document the directory as where a stable id comes from.
So the comments are what was wrong, not the route.

Removing `uuid` from the directory would be the other way to make them agree,
but it breaks `listDirectory()` for every caller and contradicts the published
recipes; that wants a decision, not a cleanup commit.
2026-10-06 16:02:13 -04:00
Juan Castro 9d8e37dc6d fix: stop an unconfirmed claim on an address from blocking invites to it
`#resolveRecipient` already refuses to treat an unconfirmed address as an
identity, and says why: "resolving it would hand the share to whoever registered
the address first." `#invite` then looked the same address up again and honoured
whatever that row said about blocking -- `blocksAllShares`, or a block on this
particular sender.

So registering an address you do not own, and never confirming it, was enough to
refuse every invite anyone sends to that mailbox. Nothing about that row is
evidence the mailbox is yours, which is the premise resolution already works
from. A team seat reaches this cheaply: those accounts are provisioned with
`requires_email_confirmation: false` and an address the owner typed.

The block is now honoured only for a row that confirmed the address. The alias
reasoning behind the lookup is unchanged -- a confirmed holder still cannot be
reached through a variant they did not type.

Not addressed here: the same unconfirmed seat still holds the address against
the real owner's signup, because `idx_user_owned_email` keys on having a
password rather than on confirmation. Narrowing that is a schema change and
wants its own ticket.
2026-10-06 16:00:32 -04:00
Juan Castro 6826071156 fix: stop mailing a plaintext team credential to an unconfirmed address
Provisioning a seat lets the owner supply any `email` that passes a format
check. Puter then mailed a working temporary password to it. Nothing confirms
the owner controls that address -- provisioned accounts are created with
`requires_email_confirmation: false`, because the team is the trust anchor for
the account, not for the mailbox. `resendActivation` reissues the credential and
mails it again, up to the 20/day `team:reset-password` budget, so the same
address can be hit repeatedly.

The credential now stays out of the mail on both paths, which is what
`resetMemberPassword` in the same file already does. Nothing is lost: all three
routes return `temporary_password` to the owner, and the SDK has always
documented that as the delivery -- "The returned password is shown once and is
not retrievable afterwards -- deliver it out of band." The mail still goes out,
naming the account and saying the team will supply the password.

The two tests that pinned the old shape are inverted rather than deleted; one of
them carried the rationale "without it this is the only copy", which the return
value contradicts.
2026-10-06 15:56:33 -04:00
Juan Castro d896250ae2 fix: make team deletion resumable, and stop it serializing its mail
`deleteTeam` tore seats down one at a time, each one awaiting a mail send, so a
full 40-seat team was 40 round trips end to end inside a single request. The
per-seat work is independent, so it now runs under the same bounded-concurrency
helper the rest of the backend uses for fan-out sends.

The loop also had no equivalent of the guard `disableMember` carries. That one
checks `suspended` before emitting, with the reason written down: a second
`team.account.disabled` opens a second byte charge that only one `enabled` will
ever close. Deleting a team emitted unconditionally, so any partial failure
followed by a retry double-charged every seat the first attempt had already
disabled. The seat teardown moves into one method both paths can state the same
way, and a retry now resumes instead of re-charging.

Failure semantics are unchanged: a seat that throws still aborts the delete and
leaves the team live, which is what makes the retry meaningful.

Touching this file reformats a few untouched spans -- it carried ten
pre-existing `prettier/prettier` errors that the repo's own pre-commit `eslint
--fix` resolves on any commit that stages it.
2026-10-06 15:56:10 -04:00
Juan Castro 665314535e docs: say what TeamStore actually caches
The cache docblock claimed `isMember` and `getByUid` were left uncached "for
that reason". Both go through it -- `getByUid` directly, `isMember` through
`getMembership` -- so the comment told a reader the opposite of the code, on the
two reads that decide authorization.

What makes it safe is the busting, not an exemption: every mutation here drops
the keys it touches, and the TTL is the backstop for the one path that does not
come through this store, the ON DELETE CASCADE on the membership row. No stale
window is constructible today; the comment now says why.

Also drops a one-line docblock stranded above it, describing a method that is no
longer in the file.
2026-10-06 15:55:40 -04:00
Juan Castro f3b05ca822 fix: page GET /teams like every other team listing
`listTeamsForUser` returned every row and the controller never read `req.query`,
so `limit`, `cursor` and `includeTotal` were accepted and ignored on the one
team route that did not paginate. The docs promise a capped page there, and
every sibling listing already honours it.

Keyset-paginated on `id` per doc/pagination.md, with the envelope the other
routes return. The app-actor directory filter still runs over the page, so an
app sees the same teams it saw before, just a page at a time.
2026-10-06 15:55:22 -04:00
404oops 96f0c51d1e fix(ai): propagate cancellation and stream termination failures 2026-10-06 20:24:19 +02:00
Filip Kujundžić a1eca46a80 Merge pull request #4087 from HeyPuter/fix/responses-sampling-capabilities
fix(ai): omit unsupported sampling options in Responses requests
2026-10-06 20:22:16 +02:00
Daniel Salazar 484c498859 fix: error cleanup (#4089)
* fix: message

* fix: error cleanup
2026-10-06 11:16:04 -07:00
Reynaldi Chernando 71399c5939 sync limits ai, fs, events, networking (#4088) 2026-10-06 23:26:53 +07:00
404oops b1da48b738 fix(ai): omit unsupported Responses sampling options 2026-10-06 18:14:13 +02:00
Filip Kujundžić 9a9972b026 Merge pull request #4079 from HeyPuter/DS/danielsalazar/anthropic-route-fidelity
feat(ai): carry the full Anthropic and OpenAI wire shapes through the chat driver
2026-10-06 18:12:04 +02:00
Juan Castro 4629b544e6 docs: say why AppDriver.select lists apps it does not own
Reads as the same enumeration surface `/query/app` refuses to be, so write down
that it is not: `/query/app` answers for apps the caller names, which is why it
must not confirm one it may not see. `select` is the listing a launcher reads,
and a listing that hid every app but your own would not be one. Protected apps
are still filtered by the permission check, and `user-can-edit` narrows it to
the caller's own.

A test already pins the behaviour by name; this is the reason behind it.
2026-10-06 11:35:20 -04:00
Juan Castro cbb8d708b4 fix: scope the rewrite collapse to fs paths
The collapse caught every `subject_does_not_exist` the rewriters raise, not just
the path resolver's. `app-root-dir:` 404s when the caller's own app has no
resolvable root directory, and that answer is theirs to have — four of its tests
said so.

Only a permission that parses as an `fs:` path can oracle someone else's tree,
so only that one collapses now.
2026-10-06 11:33:27 -04:00
Juan Castro 95809c7dd5 fix: bound the permission check, and let whoami fail
`/auth/check-permissions` took an array of any length. `checkMany` batches the
scan so it is not the fan-out the report describes, but nothing stopped a caller
sending thousands of strings; it now takes the same 16 the grant and revoke
routes take.

`whoami()` returned `resp.json()` without looking at the status, so an expired
token resolved to an error body typed as a `User`. Callers read `username` off
it as undefined — which is how a consent prompt came to ask for
`/undefined/Documents`. It throws the documented `{ status, message }` shape
now, the same one the no-token path already threw.
2026-10-06 11:17:43 -04:00
Juan Castro ac5fa20388 fix: keep the other account out of the OIDC link conflict
Linking a provider identity already bound elsewhere answered 409 with the raw
provider subject and the other account's internal id in the message. The caller
needs to know there is a conflict, not whose it is; both stay in the log for
whoever is debugging it.

Status and legacy code are unchanged — OIDCService branches on the 409.
2026-10-06 11:14:19 -04:00
Juan Castro 3389d3f8a1 fix: treat an abuse hook that could not run as a refusal
The signup paths read `allow` off a payload they initialise to `true`, so a
harness listener that throws leaves the verdict untouched and the signup goes
through. The outer try/catch the report points at is not where it happens:
`emitAndWait` already swallows each listener's error and carries on, so that
catch almost never fires and the throw never reaches it.

`emitAndWait` now records the failure on the metadata it was handed, and both
signup paths — OIDC and the ordinary one — refuse when it is set. Other emitters
are unaffected: the flag is only read where somebody is asking for a verdict.

The report named OIDC alone; the ordinary signup had the same hole, and it is
the path that matters more.

Test boots a server, registers a listener that throws, and posts a signup: 200
before, 403 now.
2026-10-06 11:13:18 -04:00
Juan Castro 39099e2056 fix: size a shared folder from its owner, and keep sub-scopes when widening
Two unrelated bugs in the same area.

`stat({ returnSize: true })` summed `fsentries` filtered on the *acting* user,
so a recipient asking for the size of a folder shared with them matched no rows
and read 0. The sum is over the owner's rows; the ACL check that let them stat
it is what decides whether they may ask at all.

`fs-access-levels` destructured the mode out of the permission but kept the
`.slice(1)` that used to skip it, so widening `fs:<id>:see:thumbnail` produced
`fs:<id>:read` and dropped the sub-scope. Anything granted below the mode stopped
being implied by a wider one. Latent — nothing ships a sub-scope today — and the
test fails without the fix.
2026-10-06 11:05:52 -04:00
Juan Castro acd6ea3c75 fix: do not spend a consent prompt on an app root dir that is not the caller's
`perms.request('appRootDir', { app: <someone else's> })` could never succeed —
only the app itself may claim its root dir — but the SDK read the refusal as
"not granted yet", prompted the user, took the grant, then polled until it gave
up and resolved undefined. A prompt the user cannot act on, and a live grant for
a directory the app will never reach.

The route now says which kind of refusal it is. `forbidden` alone could not
carry it: the SDK already treats that as the refusal a grant is about to lift,
and a unit test pins exactly that sequence, so a new `reason: 'app_mismatch'`
field rides alongside instead of changing the code clients read.

An entry refused that way is held back from the pooled grant and resolves
undefined without a prompt, which is what it resolved to before.
2026-10-06 10:55:19 -04:00
Juan Castro d6c606fb21 fix: keep the owner's path out of the ghost-file 404
A share recipient addresses an entry as /<owner>/<uuid>/<name>, which is the
point of the mask: it says where the item is without naming the folders above
it. The 404 raised when a file's backing object has gone put `entry.path` in the
error fields, so reading or copying a shared file whose object had vanished
handed back the real path.

The fields carry the uuid alone now, on both the read and the copy site. The
caller already has it — it is what they named the entry by.

The existing ghost test asserted the path was present, so it is the regression
pin; the copy one was not checking fields at all and now does.
2026-10-06 10:33:59 -04:00
Juan Castro 8c7372adc6 fix: stop the path rewrite confirming entries the caller cannot see
`fs:/path:mode` resolves through `getEntryByPath` with no ACL check, and a
missing entry answers 404 while an existing one carries on to the authorization
check and answers 403. The pair is a membership test for any path on the
deployment: grant routes take the permission as given, so anyone could ask
whether /someone-else/taxes.pdf is there.

The resolver keeps its 404 for a path inside the caller's own home, where
knowing it is missing is the useful answer and reveals nothing. Everywhere else
it answers with the same `permission_denied` the authorization check gives, so
the two cases are one case. ACLService already collapses them; this is the same
rule reaching the grant and revoke paths, which take an actor and so can tell
whose tree is being named.

Internal resolution still goes through `rewritePermission`: only the paths that
take a caller-supplied permission moved over.

The test grants on an existing foreign path and a missing one and asserts the
outcomes are identical; without the fix they are 403 and 404.
2026-10-06 10:30:58 -04:00
Juan Fernando CastroandReynaldi Chernando 3711d1e837 docs: medium teams recipe (PUT-1976) (#3976)
* docs: add the medium teams recipe

Built around what an app actually does with a team: an assignee picker
keyed on uuid, paging a large directory, scoping stored data per team,
sharing with everyone at once, and telling the three empty cases apart.
One playground example per section.

* docs: split the teams recipe into the two use cases it covered

The medium teams recipe carried six sections, which is the wrong shape for a
recipe: an agent reading llms.txt to answer "add team sharing" had to find it
inside a page called "Work With a Team", next to paging and team switching.

Two recipes, one use case each:

  share-a-file-with-a-team   the whole team, or one teammate via the directory
  show-a-team-owner-panel    gating an admin view on isOwner

Dropped rather than moved: paging and the empty/closed-directory cases, both
already in Add Team Support, and per-team data namespacing, which assumes
multiple teams we do not support yet.

Two corrections while rewriting. A team share names the team in `holder_team`;
`holder` is a username, so the old example printed the wrong field and told the
reader it meant "no one person". And `share()` takes a username for a person,
never a uuid — the picker now shares by username and keeps the uuid for what it
stores, which is what the rename advice was always about.

The old page also linked to /recipes/teams-read-membership/, which does not
exist; the slug is add-team-support.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-06 20:31:27 +07:00
Daniel Salazar 957784bdb8 fix(ai): keep existing chat callers on their previous behavior
Narrows the wire-shape changes to the callers that need them.

- request-level upstream 4xx stays strict only on the Anthropic route;
  /drivers/call and the OpenAI routes fall back on any error again
- outage chains return 400 with the old codes; the Anthropic error
  renderer maps them to 502/529 from the carried upstream status
- native Claude results report finish_reason 'stop' again (real reason
  in stopReason); OpenAI routes still map the real value, and a stream
  that carried tool calls never ends as a plain stop
- Claude usage carries thinking_tokens again, informational only and
  added after pricing and the funds cap
- reasoning_effort is clamped to values each OpenAI model accepts;
  Hoonify, MiniMax, ZAI and Meta only receive params they support
- OpenAI-form tool_choice from /drivers/call is normalized again
- /responses accepts background:false; mcp_servers is ignored; legacy
  media block shapes pass the Anthropic route's validation
- OpenAI web search holds one call; only search actions are billed;
  Azure bills Bing grounding requests; usd_cents includes search fees
2026-10-06 15:22:53 +02:00
Neal ShahandReynaldi Chernando dc029b43a8 Worker and Subdomain Recipes (#4075)
* Worker recipes medium

* subdomain recipes medium

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-06 20:09:38 +07:00
Daniel Salazar 985c76c9c3 docs: medium kv recipes (PUT-1972) (#4041)
* docs: medium kv recipes (PUT-1972)

Adds three recipes:
- Store a Set of Unique Values: an object as a set via update/remove,
  quoted paths for values with dots, like button, recent searches
- Run Code Only Once: incr as an atomic claim, once per day, quotas,
  sequential numbers, webhook dedupe in a worker
- Keep a Version History: incr version numbers, batch set with expireAt,
  reverse list paging, restore, keep last N, delete by prefix

* docs: more medium kv recipes (PUT-1972)

Adds six recipes:
- Keep an Index in Sync: batch set for record + index keys, moving keys
  on change, delete order, per-group counts vs includeTotal, search as
  you type, rebuild and stale-entry cleanup
- Share Data Between Apps: appData scopes, appUuid on every call,
  disableSharing private entries and the set-without-flag gotcha
- Save User Settings: defaults merge, nested paths, reset with remove,
  form binding, sets inside settings, renaming a setting
- Split a Large Value Across Keys: size-based read/write cost, shards
  numbered by incr, latest/older/all reads, expiring old shards
- Export or Delete Data: stream export, filtered batch import, delete by
  prefix, delete before a date, flush
- Build a Shopping Cart: quoted SKUs, incr quantities, TTL via update,
  hide-at-zero, server-side prices at checkout

Also adds a high score section to Add Counters, links Query a Collection
to the index recipe, and replaces its stale newest-first advice with
list({ reverse: true }).

* docs: simplify wording in medium kv recipes

Plainer prose throughout: shorter sentences, no bold or step-by-step
scaffolding, fewer defined terms. Code is unchanged apart from comments.
2026-10-06 19:10:27 +07:00
Daniel Salazar f21cd51edf fix(ai): retry a fast-mode 429 at standard speed on Claude
Fast mode has its own quota. Its 429 used to count as a route failure,
so the request was replayed on a reseller and the Claude route for that
model was marked unhealthy. The provider now retries once without speed.
2026-10-05 20:54:01 -07:00
Daniel Salazar c2fc45a8d1 fix(ai): keep the driver's token count off the /drivers/call surface
count_tokens is only for the Anthropic route, so the driver exposes it
under a symbol key that the RPC resolver never lists.
2026-10-05 16:56:48 -07:00
Juan Castro 614c4c9c2f test: pin the fs delegate path the escalation was reported on
The generic test covers the defect's shape; this one is the case the report
describes. `fs-access-levels` puts `manage:fs:<id>` in the higher set for
`fs:<id>:read`, so a delegate holding the manage terminates the issuer scan with
it as the leaf — which is how the old warm came to write a manage permission
under a holder granted nothing but read.

Without the fix the holder ends up with `manage:fs:<id>` in the flat view and
`canManagePermission` answers true for a read-only share, so they can re-share
the entry at any mode.
2026-10-05 18:19:25 -04:00
Juan Castro ac740984a3 fix: keep the flat permission view to the row it stands for
Two ways the flat KV view could say more than the SQL behind it.

On a miss the warm flattened the whole traversal and wrote every leaf under the
holder. Those leaves belong to the issuer: a holder granted one mode of a
resource by someone who holds the resource itself had the issuer's wider
permission written under their own key, and it answers later checks. The warm
now writes the holder's own row and nothing else, which is what the grant path
has always written.

Deleting an account cascaded the grants it issued out of SQL, but the flat
entries are keyed on the holder, so nothing removed them and they are trusted
without an issuer check. The access outlived the account, and the owner could no
longer see it to withdraw it. The refs are read before the row goes and cleared
after it, through the same batched delete a revoke uses, so peer regions hear
about it once.

Both fail without the fix: the warm test sees the issuer's wider permission
written under the holder, and the delete test reads the grant back from an
account that no longer exists.
2026-10-05 17:51:31 -04:00
Juan Fernando CastroandReynaldi Chernando 63eabf8c7c docs: medium fs recipe (PUT-1968) (#3975)
* docs: add the medium fs recipe

readdir paging, streaming, sorting and recursion, plus the write and
mkdir options that decide what happens to an existing name. One
playground example per section.

* docs: ground the medium fs recipe and examples in the UI that lists

A file browser with infinite scroll and a count footer, a gallery sort
header, an upload endpoint deciding what to do with a repeated filename,
and a report generator writing into a folder tree that does not exist.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-05 21:38:10 +07:00
Juan Fernando CastroandReynaldi Chernando 445588f857 docs: medium perms recipe (PUT-1975) (#3974)
* docs: add the medium perms recipe

Batch requests behind one prompt, appData scopes, raw permission
strings with create, and the entries rejected before any dialog.
One playground example per section.

* docs: ground the medium perms recipe and examples in the screens that ask

A first-run setup, a settings page that renders its real state, a
companion widget reading another app's data, and an importer that wants
folders created on approval.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-05 20:22:09 +07:00
Daniel Salazar 53ca2b61e5 fix(ai): review fixes for the wire-shape PR 2026-10-05 04:09:08 -07:00
Daniel Salazar a242e31343 feat(ai): carry the full Anthropic and OpenAI wire shapes through the chat driver
The Anthropic-compatible route flattened requests into the OpenAI-style
driver shape and rebuilt them for Claude, which dropped system blocks,
thinking, tool_choice, cache_control, betas and stop reasons. Claude
Code requests 400ed upstream and were silently served by a reseller.

- normalized driver args, stream chunks and results gain tool choice,
  thinking, output format, stop sequences, caching, stop reasons, usage
  details, reasoning signatures, server-tool blocks and safeguard results
- ClaudeProvider builds requests from those fields with a beta, tool and
  body-field allowlist; meters fast mode, web search and advisor calls,
  and no longer bills thinking tokens twice
- fallback only on route-level failures; outages map to 502/529
- vendor error envelopes on the Anthropic and OpenAI routes
- Anthropic route: native SSE with pings, count_tokens, /v1/models
- OpenAI-family providers and routes map the same fields; Responses web
  search is metered; reasoning_effort gating fixed; /responses rejects
  org-scoped stateful params and forces store:false
- puter.js: typedefs, new chat options, silent toString for new chunks
2026-10-05 03:35:54 -07:00
Daniel Salazar ba081e2332 feat(ddb): log the table and key of throttled DynamoDB requests (#4078)
A client middleware inside the SDK retry loop logs every throttled attempt
(including ones that succeed on retry) with its operation, table, key,
client region and DynamoDB's throttling reason, plus batch items returned
unprocessed. Each target logs at most once a minute with a count of the
throttles folded into it. Puts log only the key attributes, using key
schemas registered by the KV store at startup.
2026-10-05 02:01:38 -07:00
Juan Fernando CastroandDaniel Salazar a46eb15d5e Let the account's own API token reach the routes a privileged app needs (#4032)
* feat: let the account's own API token reach the routes it should

A privileged app is launched with the GUI session token today. Running one on a
full-access personal access token instead costs much less when it leaks, but
several routes those apps depend on refuse every access token, so the swap would
break them.

Admitted, each a read or a write the token's own HTTP reach already covers:

  - the realtime socket handshake, which refused access tokens outright; FS
    live updates and notifications stop without it
  - GET /get-dev-profile and POST /profile, both puter.js surface
  - GET /auth/list-permissions — a read of what was granted, not a grant
  - GET /share/shared-by-me/apps and GET /share/audit, over grants
    /share/shared-by-me already lists for the same credential
  - POST /rao, which puter.js calls on every setAuthToken

Gated on `isAccountContext`, not the `full_access` claim alone, so a token with
an app anywhere in its chain is still refused and a socket for one never joins
the user room.

Left session-only: mint, session, 2FA, grant, team and billing, plus two the
list named. /share/blocks is a personal safety control whose only caller is the
desktop's Blocked Senders window. /app-feedback is reachable only from our own
GUI pages, and being unsubmittable programmatically on a user's behalf is the
property it was built with.

The new HTTP suite drives a real server with a minted token and fails on all six
admissions without the change; the block routes' existing metadata check already
pins their refusal.

* test: sort the two token shapes at the events handshake

* fix: carry the account-context invariant into the HTTP gate

`requireUserActorGate` admitted on the raw `full_access` claim while the socket
and /rao use `isAccountContext`, so one shape — a token carrying the claim with
an app in its chain, or an actor built without `makeActor` — would have been
refused in two places and admitted in the third. Unreachable today, since the
mint and the auth-time read both drop the claim when `app_uid` is present, but
it is the invariant this change states and it should hold at every enforcement
point.

The payout address goes back behind a plain session: `/get-dev-profile` is open
to the account's own token now, and the rest of what it answers is a developer's
own name and incentive-program status.

Also drops a cross-reference in EventsService to the socket posture this change
relaxed, and rewords the block-routes comment, which read as if access tokens
were admitted there rather than refused.

* fix: drop a revoked token's sockets, and read the dev-profile columns

Two gaps the review found, both reachable only because this branch lets the
account's own token hold a socket and call /get-dev-profile.

`#revokeAccessTokenTail` soft-revoked the session row and told no one, so a
revoked token kept its connection — and the account's `outer.gui.*` fan with it
— until the five-minute reauth sweep. Sockets now join a per-token room and a
new `auth.access-token.revoked` event drops exactly that room, so the account's
other tabs stay up. Session revoke keeps its account-wide eviction.

`/get-dev-profile` read `first_name`, `last_name`, `paypal` and the two
incentive flags off the user row, which carries none of them: the columns have
always been `dev_`-prefixed, so the endpoint answered nulls to everyone and
`puter.apps.getDeveloperProfile()` has never returned anything. Reads the real
columns, keeping the unprefixed name as a fallback for a deployment carrying
both. The payout address stays behind a plain session, so opening the route to
the account's token hands over a name and incentive status and nothing else.

Both fail without the fix: the socket test waits five seconds on a connection
that should already be gone, and the profile test reads back a seeded row.

* fix: close /open_item to tokens, and stop the node runner leaking sockets

`/open_item` carried `allowFullAccessToken`, but it grants an app a permission
on the user's behalf and mints the app token to go with it — a grant and a mint,
which stay session-only. Pre-existing, and missed by the first sweep here, which
only asked whether routes lacking the flag should gain it and never re-read the
ones that already had it. Refused at the top of the handler too, so a credential
that gets past the gate cannot leave an ACL row behind.

The node runner loads each SDK into a vm context and never closed it, so every
test leaked a socket, and every socket holds a per-(user, origin) connection
slot. Admitting the account's own token to the handshake pushed that past the
cap, and the whole events block failed with `events_connection_failed` while the
browser and workerd runners — which do not leak — passed. Closing the FS socket
and the events channel after each test gives the slots back.

* fix: announce only the revokes that can have a socket to drop

Every access-token revoke broadcast an eviction cluster-wide, including the
read-URL tokens `revokeReadUrl` retires constantly. A scoped token is refused at
the handshake, so none of those broadcasts could ever reach a connection.

The announcement now goes out only for a token the handshake would have
admitted: full access, no app in the chain. `revokeOwnAccessToken` refuses
full-access tokens outright, so that whole path is silent; a revoke by raw uuid
says nothing about the token and still announces, which costs a no-op broadcast
rather than leaving a revoked socket up.

Fails without it: the test revokes a scoped token and a full-access one through
the same entry point and counts the announcements.

* test: pin the mint route against the account's own token

`/auth/create-access-token` carries only `requireAuth`, so it never showed up
in the sweep over `requireUserActor` routes — the refusal lives in
`AuthService.createAccessToken`, which turns away any access-token actor
outright. That holds today, and a leaked full-access token minting itself a
sibling that survives revoking the original is exactly what the token swap is
meant to rule out, so it is worth a test rather than a reading.

* fix: let the recommended-apps test follow the list it tests

Two commits on main rewrote `RECOMMENDED_APP_NAMES` without touching the test,
which seeds `editor` and `camera` and asserts the result contains them. Neither
is on the list any more, so the resolved set came back empty and `test (base)`
has failed on every PR since.

The test now seeds the first two names off the list itself and asserts on those,
so the next edit to it cannot strand the test: prepending two unknown apps keeps
all four cases green.

* feat: let an extension waive a plan gate for one route or driver method

A plan gate about to refuse a caller now emits
`subscription.gate.<surface>` through `emitAndWait` first:
`route.<method>.<path>` for a route, `driver.<iface>.<method>` for a driver
method, `share.anyone` for link shares. The payload carries the actor, the
request and the requirement, and a listener that sets `allow` waives the plan
for that call. Callers the plan already covers never reach the hook, and a
listener that throws leaves the refusal in place.

* feat: mint godmode apps their own full-access token

`/auth/get-user-app-token` answers a desktop launch of a godmode app (the
`godmode` column) with a full-access token instead of an app token. Origin
lookups, the sign-in popups for pages outside the desktop, keep getting
ordinary app tokens, and only a web session may mint one.

The token carries the app in `godmode_app_uid`, read into
`accessToken.godmodeApp` for attribution only: `effectiveApp` stays null, so
it keeps the account's reach and passes the same gates as a personal access
token, including the vendor-compatible AI routes. It is refused once the app
is gone or no longer godmode.

Its row is an `access_token` session parented to the web session that asked,
with `app_uid` set, so signing out, revoking that session or revoking all
sessions takes it along, and the sessions list shows it under the app. Each
mint re-signs onto the same row with a 12h `exp` and moves the row's expiry
out, so the desktop keeps an open app alive while a copy taken earlier still
expires on schedule.

A full-access token may now mint scoped tokens (read URLs), parented to its
own row; it still may not mint full access. Revoking a full-access token
cascades to the tokens it minted.

An access token whose session row has a parent now authenticates only while
that parent is live, so a read URL a godmode app minted lapses with the app's
token, and through it with the desktop session.

* feat: launch godmode apps on their own token and renew it

The desktop asks for a godmode app's token at launch and blocks the launch if
it can't get one, as it does for other apps; a godmode app never runs on the
desktop's session token. While the window is open the desktop re-mints the
token once less than 6h of its 12h remain, checking on a timer and when the
tab becomes visible, and answers the app's `reauth_required` with a fresh one,
posted to the app's origin.

puter.js keeps a godmode token in sessionStorage like a session token, never
in localStorage. In app mode a 401 on a godmode token, background requests
included, asks the desktop for a new token and replays, and a `puter.token`
message is only taken from the embedding frame.

The sessions list shows a godmode token row with its app's title and icon.

* fix: keep a godmode token the same across renewals

Apps hand their launch token to code that never sees a replacement (the
terminal's Drive mount, a CLI's API key), so a token re-signed every renewal
broke those copies once the first one expired. The JWT now carries no `exp`
and a renewal moves only its row's expiry, so every copy keeps working while
the desktop keeps the app alive.

What still bounds it:
- only the desktop's web session can extend it; using the token never does,
  and signing out or revoking that session revokes it;
- it lapses 12h after the desktop last asked, and a lapsed row is never
  revived: the next launch gets a new token;
- however often it is renewed, a token is replaced after 7 days, and the old
  one is revoked on the spot along with the tokens it minted and its sockets.

The desktop no longer re-posts a renewal that kept the same token, so the
app's sockets aren't rebuilt every few hours; an app that reports its token
stopped working is always answered.

* fix: reserve the feedback and no-reply mailbox names again

Signup checks names against `util/reservedUsernames`, but `fbl`, `noreply` and
`no-reply` were added to a second copy of the list in AuthController that
nothing reads any more, so they became claimable. Move them into the live list
and drop the copy.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-05 01:10:29 -07:00
Neal Shah 970092f0a0 PUT-1995 (#4020) 2026-10-05 03:26:54 -04:00
Neal Shah 6dc61b2d67 PUT-2011 (#4077) 2026-10-05 03:23:14 -04:00
Neal Shah 7976dc9a29 attach egress actor to /down (#4076) 2026-10-05 03:09:16 -04:00
Daniel Salazar a76800d693 fix(thumbnails): duplicate a copied thumbnail with GetObject + PutObject instead of CopyObject (#4072)
* fix(thumbnails): duplicate a copied thumbnail with GetObject + PutObject instead of CopyObject

CopyObject names its source as `<bucket>/<key>` in a header, which skips
any path in the client's endpoint, so it can miss an object that
Bucket/Key requests reach. The copy handler now reads the source object
(still behind the HeadObject size bound) and writes it under the copy's
key with its content type.

* fix(thumbnails): check the GetObject length before buffering a copied thumbnail

The size check runs on a HEAD and the read on a separate GET, so an object
replaced in between was read whole into memory. Reject a GET response whose
length exceeds the cap and destroy its body before reading.
2026-10-04 21:11:00 -07:00
Daniel Salazar 1f1e5674d8 chore: bump puter js ver (#4073) 2026-10-04 17:17:38 -07:00
Daniel Salazar 8db4fb4f5b fix(ai): chat credit gate ceiling, in-flight holds and disconnects (#4071)
* docs(agents): no credits or bug-report mentions in commits and PRs

* fix(ai): chat credit gate ceiling, in-flight holds and disconnects

- Cap output at min(max_tokens, context - prompt). The gate used
  max_tokens - prompt, so any prompt longer than a model's output limit
  was refused with 402 regardless of balance. Gemini 2.5 Pro's output
  limit corrected to 65,536.
- Abort the upstream stream when the caller disconnects (driver calls and
  the OpenAI/Anthropic-compatible routes), which ends the credit hold.
- A request refused only because the account's own in-flight requests
  hold its balance gets 429 `credits_reserved` instead of 402; puter.js
  retries it with exponential backoff (1s to 30s, about 2 minutes).
- When the balance caps a completion and it uses the whole cap, the
  response carries `usage_limited` (non-stream `metadata`, stream `usage`
  line), which puter.js already turns into the upgrade prompt.

* perf(ai): read the balance and its holds once per credit gate

The gate re-read holds, and the whole balance when any were held, to tell
spent from committed. getUsageHeadroom returns both halves from the same
reads getRemainingUsage already did.
2026-10-04 17:10:51 -07:00
Daniel Salazar e7c89b0639 fix(apps): keep ids longer than an app uid out of the app-stats query (#4070)
ClickHouse's app_opens.app_uid is FixedString(40) and fails the whole
query on a longer value. Marketplace callers pass item ids into
getAppsStats, so one long listing-request id turned the seller's
listing-request list into a 500. Such ids can't have opens; they now
resolve to zero without being queried.
2026-10-04 17:03:19 -07:00
Daniel Salazar 83d64ac685 fix: follow-ups from the reviews of #4047, #4049, #4051, #4053, #4055 and #4057 (#4068)
* fix(auth): issue reauth tokens only for browser sessions

A rejected token's 401 carried a signed reauth_token, which /signup turns
into a session on a temp account. Any revoked app or access token, a stale
token from a deleted app whose uid was reused, or a worker credential could
get one. Only a GUI/browser session's rejection now carries a reauth_token;
everything else still gets reauth_required without it. The stale-app check
also runs before the session is touched.

* fix(events): refuse worker creation from an events handler

workers.create minted a worker token with no handler depth, so a handler
could deploy a worker and chain writes through it at depth 0. A handler
actor can no longer create a worker; delivery, deploys and hot reload mint
from plain owner actors and are unaffected.

* fix(ai-ocr): bound the page count's parsing and close its under-counts

- Object-stream headers are capped (/N, /First) and read incrementally, and
  the inflate budget scales with the file, so a crafted PDF can't spend
  seconds and gigabytes before the credit check.
- An escaped /ObjStm type, an indirect /Count, untyped page leaves and an
  object redefined to a smaller tree no longer lower the count.
- An account with no balance is refused before the file is parsed.

* fix(fs): add a switch to stop accepting pre-binding signed URLs

Signed URLs issued before owner binding are still accepted, and owner-signed
ones never expire, so the move-to-another-tree attack stays open for them
indefinitely. legacy_file_signatures (default on) can now turn them off, a
rate-limited warning counts their use so it's visible when they've died
out, and a non-UUID uid is refused before the entry lookup.

* fix(fs): hide the issuer's home uid from a scoped token's entries

A token scoped below the home could still learn the home's uid as the
parent_uid of a direct child (e.g. stat on ~/Documents). For access-token
actors that can't list the home, a direct child's parent uid is now null in
the v2 and legacy entry shapes, using the same rule as the root listing.

* fix(gui): ask for the 2FA code on the last step, right before enabling

The setup flow verified a code before the recovery-codes screen and sent
that same code to enable afterwards; by then it had usually expired, and
the user was sent back to the start. The code is now entered last and goes
straight to enable; a wrong code keeps the user on that step, and other
failures show the server's message. Enter submits the password step.

* test(fs): expect the legacy-signature flag in the signing config

* fix: apply the review's follow-ups to the merged-PR fixes

- Reauth tokens also require the session row to be a browser ('web')
  session, not just the absence of the worker claim.
- The OCR page count only treats an untyped object as a page when it sits in
  a page tree's /Kids, so outline items, widgets and name-tree leaves can't
  inflate an honest PDF's count.
- The signed-URL uid guard checks the uuid's shape only, so a legacy row
  whose uuid has nonstandard version/variant bits still resolves.
2026-10-04 14:46:21 -07:00
Daniel Salazar 5e34258da5 fix: drop database_id from the subdomain reclaim; MySQL has no such column (#4067)
reclaimByUuid (#4064) cleared `database_id`, which exists only in the SQLite
and Postgres schemas. On MySQL the UPDATE fails with "Unknown column", so
ensureSystemSite throws: the puter-profiles row stays registered to the
admin, the site keeps 404ing, and every profile save returns 500 as it
retries the bootstrap. puter-app-icons takes the same path if its row isn't
already protected. System rows never carry a database binding, so there is
nothing to clear.
2026-10-04 02:45:03 -07:00
Daniel Salazar 51eba174b3 fix: alert errors from timeouts (#4066) 2026-10-04 02:44:40 -07:00
Daniel Salazar f533eb84ae fix(ai-ocr): check credits for the whole document before calling the provider (#4055)
* fix(ai-ocr): check credits for the whole document before calling the provider

The pre-flight checked one page's cost because the page count was only
known from the provider's answer, so a 130-page PDF passed on a balance
worth a fraction of it and drove the account negative.

The driver now estimates pages before the call: a PDF's own count (read
from its page tree, including compressed object streams), capped by a
`pages` selection; one page for images and Textract, whose synchronous
API reads single-page documents. Other documents and PDFs it can't read
count 20 pages per MB, Mistral's 1,000-page limit at its 50 MB size cap.
The estimated cost is checked and held while the provider runs, so
concurrent calls see it. Usage is still metered from the pages the
provider reports.

* fix(ai-ocr): price the credit hold at the AI cost factor

Billing scales OCR usage by the ai.cost.factor hook, but the hold was sized
at the raw cost, so a balance covering N pages passed the check and was
then charged more. Hold through reserveAiCredits like the other AI drivers.
2026-10-04 00:53:19 -07:00
Nikhil Jangid 3dfc70cab7 fix: clarify team directory app access (#4031) 2026-10-04 00:51:02 -07:00
Daniel Salazar ebf5086e1c fix(fs): keep restore metadata out of reach of anyone but the tree's owner (PUT-2050) (#4063)
Follow-up to #4048. original_path/original_name decide where a GUI restore
lands, and the owner's restore is unchecked, so whoever sets them can aim a
file at another app's AppData root and block that app's launch. #4048 only
pinned them on move for non-account actors; a write could still set them,
and a share recipient's session counts as account context.

Writes now drop the trash keys for every caller. On move, only the account
that owns both source and destination may set them; anyone else moving into
the owner's Trash gets them recomputed from the entry, and other moves drop
them, including keys already on the row. Copies drop them on every branch.
The GUI only offers Restore for direct children of Trash, since nested rows
never carry legitimate trash metadata. The AppData/Trash checks for
cross-app deletes and create grants now ignore case, like the root guard.
2026-10-04 00:48:48 -07:00
Daniel Salazar 59e28bed23 fix(metering): keep a failed lookup's fallback out of the credit cache (PUT-2002) (#4062)
Follow-up to #4052. A failed subscription lookup was cached for 5 s, but the
credit cache built from its fallback plan lived 15 s, so a paid user who
had spent past the free allowance got 402s for ~15 s after one failed read.
Credit entries built from a provisional answer (a failed lookup, or an
actor missing its email or numeric id) are now capped at that answer's
expiry and don't drive credit-state alerts.

A failed lookup that finishes after a concurrent successful one no longer
replaces the good cached answer, actors without a numeric id aren't cached
(resolvers key on it), and a failed credit refresh fails open briefly
instead of replaying a stale answer or blocking on the failing store.
2026-10-04 00:48:35 -07:00
Daniel Salazar 73ebe3f47a fix(metering): decide app-usage access in MeteringService only (PUT-2015) (#4061)
The /metering/usage/:app handler repeated the "an app reads only its own
usage" check while the service still let any app read the global bucket.
The service now refuses an app actor anything but its own app, and the
handler only resolves names. Status codes for app actors are unchanged; a
user reading os-global gets their global usage instead of a name-lookup 404.

whoami no longer sets taskbar_items to undefined for app actors, and only
keys with a value count as built, so a listener can't fill one in for an
app. getMonthlyUsage's JSDoc notes allowanceInfo is account-wide.
2026-10-04 00:47:37 -07:00
Daniel Salazar c981a51f21 fix(fs): bind signed file URLs to the owner they were issued under (#4053)
New /sign signatures are an HMAC over uid, action, expiry and the entry's
owner at signing time. /file and /writeFile resolve the entry first and
verify against its current owner, so a move that hands the entry to another
account ends its URLs and stops /file minting child URLs from them. The URL
shape is unchanged; signatures in the previous format still verify, without
the owner check.
2026-10-04 00:47:16 -07:00
Daniel Salazar 4ef5e6c110 fix(socket): hold unlisted free plans to the free connection cap (PUT-1830) (#4060)
The per-account socket cap looked a plan up directly in its map, so
org_seat_free (team seats on a free team) got the paid cap. Route and
events limits already gave an unlisted free plan the default free entry;
that rule now lives in one helper, subscriptionOverride, used by all three.

Also publishes the socket connection caps, corrects the seat-plan line in
rate-limits-and-quotas.md, and makes the isFreeSubscription test assert
literal ids.
2026-10-04 00:46:59 -07:00
Daniel Salazar 794469845d fix(thumbnails): bind thumbnail keys to their entry and size-bound uploads (PUT-1999) (#4054) 2026-10-04 00:46:35 -07:00
Daniel Salazar 9ca77be5a9 fix(auth): require a step-up to set up 2FA and a valid code to enable it (#4051)
Setup returned a fresh secret and recovery codes to any session, and enable
flipped otp_enabled without proof of the secret, so a stolen session could
enroll its own authenticator and lock the owner out. Setup now lives at
/user-protected/setup-2fa behind the same password / OIDC revalidation gate as
disable-2fa, and enable requires a live code for the stored secret. The GUI
setup window asks for the password (or opens the revalidation popup) first,
and sends the code it already collected to enable.
2026-10-04 00:26:28 -07:00
Daniel Salazar 007d5391a1 fix(events): carry handler depth onto tokens a handler mints (#4049)
An events handler could mint a fresh token for its own app through /sign or
/auth/create-access-token, and writes made with it started a new chain. An app
token minted from a handler token now carries its handler_depth and expires
with it; an access token carries the depth at the lifetime asked for.
2026-10-04 00:25:54 -07:00
Daniel Salazar c0f4fe93b1 fix(auth): stop a deleted app's sessions authenticating as the next app with its uid (PUT-2049) (#4057) 2026-10-04 00:24:55 -07:00
Daniel Salazar d57f638eda fix: keep system sites writable and their names out of reach (#4064)
Follow-up to #4056. Icons and profiles are now written as the system user,
so on a storage-limited install they counted against its allowance (100 MB
by default) and every write past that returned 413, including a user's
profile save. Both writes now pass the unmetered allowance, as writes the
system makes on users' behalf already may.

ensureSystemSite reclaims a mismatched subdomain row in place instead of
delete+create, so the name is never free for a user to register, the row
keeps its uuid, and `protected` is never cleared; the previous holder's
bindings are dropped. If the row vanished since the read it falls through
to create. puter-app-icons is now protected like puter-profiles, both names
are reserved from user registration, and they live in util/systemSite so
the subdomain driver doesn't import the services. The icon service's
shutdown wait is capped at 5 s so a stalled run can't hold up the metering
flush.
2026-10-04 00:23:01 -07:00
Daniel Salazar 26abd4e4a5 fix(fs): hide the issuer's home from scoped-token root listings (PUT-2053) (#4047) 2026-10-04 00:22:43 -07:00
Daniel Salazar 3defae3cfc fix: own system sites by the system user (#4056)
* fix: register system sites to the owner of their directory

The hosting middleware refuses a site whose user doesn't own its root
directory (absent a manage grant). The app-icons and profiles bootstraps
always registered their subdomain to the admin, but the directory can belong
to another user: a carried-over `/system` owned by the `system` user, which a
new child directory inherits. Those sites then 404 on every request.

Both bootstraps now go through ensureSystemSite, which registers the
subdomain to the directory's owner and replaces a row registered to anyone
else or pointing at another directory.

Fixes #4045

* fix: own system sites by the system user

`/system`, `/system/app_icons` and `/system/profiles` and their subdomains
belong to the system user, and icon and profile files are written as it.
ensureSystemSite re-owns directories another account created and replaces a
subdomain row registered to anyone else, so installs bootstrapped under the
admin move over on boot.

The bootstrap no longer waits for an admin account, so DefaultUserService
stops re-running it, and deployments without an admin get icons and profiles.
AppIconService lets in-flight icon runs finish on shutdown.
2026-10-03 17:06:13 -07:00
Daniel Salazar 205f19a167 fix(metering): stop caching subscription answers from email-less actors and failed lookups (PUT-2002) (#4052) 2026-10-03 16:39:08 -07:00
Daniel Salazar 2cfc1461e4 fix: scope whoami billing details and monthly usage to the calling app (PUT-2015) (#4050)
* fix: scope whoami billing details and monthly usage to the calling app (PUT-2015)

* fix(metering): refuse an app another app's detailed usage (PUT-2015)
2026-10-03 16:38:31 -07:00
Daniel Salazar 3fa23bb809 fix(fs): refuse renaming or moving AppData roots (PUT-2050) (#4048)
* fix(fs): refuse renaming or moving AppData roots (PUT-2050)

* fix(fs): refuse a forged trash original_path from apps (PUT-2050)
2026-10-03 16:37:36 -07:00
Daniel Salazar e59f6285ba refactor(metering): route free-plan checks through isFreeSubscription (PUT-1830) (#4046) 2026-10-03 16:36:05 -07:00
Filip Kujundžić c3abec8663 Merge pull request #4035 from HeyPuter/filipkujundzic/put-1895-chat-credit-gate-cost-factor
fix(ai): price the chat credit gate at the factored cost (PUT-1895)
2026-10-03 19:44:51 +02:00
Neal Shah 04406373f4 event based FBL implementation (#3969) 2026-10-03 05:54:04 -04:00
Daniel Salazar 91aff909e1 fix: keep launch tokens out of URLs and storage (PUT-2040) (#4038)
* fix: keep launch tokens out of URLs and storage (PUT-2040)

- Strip the consumed launch token param from the app URL after load
- Never persist user session (godmode) tokens to localStorage; purge any stored one at boot
- Worker log socket sends the token as its first message instead of in the URL path

* fix: strip the private-app sign-in token from top-level URLs, keep godmode tokens across reloads (PUT-2040)

- Private-app gate: a top-level GET carrying ?puter.auth.token= that passes the gate gets the sticky cookie and a 302 to the same URL without the token. Framed launches are untouched.
- Godmode session tokens go to sessionStorage (never localStorage), so a frame reload stays signed in; cleared on sign-out or when an app token replaces it.
2026-10-03 00:26:31 -07:00
Daniel Salazar a242105540 fix(events): keep app-less scoped tokens off the account's durable rows and mailbox (#4042)
A personal access token scoped below the account (e.g. the one in an
account-minted getReadURL() URL) has effectiveApp === null, which the durable
list/unsubscribe/ack scope check and notif fetch read as account context. Gate
those on isAccountContext, as workers/handlers/kv-handles already do, and refuse
such tokens on durable subscribe.
2026-10-02 22:37:07 -07:00
Reynaldi Chernando 47510897ed add ai builder link to docs (#4044) 2026-10-03 10:27:23 +07:00
dependabot[bot] d0da8c5b1e chore(deps): bump engine.io from 6.6.9 to 6.6.11 (#4017)
Bumps [engine.io](https://github.com/socketio/socket.io) from 6.6.9 to 6.6.11.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@6.6.11)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-version: 6.6.11
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 15:03:34 -07:00
dependabot[bot] 303ee6c2c7 chore(deps-dev): bump brace-expansion from 1.1.18 to 1.1.21 (#4015)
Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 1.1.18 to 1.1.21.
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.18...v1.1.21)

---
updated-dependencies:
- dependency-name: brace-expansion
  dependency-version: 1.1.21
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 15:02:56 -07:00
Daniel Salazar 37c1678897 fix(fs): keep a vacated username off-limits to other accounts for an hour (#4039)
* fix(fs): keep a vacated username off-limits to other accounts for an hour

Renaming a home rewrites descendant paths in the database, but their
cached entries keep the old path until they expire. ACL grants a home by
path prefix, so whoever takes the old name next must not be able to
before those entries are gone. renameUserHome now records the vacated
name, and the claim check every username-claim site already runs treats
it as taken for anyone but the account that left it.

* test: seed recommended apps that are still in the default list

The default list no longer includes editor or camera, so the ordering
test found neither.
2026-10-02 14:53:34 -07:00
Filip KujundžićandClaude Opus 5.5 7b8b3752e4 fix(ai): price the chat credit gate at the factored cost
The chat gate read `model.costs` directly to decide affordability, cap
`max_tokens` and size the credit hold, while usage is recorded at cost
times the AI cost factor, so all three ran low by the factor on every
request. The gate now resolves the factor and applies it to both rates.

The factor is looked up by the key the provider records usage under.
Providers expose that key through `meteringModelKey` and use it when
recording, so the gate and the charge can't drift apart; the
unreported-stream backstop records under the same key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:21:18 +02:00
404oopsandClaude Opus 5.5 521812be02 refactor(ai): move the AI cost factor out of MeteringService
MeteringService knew about AI: it built the cost-factor facade, resolved
the `ai.cost.factor.*` hook and, since the previous commit, offered
`reserveAiCredits`. Metering stays agnostic to what it meters, so all of
that now lives with the AI drivers in `drivers/util/aiCostFactor.ts`.

`withAiCostFactor(metering, events, driver)` returns an
`AiMeteringService`: the service with recording scaled by the factor,
plus `costFactor` and `reserveAiCredits` for gates. The speech providers
take that type. MeteringService itself has no AI methods left.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 21:20:24 +02:00
Nariman Jelveh bb38e8c86f Update RecommendedAppsService.ts 2026-10-02 11:29:42 -07:00
Nariman Jelveh 74d89b64fd Update RecommendedAppsService.ts 2026-10-02 11:19:37 -07:00
Daniel Salazar 31e5b64a0f fix: make app-data delete grants imply write and read (PUT-1998) (#4022) 2026-10-02 10:47:58 -07:00
Daniel Salazar e96cf77982 feat(metering): per-month allowance override for partly billed plan changes (PUT-1996) (#4027) 2026-10-02 10:40:52 -07:00
Filip KujundžićandClaude Opus 5.5 22d52cf544 fix(ai): name data-URL audio with its real extension
A data URL's filename came from its MIME subtype, so `audio/mpeg`
became `input.mpeg` and `audio/x-wav` became `input.x-wav`. OpenAI's
transcription API picks the decoder from the extension and rejected
both. Map the MIME types whose subtype isn't the extension.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 19:28:13 +02:00
Daniel Salazar 86bb94d3d9 fix: cap FS batch array lengths (PUT-1894) (#4024) 2026-10-02 10:06:21 -07:00
Daniel Salazar a4f1e665df fix: return 400 when a streamed upload is shorter than its declared size (#4023)
A streamed write sends the caller's declared size to the object store as
the body length. A body that ends short of it was rejected as
IncompleteBody and surfaced as a 500.
2026-10-02 10:05:13 -07:00
Filip KujundžićandClaude Opus 5.5 abd0d809d5 fix(ai): price speech credit gates at the charged cost and hold them in flight
Text-to-speech, speech-to-text and voice-changer providers checked
affordability against the unscaled provider cost and never reserved it,
so the gate ran low by the AI cost factor and concurrent requests could
each spend the whole balance. They now go through
`reserveAiCredits`, which scales the amount by the model's cost factor,
checks it, and holds it until the request finishes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:56:53 +02:00
Filip KujundžićandClaude Opus 5.5 82bd7a7360 chore(ai): sync model catalogs with provider listings and deprecations
Remove models providers have deprecated or retired, add newly released
ones, and move provider defaults off retired models. Gemini video (Veo
via the Gemini API) is removed; Infron and OpenRouter skip models their
listings mark deprecated. xAI bills cached prompt tokens only at the
cached rate. Claude Sonnet 4.6 max output raised to 128K.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:52:35 +02:00
Reynaldi Chernando b32ae7dcfc Add rate limit detail for contact form recipe (#4030)
* Add rate limit detail for contact form recipe

* handle ipv6
2026-10-02 23:19:41 +07:00
Juan Fernando Castro d5e6d5ea07 Merge pull request #4007 from HeyPuter/juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant
fix: refuse opaque origins cleanly instead of a logged 400 (PUT-1896)
2026-10-02 09:58:13 -04:00
Reynaldi ChernandoandCopilot Autofix powered by AI f46d187d51 contact form recipe (#4029)
* contact form recipe

* Potential fix for pull request finding 'Network failures cause unhandled fetch rejections'

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* minor

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-10-02 20:24:05 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 0fcee62884 Update recipes for storing a small list and store per id (#4028)
* Update recipes for storing a small list and store per id

* Potential fix for pull request finding 'Clarify that IDs must be path-safe or properly escaped'

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-10-02 19:23:39 +07:00
Reynaldi Chernando 384524a027 improve teams docs (#4025) 2026-10-02 14:45:54 +07:00
Daniel Salazar 42b204f46e fix: don't count failed username change towards rate limit (#4021) 2026-10-02 00:26:37 -07:00
Daniel Salazar 91c8b2385b fix: run a broadcast persistent handler in the worker or the clients, not both (PUT-1889) (#4016) 2026-10-02 00:08:02 -07:00
jelveh 05e2e7428a feat(gui): let users remove their profile picture
The account tab only offered changing the avatar. Add a "Remove photo"
action, shown while a picture is set, that clears it via
update_profile({ picture: null }) and resets every avatar to the default.
On failure the hint line says so and the picture stays.

The tab can render before the profile loads, so the profile loader also
reveals the button once a picture arrives. Also move the avatar hint
into i18n.
2026-10-01 23:21:57 -07:00
dependabot[bot] f387a7aa68 chore(deps): bump fast-uri from 3.1.7 to 3.1.8 (#4014)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.7 to 3.1.8.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.8)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 18:39:45 -07:00
Daniel Salazar e6bda950f0 perf: batch per-app origin, canonical and hosted-backing lookups in app listings (PUT-1892) (#4013) 2026-10-01 18:30:06 -07:00
Daniel Salazar 4f31388dad fix: cap app lookups and fields, dedupe icon processing, rate-limit app landing pages (PUT-1891) (#4012) 2026-10-01 17:54:25 -07:00
dependabot[bot] 376a047ecc chore(deps): bump @grpc/grpc-js from 1.14.4 to 1.14.5 (#4010)
Bumps [@grpc/grpc-js](https://github.com/grpc/grpc-node) from 1.14.4 to 1.14.5.
- [Release notes](https://github.com/grpc/grpc-node/releases)
- [Commits](https://github.com/grpc/grpc-node/compare/@grpc/[email protected]...@grpc/[email protected])

---
updated-dependencies:
- dependency-name: "@grpc/grpc-js"
  dependency-version: 1.14.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:50:14 -07:00
dependabot[bot] fe7b0ce535 chore(deps): bump axios from 1.18.1 to 1.20.0 (#4008)
Bumps [axios](https://github.com/axios/axios) from 1.18.1 to 1.20.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.18.1...v1.20.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.20.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:49:54 -07:00
dependabot[bot] e85c6917f7 chore(deps): bump fastify from 5.12.3 to 5.12.5 (#4009)
Bumps [fastify](https://github.com/fastify/fastify) from 5.12.3 to 5.12.5.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/commits/v5.12.5)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:49:42 -07:00
Juan Castro 989399a857 Merge branch 'main' into juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant 2026-10-01 20:30:41 -04:00
Juan Castro 143948ed2c Merge branch 'main' into juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant 2026-10-01 20:30:00 -04:00
Juan Fernando Castro e48dc0cbf3 Merge pull request #4011 from HeyPuter/juancastro/put-1997-bound-grant-and-token-route-inputs
fix: bound the inputs on the grant and token routes (PUT-1997)
2026-10-01 20:29:42 -04:00
Juan Castro a67cdac380 fix: bound the inputs on the grant and token routes
Three permission routes sized their work by the request body rather than by
the route, so one call could cost far more than its rate-limit slot implies.

- `extra` and `meta` were only type-checked. They ride every row a grant
  writes (up to 16) and are re-serialised into the per-(user, app) cache on
  each miss, so they are now capped at 4 KiB.
- `/auth/create-access-token` took a `permissions` array of any length: one
  permission check and one sequential INSERT each. It now uses the same
  16-per-request cap the grant routes already had, and runs every entry
  through the validator those routes use.
- Withdrawing an app's cross-app data grants looks them up by permission
  text, and no index on `user_to_app_permissions` led with `permission`.
  Indexed per engine, mirroring what mysql_mig_22 / postgres_mig_11 /
  sqlite 0067 did for `user_to_user_permissions`.

`grant-dev-app` validated none of its input — not even the type of `extra` —
so it now runs the same validator as its user-app sibling.

Two paths could also carry a permission wider than the `varchar(255)` column
it lands in. `grantUserAppPermission` already rejected that after rewriting;
`grantDevAppPermission` now does the same, and `createAccessToken` checks it
before the session row a failing INSERT would otherwise orphan.

Caps are published in rate-limits-and-quotas.md. Every in-tree caller sends
one permission and a small `extra`, so none of them change behaviour.
2026-10-01 16:51:39 -04:00
dependabot[bot] c6da95beb0 chore(deps-dev): bump undici from 7.29.0 to 7.30.0 (#3991)
Bumps [undici](https://github.com/nodejs/undici) from 7.29.0 to 7.30.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.29.0...v7.30.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.30.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 09:52:01 -07:00
Daniel Salazar ed7b8f828a fix: kv path errors and ttl hardening, events subscription end and race fixes (#4004) 2026-10-01 09:50:34 -07:00
Juan Castro 29c6f78b7c fix: refuse opaque origins cleanly instead of a logged 400
A page the browser gives no origin to — one opened straight from disk, or an
iframe sandboxed without `allow-same-origin` — serialises its origin as
`"null"`. `appUidFromOrigin` logged that at error level on every attempt and
the GUI sent it on three endpoints, so each refusal cost a 400 and an
error-level line while the user got nothing useful.

An opaque origin cannot name an app to mint a token for, and is not a valid
`postMessage` target to deliver one to, so these clients are refused rather
than supported. Make the refusal quiet, early and legible instead:

- AuthService: drop the `console.error` and say what the caller should do.
  The accept/reject set is unchanged; a 400 is already counted per route.
- puter.js: `signIn()` rejects with `unsupported_origin` before opening
  anything, which covers implicit auth too since it routes through `signIn`.
  The load-time warning now covers sandboxed iframes, console-only — a modal
  belongs in nobody else's embed.
- GUI: a popup whose opener has no attested origin says so and closes,
  instead of rendering a blank window and wedging on a failed exchange.

Also fixes two faults this made reachable: `PuterDialog.open()` prefers the
popup branch, which reaches the `puter` global before the constructor that
assigns it has returned (`puter is not defined`, no SDK at all); and
`showModal()` throws where modals are blocked. `openNotice()` does neither.

Ordinary http(s) sign-in is unaffected.
2026-10-01 11:09:04 -04:00
Juan Fernando Castro c542e4675b Merge pull request #4000 from HeyPuter/juancastro/put-2036-shared-file-issue
fix: let a share recipient create entries in a folder shared with them
2026-10-01 09:56:09 -04:00
Daniel SalazarandReynaldi Chernando d3b46a6b86 docs: easy kv recipes (PUT-1962) (#3999)
* docs: easy kv recipes (PUT-1962)

* docs: publish the append-to-list kv recipe (PUT-1962)

* docs: bare-object add rejects with invalid_path (PUT-1962)

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 18:03:07 +07:00
Juan Fernando Castro fd8b8288d3 Merge pull request #3988 from HeyPuter/juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-30 20:28:02 -04:00
Daniel Salazar fc62ea2ee1 Revert "variable color for text in input fields (#3996)" (#4001)
This reverts commit 8862420594.
2026-09-30 16:12:19 -07:00
Juan Castro d1b11c793c Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a
One conflict, in workerInvoker.integration.test.ts: #3997 fixed the same
gap-marker flake this branch did, with a different barrier. Took theirs
— waiting on the `ev:qf` counter, which `#handlerFailed` writes strictly
after `discard`, so the marker is in and the lease cleared by the time it
reads 1. Their ordering also asserts the counter before flipping `answer`
back to 200, which closes the redelivery-clears-failures hole this
branch's version had removed the line to avoid.

Converted the two `vi.waitFor` calls #3997 added to `waitUntil`, the
helper this branch introduced for the rest of the file. Neither is wrong
today, since the new block fakes no timers, but a `jump()` added near
them later would hit exactly the drained-backoff trap the helper exists
for.
2026-09-30 17:49:36 -04:00
Juan Castro d6b7c8ff52 docs: drop the replica-lag testing runbook from the open-source tree
It describes our local two-server MySQL replication setup, which is
internal tooling rather than anything a self-hoster or contributor
needs. It now lives in the heyputer repo alongside the other internal
docs.

The suite it documented stays: the header comment now states the
env-var gate directly instead of pointing at a file this repo no
longer carries.
2026-09-30 17:42:04 -04:00
Daniel Salazar 4448103ef0 fix: misc events and kv issues found during recipes (#3997)
* fix: misc events and kv issues found during recipes

* test: cover handler depth on narrowed actors and created notifications

* test: fix stale legacy batch directory upload test and seat user-agent flake

* test: wait for the refusal to be recorded before claiming its gap marker
2026-09-30 14:40:47 -07:00
Juan Castro 79e496e79d test: wait on the gap marker rather than a depth that never moves
The refused-delivery test waited for `depth(subId)` to reach 1 before
claiming the marker, but `discard` removes the entry and appends the
marker in its place, so the depth is 1 before, during and after. The
wait returned at once, and the claim came back `inflight` — the refused
delivery still held the lease it was invoked on — leaving the assertion
reading `undefined`. It only passed on the slack in `invoked`'s poll
interval, which a loaded runner takes away.

Waiting on the claim itself is the barrier the depth was standing in
for. Dropping `answer = 200` with it: `beforeEach` already resets it,
and a redelivery settling before the last assertion would clear the
failure count it reads.
2026-09-30 16:36:57 -04:00
Juan Castro 2e0bf26cac Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-30 16:31:42 -04:00
Juan Castro b126cf9c5a fix: carry the owner's path into WebDAV events and expand tilde before touch
WebDAV emits the same GUI events the FS controllers do, and had the same
confusion: the payload was masked for the actor and addressed to the owner. A
recipient mounting a shared folder in Finder reproduced the disappearing-item
symptom exactly.

/touch prefixed `/` before expanding, so `~/AppData/<app>/f.txt` became a literal
`/~/...` that no longer looked like a tilde path — the shape puter.js sends for
every app-relative write answered 404.
2026-09-30 16:21:39 -04:00
Juan Castro ec4d3b31cf fix: let a share recipient create entries in a folder shared with them
A recipient addresses the owner's entries by a masked `/<owner>/<uuid>/<name>`
path. Routes that resolve an existing row unmask it on the way in, but the ones
that create a new entry had nothing to resolve and ran the ACL check against the
mask itself, which names no row — so mkdir, touch and the batch write/mkdir/
shortcut ops answered 404 `subject_does_not_exist` inside any shared folder.
They now expand client paths the same way every other legacy route does.

The GUI events carried the same confusion the other way: the payload was masked
for whoever made the request but addressed to the entry's owner, who cannot
resolve another user's mask. The owner's client dropped the row by uid and then
failed to re-add it under a path it had never heard of, so anything a recipient
touched vanished from the owner's open window until a refresh. Those events now
publish the owner's real path; responses to the actor stay masked.

Also fixes the desktop's `item.moved` handler reading `metadata` as an object
when the wire carries a JSON string: the name of a trashed item came out empty
(trashing renames the entry to its uid), and re-encoding the string left the
restore path parsing a string instead of an object. The parse the dashboard and
the explorer already open-coded is now one helper.
2026-09-30 15:28:02 -04:00
hairyEagle 8862420594 variable color for text in input fields (#3996) 2026-09-30 11:41:32 -07:00
Neal ShahandReynaldi Chernando 3b4f3f6d06 PUT-1959 PUT-1960 subdomains & worker recipes (#3993)
* PUT-1960 worker recipes

* subdomain recipes

* fixes

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 01:33:29 +07:00
Juan Fernando CastroandReynaldi Chernando 3d1357455f docs: teams recipe and playground examples (PUT-1966) (#3967)
* docs: add the easy teams recipe and playground examples

Covers the read-only surface an app actually has: list(), listMembers()
and listDirectory(). Adds the teams recipe tag the build validates
against, and a Teams section to the playground.

Each example handles the two states that are easy to confuse: teams
turned off on the deployment (not_found) and a team that has not opened
its directory to apps (team_not_found).

* docs: open the teams recipe with what it lets an app build

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 00:11:42 +07:00
Juan Castro f8d8003fea test: stop the delivery-backoff test reading holds against a moving clock
`holds one it could not answer, for longer each time` freezes `Date` so a
hold written as `now + backoff` reads back as exactly `backoff`. But every
wait in the file went through `vi.waitFor`, which advances the faked clock
once per poll — measured at 50ms a poll. Each poll between the hold being
written and read therefore ate into the value: CI read a 2s hold as 525ms
and failed, on main one run and on this branch the next.

Polling now goes through `waitUntil`, which retries on `performance.now`
and real `setTimeout` — neither faked — so the clock only moves where
`jump` moves it. The holds then read exactly 2s/4s/8s/16s, so the
assertion is an equality rather than a one-second window that a slow
enough runner could always slide out of.
2026-09-30 12:25:40 -04:00
Juan Castro 23c8f73bcb test: verify the replica-lag fix against real MySQL replication
The unit tests inject the driver error because sqlite has neither a read
replica nor foreign-key enforcement, so neither half of this bug can
occur locally — they prove the handling, not that the situation arises.

This suite runs two MySQL servers in real replication and freezes the
follower with STOP REPLICA, so the row really is gone on the primary and
really is still served by the replica, and MySQL raises the foreign-key
errors itself. Against the pre-fix code all four cases fail, including
the unhandled "Cannot add or update a child row" the reported 500s were.

Opt-in behind PUTER_TEST_REPLICA_LAG, matching the env-gated provider
integration tests and their skipUnlessEnv helper, so CI and anyone
without the containers skip it. It builds and drops its own database
rather than touching local dev data.
2026-09-30 10:02:36 -04:00
Daniel Salazar b9efac2363 fix: misc fixes from alert logs (#3995) 2026-09-30 00:51:27 -07:00
Daniel Salazar ad12da2942 feat: add gpt 6.1 sol to our direct apis (#3994) 2026-09-29 17:38:40 -07:00
Neal Shah 9aa3989e9d fix PUT-1888 (#3992) 2026-09-29 17:06:28 -04:00
dependabot[bot] cd37cf147f chore(deps): bump socket.io-client from 4.8.3 to 4.8.4 (#3982)
Bumps [socket.io-client](https://github.com/socketio/socket.io) from 4.8.3 to 4.8.4.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@4.8.4)

---
updated-dependencies:
- dependency-name: socket.io-client
  dependency-version: 4.8.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 12:56:15 -07:00
Filip Kujundžić f60d94ef3f Merge pull request #3987 from HeyPuter/fix/put-1874-chat-test-mode
Fix chat test mode routing to fake provider
2026-09-29 19:19:44 +02:00
Juan Castro 8bce62e880 Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-29 13:13:01 -04:00
Juan Castro abd27c0192 fix: replicate tombstones, and attribute foreign-key failures correctly
Review of the first pass turned up two holes that made it ineffective.

CacheReplicationService always deleted the keys a peer broadcast and
never adopted a payload, so tombstones stopped at the region boundary:
peers dropped the marker, kept caching deleted rows off their own
replicas, and the multi-region case the fix was for stayed broken.
Tombstones are now the one payload a peer adopts — they carry no row,
only the fact of a delete.

user_to_app_permissions keys both the app and the user, and the drivers
don't all name the constraint, so a deleted *account* was being read as a
deleted app: a live app got tombstoned, the request 404'd for the wrong
reason, and the dead account was never retired, so the client retried and
re-tombstoned indefinitely. The insert site now asks the primary which
parent actually went away and answers 404 or 401 to match.

Also: re-check the tombstone after a cache write lands, since a row
cached under a live tombstone is never looked at again; extend the guard
to #refreshCache, which update() could otherwise use to undo a delete;
stop markDeleted tombstoning an address the deleted row never owned,
which was blocking the account that did own it; let the primary rather
than the tombstone decide in batched lookups; tombstone the id key when
no cached row names the others; and correct the leaf-store comment in
stores/index.ts now that SessionStore depends on UserStore.
2026-09-29 13:12:31 -04:00
dependabot[bot] 6a31b2af9e chore(deps): bump libphonenumber-js from 1.13.13 to 1.13.14 (#3984)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.13 to 1.13.14.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.13.13...v1.13.14)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 10:11:03 -07:00
dependabot[bot] 3ebb753eb1 chore(deps): bump socket.io from 4.8.3 to 4.8.4 (#3985)
Bumps [socket.io](https://github.com/socketio/socket.io) from 4.8.3 to 4.8.4.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@4.8.4)

---
updated-dependencies:
- dependency-name: socket.io
  dependency-version: 4.8.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 10:08:32 -07:00
Juan Fernando CastroandReynaldi Chernando ee4d55027d docs: easy perms recipe (PUT-1965) (#3972)
* docs: add the easy perms recipe

Covers the supported surface: request() and check(). Adds the perms
recipe tag the build validates against.

* docs: add a playground example per operation in the perms recipe

One runnable example for each section: request a folder, request the
email address, request apps or subdomains, and check before prompting.

* docs: open the perms recipe with what it lets an app build

* docs: ground the perms examples in the UI that would ask

A photo editor exporting to Pictures, a checkout emailing a receipt, a
launcher listing apps, and a settings toggle that reads its own state
without raising a prompt.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-09-30 00:00:34 +07:00
dependabot[bot] 7eb9ace733 chore(deps-dev): bump prettier from 3.9.8 to 3.9.9 (#3986)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.8 to 3.9.9.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.8...3.9.9)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 09:51:06 -07:00
dependabot[bot] 14ae6a69ee chore(deps): bump postal-mime from 3.0.0 to 3.0.1 (#3983)
Bumps [postal-mime](https://github.com/postalsys/postal-mime) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/postalsys/postal-mime/releases)
- [Changelog](https://github.com/postalsys/postal-mime/blob/master/CHANGELOG.md)
- [Commits](https://github.com/postalsys/postal-mime/compare/v3.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: postal-mime
  dependency-version: 3.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 09:50:45 -07:00
Juan Fernando CastroandReynaldi Chernando 3b00cd2d14 docs: easy fs sharing recipe (PUT-1958) (#3973)
* docs: add the easy fs sharing recipe

share(), listShared(), getShares() and unshare() — the fs ground the
existing recipes do not already cover.

* docs: add a playground example per operation in the fs sharing recipe

One runnable example for each section of the recipe: share, mode,
listShared, getShares and unshare.

* docs: open the sharing recipe with what it lets an app build

* docs: ground the sharing examples in a document-editor flow

Each example now sits in the UI that would call it — a Share button, a
role picker, a shared-with-me inbox, an access list, ending a review.

* remove redundant examples and refine recipe

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-09-29 23:47:23 +07:00
Juan Castro 785304e807 fix: stop deleted apps and users from being resurrected by a lagging replica
A delete lands on the primary and invalidates the cache, but a concurrent
request that misses the cache reads a replica that hasn't applied it yet,
finds the row, and writes it back under a fresh TTL — 24h for apps, 15m
for users. Every write referencing that resurrected row then fails its
foreign key and escapes as a raw driver error, so the route 500s until
the TTL expires. Origin-bootstrapped app uids are deterministic, so the
same origin keeps hitting the stale row and never re-bootstraps.

Deletes now tombstone their cache keys before dropping them. While a
tombstone is live the read path goes to the primary and the write path
refuses to cache; re-creating a row clears it. As a backstop, a write
rejected for a missing parent is answered as one: 404 for a vanished app,
401 for a vanished account, with the stale entry retired either way.

Also fixes cascadeDelete's identity lookup, which selected a column some
schemas lack and so threw into its catch, skipping the invalidation
entirely; and drops SessionStore's private copy of isUniqueViolation in
favour of the shared helper.
2026-09-29 10:16:48 -04:00
Juan Fernando Castro 8e31d9149a Merge pull request #3966 from HeyPuter/juancastro/put-1994-group-scan-cycle
fix: stop group permission scans re-entering the sharer's own grant (PUT-1994)
2026-09-29 10:12:54 -04:00
Juan Castro 9539c7b553 test: cover team-share access through the real authorization path
The existing team-share suite resolves shares through listings and store
reads, neither of which enters the permission scan, so a scan that never
terminates leaves it green. These ask the question the product asks —
can this member open the file — via ACLService.check.
2026-09-29 09:41:05 -04:00
Juan Fernando Castro 27c4873042 Merge pull request #3971 from HeyPuter/juancastro/put-2033-team-create-read-after-write
fix: read a team back from the primary after writing it (PUT-2033)
2026-09-29 09:06:43 -04:00
404oops 0b1a4283f5 fix(ai): honor chat test mode 2026-09-29 12:17:33 +02:00
Daniel Salazar 1cc091206c fix: PUT-1885 PUT 1950 (#3981)
* fix: app uuid fixes

* fix: bound kv update/add createPaths writes (PUT-1885)

* fix: upload progress socket events on the v2 tracker (PUT-1950)
2026-09-28 23:38:34 -07:00
Daniel Salazar 09887747e0 fix: app uuid fixes (#3980) 2026-09-28 19:43:23 -07:00
Daniel Salazar f80d4ae136 fix: events docs (#3968)
* fix: events docs

* fix: docs
2026-09-28 17:24:37 -07:00
Filip KujundžićandClaude Opus 5.5 42e0c28d0a feat(ai): add Claude Sonnet 5.5 (#3970)
Add claude-sonnet-5-5 ($2/$10 per MTok, 1M context, 128K output) and move
the bare claude-sonnet / claude-sonnet-latest aliases to it. Sonnet 5.5
rejects non-default sampling params and disabled thinking, so it joins the
temperature-omit, effort, and adaptive-thinking gates.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-09-28 15:29:21 -07:00
Juan Castro d888377138 Merge remote-tracking branch 'origin/main' into juancastro/put-2033-team-create-read-after-write 2026-09-28 17:20:43 -04:00
Juan Castro 8a53d14540 fix: read a team back from the primary after writing it (PUT-2033)
create() inserted on the primary then read the row back through the
replica, so a replica that had not caught up answered the same as a team
that was not there and the call threw. Nothing rolled the insert back,
and the owner membership was never written, leaving a team its owner
could not see but which still counted against their cap.

Read back through pread in create() and update(), seed the row cache
with the result, and delete the row if the read-back still finds
nothing. #cached no longer stores a null, which would otherwise hold a
lagging replica's answer for the full TTL.
2026-09-28 16:55:19 -04:00
Juan Fernando Castro 12e6bc3d71 Merge pull request #3950 from HeyPuter/juancastro/put-1988-team-force-2fa
feat: let a team require two-factor authentication of its accounts (PUT-1988)
2026-09-28 16:52:34 -04:00
Filip Kujundžić 359d68e51f Merge pull request #3949 from HeyPuter/fix/ai-ocr-model-sync-and-normalization 2026-09-28 21:23:28 +02:00
Filip Kujundžić 020e61065f Merge pull request #3947 from AnujShrivastava01/anuj/fix/remove-deprecated-ai-chat-models 2026-09-28 21:22:33 +02:00
Juan Castro 480d9d032b Merge remote-tracking branch 'origin/main' into juancastro/put-1988-team-force-2fa
# Conflicts:
#	src/backend/clients/database/SqliteDatabaseClient.test.ts
#	src/backend/clients/database/SqliteDatabaseClient.ts
#	src/backend/clients/database/migrations/mysql/mysql_mig_42.sql
#	src/backend/clients/database/migrations/postgres/postgres_mig_31.sql
2026-09-28 10:24:39 -04:00
Juan Castro 528eb192b5 fix: stop group permission scans re-entering the sharer's own grant
The group path recursed without the anti-cycle state, so a grant issued
into a group the issuer belongs to resolved back into itself and the
scan did not terminate. Thread the scan state through the group scanner
and skip an issuer already on the path, matching the user-to-user path.

Readings produced under a non-empty path are pruned for that path only,
so they are no longer written to or served from the shared scan cache.
2026-09-28 10:05:54 -04:00
Rupanjana ChoudhuryandDaniel Salazar ca52b5236c Fix/3638 shared link account (#3845)
* fix: open shared links with the intended account

* refactor: validate shared account hints in the GUI

* fix: scope the shared-link account hint to plain top-level visits

The hint no longer switches accounts in popups, embeds or action flows, and
only counts next to a share path the GUI would open. It is dropped from the
URL before any switch or prompt, so a reload never asks again. An unsaved
recipient gets a dismissible choice (sign in, or continue as the current
account) instead of a picker with no way out.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-27 18:25:28 -07:00
Neal Shah 153f581011 fix PUT-1872 (#3952) 2026-09-25 22:34:53 -04:00
Neal Shah 1fc2eae1ae self hosted mail (#3932)
* self hosted mail

* update lock

* address daniel review
2026-09-25 21:48:15 -04:00
Daniel Salazar dcb61bc4fd fix: PUT-1824 + PUT-1884 + PUT-1952 + PUT-1827 (#3951) 2026-09-25 17:47:03 -07:00
Juan Fernando Castro b445680f10 Merge pull request #3937 from HeyPuter/juancastro/put-1879-oidc-popup-return-proof-binding
fix(auth): bind the OIDC popup-return proof to its purpose, browser, and popup
2026-09-25 18:04:59 -04:00
Juan Castro 95ee796157 Merge branch 'main' into juancastro/put-1879-oidc-popup-return-proof-binding 2026-09-25 17:22:12 -04:00
Nariman JelvehandDaniel Salazar 3d3678a555 feat: let the Contact Us form carry screenshots and recordings (#3563)
* feat: let the Contact Us form carry screenshots and recordings

The form doubles as our bug-report channel, and the bugs worth reporting
are often the ones that need a picture: a visual glitch, or something that
takes five steps to reach. Up to 5 images or videos may now ride along
with the message, delivered as attachments on the support email.

Nothing the client says about a file is believed. The endpoint takes bare
base64 and re-derives all three of the things that matter from the decoded
bytes:

- Type, sniffed from magic numbers and matched against an allow-list of
  PNG/JPEG/GIF/WebP and MP4/QuickTime/WebM. A declared MIME is never read,
  so it cannot smuggle anything past the list. SVG is excluded on purpose
  (it carries script, and support tooling renders what it is sent), as are
  the non-video brands that share MP4's `ftyp` box -- HEIC, M4A, JPEG 2000.
- The file name, reduced to a display label with the extension taken from
  the sniffed type. PNG bytes named `payload.html` arrive as
  `payload.png`; a name can never carry the CR/LF that would break out of
  a Content-Disposition header, nor the bidi overrides that make
  `report<RLO>gnp.exe` render as `report.exe.mp4`.
- Size: 10 MB per file, 15 MB per submission, counted on decoded bytes.
  Encoded length is capped before decoding, so an oversized payload costs
  a length check rather than a 10 MB allocation. The total stays well
  under the 25 MB most providers enforce, since the outgoing mail base64s
  these again.

Base64 is decoded strictly, reusing the round-tripping decoder that
already guards app icons -- `Buffer.from(s, 'base64')` silently drops
characters it does not recognise, and the round-trip is what rejects
bytes smuggled after the payload. That decoder and the image sniffer move
from appIcon.ts to a new mediaSniff.ts, which gains the video counterpart.

One request is now worth megabytes of parsing and outbound mail, so the
existing per-user rate limit gains a per-IP backstop (which the per-user
counter cannot see through freshly minted accounts), a concurrency cap,
and a Content-Length gate that refuses an impossible body before anything
decodes it.

Payloads are not stored. They ride the email; the new
`feedback.attachments` column records names, types and sizes only, so an
abusive submission stays attributable once the mail has been dealt with.

Also fixes the form posting an empty message when Send was pressed with
nothing typed, and surfaces submit failures instead of leaving the button
disabled with no explanation.

* fix: stream Contact Us attachments as multipart instead of base64 JSON

Base64 in a JSON body went through the global JSON parser before the route
ran: raw buffer, rawBody copy, decoded string, parsed strings, then decoded
Buffers plus a re-encode for the strict check. A max-size submission peaked
around 110-135 MB of heap for 15 MB of files, and the route's Content-Length
check ran after all of it.

Attachments now arrive as multipart/form-data, which the global parser skips.
Auth, rate limit, concurrency and the Content-Length 413 all run before the
body is read, and busboy enforces the count, per-file and total caps while
streaming, so only the decoded file bytes are held (~16 MB peak). On a broken
limit the reader stops and the response closes the connection.

JSON stays supported for message-only posts; a JSON `attachments` field is
rejected.

* fix: deliver Contact Us 413s instead of resetting the upload

Closing the socket on a mid-stream limit sent the 413 and then reset the
connection with the client still uploading, so the client could see a
reset instead of the error. After a limit trips, the rest of the body is
now read and discarded, bounded by the body budget; past the budget the
request is destroyed.

Adds an HTTP-level test through the full middleware stack (FormData
upload, unauthenticated refusal, 413 on declared length before the body
is sent, 413 on a chunked upload over the per-file cap), trims comments
to the AGENTS.md length rule, and drops a box-drawing test divider.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-25 14:19:35 -07:00
hairyEagle a9ee3d1ea6 fix(selfhost): configure bucket CORS in s3-init so browser uploads work (#3934)
Browser uploads (Dev Center deploy, puter.fs.upload) PUT file bytes
directly to presigned URLs on the public S3 endpoint. That cross-origin
PUT is preflighted, and the bucket had no CORS rules, so RustFS answered
without Access-Control-Allow-Origin and the browser blocked the upload.

s3-init now runs put-bucket-cors on every boot after ensuring the bucket
exists (idempotent). Origins are open because the presigned URL is the
credential and the SDK sends no cookies. Document how to re-run and
verify in doc/self-hosting.md.
2026-09-25 12:12:02 -07:00
Juan Castro ee3f928b0e fix: annotate the seat-lookup catch so the baselined typecheck passes 2026-09-25 14:39:23 -04:00
Juan Castro 176c8848f5 fix: let a seat enrol in 2FA from the console, and hold it once required
The Security tab only rendered the 2FA card for accounts with a confirmed
email, so a seat — provisioned without one by design — had no way to
enrol from the UI at all. It was the GUI twin of the enable-route check
fixed earlier: the rule could be required of accounts that could not
satisfy it. The card now shows for a seat too, giving it the ordinary
authenticator flow: QR, manual secret, six-digit code, recovery codes.

Disabling is refused while the seat's team requires 2FA. Without that a
member could switch off the factor their team mandates and lock
themselves out of everything until they enrolled again.
2026-09-25 14:31:58 -04:00
Juan Castro beaf4a9a7c docs: record the team 2FA rule where the other member-facing gates are
No method pages: requiring 2FA and resetting a member's factor are team
administration, which the docs deliberately do not publish — only
`list`, `listMembers` and `listDirectory` are app-callable.

But an app acting for a gated seat does meet `two_factor_required`, and
an undocumented 403 leaves it nothing to go on. It goes beside
`password_change_required` in rate-limits-and-quotas, which already
explains what a team's administration does to a member's requests.
2026-09-25 14:18:30 -04:00
404oopsandClaude Opus 5.5 d36da7522b fix(ai-ocr): address review: playground examples, Textract options, inline size
Adds the Mistral and document-annotation img2txt examples to the
playground, states that Textract takes no options, and drops the
normalize mentions from the img2txt docs and types.

The SDK now checks Mistral inline sources against the size a base64
upload can reach inside the 50 MB JSON body (36 MB) instead of the
10 MB Textract limit; 50 MB let 38-50 MB inputs through to a body-parser
413. Retired-model messages no longer cite vendor dates.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 18:56:29 +02:00
Daniel Salazar e6cfe8c492 fix: codeowners (#3948) 2026-09-25 09:23:26 -07:00
404oopsandClaude Opus 5.5 eef8258539 fix(ai-ocr): sync OCR models and prices, route by model, cap input size
Model sync against Mistral's listing, docs and a live call per id:
mistral-ocr-latest now serves OCR 4.1, so the catalog pins it (with
mistral-ocr-4 and the retired-but-redirected 2503) to mistral-ocr-4-1,
adds mistral-ocr-4-0 and OCR 3 (mistral-ocr-2512 / -3 / -3-0), and
rejects the retired mistral-ocr-2505 before any upstream call. Mistral
usage is metered per model at vendor rates: $4/1000 pages for OCR 4.x
with $5 annotations, $2 and $3 for OCR 3, replacing the flat $1 and $3.

Textract now calls DetectDocumentText instead of AnalyzeDocument with
LAYOUT. The layout feature cost $4/1000 pages while we billed the
$1.50 detect rate, and it only added text-less LAYOUT_* blocks that put
blank lines into img2txt output. Multi-page PDFs and unsupported
formats become a 400 that names Mistral as the alternative.

Request shape: a model name alone now selects its provider, so calls
differ only by model; an explicit provider still picks its default and
must agree with the model. Mistral options gain tableFormat,
extractHeader, extractFooter and documentAnnotationPrompt; annotation
formats accept the REST (json_schema.schema) or SDK
(jsonSchema.schemaDefinition) spelling and are validated before
billing. Non-image documents such as DOCX go as document_url.

Response shape: both providers return model, text and blocks with a
0-based page; Mistral also returns document_annotation, which was
billed but dropped. img2txt resolves to that annotation when
documentAnnotationFormat is set and to the text otherwise; the return
is a string for every model, so normalize has no effect on it.

Input size: OCR input is capped at the provider limit (10 MB Textract,
50 MB Mistral), and web inputs in loadFileInput are streamed under the
cap instead of buffered whole, which also bounds speech2txt URLs.

Docs: img2txt models, options, input limits, return value and errors;
OCR limits in the quotas page.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 18:19:54 +02:00
Juan Castro 3105378fdc fix: let a team seat enrol in 2FA, and fit the rule's note on its card
Found by driving the flow against a live instance: a provisioned seat
could reach `configure-2fa/setup` but never `enable`, which demands a
confirmed email. Seats are provisioned without one on purpose (PUT-1792,
the team being the trust anchor), so requiring 2FA of a team would have
locked out precisely the accounts the rule is for — permanently, since a
seat cannot enrol and the rule never lifts.

The check now passes for an account a team provisioned; every other
account still needs its address confirmed, with tests either way.

The card's note was also a line too long and rendered clipped, so it says
the same thing in the space the layout gives it.
2026-09-25 11:58:32 -04:00
Juan Castro 695a26b272 feat: surface the team 2FA rule in the console and SDK (PUT-1988)
An owner toggle beside the directory one, confirmed on the way on since
it locks out every account without 2FA until they enrol, and unconfirmed
on the way off since that only lifts a requirement.

A per-member Reset 2FA action, shown only while the rule is on — with it
off a member can clear their own factor and needs nobody's help. The
confirmation says what it does and what it leaves alone: signed out
everywhere, told about it, password unchanged.

`require2fa` rides the Team shape and `update()`, and `resetTwoFactor()`
joins the module beside `resetPassword()`, whose docblock already says
2FA is left alone — the two now read as the pair they are.
2026-09-25 11:28:59 -04:00
Juan Castro cab70254b1 feat: let a team owner reset a seat's second factor (PUT-1988)
Forcing 2FA makes lockout materially likelier, and nothing could clear a
factor except the account holding it — `resetMemberPassword` leaves 2FA
alone on purpose, and admin only displays it. A seat that lost both its
authenticator and its recovery codes had no way back.

Its own route and its own audit action rather than folded into the
password reset, so that method keeps the property its comment claims: a
password reset alone is not takeover. The two together are, and for an
account the team created and pays for that is the team's to do — but it
has to be chosen and recorded, not arrived at sideways.

Bounded to `org_owned = 1`: a joined member's own login is not a team's
to reach into. The member is told, and signed out everywhere.
2026-09-25 11:15:23 -04:00
Juan Castro db1a95260a feat: let a team require 2FA of the accounts it provisioned (PUT-1988)
`group.require_2fa`, toggled by the owner through `PUT /teams/:uid` and
recorded in the audit log like `directory_enabled` — it changes what
every seat must hold to sign in, so it is not something a team should be
able to alter silently.

Enforced by deriving, not by stamping a flag on the user: a stamp left
behind by a member leaving, or by the rule being turned off, would lock
someone out of their own account with no team left to clear it. The seat
row already joins `group` and is already read per request, so it carries
the flag and the gate costs nothing extra. Toggling busts those rows, or
turning the rule off would leave members locked out until the TTL lapsed.

The gate rides the same `allowUnconfirmed` opt-out as the other pending-
requirement gates, so the routes a gated member must still reach stay
reachable — `configure-2fa` among them, which would otherwise be walled
off behind the very requirement it clears.

Scoped to `org_owned = 1`: the owner is not a seat of their own team, and
a joined member is an ordinary self-paying account whose personal login
is not a team's to dictate. The owner is held to the rule at the point it
matters — they cannot turn it on without 2FA of their own.
2026-09-25 10:35:24 -04:00
Anuj Shrivastava e041fb6b9b Remove deprecated AI chat models flagged by live model check
OpenAI's o3-pro, Mistral's mistral-large-2512 and zai-glm-5-2, and
DeepSeek's deepseek-v4-flash / deepseek-v4-flash-vision-exp are no
longer served by their upstream APIs, so drop them from the hardcoded
provider model lists (closes HeyPuter/puter#3945).

DeepSeek's legacy chat/reasoner aliases and default model previously
pointed at deepseek-v4-flash; they now resolve to deepseek-v4-pro, the
remaining model in that catalog, so existing callers keep working.

Signed-off-by: Anuj Shrivastava <[email protected]>
Signed-off-by: Anuj Shrivastava <[email protected]>
2026-09-25 19:40:02 +05:30
Nariman Jelveh 5721bdeb1d Add link to AI Builder in README (#3946) 2026-09-24 21:00:04 -07:00
Daniel Salazar 132cae9f6a Bug bash fixes (PUT-1876, 1877, 1880, 1881, 1948, 1949, 1951) and KV event fan-out by plan (#3943)
* fix(kv): keep disableSharing entries out of other apps' events (PUT-1876)

Private kv keys and values are no longer delivered to cross-app subscriptions, including forwarded and durable deliveries.

* fix(fs): refuse usernames whose home path still holds another account's rows (PUT-1877)

Username claims (signup, save_account, change-username, OIDC, team provisioning) now also check for leftover rows under /<name>/, and the root listing only returns the actor's own home.

* fix(puter-js): reconnect the events channel after a server-side disconnect (PUT-1881)

A server hang-up is retried with a bounded backoff; persistent handlers survive it and get an optional onError when the connection can't be restored.

* fix(backend): let the graceful shutdown finish before telemetry exits (PUT-1948)

The telemetry preload no longer exits on signals; the entry point drains, tears down top-down, flushes telemetry last, then exits. DB pools stay open until teardown.

* fix(fs): reserve quota for signed uploads in progress (PUT-1880)

startWrite/startBatchWrite hold each upload's declared size in a per-owner cache reservation until it completes, aborts or expires, so a burst of starts can't all read the same committed usage. Caps in-flight uploads at 10,000 per account.

* fix(fs): refuse to complete a signed upload whose object never arrived (PUT-1951)

Completion now 400s when the object store reports the key missing (transient errors stay lenient), abort no longer deletes an object a live entry still uses, and a ghost entry is only removed after a second miss at its recorded location.

* fix(metering): move usage to v2 keys with a small totals item, per-model detail shards and a 3-month TTL (PUT-1949)

The billed item holds only totals; per-model detail lives in 100 hash-sharded items read in one batch and cached for 60s, and more than 5,000 distinct usage types in a month fold into other. Global and per-app aggregates store totals only, appTotals comes from a prefix listing, a refused path only skips the key that refused it, exact reads near the allowance are throttled per key, and every metering record expires after 3 months. September usage restarts at deploy; the monthly-charge claim stays on the v1 key through September so recurring charges don't fire twice.

* test(events): let the worker backoff test tolerate a loaded run

* feat(events): scale KV event fan-out with the key owner's plan

A KV change delivers to up to 512 matching subscriptions (2,048 filter evaluations) for a paid key owner and 128 (512) otherwise, inline values are dropped once more than 128 match, and paid accounts may hold up to 512 share handles per app. The owner's plan is only looked up when a change has more candidates than any cap.

* test(metering): seed the 150-app listing test directly so it fits under coverage
2026-09-24 16:00:52 -07:00
Neal Shah 4a9e5a2440 Ns/worker no socket (#3944)
* remove socket creation from puter worker startup

* Remove really extra comment
2026-09-24 18:55:10 -04:00
Juan Fernando Castro 600064dcee Merge pull request #3942 from HeyPuter/juancastro/put-1944-teams-hardening
fix: Teams lifecycle robustness — owner deletion, cap serialization, provisioned-account scope (PUT-1944)

fixes requested by Daniel pushed
2026-09-24 16:51:43 -04:00
Juan Castro 66ee50317c style: restore prettier formatting displaced by the main merge 2026-09-24 16:35:09 -04:00
Juan Castro 8f0a317a97 Merge remote-tracking branch 'origin/juancastro/put-1944-teams-hardening' into juancastro/put-1944-teams-hardening 2026-09-24 16:33:40 -04:00
Juan Castro 8c816060a3 fix: answer a cap-lock failure as 409, like contention, per review 2026-09-24 16:33:12 -04:00
Juan Castro 0131ebfe9b Merge branch 'main' into juancastro/put-1944-teams-hardening 2026-09-24 12:20:12 -04:00
Juan Castro ca48477db9 Merge branch 'main' into juancastro/put-1879-oidc-popup-return-proof-binding 2026-09-24 12:19:59 -04:00
Juan Fernando Castro 2cb7516141 Merge pull request #3936 from HeyPuter/juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and
fix(hosting): keep alternate-host bootstrap stubs from shadowing private apps
2026-09-24 12:19:31 -04:00
Juan Castro 0c5c1be777 fix: Teams lifecycle robustness ahead of wider enablement
Closing an account that owns a team left `owner_user_id` NULL while
`deleted_at` stayed NULL, so the team remained live while every team
route resolves by membership — which the deletion cascades away. The team
could then not be administered by anyone, and the accounts it provisioned
could not be closed. Declined now in both directions: live teams via
`countOwned`, and soft-deleted ones still holding accounts via a new
store read, since a deleted team's accounts are suspended rather than
removed.

The cap lock gave up after 200ms and proceeded unserialized, so
overlapping provisioning could each read the seat count before any insert
committed. It declines instead; a single create never contends. An
unreachable lock backend is a 503 rather than a silent pass.

An account a team provisioned can no longer create a team of its own
(PUT-1890) — that is not the intended lifecycle, since such accounts do
not go through the normal sign-up path. Keyed on `org_owned = 1`, so the
owner and joined members are unaffected, and not via `teamsAvailableTo`,
which is permissive by design.

The directory declines scoped access tokens, as `listMembers` already
bounds non-account callers.
2026-09-24 11:06:00 -04:00
Juan Castro 8ecaa77a0e Merge remote-tracking branch 'origin/main' into juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and
# Conflicts:
#	src/backend/controllers/auth/AuthController.test.ts
2026-09-24 10:04:00 -04:00
Neal Shah 07a11240ea PUT-1868 (#3940) 2026-09-24 00:30:47 -04:00
Daniel Salazar 7758aea78c fix: acl check (#3939) 2026-09-23 19:03:16 -07:00
Rasad RegmiandDaniel Salazar 83de67cca9 fix(tools): spawn npm.cmd through a shell on Windows (#3923)
* fix(tools): spawn npm.cmd through a shell on Windows

* fix(tools): join the npm command line on Windows to avoid DEP0190

Passing args alongside shell: true warns on Node 24. Test the spawn
tuple as data and run a real npm spawn instead of mocking child_process.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-23 18:54:43 -07:00
Echa ApriliyantoandDaniel Salazar 9f69483593 feat: allow browser extension origins in auth requests (#3907)
* feat: allow browser extension origins in auth requests

Add chrome-extension://, moz-extension://, safari-extension://, safari-web-extension://, and extension:// to the protocol allow-list so browser extensions can obtain app tokens via /auth/get-user-app-token.

Extract WEB_AND_EXTENSION_PROTOCOLS constant in validation.js so the allow-list is defined once and shared by AuthService, AppStore, and AppDriver.

* fix: harden the extension-origin allow-list

Review follow-ups on the extension-origin change:

- Require a host in `validateUrl`. Only "special" schemes need an
  authority, so `chrome-extension:` parsed with an empty hostname and
  slipped past the reserved-system-host guard in AppDriver.
- Lowercase the host in `AuthService#normalizedOrigin`. `new URL()`
  lowercases http(s) hosts but leaves opaque ones alone, so one
  extension in two spellings resolved to two app uids — two AppData
  trees, two permission sets — and missed the origin blocklist.
- Drop `extension:`. No browser emits it, and it accepted
  `extension://evil.com` as an app origin.
- Freeze the allow-list and derive `validateUrl`'s http(s) default from
  `WEB_PROTOCOLS` so the two spellings can't drift apart.
- Pin the tests to the real uid derivation, use unique extension ids so
  a row left by another test can't mask the bootstrap path, and cover
  the host-less and near-miss schemes.
- Fix the prettier/eslint failure in AppStore.js.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-23 18:24:31 -07:00
Daniel Salazar 606f5a27a3 chore: CODEOWNERS (#3938) 2026-09-23 18:24:09 -07:00
Daniel Salazar 58ed2f485f feat(auth): signup bonus codes
Signup links can carry a bonusCode. Core checks its shape and previews it through `puter.signup-bonus.check`, refuses a dead code before the abuse hook records the attempt, then hands a live one to `puter.signup-bonus.validate` after `puter.signup.validate`, which may raise the phone/card gates; the accepted code rides on `puter.signup.success`, and OIDC carries it in the signed state. `user.card-verified` is now awaited like `user.phone-verified`. The signup form shows the offer and sends the code.
2026-09-23 14:47:34 -07:00
Juan Castro d73933b8a5 fix(auth): bind the OIDC popup-return proof to its purpose, browser, and popup
Tokens under the `oidc-state` scope share one signing key, so the payload is
what says which job a token belongs to. The popup-return proof now carries a
`purpose` claim, and each verifier accepts exactly one kind of token.

The return leg also sets a single-use companion cookie, mirroring the nonce
`/start` already uses, and stamps the action its redirect lands on. Redeeming
a proof requires the matching cookie — consumed on success only, so a rejected
call can't invalidate an in-flight return — and the popup honors only a proof
minted for its own action.

The cookie is host-only, so `/auth/oidc/verify-popup-return` is also served on
the GUI origin and the popup redeems it same-origin. The return leg always
lands the popup on `config.origin`, where both the cookie and the route live.
2026-09-23 17:21:35 -04:00
Daniel Salazar 4c3c684cb4 fix: refuse app-issued scoped tokens on events handler and worker routes (#3935)
* fix: new cache costs for claude

* fix: refuse app-issued scoped tokens on events handler and worker routes

An access token an app mints (e.g. a getReadURL() token) resolves
effectiveApp to the issuing app, so #handlerApp and listEventsWorkers
treated it as the app: it could publish, remove and list handlers, and
list or destroy the app's worker, regardless of its permission manifest.
Refuse any scoped token there, as the kv-handle routes already do.

* test: freeze the clock before the first backoff read in workerInvoker

The first hold was read on real time, so a slow run measured the 2s
backoff as 1s.
2026-09-23 13:58:49 -07:00
Juan Castro 438a4584de fix(hosting): keep alternate-host bootstrap stubs from shadowing private apps
A public origin-bootstrap app row planted on an alternate hosting domain
could resolve ahead of the owner's private app and skip the private
access gate (PUT-1883).

- get-user-app-token canonicalizes hosted-subdomain origins before uid
  derivation and bootstrap, so every hosting variant shares one app row
- the app create/update conflict check crosses hosting-domain variants,
  so an existing alt-host stub is absorbed by the owner's create
- resolveOwnedAppForHostedSite and the canonical-uid lookup order
  matches private-first with a deterministic id tiebreak
2026-09-23 16:39:22 -04:00
Daniel Salazar b3fda186e7 fix: let apps read opted-in teams, scope worker listing, fix revokeReadURL (#3933)
- GET /teams and /teams/:uid/members admit apps for teams whose owner opened
  the directory; apps get active members' usernames only
- events.workers.list() from an app returns only its own worker
- hide account-session-only SDK methods from the public types
- unpublish the updateProfile docs page (redirects to getProfile)
- revokeReadURL() works through a new /auth/revoke-own-access-token route
2026-09-23 12:06:56 -07:00
Filip KujundžićandClaude Opus 5.5 bc0f4dc279 feat(ai): add GPT-6 Sol and Luna and sync Claude pricing (#3931)
* feat(ai): sync GPT-6 Sol, GPT-6 Luna, and Claude Opus 5.5

* chore: remove documentation changes from model sync

* fix(ai): bill OpenAI cache writes and long-context pricing

GPT-5.6 and later bill prompt-cache writes at 1.25x input and report them
in `cache_write_tokens`, inside the input total. Both OpenAI calculators now
split them out of the prompt count and meter them under their own key, and
the six GPT-5.6+ models carry the rate.

GPT-6, GPT-5.6, GPT-5.5 and GPT-5.4 (incl. Pro) bill a request with more
than 272K input tokens at 2x input (cached reads and cache writes included)
and 1.5x output for the whole request. Models declare this as
`long_context_pricing`, and the ledger overrides, the reported `usd_cents`
and the credit gate all apply the multipliers.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ai): bill GPT-5.6 Sol at OpenAI's promotional pricing

The catalog carried GPT-5.5's $5/$0.50/$30, but OpenAI bills GPT-5.6 Sol
at $4/$0.40/$20 per million input/cached/output tokens, promotional
through at least 2026-11-21. PUT-1943 tracks re-checking before then.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-23 01:59:10 -07:00
Juan Fernando Castro 17f5454265 Merge pull request #3915 from HeyPuter/juancastro/put-1871-authcheck-app-reports-every-app-as-already-authorized
fix: check-app reported every app as already authorized
2026-09-22 18:24:20 -04:00
Filip Kujundžić 5ddff2b3d6 Merge pull request #3927 from HeyPuter/404oops/put-1870-ai-upstream-credit-exhaustion-is-the-largest-error-class-in
fix(ai): classify upstream credit exhaustion, contain the route, redact vendor text (PUT-1870)
2026-09-22 22:29:58 +02:00
404oops 52f0176854 fix(ai): classify and sanitize upstream credit exhaustion 2026-09-22 21:51:44 +02:00
Daniel Salazar 300fb115a1 feat: opus 5.5 (#3930)
* feat: opus 5.5

* fix: bad typing
2026-09-22 12:31:44 -07:00
dependabot[bot] 6debd7df76 chore(deps-dev): bump prettier from 3.9.6 to 3.9.8 (#3925)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.6 to 3.9.8.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.6...3.9.8)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-22 11:33:49 -07:00
dependabot[bot] 70404579f2 chore(deps-dev): bump @types/node from 24.13.4 to 24.13.6 (#3926)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.4 to 24.13.6.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-22 11:33:40 -07:00
Juan Fernando Castro 7462a8eda9 Merge pull request #3914 from HeyPuter/juancastro/put-1869-unhandled-rejection-on-the-hottest-permission-path-can-take
fix: stop leaking an unawaited permission traversal on every check
2026-09-22 14:28:44 -04:00
Daniel Salazar 718de8a9c6 fix: bump server timeouts for better error handling (#3929) 2026-09-22 10:13:46 -07:00
Juan Fernando Castro 8abdcf4f7f Merge pull request #3928 from HeyPuter/juancastro/put-1873-restrict-godmode-app-launches-to-godmode-callers
fix: restrict godmode app launches to godmode callers
2026-09-22 12:32:32 -04:00
Juan Fernando Castro 476f7e075d Merge pull request #3913 from HeyPuter/juancastro/put-1865-apps-repeatedly-ask-permission-when-it-wants-to-access-files
fix: settle a URL file-access prompt from a grant the app already holds
2026-09-22 12:32:06 -04:00
Juan Castro 1bc8cb964e fix: restrict godmode app launches to godmode callers
`ExecService.launchApp`, reachable from `puter.ui.launchApp`, launched any
named app with no check on the target. Gate it: a godmode target may only
be launched by a godmode caller. Desktop launches (tile, double-click,
URL) call `launch_app` directly and are unaffected; `connectToInstance`
already had its own allowlist.
2026-09-22 09:12:25 -04:00
Daniel Salazar 2690f4934c fix: tests (#3922) 2026-09-21 23:33:23 -07:00
Daniel Salazar d07ddeaaf5 fix(fs): keep a home directory on its account's username (#3920)
A root was free to be renamed or moved, so it could be parked on a
username no account held yet. The next account to take that name
provisioned a second root at the same path, and from then on either row
could answer a lookup of it — new entries inherit their owner from
whichever row resolved as the parent, so one account's files were
created owned by the other, and renaming the parked root dragged the
other account's subtree along with it.

Rename and move now refuse a root, every username claim checks the home
path before the user row is written, and renameUserHome refuses to heal
onto a path another account holds.
2026-09-21 23:29:09 -07:00
Daniel Salazar 1261976088 feat: user profiles in a system-owned store, public only for paid accounts (#3919)
- ProfileService keeps each profile as /system/profiles/<uuid>.profile,
  admin-owned and served by the protected puter-profiles subdomain
- GET/POST /profile; puter.auth.getProfile/updateProfile, with
  getProfilePicture reading through them
- another user's profile is served only while its owner is on a paid
  plan (profileGate.enabled kill switch); the hosted file is gated
  through the new site.access.check hook the hosting middleware asks
  before streaming any file
- SubdomainStore.create takes isProtected
- the GUI reads and writes the profile through the SDK
- docs: getProfile, updateProfile, UserProfile, limits

PUT-1859 PUT-1860 PUT-1861 PUT-1862
2026-09-21 23:28:56 -07:00
Daniel Salazar 2abd5533ac fix(puter-js): restore file pickers on third-party sites (#3917)
The clickjacking guard in #3912 pinned `event.source` to `messageTarget`,
which is only assigned when `env === 'app'`. On a third-party site it is
undefined, so the UI listener dropped every message and
showOpenFilePicker, showSaveFilePicker and showDirectoryPicker never
settled.

Split the two environments. `app` still pins the host frame, whose origin
is whatever the deployment is served from. `web` has no host frame — the
picker popups post back directly — so pin the GUI origin we opened them
on, plus the set of popups we opened. Origin pinning is safe there,
unlike the parent-frame case, because we chose the URL, so self-hosted
and local deployments keep working.

Falls back to the origin alone when `event.source` is null: the picker
calls window.close() right after posting and a discarded browsing
context can drop the source.

PUT-1867
2026-09-21 21:43:58 -07:00
922d203e18 fix(ai): send stable, non-sequential user identifiers to AI providers (#3856)
* fix(ai): send stable, non-sequential user identifiers to AI providers

A precedence bug in the AI providers' identifier expression made every
request send `user: ":undefined"` (the ternary bound the app-uid suffix to
the whole `actor.user.id + actor.app?.uid` sum instead of just the suffix),
or read `actor.user.id` on a missing user. The same expression also shipped
the sequential internal user id, letting AI vendors correlate a single
account across apps and sessions.

All eight OpenAI-, Azure-, xAI-, Meta- and ZAI-style providers now build
the identifier through one shared helper, `aiUserIdentifier()`:

- `puter-<user-uuid>[-<app-token>]`: the random user UUID is always
  preserved in full; `maxLength` constrains only the app-bearing form
- app attribution reads `effectiveApp`, so access-token requests name the
  issuing app instead of looking like direct user traffic
- the app token is truncated to fit the budget, and omitted entirely when
  the remaining budget is below 8 chars, where a truncation could collide
  with another app's uid
- nothing is sent for the system actor
- Meta and ZAI keep a caller-supplied `safety_identifier` / `user_id`
  override, applied before the helper result

`user` is deprecated by OpenAI; the SDK types direct callers to
`safety_identifier` (abuse detection) and `prompt_cache_key` (cache-hit
bucketing). The four OpenAI/Azure chat providers and MetaProvider now send
`prompt_cache_key` as well, defaulting it to the same per-user identifier
unless the caller supplies one; Azure's Grok branch drops both fields,
matching its rejection of unknown args. The cap comment cites only verified
limits: OpenAI's 64 for `safety_identifier` (from the SDK types) and Z.AI's
6-128 for `user_id` (from Z.AI's docs); Meta and xAI document none, so none
is claimed.

The xAI image `#edit` path now carries the identifier like generation, and
takes a named-options param so `user` cannot be transposed with the
adjacent same-typed `aspectRatio`.

Tests share a four-actor matrix (`user` / `user+app` / `access token` /
`system`) with `assertActorMatrixIdentifiers()` across the six
OpenAI-style suites; the helper has exact-string and boundary coverage
(size caps, zero-budget and sub-base cases, no dangling separator, UUID
never truncated, collision guard); the Azure Grok assertions run under a
real user actor so they cannot pass vacuously. 212 provider-suite tests
pass; typecheck and ESLint are clean.

* fix(ai): lock the vendor identifier down and keep vitest out of the test util

Meta and Z.AI no longer let `custom` override the abuse identifier; it is
Puter's attribution, not the caller's. The shared test util exposes pure
field pickers instead of importing vitest into a file the production
tsconfig compiles. The helper's length-cap comment now matches vendor docs
(Meta does cap `safety_identifier` at 64), the redundant budget branch and
the unused export are gone, and the per-user `prompt_cache_key` trade-off is
stated once in the helper instead of five times in providers.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 16:21:47 -07:00
404oopsandClaude Fable 5.1 027e9a71f3 fix(ai-image): refresh model catalogs, unify txt2img requests, expand docs (#3889)
* fix(ai-image): sync image catalogs, unify request shape, document models

Model sync against every vendor listing and a live generation sweep:
Gemini stable ids replace the retired preview spellings (2.5 Flash Image
delisted ahead of its 2026-10-02 shutdown), OpenAI gpt-image-1/-1-mini/-1.5
are delisted but routable until their shutdown dates with gpt-image-2 as
the default, xAI gains grok-imagine-image-2.0 with its quality tiers,
BytePlus gains the 3K/4K tiers and per-model pixel bounds, Cloudflare
gains SDXL Lightning/Base and SD 1.5 Inpainting, Replicate gains a
schema-driven catalog of 88 additional models with version-pinned
community predictions, and every Together image route is excluded for
the third-party data-sharing requirement.

Request normalization: the driver collapses ratio/width/height/aspect_ratio
into one imageSize with aspect-versus-pixel intent, validates prompt,
quality and resolution once, resolves provider hints (short names or
full driver ids) and aliases with exact ids winning over resellers, and
hands each provider an immutable copy of the caller's args. Providers
share prompt validation, aspect snapping, and content-sniffed data URIs
so responses carry the right MIME type. Replicate predictions are created
once, cancelled on abort or deadline, and bounded in input fan-out.

SDK: txt2img copies caller options, rejects blank prompts with
prompt_required in every call form, and documents that normalize has no
effect because the image return shape is already uniform.

Docs: txt2img options per provider, provider defaults and discovery,
availability notes, a new image model catalog and pricing page, and the
image-generation limits in the quotas page.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai-image): sniff SVG outputs correctly, reject hex/exponent dimensions

imageDataUri only decoded the first 24 base64 chars (18 bytes) before
sniffImageMime, which cannot see an <svg> root behind an XML prolog -
SVG outputs from recraft-v*-svg models were being labeled image/png.
Decode enough of the payload to cover the 8 KB SVG sniff window.

dimension() accepted string number literals ('0x10', '1e3') as if they
were decimal dimensions; restrict to plain decimal notation.

* docs(ai): link to the model directory instead of a static catalog

The image model catalog and pricing page duplicated the always up to date
model directory at developer.puter.com/ai/models. Link to the directory
from txt2img-related docs and keep the Together data-sharing exclusion
note self-contained.

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 15:00:02 -07:00
Juan Castro 259cb69fde fix: check-app reported every app as already authorized
The token grant writes `flag:app-is-authenticated` on the user->app row,
but the check asked for `service:<app_uid>:ii:flag:app-is-authenticated`.
Those never match, and the parent walk reduces the question to a bare
`service` — a root every user holds by default — so the check answered
true for any app_uid and handed back an app-under-user token for an app
the user had never opened.

Read the row the grant writes, through one shared constant so the two
cannot drift again, and mint against the resolved uid.

A cancelled account picker no longer leaves `popup_signin_consent` set:
with no relationship there is no token to flip it.
2026-09-21 16:13:53 -04:00
404oopsandClaude Fable 5.1 081941a6f3 fix: null-prototype model maps in chat/video drivers; txt2vid prompt and option guards (#3887)
Chat and video drivers keyed their provider and model maps on plain objects,
so `model` or `provider` values such as `__proto__` reached Object.prototype
and surfaced as 500s. Both maps are now null-prototype objects and reject
those names as ordinary unknown models.

txt2vid now rejects blank or non-string prompts with `prompt_required` before
any request, tolerates `null` in either argument slot, and copies the caller's
options before resolving the `duration` alias and output path so frozen
option objects work and caller objects are never mutated.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 13:12:35 -07:00
404oops e2a5ab8cdc docs: document local Docker builds with Compose overrides (#3867) 2026-09-21 13:12:16 -07:00
Juan Castro 07e889f135 fix: stop leaking an unawaited permission traversal on every check
`validateUserPerms` started the linked SQL traversal beside the flat read
and returned on a flat hit without awaiting it. The hit is the common
case, so nearly every user-permission check left a floating promise: with
`keep_alive_on_uncaught` unset no `unhandledRejection` listener is
registered, so a DB wobble inside the traversal exits the process.

Run it only when the flat read misses. That also stops a SQL walk the
cache had already answered.
2026-09-21 16:07:26 -04:00
Juan Castro ab7a5f325a fix: refuse an empty app_uid and survive a failed grant check
A present-but-empty `app_uid` fell through to checking the user, where
every `fs:` scope on their own file answers `true` — a prompt settled by
a question nobody answered. It is a 400 now.

The launch-path check also sat outside any catch, so a rejecting read
would have taken the whole launch down instead of falling through to the
prompt. Renamed the seam it is injected through to match the function it
defaults to, since wiring the token-based sibling in its place would send
an app uid as a bearer token and silently always prompt.
2026-09-21 15:56:39 -04:00
Juan Castro b4b464554b fix: settle a URL file-access prompt from a grant the app already holds
Opening `/app/<name>?file=<path>` asked for consent on every launch: the
gate prompted unconditionally and never read the `fs:<uid>:write` grant
its own Allow had written.

`/auth/check-permissions` takes an optional `app_uid` so the account can
ask what one of its apps holds, and the launch gate skips the dialog when
the answer is yes. Sessions only — an app or a scoped token asking would
be a window onto its neighbours' grants.
2026-09-21 15:00:02 -04:00
Nariman Jelveh 4de1d1fb39 Redesign the Teams tab in the dashboard (#3901)
* Redesign the Teams tab in the dashboard

Hero card with a lettered tile, role pill, handle and headcount; the
directory setting becomes a settings row with the shared toggle switch;
the add-account form gets labels, Enter to submit, and a copy button on
the one-time password. The accounts and record tables drop the grid for
hairlines, avatar tiles and status pills, and stack into labelled rows
on phones so actions stay reachable. Loading, empty and error states
get a skeleton, a create call to action, and a retry.

Also fixes audit dates showing 1970 (the audit routes send unix
seconds), the actions cell breaking the row border by being a flex
table cell, and the "0 suspended" clause on a healthy team.

* Use the dashboard's overlay dialog in the Teams tab, not UIAlert

Adds UIDashboardDialog, a confirm / alert / prompt card built like the
share and properties modals: mounted in the dashboard window, revealed
after a frame, dismissed by Escape, the close button or a backdrop
press-and-release, with a focus trap and focus restored on close. On
phones it docks to the bottom as a sheet. The Teams tab's confirms,
error alerts, seat-limit warning and rename/create prompts all go
through it; UIAlert and UIPrompt are no longer imported there.
2026-09-21 09:30:03 -07:00
Daniel Salazar 4e9b03bf5c fix: harden clickjacking (#3912) 2026-09-21 09:19:37 -07:00
Juan Fernando Castro a63172e9ae Merge pull request #3892 from HeyPuter/juancastro/put-1806-invite-email-addresses-are-disclosed-to-apps-manage
fix: stop disclosing invite addresses to apps, tokens and delegates

approvals already in place
2026-09-21 12:05:08 -04:00
Reynaldi Chernando 29d805d793 show app icon when login from external app (#3905)
* show app icon when login from external app

* fix
2026-09-21 22:33:38 +07:00
Juan Fernando Castro 059fae3d15 Merge pull request #3872 from HeyPuter/juancastro/put-1813-team-share-blocklist-and-unshare-paging
fix: make the block list bite on team shares, and unshare every re-share (PUT-1813)

Approvals in place to move with the merge
2026-09-21 11:31:24 -04:00
Juan Castro 18ada4764a chore: drop a stray lockfile marker
A local `npm install` (needed to pick up main's new `postal-mime`) wrote
`"extraneous": true` onto a bundled `@clack/prompts` dependency. It was the
branch's only lockfile change and it does not belong to this PR.
2026-09-21 11:26:27 -04:00
Juan Castro 72203152d9 fix: bound a scoped token to what it issued, which is nothing
CI caught three HTTP-level tests the earlier merge left behind, and they
were right to fail: dropping this branch's `manage` gate in favour of
main's row filter lost a case main's filter does not cover.

Main bounds an app to the rows it issued. A *scoped* access token is not
an app, so it was falling through unbounded — `/fs/stat` with
`return_shares` handed a `fs:<uid>:list` token the owner's whole share
list. Addresses stayed withheld, but who else can reach a file is no
more a narrow token's business than the addresses are.

So the filter is generalised rather than the gate restored: a scoped
token is bounded to nothing, since it issues nothing under its own name.
Filtering it by a null app would have been worse than not filtering —
that matches the owner's own rows. Apps and sessions behave exactly as
they do on main, and a full-access token still holds the account's reach.

The three tests now assert the answer instead of a refusal, including
the one whose name had always promised a refusal its body never checked.
2026-09-21 11:12:37 -04:00
Juan Castro cec8382d4d Merge branch 'main' into juancastro/put-1806-invite-email-addresses-are-disclosed-to-apps-manage
Two textual conflicts, both additive on each side: `isAccountContext`
(here) and `isPlainUserActor` (main) are both imported and both used,
and the `stat()` note keeps both sentences — this branch's on who may
read an invite address, main's on the share-read limit it spends.

The rest is adapting to main, which grew its own answer to half of what
this branch was for. `listSharesOf` there bounds an app to the rows it
issued itself; this branch instead required the credential to hold
`manage` and refused it otherwise. Main's is the better mechanism — it
answers the app rather than turning it away, and it hides other
issuers' rows outright rather than redacting a field on them — so the
`manage` gate goes, and with it the two tests that asserted the
refusal. They are replaced by tests that hold main's line: an app sees
none of the invites it did not send, with or without `manage`.

What this branch still carries is the gap main does not close. Its row
filter only applies to apps, so a plain manage delegate still reads the
owner's invite addresses; `#maySeeInviteAddress` is what withholds
those, and its delegate tests pass unchanged. The `stat()` note is
corrected to describe main's behaviour rather than the removed gate.
2026-09-21 10:46:07 -04:00
Juan Castro 3f335939b3 fix: team shares are not subject to a recipient's per-sender block
Review call from the ticket's author: a team share is the team's, not
one colleague's to withhold from another, so a personal block should not
hide it. This drops the enforcement added earlier on the branch — the
listing, its total and the fs-event fan-out no longer filter team rows
by the recipient's block list, and the SQL fragment that did it goes
with them.

What a block still does for a team share is suppress the notification,
which was already true before this branch: it stops the interruption
without pretending to stop the access. Leaving the team is what ends
that. Said so in the block API docs, the settings copy and the code,
since the mismatch between the two was the original complaint.

The unshare sweep is untouched — that half of the ticket stands.
2026-09-21 10:28:50 -04:00
Juan Castro 4e821c128c Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging
# Conflicts:
#	src/backend/services/share/ShareService.ts
#	src/backend/stores/share/ShareStore.js
2026-09-21 10:28:33 -04:00
Juan Fernando Castro c313a381b6 Merge pull request #3896 from HeyPuter/juancastro/put-1843-plan-card-unknown-not-free
fix: a team seat's plan card names its team, and an unreadable plan is not Free (PUT-1843)
2026-09-21 09:58:41 -04:00
Juan Fernando Castro 76e0d2ec93 Merge pull request #3871 from HeyPuter/juancastro/put-1798-sharing-email-notification-changes
feat: name the issuing app in share emails, and cut the digest window to 5s
2026-09-21 09:58:09 -04:00
Juan Fernando Castro 24fb679dd3 fix: charge stat's return_shares against the share-listing budget (#3870)
* fix: charge stat's return_shares against the share-listing budget

`return_shares` on /fs/stat and the legacy /stat runs the same work as
GET /share/shares, but was only metered under fs:stat's far more
generous limit — and the two scopes stacked instead of sharing one
counter.

Adds consumeRouteRateLimit(req, spec), an imperative charge that
resolves the key and per-subscription limit exactly as rateLimitGate
does, so a handler can conditionally spend a second scope when a
request flag makes the route expensive. Both stat handlers now charge
share:list before doing the listing work; SHARE_LIST_LIMIT moves to a
shared share/limits.ts so all callers pin the same spec.

Closes PUT-1597.

* feat: consumeRouteRateLimit takes the array spec form too

Review follow-up on #3870: a multi-window spec passed whole would have
read an undefined window and silently never pruned. Charge each window
in order instead, refusing on the first refusal, matching the gate.
2026-09-19 14:11:29 -07:00
Daniel Salazar eb9f03e984 fix: misc hardening + other fixes (#3906) 2026-09-19 12:57:57 -07:00
Daniel Salazar 9292771554 fix: hardening (#3904) 2026-09-18 11:22:39 -07:00
Reynaldi Chernando f4974581d0 Add auth message for external apps using puterjs (#3893) 2026-09-18 08:56:02 +07:00
Daniel Salazar b39bb771d9 feat: sortby for recursive readdir (#3900) 2026-09-17 17:29:55 -07:00
Nariman Jelveh 766379ae9f feat(puterjs): add defensive profile picture lookup (#3899) 2026-09-17 17:16:54 -07:00
Daniel Salazar 0be3bc55c2 feat(metering): AI cost multiplier hook for AI drivers (#3898)
Emits ai.cost.multiplier.<driver>.<provider>:<model> before recording AI
usage, so what a model costs to charge is policy an extension owns rather
than a number in core. Nothing listening records the provider cost.

MeteringService.withAiCostMultiplier(driver) returns a view of the service
whose recording paths scale costOverride by the hook's answer; every AI
driver hands that view to its providers, so all of them are covered without
touching provider code.
2026-09-17 15:18:38 -07:00
Nariman Jelveh d1484af754 One recipient field in the sharing dialogs, with suggestions (#3897)
* feat: one recipient field in the sharing dialogs, with suggestions

Both sharing dialogs asked twice: a text field for a person, and a
separate select, label, note and button for a team. Which control to use
was a fact about the API — a bare string is read as an email or a
username, so a team could not be typed — not something a user should
have to know.

Now there is one field. Clicking it offers who this account shared with
before, the teams it belongs to, and the people in them; typing narrows
the list. Choosing an offer locks the field to that recipient, so a team
picked by name still goes out by uid, and the note about a team grant
reaching everyone in it appears at the moment it applies. Anyone already
on the access list is left out of the offers, and a typed address still
works untouched.

Past recipients are kept in the account's key-value store, so they
follow the user between browsers; a typed address that turned out to
belong to an account is filed under the username the backend resolved.
Colleagues are read one page per team, cached for five minutes, and only
looked up once the field is actually used.

The picker is one helper shared by both dialogs so the two can't drift,
styled through `share-suggest-*` tokens each host restates in its own
palette. It opens in the flow rather than floating, which a scrolling
modal body and a mobile bottom sheet would otherwise clip.

* fix: float the recipient suggestions instead of resizing the dialog

The list opened in the flow, which pushed the Share button and everything
under it down and back up as it opened and closed — a dialog that resizes
under the cursor. It now hangs off the recipient row, out of the flow, and
floats over what follows it.

It goes inside the row rather than after it, so the row is its containing
block; as a sibling it resolved against whatever was positioned further up
and landed at the bottom of the scrolling body. Being out of flow it can
be cut off by that scroll container rather than scrolling itself, so on
open it measures the room between the row and the nearest clipping
ancestor, caps the list to it, and opens upwards where below is too tight.

* fix: keep the recipient field focused when the picker is wired onto it

Both sharing dialogs focus the recipient field as they open, and the
picker then moves that field into its wrapper. Moving an element takes
it out of the document for an instant, which drops its focus, so every
dialog opened with the caret in the field opened with the caret on
<body> instead.

* fix: keep the recipient suggestions inside the screen

The list sized itself against the nearest clipping ancestor alone. That
ancestor is the dialog's own scrolling body, which reaches past the
bottom of a short screen — or of any screen once the dialog has been
dragged low — so the list was placed below the fold and the user saw
nothing at all. The viewport bounds it too.

* fix: stop the suggestions flashing a loading box over the dialog

Clicking the recipient field put up a panel saying it was loading before
it knew there was anything to load. For the many accounts with no teams
and nobody shared with before, that panel covered the Share button for
as long as the round trip took and then vanished again. It now waits a
beat before saying anything, and leaves whatever is already listed in
place while a reload is in flight. A list still on its way also no
longer swallows the Escape that closes the dialog.

* fix: do not double-encode the recipient field's placeholder

`i18n()` encodes what it returns, which is right for a string dropped
into markup and wrong for one handed to `.attr()` — the entities show up
as themselves. Nothing is lost today because only English carries the
key, but the next translation with an apostrophe in it would read
`l&apos;équipe`.

* chore: drop the styles for the team picker the field replaced

The desktop dialog's separate team control went with the one recipient
field; its rules stayed behind. The dashboard's equivalents were already
removed with its own markup.

* fix: let Enter share from the desktop dialog's recipient field

Typing a name and pressing Enter did nothing there: the field is in no
form, so the only way to send was to reach the button. The Dashboard's
dialog has always submitted on Enter, and the field now takes Enter to
choose a suggestion, which makes a second press that does nothing read
as a dead key. The picker still gets the press first while it is
choosing a row.
2026-09-17 13:54:03 -07:00
Juan Castro dfd7925872 fix: say which team a seat's plan belongs to, paid or free
A provisioned account read as a plain "Free" plan with an "Upgrade for
more features" badge: nothing said the account belongs to a team, and
the prompt asked for an upgrade only its owner can buy. Its free tier is
not the free plan either — org_seat_free is half the allowance.

The card now names the team on both paths: a paid seat keeps its tier
and status and gains "Managed by <team>", and a seat with no tier reads
"Team account" instead of an upgrade pitch.
2026-09-17 14:15:01 -04:00
Juan Castro e50d210423 fix: an unreadable plan is not a free plan (PUT-1843)
The Home tab read /marketplace/subscriptions/current and folded every
failure into the free state: a non-OK response left `subscription` null,
and the catch said so explicitly. So a refusal rendered as a confident
"Free" with an Upgrade badge.

That is reachable: verification gates are default-on for authenticated
routes, and a team seat that still owes its password change is refused
there — it then reads Free while its team pays for a tier. A rate limit
or a blip does the same to anyone.

The plan card moves into its own method so a failed read can return
without touching it, leaving the card as it was rather than naming a
plan the account does not have. A 200 carrying no subscription still
reads Free, which is the one case that actually means it.
2026-09-17 13:52:03 -04:00
Daniel Salazar 51ce868d1e fix: shared kv handles also allow value opt in (#3895) 2026-09-17 10:34:35 -07:00
Juan Castro 461cb7d0a9 fix: close two gaps the review found in the invite-address fix
- A full-access token was denied the address while `shared-by-me` still
  handed it the same rows, so the clause bought no privacy and cost an
  API client the address `unshare()` takes. `isAccountContext` is the
  boundary the rest of the codebase already uses for 'acting as the
  account': plain session or full-access token, never a scoped one.
- The Dashboard share modal dropped a withheld invite entirely, since
  its aggregate keys a pending row on the address — so a delegate saw no
  sign of an outstanding invite and accessCount under-reported who could
  reach the item. It is now kept, keyed on the share uid, labelled, and
  without the controls that would need a recipient to address.
2026-09-17 13:21:13 -04:00
Daniel Salazar bb97660ad7 feat: kv events value opt in (#3894) 2026-09-17 00:17:12 -07:00
Juan Castro 3e3d02bafe fix: stop disclosing invite addresses to apps, tokens and delegates
getShares (and stat's return_shares, which runs the same listing) gated
on #assertCanManage's default 'see' mode, so any credential that could
see the node got every unclaimed invite's raw email — including an app
handed one file by the picker, a list-scoped token on the stat surface,
and a manage delegate reading the owner's invitees.

Two bounds, matching the invariant clientShare.ts already claimed:

- An invite's address goes only to the item's owner and to whoever sent
  it, and never to an app or token. A delegate can revoke only what they
  issued, so withholding costs them nothing they could act on.
- An app or token must hold manage reach of its own to read the listing
  at all; it answers for the ancestors too, which is not what being
  handed one file grants. tryListSharesOf turns that into an empty
  shares array, so stat itself keeps working.

The share dialog names an unattributable invite rather than rendering a
blank row with a dead revoke button.

Closes PUT-1806.
2026-09-16 18:59:10 -04:00
Neal Shah b2fef45fd6 email read api (#3869)
* email read api

* address docs changes

* remove playground examples
2026-09-16 18:38:58 -04:00
Juan Castro 728966bfb1 Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 18:05:58 -04:00
Juan Castro 3f7ee34706 Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 18:05:39 -04:00
Juan Fernando Castro ff333d05e8 Merge pull request #3890 from HeyPuter/juancastro/put-1819-keep-plan-console
feat: tell the plan picker when a seat's plan is on the way out
2026-09-16 18:05:12 -04:00
Daniel Salazar 194ca7c789 fix: list a link share only while the owner's plan covers it (#3891)
* fix: list a link share only while the owner's plan covers it

* chore: types + jsdoc cleanup
2026-09-16 14:42:52 -07:00
Juan Castro 0b85203edb feat: tell the plan picker when a seat's plan is on the way out
The billing view now names each seat's status; the console passes it
through to the picker so a cancel-pending plan can be kept, plus the
two strings that flow renders.
2026-09-16 15:55:52 -04:00
Juan Castro 23c33b2e06 Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging
# Conflicts:
#	src/backend/stores/share/ShareStore.js
2026-09-16 15:27:09 -04:00
Daniel Salazar 1b6334c02c feat: a paid plan counts as a verified card for the sharing gate (#3886)
* fix: list a link share only while the owner's plan covers it

* fix: fs limits for signed urls

* feat: a paid plan counts as a verified card for the sharing gate
2026-09-16 10:18:47 -07:00
Juan Castro 0593844acf Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 13:05:03 -04:00
Juan Castro 67d048148c Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 13:04:43 -04:00
Juan Fernando Castro 05cbd33211 Merge pull request #3888 from HeyPuter/juancastro/bump-sqlite-schema-version-80
test: schema version is 80 since 0084
2026-09-16 13:04:15 -04:00
Juan Castro 5c8f02384c test: schema version is 80 since 0084
#3874 added 0084_share-anyone-with-link.sql, taking the sqlite chain's
target user_version to 80, but CURRENT_SCHEMA_VERSION stayed at 79 —
and the test workflow only runs on PRs, so main broke silently and every
open PR's 'test (base)' job now fails on it.
2026-09-16 12:48:07 -04:00
Juan Castro 5d20e054ac Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 10:33:03 -04:00
Juan Castro 3bd3955e0d Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 10:32:05 -04:00
Reynaldi Chernando 79c856386f document model variants (#3875) 2026-09-16 19:31:28 +07:00
Daniel Salazar 59f73528cb feat: share with anyone with the link (PUT-1580) (#3874)
* feat(email): inline cid attachments and Puter mailbox delivery for sendTransactional

EmailAttachment gains cid/contentDisposition so the transactional driver can send inline images. The SDK's EmailAttachment typedef now comes from types.js, which already carried cid. Docs describe delivery to <username>@puter.email recipients and the not_found code.

* feat: share with anyone with the link (PUT-1580); gate sharing on a verified phone or card
2026-09-15 21:06:12 -07:00
Juan Castro 27e858d760 fix: gate the group-share index delete on authority, qualify the live filter
Review follow-ups on #3872: deleteActiveGroup ran even when every
canManagePermission check was denied, deleting the index row while the
team's grant survived — unlisted live access, the class this PR exists
to remove. And memberIdsAmong's join left deleted_at unqualified, which
only works while jct_user_group has no such column; #live now takes an
alias.
2026-09-15 18:57:50 -04:00
Juan Castro 0bd6458441 test: pin the mixed-digest button link carrying no app
Review follow-up on #3871: the flush-level rule — shared_app on the
button only when every entry came through one app — had no direct
coverage. An app share and a plain share folded into one digest now
prove the line and item link keep their attribution while the button
stays clean.
2026-09-15 18:56:38 -04:00
Daniel Salazar 45aff36f0c fix: auto create folders for fs perms (#3873) 2026-09-15 15:11:13 -07:00
Nariman Jelveh a1f7577acb fix: warn before closing the tab while an upload is in flight
Uploads stream from the page, so closing the tab loses a nearly-finished
one with no warning. The beforeunload handler only existed behind
prompt_user_when_navigation_away_from_puter, which is off by default.

Install it unconditionally and have it consult window.active_uploads,
the registry the progress code already maintains. The feature flag keeps
its open-window behavior.

Uploads now register in active_uploads before the first await in the
init callback: an upload failing while the progress window was still
opening used to run its delete ahead of the insert, leaving a phantom
entry that would have made the tab unclosable.
2026-09-15 13:07:43 -07:00
Juan Castro af6547724d Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-15 15:23:22 -04:00
Juan Castro e9922aea26 Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-15 15:23:11 -04:00
Juan Fernando Castro b136c56cb5 Merge pull request #3846 from HeyPuter/juancastro/put-1792-optional-seat-email
feat: the team seat experience — no email required, forced password change, team label, and plan-based limits (PUT-1792)

Note: Bypassing the code owners rule, since there are couple approvals in place for this.
2026-09-15 14:53:06 -04:00
Juan Castro aadcda073d fix: keep blocks out of the authority graph, and sweep what unshare missed
Code review on the previous commit confirmed three ways the block filter
inside readUserGroupPerms turned a reversible block into permanent loss:
canManagePermission reads the same rows, so a block-suspended grant
looked revoked to #revokeDownstream's cascade, to unshare's authority
gate, and to invite claiming — deleting re-shares and invites that were
supposed to come back on unblock. The scan is now deliberately blind to
blocks; a block suspends delivery only (listing, count, fan-out,
telling), which the share store filters itself through one shared
notBlockedSql fragment owned by UserBlockStore.

The team-unshare sweep also now covers what it claimed to: a member's
re-share *to another team* is revoked with them (group rows swept in
 #revokeDownstream, user path included), and the sweep is skipped
entirely while another issuer's grant still backs the team — members'
re-shares rest on that authority and revoking them would orphan live
access. The blanket block-all note in settings now says team shares
still arrive, matching what the code deliberately does.
2026-09-15 14:49:54 -04:00
Juan Castro 9da03555ae fix: make the block list bite on team shares, and unshare every re-share (PUT-1813)
A sender a member blocked could still land items in that member's
shared-with-me — and grant them real access — by sharing with a common
team. The group grant is one row and cannot exclude a member, so the
block is now enforced where delivery is derived per member: the
permission scan, the inbound listing and its count, and the fs-event
fan-out all skip team rows whose issuer the member blocked. Nothing is
revoked, so unblocking restores everything. Blocking now also bumps the
blocker's permission cache so it bites immediately. Direct shares keep
their contract: existing ones stand.

#unshareTeam swept re-shares by listing members with a 200 cap and
never following the cursor, so members past the cap kept their orphaned
rows. It now walks the share rows in the subtree — the set that can
actually need sweeping — and filters those issuers to members, which
has no cap by construction.
2026-09-15 14:49:54 -04:00
Juan Castro f428797a79 feat: render the teams UI for allowlisted users without the global switch
`gui_params.teams_ui` stays the deployment-wide switch; with it off, the
tab now also renders for a signed-in user who passes the email-domain
allowlist (membership included, so seats see their roster). Anonymous
renders hide it, and the API keeps deciding real access either way.
2026-09-15 14:48:11 -04:00
Juan Castro ad3d15e7e2 feat: stage the teams rollout behind an email-domain allowlist
`teams_allowed_email_domains` limits who may enter the teams surface;
unset keeps today's behavior. Gated on the two routes that constitute
entry — creating a team, and the listing that shows the tab — with the
same 404 a teams-off deployment answers, so the GUI needs no change and
a staged rollout is indistinguishable from the feature being off.
Members of an existing team always pass, whatever their domain: an
allowed owner brought them in, and the surface follows the team.
2026-09-15 14:23:49 -04:00
dependabot[bot] 07bf0be9c7 chore(deps): bump peter-evans/create-pull-request from 6 to 8 (#3865)
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 8.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6...v8)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:39:50 -07:00
dependabot[bot] d116f49669 chore(deps): bump dorny/paths-filter from 3 to 4 (#3866)
Bumps [dorny/paths-filter](https://github.com/dorny/paths-filter) from 3 to 4.
- [Release notes](https://github.com/dorny/paths-filter/releases)
- [Changelog](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/dorny/paths-filter/compare/v3...v4)

---
updated-dependencies:
- dependency-name: dorny/paths-filter
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:35:19 -07:00
dependabot[bot] a5f267c2d4 chore(deps): bump docker/setup-qemu-action from 3 to 4 (#3864)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:35:07 -07:00
Daniel Salazar 2fdd67c72e fix: tighten up fs perm strings (#3860) 2026-09-15 10:33:47 -07:00
dependabot[bot] 92dfe4019e chore(deps): bump compression from 1.8.1 to 1.8.2 (#3861)
Bumps [compression](https://github.com/expressjs/compression) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/expressjs/compression/releases)
- [Changelog](https://github.com/expressjs/compression/blob/master/HISTORY.md)
- [Commits](https://github.com/expressjs/compression/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: compression
  dependency-version: 1.8.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:30:41 -07:00
dependabot[bot] 61a6820767 chore(deps-dev): bump @types/node from 24.13.3 to 24.13.4 (#3862)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.3 to 24.13.4.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:29:37 -07:00
dependabot[bot] 38b131e5e8 chore(deps): bump libphonenumber-js from 1.13.12 to 1.13.13 (#3863)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.12 to 1.13.13.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.13.12...v1.13.13)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:29:09 -07:00
Juan Castro 230241d6d2 fix: let an owner retire the seats of a deleted team, and only those
Deleting a team suspends every provisioned seat, but every member route
resolved live teams only — so the suspended seats could never be
deleted afterwards, stranding the accounts and, on the billing side,
their paused subscriptions. deleteMember now resolves the team
soft-deleted or not, matching the audit reader.

It also gains the guard enableMember always had: only the team's own
suspension qualifies. A platform-suspended seat could previously be
cascade-deleted by the owner, destroying an account Puter had frozen.
2026-09-15 13:22:13 -04:00
Juan Castro 577693bf84 feat: name the issuing app in share emails, and cut the digest window to 5s
When an app shares on a user's behalf, the mail now says so — the
digest line reads 'alice shared report.txt via Mail App', preferring
the app's title, then its name, then its index_url host. Both the
per-item links and the 'Open Puter' button carry a new shared_app
query param (the app's name, or its uid) so the GUI can match the
share back to the app; the button only carries it when the whole
digest came from one app. Plain shares are byte-for-byte unchanged.

Also drops SHARE_EMAIL_BATCH_SECONDS from 30 to 5: the window only
has to fold one gesture's worth of calls, and every second there is
a second the common single-share email arrives late.

Closes PUT-1798.
2026-09-15 12:15:30 -04:00
Juan Castro 27dd0f8ce6 docs: publish only the team surface an app can actually call
Every team route except the directory refuses app and API-token callers —
administration belongs to the account console. Publishing provisioning,
credentials, suspension and audit in the app-developer docs invited
integrations that would 403 on their first request, so those thirteen
pages are gone and the overview is rewritten around what an app can do:
detect team context with list(), look up colleagues with listDirectory(),
and share with a whole team.

listDirectory — the one app-callable method — was also the only one with
no page; it has one now, leading with its consent gate. The team
recipient joins the share() docs, where the integration actually happens.
2026-09-15 09:34:17 -04:00
Daniel Salazar 50d3794283 feat: handle sub required endpoints better (#3859) 2026-09-14 21:39:01 -07:00
Daniel Salazar 7fdbc0247f fix: bug bounty dupes (#3858) 2026-09-14 16:04:00 -07:00
Juan Castro 3bf5ba3525 feat: i18n keys for the seat cancel flow 2026-09-14 18:40:37 -04:00
Juan Castro 018196f8b4 feat: i18n keys for the team billing surfaces
The private billing extensions render user-facing strings; their keys
live here with every other translation.
2026-09-14 17:22:04 -04:00
Juan Castro c3e2717e8e fix: close the review findings on the seat experience
From the adversarial review of this PR.

The forced password-change gate could deadlock: it POSTed to the
cookie-only route with a bare fetch, and both initgui call sites open it
before update_auth_data mints the session cookie — a fresh browser with a
token URL 401'd every submit inside a non-dismissible loop. It uses the
session-cookie retry wrapper now, taking the caller's token because
window.auth_token does not exist yet on that path. It also gains the
logout footer its sibling gates have; a lost temporary password was a
hard lock with devtools as the only exit.

Password recovery refused for seats: the address is admin-supplied and
never verified, so whoever holds that inbox could take the seat over at
any later time. A seat's recovery channel is its admin's reset.

change-email gets the same seat guard as change-username and deletion —
the address is where admin-issued credentials go.

The login response now carries `team` alongside requires_password_change:
no-reload logins store that payload as window.user verbatim, and every
seat restriction keys on it.

Smaller: the team-badge tooltip no longer double-encodes; the create-token
hint for an emailless account stops pointing at a verification it can
never perform; the quotas doc records the halved org_seat_free allowance;
the config template tells upgrading operators how to keep the old flat
cap; the SDK suite covers emailless provisioning and the owner-only uuid.
2026-09-14 17:00:18 -04:00
Juan Castro 15527eac4c fix: hold a team seat to the free caps it was meant to have
Review catch by @Salazareo: `org_seat_free` reached `FREE_SUBSCRIPTION_IDS`
and so the `requireSubscription` gate, but two other surfaces decide on
plan and neither consults that set.

`bySubscription` maps name `user_free` and `temp_free`. A plan that
matches no key fell through to the top-level `limit` -- the paid cap --
so a seat outranked an ordinary free account: 240 event listings a minute
against their 120, and the same shape across the kv, notification,
subdomain and worker drivers. Both resolvers now fall back to the
`user_free` entry for anything in the free set, which covers every driver
at once and any free plan added later.

`subscriberOnly` compared against the two named ids, so a seat could
reach a paid-only model. It asks the set now.

A paid plan that names no cap of its own still takes the base, and an
unresolved plan still takes the base; there are tests for both so the
fallback cannot widen into "free by default".
2026-09-14 09:44:32 -04:00
Nariman Jelveh 4a23c296f0 Fix/dashboard apps files audit (#3857)
* Dashboard Files: keep rows in sorted order when icons resolve late

renderDirectory appended each row as soon as its icon resolved. Icons for
weblinks and .app files are read from the file itself, so those rows always
landed at the end of the listing regardless of the sort column or direction.
Resolve every icon first, then append the rows in sorted order; renderItem
takes the pre-resolved icon and still looks it up itself for single-row
callers (socket adds, instant folder creation).

* Rename: store the new item name raw in data attributes and the editor

rename_file wrote html_encode(new_name) into data-name, title and the
name editor's value. jQuery's attr()/val() store strings literally, so a
file renamed to "a&b.txt" carried data-name "a&amp;b.txt": the dashboard
Files tab, which re-reads data-name for column truncation, then showed the
entity on screen, type-to-select and sorting compared the encoded string,
and the editor reopened on the encoded name. The initial render already
stores these raw; make the rename path match.

* Dashboard Files: Enter with several folders selected enters only the first

The Enter handler called pushNavHistory + renderDirectory for every selected
folder. Only the first render runs (renderDirectory drops calls while one is
in flight), but every folder was pushed onto the navigation history, so after
selecting two folders and pressing Enter, Back led to the folder already on
screen and Forward to one that was never opened. Enter now navigates into the
first selected folder only; selected files still open in their apps.

* Dashboard Files: don't offer to move items the user can't move

Trashing or cutting sends an item to its owner's trash or a new folder, which
a shared root or a read-only share can't do — the server answers Forbidden and
the user gets an error alert for an action that should not have been offered.
The single-item context menu already leaves Delete out for such rows; the
Delete key, Ctrl/Cmd+X, the multi-selection menu and the mobile selection bar
did not check. Route all of them through can_restructure, skipping rows that
can't move and hiding Cut/Delete when the whole selection can't.

* Dashboard Files: hide New Folder and Upload in the Shared view

Shared is a query over other people's items, not a directory. Both buttons
stayed visible there: New Folder did nothing, and Upload opened the OS file
picker and then silently dropped whatever the user picked. Hide them the same
way Trash already does.

* Dashboard Files: let the same file be picked again after a failed upload

The upload input was only cleared in the success callback, so after an upload
that failed, was cancelled, or was blocked (Shared view), choosing the same
file again fired no change event and nothing happened. Snapshot the picked
files and clear the input as soon as the change fires. This also drops the
document.querySelector('form').reset() that reset whichever form came first
in the document rather than this one.

* Dashboard Files: read saved preferences in parallel and re-sort immediately

init awaited four independent kv reads one after another before the first
listing could start, and handleSort awaited two kv writes before re-rendering,
so a sort click cost two round-trips before anything moved. Issue the reads
together and let the writes settle in the background. While here, set
$el_window before the first await (renderDirectory reached through a route
change or socket event during init threw and left renderingDirectory stuck),
and tolerate a corrupt saved column_widths value instead of failing init.

* Dashboard Files: show the loading spinner on the first directory load

showSpinner keyed off a loading flag, but clearing the listing at the start of
a navigation removed the overlay without resetting it: init's spinner was
wiped by the first render's clear, and the render's own showSpinner call then
no-oped, so the initial load ran with no indicator. Guard on the overlay's
presence instead.
2026-09-13 11:31:16 -07:00
Ayoub Ait ChikhandDaniel Salazar 790db87e13 Fix/node signed batch upload (#3816)
* puter_signed_upload_error_correction

* docs: update signed upload behavior

* fix(puter-js): complete a bodiless response in the XHR shim

A response with no body at all (`fetch` gives `null`) threw inside the
shim's own `then`, the same way a missing content-type did: no 'load',
no 'error', the request hangs forever. Signed storage PUTs answer in
exactly that shape.

Adds the regression tests the content-type fix was missing — all three
hang against the shim as it was.

* refactor(puter-js): drop the signed batch-write env allowlist

With `nodejs`, `service-worker` and `web-worker` added, the list held
every value `puter.env` can take, so the gate decided nothing — and a
new env would have been silently routed to the legacy path.

The backend capability check (`signedBatchWriteSupported`, set from a
404/405/501 on `startBatchWrite`) is the one that still does work.

* test: run the directory-upload suite on node and workers

Both were pinned to browser because the legacy `/batch` fallback cannot
create a directory tree. Now that every platform takes the signed path
they are the regression test for it, and the stale comment goes with
them.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-12 16:32:36 -07:00
dependabot[bot] b901040c14 chore(deps): bump nodemailer from 9.0.6 to 9.1.1 (#3832)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.0.6 to 9.1.1.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.0.6...v9.1.1)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 9.1.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-12 00:32:34 -07:00
Daniel Salazar ed7e0b9cc4 fix: sharing, events and worker-credential authorization hardening (#3855)
- PUT-1800: gate `createWorkerSessionToken` on actor type, so an app or an
  access token can no longer mint an app-less, root-shaped worker session;
  `WorkerDriver` binds on `effectiveApp` instead of `app`.
- PUT-1799: add `isAccountContext` and read it where "no app" was being read
  as "the account" — handler publish, events-worker listing, kv handle
  mint/revoke/list. A scoped API token is no longer an account session.
- PUT-1802: re-authorize a durable row before its backlog drains, settling it
  permanently when the grant is gone. Covers an ancestor-level unshare, which
  the revoke settle deliberately leaves to the delivery re-check.
- PUT-1803: mask the owner's absolute path out of deliveries and subscription
  anchors on a foreign node, the way every FS surface already does.
- PUT-1804: let a revoke reach rows already suspended for a resumable reason,
  re-stamping them so a resume cannot hand over the held backlog.
- PUT-1805: apply the subscribe path's audience gate to `/events/fetch` before
  the query, so a cursor can no longer count and name invisible notifications.
- PUT-1807: refuse `mode: 'manage'` from any actor holding an app — inside its
  own AppData the ACL short-circuit would otherwise supply the reach.
- PUT-1808: take the sending peer from the verified signature header rather
  than the request body.
- PUT-1810: re-base a kv share-handle row's stored match filter on the handle,
  so the owner's absolute key prefix stays hidden.
- PUT-1814: escape LIKE wildcards and anchor the issuer-prefix queries on a
  segment; anchor `manage:` stripping; reject a backslash in a share prefix;
  assert a resolved actor in `subscribeDurable`.
- PUT-1815: bound the char/varchar columns behind `event_subscriptions` and
  `kv_share_handles` at the store layer.
2026-09-11 21:18:49 -07:00
Neal Shah 883072d144 add email composer api (puter.email.send) (#3854)
* add email composer

* fix imports

* add import for fetchUrl

* add import for compose

* add API origin to fetchUrl

* CI related fixes

* types fix

* address leaf function nitpick

* tests and validation checks

* Fix lockfile
2026-09-12 01:20:41 +00:00
Juan Castro 8d90418c9e fix: don't offer a seat a plan link it cannot use
The Usage card's Upgrade link relabels to "Manage →" for anyone on a
plan, including a seat whose plan is its team's. Following it only
reaches a dialog saying so. Hidden for a seat, unchanged for everyone
else.
2026-09-11 18:00:53 -04:00
Juan Castro de74b0c488 fix: name the plan a team bought, not its policy id
`i18n()` echoes a key it has no translation for, and a team tier has
none — so a seat's dashboard read "team-basic" where a personal plan
reads "Basic". The offering already travels with the subscription, so
its display name is right there; `i18n` stays the fallback for the
personal tiers that do have keys.
2026-09-11 17:44:30 -04:00
404oops 967b974fe9 Merge pull request #3852 from HeyPuter/404oops/infron-fix-tiers
fix: quote and pin Infron service tiers
2026-09-11 23:14:52 +02:00
Daniel Salazar 93885dfc54 fix: email driver rename (#3853) 2026-09-11 13:58:53 -07:00
Filip KujundžićandClaude Opus 5 ab077f047b Revert "docs: document Infron service tiers"
This reverts commit 5fbdc7cac5. Docs will land
in a separate PR.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-11 21:32:42 +02:00
Juan Castro 63f857ce3c fix: make the seat cap the owner's plan decides actually apply
`config.default.json` shipped `max_seats_per_team: 50`, and that flat
override wins over the plan, so the free/paid caps were dead code on
every deployment that did not delete the key. A free owner got 50 seats.
The defaults now carry the two per-plan numbers and leave the flat
override unset; a deployment that wants one number for everyone still
sets it and still wins.

`#ownerPays` asked metering about an `{id, uuid}` stub, so a resolver
keyed on any other field missed and the answer came down to whether
something else had cached that user's plan first — the same team, at the
same seat count, was refused or allowed depending on nothing.

Running out of seats now says what raises the limit and offers the plan
picker, rather than reporting the number and stopping there. The record
table pages at ten rows instead of running off the bottom, and the team
card no longer labels the absence of a handle nothing here can set.
2026-09-11 14:51:54 -04:00
Juan Castro 480d6bc3b5 feat: give a team seat the account surface that is actually its own
A provisioned account could rename itself, delete itself, and see a
Billing tab for a subscription it does not hold — all of it the team's,
not the account's. Each is now refused server-side and dropped from the
UI, keyed on one predicate: whoami reports a team only for an org-owned
seat, and the owner joined their own team.

The Teams tab showed a seat nothing but its own audit rows, so a member
could not see who else was on the team they were told they shared it
with. It now lists them; listMembers was already membership-gated and
already withholds from a member what is not theirs.

The dashboard share modal had no way to reach a team, though the desktop
dialog has had one since team sharing shipped and the SDK has always
taken `{ team }`. Same control, same copy, same helper. The access list
needed a team bucket to go with it: a team share names no holder, so the
aggregate dropped it and a team you had just shared with vanished.
2026-09-11 14:09:07 -04:00
Juan Castro 1efe086565 fix: give the owner the seat uuid the plan action is keyed on
The per-account subscription button did nothing. `listMembers` never
returned a member's uuid, the SDK's `toMember` dropped it, and the Plan
column read `seatTiers[undefined]`, so every seat rendered as Free and
the action dispatched with no seat.

The uuid is owner-only: it is what billing keys a seat's plan on, and one
member has no business identifying another.
2026-09-11 13:14:09 -04:00
Juan Castro 06bdc26efd refactor: let the billing extension own the plan picker
The Teams tab was asking which tier with a UIAlert, which looked nothing like
the personal plan modal. It now dispatches the seat and lets the extension
render the grid, so the two match and OSS stays free of prices.

Carries `username` and `currentTier` so the picker can title itself and mark
the plan the account is already on.
2026-09-11 10:51:57 -04:00
Juan Castro c2a975b3f7 refactor: drop the team plan card
It described a team-wide tier, which stopped existing when plans moved to the
account. What it still showed -- what each tier costs -- is in the Change plan
picker, next to the account it applies to, so nothing is lost by removing it.

`teamPlan.js` and its tests go with it, along with twelve i18n keys nothing
reads any more. `state.plan` stays: the Plan column and the picker both need
the catalogue and the seat assignments.
2026-09-11 10:20:09 -04:00
Juan Castro b8b0650990 feat: list the uuids of the seats a team is billed for
Per-tier quantities need to know *which* seats are active, not just how many:
the count has to be split by the tier each one is on. `countActiveSeats` answers
the old question and stays for the cap and the console copy.

Keyed by `uid` rather than the numeric id, because the billing side holds uids
and would otherwise have to look the team up twice. Same suspension filter, and
a test asserts the two agree so they cannot drift apart.

Falsified: dropping the suspension clause fails "leaves out a suspended seat,
matching countActiveSeats".
2026-09-11 10:11:34 -04:00
Juan Castro bafe8af71b feat: icon buttons for the per-account actions
A row now carries up to five actions -- change plan, reissue, suspend or enable,
delete -- and five labelled buttons do not fit the cell. Each becomes a 30px
icon button.

The label is not dropped, only hidden: it stays as `title`, as `aria-label`, and
as visually-hidden text, so a screen reader and a hover both still get it. The
glyphs are `aria-hidden`, since the button already carries the name.

Inline SVG rather than files under `icons/`: these are one-place 24px line
glyphs, and the dashboard already inlines its sidebar chevron the same way. An
`edit` glyph is defined but unused, ready for the account-edit action.

Falsified: removing the visually-hidden label fails "keeps the label reachable
without showing it".
2026-09-11 10:09:39 -04:00
Juan Castro 187e4c8a2f feat: set a plan per account, not one tier for the whole team
Reverses PUT-1788 D1 at the UI. A team keeps one subscription; each account sits
on its own tier within it.

The plan card stops being the chooser and becomes a summary: what each tier
costs per account, and how many accounts are on it. Choosing happens on the
account, with a Change plan action per row, because that is the thing the tier
now belongs to.

`memberPlanLabel` reads the seat's own assignment rather than the team's single
tier, so two accounts can honestly show different plans. The other three states
are unchanged and still matter: the owner is the payer, a suspended seat is not
billed whatever it was on, and a seat nobody bought a tier for is free.

The action only appears where a billing extension is present and a catalogue
came back, so a deployment that sells nothing shows prices and no dead buttons.
2026-09-11 10:04:40 -04:00
Juan Castro f2c9737bd8 feat: show each account's plan in the team's accounts table
Per-seat billing context without per-seat subscriptions. PUT-1788 D1 stands: one
tier per team, one Stripe subscription, quantity = seat count. The tier is still
changed once, in the plan card.

Four states, because "on Team Basic" is not true of every row. The owner is the
payer and keeps their own personal plan, so they show as such rather than
inheriting the team's. A suspended seat reads "not billed" -- it stops costing a
per-account charge, which is the same rule the billing summary and the plan
card's seat count already use. A seat of a team that bought nothing reads free,
which is the reduced org-seat allowance. Everyone else shows the tier.

The rule is a helper in teamsConsole so it is testable, like the rest of that
file.

Falsified: dropping the payer branch fails "says the owner is the payer, not a
seat"; dropping the suspended branch fails "says a suspended seat is not
billed".
2026-09-11 09:37:34 -04:00
Filip KujundžićandClaude Opus 5 5fbdc7cac5 docs: document Infron service tiers
Tiers beyond a model's default are reachable as `<model>:<tier>` ids, so
the chat docs gain a Service tiers section covering what the suffixes
mean, that the tier named is the tier billed, and that a tier can carry
a smaller context window than its siblings.

The listModels return value described a `cost` object with `input` and
`output` keys. No such field is returned — pricing comes back as `costs`
alongside `costs_currency`, keyed per vendor and named by the model's
own `input_cost_key` / `output_cost_key`. Correct the prose and replace
the example with a real entry.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-11 14:52:28 +02:00
Filip KujundžićandClaude Opus 5 3d2bdfceef fix: quote and pin Infron service tiers
Infron sells the same model at several service tiers. `min_prompt_price`
and `min_completion_price` are the floor across all of them, so every
model with a flex tier was advertised at a batch-job price nobody gets
by default: 25 of 286 chat models, among them gpt-6-astra at $5/$25
against the $7.5/$37.5 a default request actually bills.

Price each tier from its own row in `providers[]` and pin that tier on
the request, so the price quoted is the price charged. Tiers beyond the
default are listed under their own `<model>:<tier>` ids, letting callers
opt into flex or priority by model name:

    puter.ai.chat(prompt, { model: 'infron:openai/gpt-6-astra:flex' })

Suffix parsing matches the catalog exactly before reading a trailing
segment as a tier, since catalog ids can carry a colon of their own
(`deepseek/deepseek-v4-flash:free`). Tier variants take the context
window of their own offering, which differs from the model-level figure
for 11 of them.

Billing was never wrong — it bills Infron's reported `cost` — but the
understated prices fed the credit gate's output cap, which let a request
run roughly 50% past the balance it was gated against, and the fallback
path that prices per token when a response carries no cost.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-11 14:47:41 +02:00
Juan Castro 2083ef47c0 fix: the plan card counted no seats
It filtered on `org_owned`, but the GUI annotates members to `orgOwned` --
so the count was always zero and the card read "billed for 0 accounts" beside
an accounts table saying two.

Uses `membersBillingSummary` now, the same count the table shows, which also
excludes suspended seats: they stop costing a per-account charge, which the
naive filter would have billed for.
2026-09-10 18:32:31 -04:00
Daniel Salazar 9940beb4bb fix: card verification missing email (#3850) 2026-09-10 15:32:01 -07:00
Juan Castro fa36f2e3cf fix: run the forced password change on the login path, not only token-in-URL
`initgui` runs the verification gates in two places: the token-in-URL branch and
the session-restore/login branch. The password gate went into the first only, so
it never fired for the case it exists for -- a seat signing in through the login
form with the credential its administrator issued. The account reached the
desktop and then failed at every gated call with no prompt, which is the state
the gate was written to prevent.

Caught by manual testing, not by any test: both chains looked right in
isolation. Added an invariant test over initgui's source asserting each gate
appears on both paths and each loops until cleared, which is the shape of the
mistake rather than the instance of it.

Falsified: removing the login-path gate fails "runs the forced password change
on both paths" with `expected 1 to be 2`.
2026-09-10 18:27:01 -04:00
Juan Castro efe6ef09bd feat: show a team's plan in the Teams tab, and let the owner change it
PUT-1796's UI half. The owner already manages accounts here, so the plan they
are billed for belongs on the same page rather than a tab away.

Pricing stays out of OSS. The card is drawn from whatever catalogue the server
returns, so this code knows no tier, no price and no payment provider -- a
deployment that sells nothing serves no catalogue and the card does not render.
`TabHome` already reads `/marketplace/subscriptions/current` the same way, so
OSS reading a prod-served endpoint is not a new idea here.

Two things are deliberately not offered rather than offered and broken. A tier
the server marks unavailable gets no button, because no Stripe price is
configured for it and buying would 422. And no button appears at all unless a
billing extension has set `window.team_billing_ui` -- the prices still render,
which is useful on its own, but nothing invites a click nobody can handle.
Checkout is Stripe's, so the button only dispatches `team-plan-purchase` with
the team and item id and lets the extension take it from there.

The markup is a helper rather than another branch in TabTeams, matching
teamBadge/credits/usageBudget, so it can be tested without the window stack.

Falsified: offering an unavailable tier fails "offers no button for a tier with
no configured price"; rendering buttons regardless of the extension fails
"offers no button without a billing extension to act on it".

342 GUI/SDK tests, 136 team backend tests, typecheck clean.
2026-09-10 17:42:51 -04:00
Daniel Salazar 367d20a55c test: kv tests (#3844)
* test: kv tests

* chore: bump deps
2026-09-10 14:35:46 -07:00
Juan Castro e886e86cb5 feat: size a team by whether its owner pays, and halve a free seat's allowance
Three related limits.

Seats per team now depend on the owner's plan: 4 free, 40 paid, decided by
`subscriptionSatisfies(id, true)` -- which is `!FREE_SUBSCRIPTION_IDS.has(id)`,
so a plan an extension adds counts as paid without core knowing its name. The
existing `max_seats_per_team` still overrides both, so a deployment that already
set it keeps what it asked for, and `max_seats_per_team_free` / `_paid` tune
each. An unreadable plan takes the smaller cap: over-provisioning a free team is
the worse failure.

A seat of a team that pays for nothing resolves `org_seat_free`, half the
registered free plan. Without it a team is a way to mint free tiers -- provision
four seats and each arrives with a full free allowance nobody paid for. The
figures are derived from `REGISTERED_USER_FREE` rather than restated, so the two
cannot drift, and the id joins `FREE_SUBSCRIPTION_IDS` because nobody paid for
it either and it must not satisfy a plan gate.

It is a *default* resolver, so a paid team tier -- which only prod knows about,
through `registerSubscriptionResolver` -- still outranks it. The lookup is
`getOrgSeat`, already cached with its negatives, because almost nothing is a
seat.

Found while testing: the suite was reading `max_seats_per_team` out of the
developer's own config.json, so the cap under test was whatever that file said.
It would have passed here and failed in CI, which has no such file. The suite
now pins the value and the cap tests set their own.

Falsified three ways, each breaking only its own test: equal caps fails "lets a
paid owner past four"; a resolver returning null, and an unhalved allowance,
both fail "resolves half the free plan".

186 team/whoami tests, 159 metering tests, typecheck clean.
2026-09-10 17:04:43 -04:00
Juan Castro 106978e714 feat: tell a seat which team its account belongs to
A provisioned account had no way to know it was one. That matters: the team can
reset its password and close it, which is exactly what the account-created email
already warns about, and nothing in the product repeated it afterwards.

`whoami` now carries `team: { uid, name }`. Two gates on it. Only user actors --
a seat's employer is no more an app's business than its phone number, which the
same handler already withholds. And only where `teams_enabled` is on, so a
deployment without teams is byte-identical.

It rides whoami rather than a route of its own because the sidebar needs it at
first paint. A `/teams/whoami` would add a request to every page load for every
user, and almost none of them are seats. The lookup costs nothing either way:
`getOrgSeat` is already cached, negative results included, precisely because
almost nothing is a seat. `team_name` comes off a join the query already made.

In the sidebar it sits under the Puter wordmark -- the conventional slot for
workspace context -- as a muted second line, hidden when the sidebar collapses.
Owners see nothing: they already know, and one may own several teams, so there
would be no single name to show.

The markup is a helper rather than another branch inside UIDashboard, matching
how appGroups/credits/usageBudget were pulled out, so it can be tested without
mocking the window stack.

Falsified: dropping the `isUser` gate fails "withholds it from an app actor" and
nothing else.

181 backend tests, 331 GUI/SDK tests, typecheck clean.
2026-09-10 16:52:10 -04:00
Juan Castro fb74789de1 feat: email a seat its username and temporary password, when an address is given
Follows the previous commit. Dropping the email field entirely went one step too
far: without an address the temporary password shown once in the panel is the
only copy, and an admin who closes that panel has to issue a new one. The field
is back, marked optional, and now it buys something concrete.

`team_account_created` carried no credential -- it said the team "will send you
a temporary password separately". It now carries the username and the temporary
password, so an admin who supplies an address hands nothing over by side channel.

`#notifyUser` takes extra template variables, and both credential-issuing paths
pass the one they just minted: provisioning and re-issue. Re-issue passing the
fresh credential rather than the stale one is the case worth checking, and there
is a test that asserts the old password is absent from that mail.

With no address nothing is sent, which was already true -- `#notifyUser` returns
early without one -- and is now covered.

The docs said the notice carries no credential in two places. Both corrected.

Falsified: dropping the credential from the re-issue call fails "emails the fresh
credential on re-issue, not the old one" and nothing else.

178 backend tests, 326 GUI/SDK tests, typecheck clean.
2026-09-10 16:41:03 -04:00
Juan Castro a983969f32 feat: prompt a team seat to choose its own password on first sign-in
The backend already refused every route with `password_change_required` until a
provisioned account replaced the password its administrator chose, and
`/user-protected/change-password` was already exempt so the account could act.
The client half was missing entirely: nothing in the GUI referenced that code,
so a seat signed in and then failed at everything with no prompt and no way out.

Two gaps, both closed here.

The flag never reached the client. Neither the login response nor `/whoami`
carried `requires_password_change`, so the GUI could not have known even if it
wanted to. It ships from both now, alongside the other three verification flags
the whoami extension already describes as "the flags the GUI acts on".

There was no window to show. `UIWindowChangePassword` is a settings dialog --
closable, and it never resolves on success -- so it cannot act as a gate.
`UIWindowPasswordChangeRequired` mirrors the existing `*Required` windows: it
resolves true only once the change lands, and `initgui` loops on it. It runs
last in the boot chain, matching the server order in assertVerifiedAccount.

It refuses a new password equal to the current one. Without that the account
stays on the credential its administrator still holds, which is the entire thing
the gate exists to end.

Falsified twice: dropping the same-password guard fails "refuses to reuse the
password the admin handed over", and resolving on a rejected response fails
"stays open on a rejected change, so the gate cannot be escaped" -- each
breaking only its own test.

175 backend tests, 326 GUI/SDK tests, typecheck clean.
2026-09-10 16:35:05 -04:00
Juan Castro b19b312e30 feat: a team seat needs no email address, and is never asked to confirm one
PUT-1792. Two separate problems, both from treating a provisioned account like
a self-registered one.

`email` was required, so an admin creating ten seats had to invent ten addresses
and then keep track of ten uniqueness constraints -- for accounts that sign in
by username and never use the address. It is now optional at every layer, and
the add-account form does not ask for it at all: username is the only thing a
seat needs.

`requires_email_confirmation` was set to true, with the reasoning that an
admin-supplied address is unverified. True, but `requireVerifiedAccount` turns
away on exactly `requires_email_confirmation && !email_confirmed`, so a
freshly created seat was asked to confirm an address it may not hold and could
not use the product until it did. The team creating the account is the trust
anchor, not the mailbox, so this is now false either way.

An address is still accepted and still stored when given, because the notices
are worth delivering. `#notifyUser` already returned early on a missing
address, so `team_account_created`, `team_account_disabled`,
`team_password_reset` and `team_closed` degrade quietly with no new branching --
the temporary password is in the API response, which is the documented delivery.

`idx_user_owned_email` is partial and skips password-null rows, so omitting the
address sidesteps it rather than creating a collision surface. Two seats with no
address do not conflict, and there is a test for it.

Docs now say an emailless seat is recoverable only through its team's owner.
That falls out of the design rather than being a limitation of this change, but
it should be written down rather than discovered.

Falsified: putting `requires_email_confirmation: true` back fails
"never demands confirmation, with or without an address" with
`expected true to be false`, and nothing else.

164 team tests, 40 SDK tests, typecheck clean.
2026-09-10 15:55:50 -04:00
Daniel Salazar a8a78736bb feat: kv list ordering toggle + addressable path improvements (#3843)
* fix: put-1787

* feat: kv list reverse
2026-09-09 17:24:03 -07:00
Juan Fernando Castro 9b465aba9f Merge pull request #3837 from HeyPuter/juancastro/team-count-active-seats
feat: add TeamStore.countActiveSeats for the billable seat count
2026-09-09 19:33:55 -04:00
Daniel Salazar 0edee23dd9 Update CODEOWNERS (#3842) 2026-09-09 15:55:19 -07:00
Daniel Salazar 99f3037110 Update CODEOWNERS (#3841) 2026-09-09 15:53:36 -07:00
Juan Castro 7ab48c590e feat: add TeamStore.countActiveSeats for the billable seat count
Per-account billing needs "how many accounts is this team getting the
benefit of", and neither existing count answers it. `countSeats` includes
suspended seats, because it bounds the seat cap and a suspended seat still
holds its username. `countPayers` counts the owner. Billing off `countSeats`
overcharges for suspended accounts, contradicting what the admin console
tells the owner: "0 suspended accounts cost nothing per account."

So this is a third, separate count rather than a change to either — the cap
must keep counting suspended seats.

An unactivated seat is deliberately still counted: it holds a temporary
password, not a suspension, and it is paid for. That is why the filter is
narrower than `listDirectory`'s, which also excludes
`requires_password_change`.

Falsified: dropping the suspended clause fails exactly one test
("drops a suspended seat from the active count but not from the cap",
expected 3 to be 2) and leaves the other 34 green.
2026-09-09 18:13:04 -04:00
404oopsandClaude Fable 5.1 6146ab9dd3 feat(ai): refresh txt2vid catalogs, retire Sora, default to Veo 3.1 Lite (#3828)
OpenAI shuts the Sora Videos API down on 2026-09-24 and sora-2 was the
default txt2vid model, so the default moves to Veo 3.1 Lite on Gemini
and the OpenAI video provider goes. While there, the video catalogs are
brought in line with what each vendor serves today, the request options
are unified across providers, and the txt2vid docs are rewritten.

- driver: default provider gemini-video-generation with
  veo-3.1-lite-generate-preview; a request under the generic `ai-video`
  driver name lands on the default instead of the first-registered
  provider; `WIDTHxHEIGHT` sizes map onto tier catalogs by the shorter
  side and fill width/height
- openai video provider, the `openai-video-generation` alias, its
  config template and migration entries, and Together's openai/sora-2*
  rows removed
- gemini: Veo 3.1 Fast rates 10/12/30 cents per second for
  720p/1080p/4K, Veo 3.1 Lite accepts reference images, URL image
  inputs are fetched server-side through the SSRF-guarded fetch
- together: drop nine models retired upstream, add eighteen from the
  live listing; per-second models are estimated from the catalog rate,
  clamped to remaining credit and billed at the cost Together reports
  on the job; tier-sized models take resolution/ratio;
  input_reference/last_frame map onto keyframes; generate_audio is
  forwarded
- byteplus: Seedance 2.5 (dreamina-seedance-2-5-260628) with per-model
  reference-image caps
- util/imageInput: string-level image helpers shared by the image and
  video drivers; ai-image/inputImage re-exports them unchanged
- puter.js types: provider and generate_audio options; docs: txt2vid
  page rewritten with per-provider model tables, unified options and
  four new playground examples

Known follow-up: Veo returns a key-protected Google file URL, so the
default clip cannot be played directly by a browser until the provider
fetches it server-side.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-09 14:56:17 -07:00
404oopsandClaude Fable 5.1 c8f4906af4 fix(ai): keep per-provider failures when every chat route fails (#3836)
When the chat fallback chain is exhausted, the driver already records
each attempt (model, provider, status, code, message, timeout) in the
error's `fields.attempts`, but the alarm keyed on the classified message
alone and the alarm client printed the error at inspect depth 2, so the
log and Slack line read `internal_error:All providers failed` with
nothing about which providers failed or why. Deduped repeats printed
only a count.

- The HTTP alarm gate now attaches an HttpError's `fields` to the alarm
  under a single `details` key. One key can't shadow the gate's own
  request fields, and a repeat from another thrower on a shared id
  replaces it instead of merging into it.
- The chat driver logs one warn line per exhausted chain with the
  completion id, the resolved route, the classified code and the
  attempts as JSON, so every occurrence is greppable by trace even when
  the alarm dedupes it. Chains marked `noAlarm` don't log.
- Docs: `fields` reaches both the client and the alarm, so it has to be
  safe to show the caller.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-09 14:56:11 -07:00
Reynaldi Chernando 435629afc5 Add gpt 2.5 image (#3839) 2026-09-09 14:29:41 -07:00
Juan Fernando Castro e132f765d6 Merge pull request #3745 from HeyPuter/juancastro/put-1736-team-directory
🏗️ PUT-1736: a team directory apps can read, once the team opens it
2026-09-09 16:09:03 -04:00
Juan Castro 4e45fdc172 feat: a team directory apps can read, once the team opens it
Members can already enumerate each other: `/teams/:uid/members` needs a user
actor and nothing more. The only thing this adds is admitting an app actor to
the same names, so an app can offer colleagues without the member driving it.

That is the whole risk, so it is off until the team owner turns it on.
`group.directory_enabled` defaults to 0, and a team that has not opted in
answers 404 rather than 403 -- whether a team has this on is not something an
app should be able to probe for either.

Three things bound what an app sees. The membership tested is always the
person's, never the app's, so an app installed by a member of one team can
never read another's. The page carries username and uuid and nothing else --
no email, activation state, usage or role. And suspended accounts and ones
that never took up their credential are left out, since offering someone who
cannot sign in is noise and their existence is not this list's to disclose.

Activation is the forced-change flag clearing, not the password existing: a
provisioned seat holds its temporary password from birth, so testing
`password IS NOT NULL` would have leaked exactly the accounts meant to be
excluded. A test covers that distinction.

Turning the directory on or off writes an audit row, because it changes who
can read the member list and that is not something a team should be able to
alter silently. Setting it to the value it already has records nothing.

The toggle lives in TabTeams, and turning it on asks for confirmation while
turning it off does not -- one grants access, the other only takes it away.

Closes PUT-1736.
2026-09-09 14:51:05 -04:00
Juan Fernando Castro f6b519874e Merge pull request #3742 from HeyPuter/juancastro/put-1740-1762-workspace-gui
🏗️ PUT-1740 + PUT-1762: the workspace surfaces in the GUI
2026-09-09 14:50:56 -04:00
Reynaldi ChernandoandCopilot Autofix powered by AI 59be4ad237 more recipes (#3819)
* more recipes

* addition

* final

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* styling fix

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-10 01:07:26 +07:00
Juan Castro 9121e66c3f feat: the team surfaces in the GUI
Two places a team becomes visible to a person rather than an API: the
Dashboard tab that administers it, and the share dialog that shares with it.

Squashed because they are one change to a reviewer. Both are the first GUI
consumers of `puter.teams`, both add strings to the same `en.js`, and both had
to answer the same question -- what a team looks like to someone who has
never seen one. Splitting them would mean reviewing that answer twice and
resolving the same translation conflict twice.

TabTeams (PUT-1740)

  Registered in `builtinTabs` after `TabFiles`, as a plain object with
  `html()` and `init($el_window)` like every other tab. The member table
  carries the operations the API allows and nothing it does not: provision,
  resend an activation, disable, restore, delete.

  Deleting asks for confirmation naming the account, because the API deletes
  on a single call -- PUT-1732 places the reversible step at disable, not in
  front of the request, so the client is where a confirmation belongs.

  The wording says plainly that disabling stops the per-account charge and
  keeps the storage one, and that deleting is the only thing that ends it.
  This is the one place a person decides between the two, so leaving them to
  infer the difference costs them money.

  Rendering and the billing arithmetic are pure functions in
  `teamsConsole.js`, with tests beside them; the tab file is the DOM.

Share dialog (PUT-1762)

  `UIWindowShare` gains the teams the caller owns as recipients. A
  team is one entry in the list, not its members expanded -- sharing
  reaches every member, and the per-person wording would understate what the
  grant does.

  `shareWorkspaces.js` holds the resolution and the deduplication, so the
  dialog does not gain a second source of truth about who a recipient is.

Closes PUT-1740 and PUT-1762.
2026-09-09 11:51:44 -04:00
Juan Castro afb5b0d82a feat: add the puter.teams SDK module
Team administration reaches the backend through the `/teams` routes
rather than a driver interface, because the gates it needs are route-level:
a user actor, a verified account, and a dual-window rate limit.

The module follows `apps/` and `perms/` in layout -- one file per method,
a thin `index.js`, a JSDoc-only `types.js`, and the `METHODS` rebinding so a
destructured method keeps its `this`. It does not follow `perms/lib/req.js`:
those endpoints resolve `{ error: true }` for backward compatibility, and
nothing here has callers to keep compatible, so this throws `PuterJSError`
with the backend's own code.

Every method takes a team `uid`. A handle is a mutable label that
deleting the team releases, so a stored handle can later resolve to a
different team.

The list methods offer the three forms `puter.apps.list()` does -- an array
by default, the page envelope under `cursor`/`includeTotal`, an async
iterator under `stream`. They refuse `offset`: these routes are keyset-only
and would otherwise return page one however far you asked to skip.

The surface covers the routes that exist today. Usage totals and member-email
correction have no backend route yet and are deliberately absent rather than
shipped as methods that 404. `deleteMember()` is here because the route it
needs lands in the commit below this one.
2026-09-09 11:51:44 -04:00
Juan Castro a60a4a98b4 feat: delete a team seat for good, once it is disabled
A disabled account persists indefinitely. Removing it is an explicit request,
never a timer, and it is refused on a live account with
`account_must_be_disabled_first` — which puts a reversible step in front of the
only irreversible operation in the feature.

The audit row is written before `cascadeDelete` runs. The FKs are ON DELETE SET
NULL and the `_keep` columns carry the identifiers, so the record of what was
done survives the account it names.

No second billing emit here: `cascadeDelete` already captures the seat and
fires `team.account.deleted` through UserAccountService, and emitting again
would close the storage charge twice. Disabling closed the per-account charge;
this closes the storage one, and it is the only thing that does.

There is no restore window, and none was wanted: the reversible step already
exists earlier at disable, a disabled account costs only the bytes it holds so
nothing pressures a hasty delete, and a restore promise means retaining data
the team explicitly asked to be rid of.

Published in rate-limits-and-quotas.md alongside the team-deletion note,
since the two are easy to confuse and only one of them frees a seat.

Closes PUT-1732.
2026-09-09 11:51:44 -04:00
Juan Castro 81d15f412b feat: tell a team member what was done to their account
Two halves of the same question -- what did the team do to me, and how do
I find out. One is pull, the other push.

The activity view (PUT-1746) was already user-scoped and already 404'd a
non-member; what was missing is the load-bearing half. A member is told a reset
happened, but a reset only matters alongside who signed in afterwards, so
`SessionStore.listSignIns` merges their own sign-ins into the same stream,
newest first, with a per-stream cursor. Without that row the audit says a
credential was issued and never says whether it was used.

The notices (PUT-1733) cover the two things a member cannot discover for
themselves: their account being disabled, and their team being closed.
Deliberately one each -- closing a team disables every account in it, so
sending both would tell one person twice about one event. Both say plainly that
nothing was deleted; the accounts persist, suspended, holding their files.

Squashed because they are one change to a reviewer: same audience, same
purpose, seven files, overlapping only in TeamService. Neither touches shared
platform code.
2026-09-09 11:51:44 -04:00
Juan Castro f5b2365e83 feat: enforce the forced password change on team seats
`user.requires_password_change` shipped with the team columns but nothing
enforced it and nothing ever cleared it, so a provisioned seat kept its
administrator-issued password indefinitely and `reissueCredential`'s
"already activated" 409 was unreachable.

Adds the fourth clause to `assertVerifiedAccount`, the only place a
verification gate may live -- WebDAV builds its own actor and calls that
function directly, so a second implementation would bypass it the way the
phone and card gates once were bypassed.

A gate that refuses everything also refuses the endpoint that clears it,
so `/user-protected/change-password` opts out with `allowUnconfirmed`.
That widens the route: an account pending email, phone or card
verification can now change its password, which it could not before. The
caller is authenticated and proves the current password, so this is
benign, but it is a behaviour change to a shared route.

Also here, because the gate is worthless without them:

- change-password and the recovery-token path clear the flag, and record
  an `activate` entry when the account is a seat.
- Reset takes a live account back with a fresh credential, capped at 20
  per day and audited as `reset_member_password` with no credential in
  the row. Re-issue is audited the same way; it stays closed once a seat
  has chosen its own password.
- An issued credential expires after 24h (new `temp_password_expires_at`
  column, three dialects) and login refuses it after that, so an unused
  reset dies instead of becoming a standing credential.
- 2FA is untouched by a reset, so a reset alone is not takeover.
2026-09-09 11:51:44 -04:00
Juan Fernando Castro f3943f6a55 Merge pull request #3729 from HeyPuter/juancastro/put-1726-put-1727-put-1729-team-share-recipient
🏗️ PUT-1726 + PUT-1727 + PUT-1729: share with a workspace
2026-09-09 10:41:21 -04:00
Juan Castro 7a02fa7741 feat: tell every team member when something is shared with them
`notifyShared` keyed every map off `ResolvedShare.holderId`. A team share
has no individual holder -- the grant is one row against the group and
`holder_user_id` is NULL -- so it was skipped everywhere and the call returned
having told nobody. Nothing errored: the share was created, resolved and listed
correctly. The failure mode was silence.

`#recipientsOf` now expands a team share to its live members at
announcement time, so the grant stays one row and only the telling fans out.
`#announce` and `#emailHolder` are both inside the per-recipient loop, so
members get the in-app notification and the email digest.

Bounded by `NOTIFY_FANOUT_CAP`, which logs when it bites rather than truncating
silently -- a shortened fan-out otherwise reads as "everyone knows". The
existing per-recipient budgets absorb the volume from there.

Wording is unchanged: "shared N items with you", not the team name.

A member who joins later resolves the grant through the scan but is not
retroactively told; announcements describe a moment, not a state.
2026-09-09 10:22:56 -04:00
Juan Castro da68074401 feat: share with a team
Covers PUT-1726, PUT-1727 and PUT-1729. Together because a recipient without
the listing work produces a share that is created correctly, resolves
correctly, and never appears in "Shared with me" — a state nobody would ship.

The recipient. `ShareRecipient` gains `team` (uid) and `teamHandle`, resolved
before the email and username branches and never falling through to them.
Passing both is an error rather than a precedence rule, so a call site always
shows which was chosen: handles are released on soft delete and can be
reclaimed by an unrelated team, and a scripted share to a handle would
silently retarget. There is no bare-string spelling, since that would change
how existing strings are interpreted.

ACLService.setUserGroup mirrors setUserUser: same read-modify-write, same
one-mode-per-node rule, under a node lock keyed on the group.

One grant against the team, not one per member, so membership changes
apply without touching the grant and a share spends one unit of the daily
quota however many members there are. A member who joins afterwards gets
access, which is asserted.

The index row carries `holder_group_id` and leaves `holder_user_id` NULL, so
`0077`'s group index constrains it rather than the user-holder one.

Listing. `listByHolder` and `countByHolder` union the caller's teams into
the same keyset page — `ORDER BY id` still holds — and `#grantEvidence` gains
group grants as a third source. Without that third source the share is
filtered out of every listing as dead: nothing errors, the share simply is not
there, which is the one place a user would look for it.

Unsharing revokes the grant as well as deleting the index row. Deleting the
row alone would hide the share while leaving every member holding real access.

Closes PUT-1726, PUT-1727 and PUT-1729.
2026-09-09 10:22:56 -04:00
Juan Castro 9f615cc632 feat: add the two-team fixture and group permission grants
Two tickets, together because the fixture has no behaviour of its own — the
grant tests are its first real use, and "a grant to team A must not
resolve for B's members" is exactly a two-team assertion.

PUT-1719. Every query resolving team-scoped data has to take the
team as a parameter rather than infer it. One that forgets returns the
right rows against a database holding a single team, so a one-team
fixture does not give weaker coverage — it gives false confidence.

The fixture provides two teams, each with its own master and two
activated seats, plus a user in neither. Each team having its own master
is also what the shipped `max_teams_per_user: 1` requires, so it runs
against the real cap rather than lifting it the way the existing team suites
have to. Seats are activated before use: provisioning leaves
`requires_email_confirmation` set and `requireVerified` rejects them outright,
so an unactivated seat cannot call anything and every authorization assertion
built on one would be vacuous.

PUT-1725. The write half of group grants; the read half already worked, since
`#scanUserGroup` joins `jct_user_group` itself and resolves for whoever is a
member at scan time.

  PermissionStore   resolveGroupId, listGroupMemberUuids,
                    upsertUserGroupPerm, deleteUserGroupPerm,
                    auditUserGroupPerm
  PermissionService grantUserGroupPermission, revokeUserGroupPermission

Both rewrite the permission first, so `fs:/path:mode` collapses to
`fs:<uuid>:mode` and a revoke names the same string the grant wrote; both gate
on `canManagePermission`; both audit into the existing
`audit_user_to_group_permissions`.

The delete is scoped to the issuer, so one issuer's revoke cannot drop
another's identical grant — the PK is (user_id, group_id, permission) and
user_id is the issuer.

Cache invalidation goes through `bumpCacheGenerations` with every member uuid
at once. That already announces to peer regions in a single event, so the
fan-out costs one broadcast rather than one per member.

There is no flat-KV equivalent for group grants, so unlike the user-to-user
path there is no second write to keep consistent and no second invalidation
surface.

Closes PUT-1719 and PUT-1725.
2026-09-09 10:22:56 -04:00
Daniel Salazar 8a70fa2e5c fix: CODEOWNERS + perm dialogs outside puter dashboard/desktop (#3833) 2026-09-08 22:56:03 -07:00
Felix-Ayush 9bb5bffd01 fix(puter-js): account for base64 padding in estimateDataUrlSize (#3805)
Math.ceil(len * 3 / 4) over-counts by 1-2 bytes on padded payloads, so thumbnails at the size limit were wrongly rejected. Subtract padding like ai dataUriByteLength.
2026-09-08 22:33:33 -07:00
Felix-Ayush d7877359d4 fix(puter-js): return key_undefined for missing kv keys (#3803)
get/del/incr/decr/add/expire/expireAt only checked size, so undefined keys threw TypeError. Add assertKeyPresent first, matching set/update/remove.
2026-09-08 22:32:53 -07:00
Felix-Ayush cdc524f504 fix(puter-js): preserve zero amount in kv incr/decr (#3802)
parseCounterArgs used a falsy check, so incr(key, 0) sent amount 1. Only default when amount is undefined/null.
2026-09-08 22:30:02 -07:00
dependabot[bot] 8b872545dc chore(deps): bump docker/setup-buildx-action from 3 to 4 (#3823)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:27:06 -07:00
dependabot[bot] 52f9fa57a6 chore(deps): bump docker/metadata-action from 5 to 6 (#3822)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:26:49 -07:00
dependabot[bot] eb28012897 chore(deps): bump peter-evans/dockerhub-description from 4 to 5 (#3826)
Bumps [peter-evans/dockerhub-description](https://github.com/peter-evans/dockerhub-description) from 4 to 5.
- [Release notes](https://github.com/peter-evans/dockerhub-description/releases)
- [Commits](https://github.com/peter-evans/dockerhub-description/compare/v4...v5)

---
updated-dependencies:
- dependency-name: peter-evans/dockerhub-description
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:26:27 -07:00
Daniel Salazar e5f322f68c fix: misc event hardening + perm fixes (#3831) 2026-09-08 19:16:21 -07:00
Neal Shah ab3bba5525 add file query param (#3829) 2026-09-08 21:39:19 -04:00
Juan Fernando Castro 9f3c658975 Merge pull request #3830 from HeyPuter/juancastro/phase3-land-on-main
Teams phase 3: land the remaining three PRs on main (#3733, #3724, #3725)
2026-09-08 18:27:11 -04:00
Juan Castro bd19fd18ec feat: cap teams per user and seats per team
A seat is a real Puter account: it takes a name from the global username pool
and gets a home directory. Nothing charges for one — that is prod's job — so
until it does, the only bound on creation is the request rate limit, which
bounds the rate and not the total.

  max_teams_per_user   default 1
  max_seats_per_team   default 50

Ordering is the substance of both checks. The team cap is tested before
the handle, so a capped user is told they are capped rather than that the name
they picked was unusable. The seat cap is tested before any account state
exists, so a refused provision does not burn a global username.

Soft-deleted teams do not count toward the owner's cap, so deleting frees
the slot — which does mean create, provision, delete, repeat still consumes
usernames over time, bounded by the daily rate limit. The caps raise the cost
and make the cycle audited; they do not close it.

Lowering the seat limit blocks new provisioning and disables nobody.

Both limits are published in rate-limits-and-quotas.md, and both keys are
documented in config.template.jsonc and config.default.json.

Closes PUT-1758.
2026-09-08 16:59:29 -04:00
Juan Castro 41a05b68fd feat: announce credit-state transitions so billing can act on them
A seat that exhausts its allowance is refused and cannot fix it itself — only
the team owner can add capacity. Something has to tell the owner, and that
something lives in prod, so OSS's job is to say when it happened.

`MeteringService` emits `metering.credit-state` from `rememberRemainingCredits`,
the single point where the verdict is computed, at 90% of the allowance and
again on exhaustion.

Emitted on a transition, never per request. A blocked account keeps trying and
every retry recomputes the verdict, so without that a retry loop would announce
hundreds of times. The state is tracked per uuid in process, under the same FIFO
bound as the credit cache, and dropped in `#dropCachedCredits` — so added
capacity re-arms the alert for the rest of the month.

The dedup that turns this into exactly one notice per member per month belongs
with whoever sends the mail, and needs a cross-region marker rather than a
per-cluster one.

Part of PUT-1750; the notification half is prod's.
2026-09-08 16:59:29 -04:00
Juan Castro 0c6045ed90 feat: emit team billing events for the payment integration
The charge lives outside this repo, the way the marketplace extension cancels
Stripe subscriptions off `user.delete`. This is the trigger, and it is the
whole of what OSS owes billing.

  team.account.created    a seat exists and can be used
  team.account.disabled   it stopped, and still holds its bytes
  team.account.enabled    it resumed
  team.account.deleted    it is gone
  team.deleted            the team is gone; its accounts are not

Each carries the team uid, the affected account, and the owner's
`stripe_customer_id`. That column ships in the mysql and postgres schemas but
not sqlite, so the read is guarded and degrades to null, as `cascadeDelete`
already does.

Deleting a team emits one `team.account.disabled` per seat plus the
team event, rather than one bulk event: the accounts persist, suspended,
holding their files and their usernames. Deleting a team is not a way to
stop paying for the accounts in it.

`team.account.deleted` is captured before the row is deleted.
`jct_user_group.user_id` is ON DELETE CASCADE, so by the time a listener on
`user.delete` runs, nothing can say which team paid for the account.

`getOrgSeat` deliberately admits soft-deleted teams: their accounts still
exist, so the charge is still running.

Closes PUT-1712.
2026-09-08 16:59:28 -04:00
Juan Fernando Castro 00afea1285 feat: a seat is created, never adopted (#3720)
`addMember` refuses to turn an account that already has a password into a
team seat. No service path did this — `provisionAccount` always creates —
but the store permitted it, and the design rules out existing accounts joining
a team.

Provisioning passes the guard because it admits the account before setting its
temporary password.

There is no bypass parameter. Both HTTP suites now provision a real seat and
authenticate as it, using the same token-minting the harness uses for
`POST /login`. That surfaced something worth knowing: an unactivated seat
cannot call the API at all. Provisioning leaves `requires_email_confirmation`
set and `requireVerified` rejects it, so the suites activate the seat first —
which is the state a member is actually in when making requests.

`listMembers` and `getMembership` also return `u.uuid`, which the billing
events need in order to name the account without a second lookup.
2026-09-08 16:45:16 -04:00
dependabot[bot] 84368f8d02 chore(deps): bump otpauth from 9.5.1 to 9.5.2 (#3820)
Bumps [otpauth](https://github.com/hectorm/otpauth) from 9.5.1 to 9.5.2.
- [Release notes](https://github.com/hectorm/otpauth/releases)
- [Commits](https://github.com/hectorm/otpauth/compare/v9.5.1...v9.5.2)

---
updated-dependencies:
- dependency-name: otpauth
  dependency-version: 9.5.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 13:26:41 -04:00
Daniel Salazar 86050f131b fix: hardening events for shared kv (#3827) 2026-09-08 09:04:50 -07:00
dependabot[bot] e50f2911b6 chore(deps): bump actions/upload-artifact from 4 to 7 (#3824)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:58:37 -07:00
dependabot[bot] 414c121ae4 chore(deps): bump actions/setup-node from 4 to 7 (#3825)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 7.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:58:24 -07:00
Toshit ChawdaandClaude Opus 5 1d6d0da950 feat(puter.js): transferables in AppConnection.postMessage (#3776)
`postMessage(message, transfer)` (or `{ transfer }`) moves ArrayBuffers,
MessagePorts, streams and the rest to the target app instead of copying
them. The second argument is optional, so existing callers are unchanged.

The transfer list rides in the message body as well as the real transfer
list: structured clone's memory map keeps those objects identical to the
ones inside `contents`, which is how the desktop picks them out mid-relay
and keeps transferring them onward rather than leaving copies behind. Both
`messageToApp` paths carry it — the direct iframe relay and the connection
path that `launchApp()` between apps actually uses.

The desktop forwards the list as-is and lets postMessage judge it. A list
that arrived through the SDK was already validated by the browser on the
first hop, and validating again here would mean an allowlist of
transferable types that silently downgrades an unrecognised one to a copy.
A bad list only reaches us from an app that hand-wrote the envelope; that
throws without detaching anything, and is caught so it cannot escape
`ipc_listener` as an unhandled rejection.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-09-08 08:57:52 -07:00
dependabot[bot] 72889d72c8 chore(deps): bump undici from 7.29.0 to 7.29.1 (#3821)
Bumps [undici](https://github.com/nodejs/undici) from 7.29.0 to 7.29.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.29.0...v7.29.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.29.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:52:50 -07:00
Nariman Jelveh 8a6daee9ab feat: let extensions customize recommended apps 2026-09-07 16:44:41 -07:00
404oops e980b131a0 Merge pull request #3815 from HeyPuter/filipkujundzic/ai-vision-canonical-media-parts 2026-09-08 00:54:41 +02:00
404oopsandClaude Fable 5.1 1cb5472c4c fix(ai): canonicalize image parts once and translate them per provider
The SDK shorthand sends `{ image_url: { url } }` with no type. Claude
forwarded it untouched (Anthropic: "content.1.type: Field required"),
the OpenAI/Azure Responses providers sent a Chat Completions part the
Responses API rejects, and Mistral emitted snake_case `image_url` where
its SDK validates `imageUrl`. All three only appeared to work because
the fallback loop re-served them through OpenRouter.

- utils/mediaParts: canonicalise every inbound media part (untyped
  shorthand, bare string URLs, Responses input_image, Anthropic image
  blocks, Gemini inline_data) to the Chat Completions form in the
  driver, so providers translate from one shape
- Claude → Anthropic image blocks (url / base64 sources), copy-on-write
- Responses processor → input_image with a string URL and detail, and
  copies messages instead of mutating the caller's objects
- Mistral → camelCase imageUrl; the test that asserted the old shape
  is corrected
- driver: 400 up front when the catalog says the model has no image or
  video input; resolve puter_path parts per attempt for providers
  without their own upload path (Claude keeps the Files API)
- move the Moonshot http→data-URL inliner to utils/inlineImages and
  apply it to Grok-via-Azure and Gemini, whose upstreams cannot fetch
  http image URLs

Fixes #3409

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-07 15:23:40 +02:00
Daniel Salazar 927317bc4e fix: events hardening (#3814) 2026-09-06 22:44:07 -07:00
Daniel Salazar a441e7f748 fix: harden events (#3813)
* fix(events): rename carries from, one retry on deploy-timeout, forward-path counters

- FSService.rename passes the pre-rename path so a folder subscription
  sees op move with from, like a real move does
- deploy-timeout from the dispatcher gets one retry with the deployed
  header and no second upload (ALREADY_DEPLOYED_MISS_REASONS)
- OTel counters events.forward.sent/received and events.single.attempt
- cross-app KV subscribe error hints at the three-segment parse
- docs: lease is 60 s, kv prefix example is fully qualified, move covers
  rename

* feat(events): forward session subscriptions across regions, fast bumps, worker session cleanup

- session (onLocal) subscriptions now receive writes committed in other
  regions: a transition-maintained remote-watch index (ev:sc / ev:rw),
  watch/event forward items, replay through dispatchForwarded against
  session rows only; events.forwardSession=false is the kill switch
- subscription and presence generation bumps also ride the addressed
  forward channel (kind bump) so a peer sees a new durable row within
  a queue window; the webhook fan stays as backstop
- workers.destroy revokes every holder's events:handlers session; app
  deletion reaps the app's rows, backlog and handlers and revokes the
  sessions; an hourly sweep revokes sessions whose app is gone
- docs: cross-region latency, per-region caps footnote, session
  lifecycle
2026-09-06 16:44:07 -07:00
dependabot[bot] 2784feb47d chore(deps): bump fast-uri from 3.1.5 to 3.1.7 (#3810)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.5 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.5...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-06 16:43:02 -07:00
Daniel Salazar 2f70cd1a14 feat(email): sendTransactional alias and streaming file inputs (#3812)
- puter.email.sendTransactional is the new name; send stays as a
  deprecated alias with the same arguments and result
- fileInput: openFileInputStream / resolveFileInputEntry expose the
  ACL-checked FS read as a stream; loadFileInput wraps them
- EmailAttachment accepts a `path` the transport streams on its own
2026-09-06 16:41:02 -07:00
Felix-Ayush b1d3a7c191 fix(puter-js): require prompt in ai.txt2img (#3804)
txt2vid rejects missing prompts client-side with prompt_required, but txt2img sent the request to the backend. Add the same guard before the driver call.
2026-09-06 16:39:35 -07:00
Nariman Jelveh 092a9245ff Fix PDF thumbnail worker loading from CDN bundles 2026-09-06 14:31:42 -07:00
Nariman Jelveh e6cedde732 Add resilient PDF thumbnails to GUI uploads (#3809)
* Add resilient PDF thumbnails to GUI uploads

* Keep GUI image thumbnails working with SDKs lacking the callback context

The desktop loads puter.js from js.puter.com by default, and the SDK
there predates the thumbnail callback context, so passing the PDF
generator made every image upload lose its thumbnail until the SDK
deploys. Fall back to the SDK's bundled image generator whenever the
running SDK passes no usable context, and cover both paths in the unit
and browser tests.

* Ignore preparation failures that land after an upload is cancelled

Cancelling during preparation already rejects the upload and fires the
abort callback. If the step that was in flight then fails, such as a
dropped directory that cannot be read, the error callback also fired and
the GUI showed an upload error for an upload the user had just
cancelled. Skip error reporting once preparation has been aborted.

* Give each PDF thumbnail worker four seconds

The per-PDF budget covers downloading PDF.js as well as rendering, and
the first PDF of a session on a slower connection ran out of time
before its assets had even loaded. Four seconds fits that first load on
ordinary connections while staying under the five-second batch cap, so
one stuck PDF still leaves the rest of the batch a chance.
2026-09-06 13:51:02 -07:00
404oops 9b4d8f11d4 Merge pull request #3801 from HeyPuter/filipkujundzic/remove-retired-imagen-moonshot-models
fix(ai): remove retired Imagen 4 and legacy Moonshot models
2026-09-06 13:26:14 +02:00
dependabot[bot] bf4e810f9c chore(deps-dev): bump browserstack-local (#3808)
Bumps [browserstack-local](https://github.com/browserstack/browserstack-local-nodejs) from 1.5.5 to 1.5.9.
- [Release notes](https://github.com/browserstack/browserstack-local-nodejs/releases)
- [Commits](https://github.com/browserstack/browserstack-local-nodejs/compare/v1.5.5...v1.5.9)

---
updated-dependencies:
- dependency-name: browserstack-local
  dependency-version: 1.5.9
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:43:42 -07:00
dependabot[bot] 89a4947f68 chore(deps): bump webpack from 5.101.3 to 5.110.3 in /src/puter-js (#3800)
Bumps [webpack](https://github.com/webpack/webpack) from 5.101.3 to 5.110.3.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Changelog](https://github.com/webpack/webpack/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack/compare/v5.101.3...v5.110.3)

---
updated-dependencies:
- dependency-name: webpack
  dependency-version: 5.110.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:47 -07:00
dependabot[bot] 9ebe1092f1 chore(deps): bump dependabot/fetch-metadata from 2 to 3 (#3763)
Bumps [dependabot/fetch-metadata](https://github.com/dependabot/fetch-metadata) from 2 to 3.
- [Release notes](https://github.com/dependabot/fetch-metadata/releases)
- [Commits](https://github.com/dependabot/fetch-metadata/compare/v2...v3)

---
updated-dependencies:
- dependency-name: dependabot/fetch-metadata
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:22 -07:00
dependabot[bot] 1caf3e121f chore(deps): bump actions/checkout from 3 to 7 (#3764)
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:13 -07:00
dependabot[bot] 3620f7bd85 chore(deps): bump docker/build-push-action from 5 to 7 (#3765)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:39:57 -07:00
dependabot[bot] e48802252e chore(deps): bump actions/download-artifact from 4 to 8 (#3767)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:39:21 -07:00
Daniel Salazar da65b7f569 feat: the invoking backend deploys an events worker the dispatcher cannot find (#3807)
The dispatcher's rehydrate callback reaches whichever backend answers the
API's public hostname. A backend with the runtime flag on that is not behind
that hostname, or the only one in the fleet with it on, could never get its
scripts deployed that way — the callback answered "disabled". The invoking
backend already knows the app and script, so on a dispatcher miss it deploys
the set itself and retries once, telling the dispatcher to skip its callback
and negative cache. The callback stays the path for evicted scripts.
2026-09-05 14:40:44 -07:00
404oopsandClaude Fable 5.1 ded3d3695b fix(ai): remove retired Imagen 4 and legacy Moonshot models
Google shut down imagen-4.0-fast/standard/ultra in the Gemini API on
2026-08-17; live calls now return 404 "not found ... or is not supported
for predict". Drop the three entries, the generateImages code path that
only they used, and their tests; the integration test moves to
gemini-2.5-flash-image.

Moonshot's live /models listing and the Kimi pricing docs now carry only
kimi-k3, kimi-k2.7-code(-highspeed) and kimi-k2.6. Drop kimi-k2.5 and the
whole moonshot-v1-* family and repoint the unit and integration tests at
current models. kimi-k2.5 still resolves through OpenRouter.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-05 22:22:16 +02:00
dependabot[bot] e451d507ad chore(deps): bump lodash from 4.17.21 to 4.18.1 in /src/puter-js (#3797)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.21 to 4.18.1.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.21...4.18.1)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.18.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 12:21:36 -07:00
dependabot[bot] 7099cadc6f chore(deps): bump shell-quote from 1.8.1 to 1.10.0 in /src/puter-js (#3774)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.8.1 to 1.10.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.8.1...v1.10.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.10.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 12:19:35 -07:00
dependabot[bot] 73cb60f5ae chore(deps): bump docker/login-action from 3 to 4 (#3762)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:58:57 -07:00
Daniel Salazar e478087260 fix: keep matrix legs of test/coverage jobs always instantiated (#3799)
Skipping the whole matrix job via a job-level `if` collapses it to a
single check with the unevaluated name template ("test (\${{
matrix.artifact }})") instead of expanding to "test (base)"/"test (pr)",
since there's no matrix context to fill in when the job never runs. That
check never matches a required-status-check pinned to "test (pr)", so
PRs that skip backend/puterjs testing block forever waiting on a check
that will never be reported. Gate the steps instead so the matrix always
expands with stable names, doing no real work when irrelevant.
2026-09-04 23:58:12 -07:00
dependabot[bot] b6fb4f3cae chore(deps-dev): bump @eslint/js from 9.39.4 to 9.39.5 (#3791)
Bumps [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js) from 9.39.4 to 9.39.5.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/commits/v9.39.5/packages/js)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.39.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:56:03 -07:00
dependabot[bot] 61566431cc chore(deps): bump fs-extra from 11.3.5 to 11.3.6 (#3785)
Bumps [fs-extra](https://github.com/jprichardson/node-fs-extra) from 11.3.5 to 11.3.6.
- [Changelog](https://github.com/jprichardson/node-fs-extra/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jprichardson/node-fs-extra/compare/11.3.5...11.3.6)

---
updated-dependencies:
- dependency-name: fs-extra
  dependency-version: 11.3.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:55:31 -07:00
dependabot[bot] d3baf0ef7e chore(deps): bump js-yaml from 4.3.1 to 4.3.2 (#3792)
Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.3.1 to 4.3.2.
- [Changelog](https://github.com/nodeca/js-yaml/blob/4.3.2/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/4.3.1...4.3.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:54:54 -07:00
dependabot[bot] 4c3fe8ef7b chore(deps-dev): bump @vitest/coverage-v8 from 4.1.10 to 4.1.11 (#3793)
Bumps [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) from 4.1.10 to 4.1.11.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.11/packages/coverage-v8)

---
updated-dependencies:
- dependency-name: "@vitest/coverage-v8"
  dependency-version: 4.1.11
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:53:49 -07:00
dependabot[bot] c164b5724f chore(deps): bump nodemailer from 9.0.3 to 9.0.6 (#3795)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.0.3 to 9.0.6.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.0.3...v9.0.6)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 9.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:52:48 -07:00
Daniel Salazar bc9cb2d7e7 fix: withdrawing background consent revokes the app's events session (#3777)
* fix: withdrawing background consent revokes the app's events session

A background handler runs as a worker session for the subscriber and app.
Revoking `events:background` or uninstalling the app suspended the
subscriptions but left that session valid, so a token a handler had copied
out kept working until the user found the row in the sessions list. The
revocation settle now revokes the session too; the next consented delivery
mints a fresh one.

* fix: cleanup docs
2026-09-04 21:10:27 -07:00
dependabot[bot] 5669a2d960 chore(deps): bump fastify from 5.10.0 to 5.12.3 (#3749)
Bumps [fastify](https://github.com/fastify/fastify) from 5.10.0 to 5.12.3.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/commits)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 21:01:11 -07:00
dependabot[bot] 50aea0cc64 chore(deps-dev): bump vite from 8.1.3 to 8.1.5 (#3790)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.1.3 to 8.1.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.1.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.1.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:36 +00:00
dependabot[bot] c67f1c3653 chore(deps): bump uuid from 14.0.1 to 14.0.2 (#3789)
Bumps [uuid](https://github.com/uuidjs/uuid) from 14.0.1 to 14.0.2.
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/uuidjs/uuid/compare/v14.0.1...v14.0.2)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 14.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:13 +00:00
dependabot[bot] 78f44c8f8c chore(deps-dev): bump vitest from 4.1.10 to 4.1.11 (#3788)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 4.1.10 to 4.1.11.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.11/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-version: 4.1.11
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:03 +00:00
dependabot[bot] 45ee0ef102 chore(deps-dev): bump @types/node from 24.13.2 to 24.13.3 (#3787)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.2 to 24.13.3.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:52:45 +00:00
dependabot[bot] c4da663533 chore(deps): bump sharp from 0.35.3 to 0.35.4 (#3786)
Bumps [sharp](https://github.com/lovell/sharp) from 0.35.3 to 0.35.4.
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.35.3...v0.35.4)

---
updated-dependencies:
- dependency-name: sharp
  dependency-version: 0.35.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:52:31 +00:00
dependabot[bot] 69b5901278 chore(deps): bump mysql2 from 3.22.5 to 3.22.6 (#3784)
Bumps [mysql2](https://github.com/sidorares/node-mysql2) from 3.22.5 to 3.22.6.
- [Release notes](https://github.com/sidorares/node-mysql2/releases)
- [Changelog](https://github.com/sidorares/node-mysql2/blob/master/Changelog.md)
- [Commits](https://github.com/sidorares/node-mysql2/compare/v3.22.5...v3.22.6)

---
updated-dependencies:
- dependency-name: mysql2
  dependency-version: 3.22.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:54 +00:00
dependabot[bot] 99fb4c2dad chore(deps-dev): bump prettier from 3.9.4 to 3.9.6 (#3783)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.4 to 3.9.6.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.4...3.9.6)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:41 +00:00
dependabot[bot] 25b6a80cb5 chore(deps): bump libphonenumber-js from 1.13.6 to 1.13.12 (#3782)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.6 to 1.13.12.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/commits/v1.13.12)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:34 +00:00
dependabot[bot] b334961f75 chore(deps): bump @esbuild/linux-x64 from 0.25.11 to 0.25.12 (#3781)
Bumps [@esbuild/linux-x64](https://github.com/evanw/esbuild) from 0.25.11 to 0.25.12.
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG-2025.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.11...v0.25.12)

---
updated-dependencies:
- dependency-name: "@esbuild/linux-x64"
  dependency-version: 0.25.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:10 +00:00
dependabot[bot] aa95a76253 chore(deps): bump @prelude.so/js-sdk from 0.12.0 to 0.12.1 (#3780)
Bumps [@prelude.so/js-sdk](https://github.com/prelude-so/web/tree/HEAD/packages/js-sdk) from 0.12.0 to 0.12.1.
- [Changelog](https://github.com/prelude-so/web/blob/main/packages/js-sdk/CHANGELOG.md)
- [Commits](https://github.com/prelude-so/web/commits/HEAD/packages/js-sdk)

---
updated-dependencies:
- dependency-name: "@prelude.so/js-sdk"
  dependency-version: 0.12.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:50:51 +00:00
dependabot[bot] 85bf595d2d chore(deps): bump highlight.js from 11.11.1 to 11.11.2 (#3779)
Bumps [highlight.js](https://github.com/highlightjs/highlight.js) from 11.11.1 to 11.11.2.
- [Release notes](https://github.com/highlightjs/highlight.js/releases)
- [Changelog](https://github.com/highlightjs/highlight.js/blob/main/CHANGES.md)
- [Commits](https://github.com/highlightjs/highlight.js/compare/11.11.1...11.11.2)

---
updated-dependencies:
- dependency-name: highlight.js
  dependency-version: 11.11.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:50:38 +00:00
Daniel Salazar 2f15e40a77 ci: add CODEOWNERS, restrict Dependabot to patch bumps, make test workflows always-run (#3778)
- CODEOWNERS: default owners @Salazareo @ProgrammerIn-wonderland @jelveh
  @jfcastro92, plus @reynaldichernando for src/docs.
- dependabot.yml: ignore semver-minor/major npm updates, patch only.
- backend-tests.yaml / puterjs-tests.yaml: trigger unconditionally and gate
  the real work behind a dorny/paths-filter job instead of a workflow-level
  path filter, so the checks always report (skipped when irrelevant) and
  can be marked required. Also pins matrix job names to the artifact label
  instead of the branch ref, which previously made the check name change
  per PR and unusable as a required check.
2026-09-04 20:49:14 -07:00
Daniel Salazar b784b51cf3 fix: harden the events stack for flag-on (#3752)
* fix: harden events dispatch, single delivery and KV share handles

Dispatch: a filtered subscription used the anchor path stored at subscribe
time, so renaming or moving the anchor folder silently ended its deliveries;
dispatch now resolves the anchor's live path from the event's own ancestor
chain. A move out of a watched folder now reaches that folder's subscribers,
with `from` only for rows that watched the source side. Gap markers are
authorized like deliveries and coalesced per subscription and subject instead
of fanning per lost event. Session subscriptions: the per-socket cap decides
on the write, not before it; an orphaned watched-set token heals on refresh;
durable rows keep their watch window when a session subscribe touches the
same keys. `self` is false when the acting user is unknown.

Single delivery: a subscription in backoff or suspended with a backlog pinned
the sweeper's head and starved everyone behind it — the sweep now defers it.
Only a settled handler run bills a delivery. A socket-only account row no
longer wedges after two attempts nobody received. The lease is twice the
handler timeout; remote candidates have their own attempt counter; the region
depth reconcile runs once a minute region-wide with a bounded scan.

KV share handles: a grantee no longer sees the owner's namespace and absolute
prefix on the subscribe answer or listing, nor in the delivery token; revoking
a wider handle retires the handles it covers; minting the same handle twice
returns the existing one, after the delegation check; a row whose event
cannot be re-based onto its handle is dropped rather than delivered raw.

* fix: presence survives replication, long sessions and region churn

One presence item per (user, app) with per-region map fields lost a region
whenever two regions joined inside the replication window, and nothing ever
put it back. Presence is now one item per (user, app, region): each region
writes only its own, a leave or repair retires it conditionally on its own
write stamp, and a read is a prefix query. Items carry a 48 h ttl refreshed by
a claim-gated write off the existing socket renew path, at most once per
12 h, so a tab that stays connected keeps its region in the row. A region
that answered "no socket" or completed a leave releases a shared pin, so a
reconnect on another node rejoins and a flapping client cannot force a
replicated write per cycle. Cached rows expire after a minute; unaddressable
region names are filtered and pruned; relayed acks settle under a bounded
concurrency; the forward queue is bounded in bytes as well as items.

* feat: indexes for the event_subscriptions hot queries

Handler publish, remove and listing, and the hourly expiry and suspension
sweeps, all scanned `event_subscriptions`. Adds (app_uid, handler_name),
(expires_at) and (suspended_at, id), guarded on every engine. Existing
migrations: the postgres widens are now guarded so a boot does not take an
exclusive lock for a no-op, the kv_share_handles grantee FK gets an index,
the sqlite notification rebuild is transactional and idempotent.

* fix: notification writes go through the registry

The driver's `create` bypassed the type registry, producing uncatalogued
rows with no size bound; it now requires a registered type, caps the payload,
and answers 400 rather than 500 for a bad one. `mark_acknowledged` emits the
ack other tabs listen for, and only when a row was actually changed.

* fix: the handler scanner, unsubscribe, and the in-tab handler environment

The free-variable scanner skipped arrows inside a declaration's initializer,
so `const ids = event.items.map(x => x.id)` was refused, and treated a name
after a comma in a nested initializer as bound, so a real free variable slipped
through to fail on first delivery. `unsubscribe()` now drops the durable
routing entry so the events socket can close. A broadcast handler running in
the tab gets `user` and `fetch` like the worker gives it. `single` without
a handler name is refused before the round trip.

* docs: events limits, error codes and the background-workers section

Retention is deployment-configured rather than a fixed 14 days, and the
template no longer ships it armed. Documents `events_terminal`, the two
per-event gap reasons, the subject length and listing caps, the `from` field
on moves, and the handle-relative anchor. The sessions manager hides the
background-workers section when the server has none to show.

* feat: a background handler acts as the app does for its user

A handler's `user` was a five-minute access token scoped to the subscription's
`list` grant, which could stat the changed file but not read it, and could
not reach the app's KV or AppData — so an app told that a file was written
could do nothing with it. It now runs with the same authority the app has for
that user in a tab: an app-under-user worker session, one row per (user, app)
named `events:handlers`, visible and revocable in the sessions list. The
`events:background` consent is what authorizes running it unattended, and is
re-checked before every mint.

The wider token exposed two things: puter.js opens a filesystem socket the
moment it has a token, which would have parked the isolate in the app's own
delivery room and steered deliveries at it; the events client now opts out of
sockets (and the per-open bookkeeping) before construction, and is memoized
per token in the isolate. And four filesystem operations assumed a socket
exists; they no longer do.
2026-09-04 17:32:57 -07:00
Daniel Salazar f30baa2a1c feat: the per-app events worker runtime (#3697)
* feat: bake published handlers into a generated events worker

* feat: deploy and address the per-app events worker behind a flag

* test: single delivery end to end through a real local worker

* feat: events workers run their own runtime, in their own namespace

An events worker was being deployed as an ordinary worker: default dispatch
namespace, a `subdomains` row, the router preamble, and an app-scoped worker
token baked in. The public dispatcher resolves any script in that namespace
straight off the hostname, so the worker answered at `<name>.puter.work`, and
the only thing in front of it was an unguessable name plus a check that a
`puter-auth` header was present — which the router never validates. Anyone who
learned the hostname could run an app's handlers with a body of their choosing,
in an isolate holding the owner's token as `me`.

Instead:

- Handlers run on their own runtime (`src/worker/src/events-runtime.js`), which
  provides no `router` and no `me`, owns the single invoke route, and hands a
  handler only `{ event, ctx, user, fetch, ack }`. `user` is built from the
  invocation's delivery token, so a handler acts as the subscriber whose
  delivery it is and nothing wider. The preamble build emits one bundle per
  runtime; the shared half of the template is now included by both.
- The deploy target carries the runtime to prepend, the source to deploy, and
  whether to mint a worker token at all, so an events worker deploys into the
  `events` dispatch namespace from generated source with no token binding, no
  `subdomains` row, and no claim on the owner's worker quota or worker list.
- An invocation carries a key derived from the deployment secret and the script
  name, bound as a secret and checked in constant time inside the isolate,
  which reads it once and drops it before handler code runs.
- Scripts are named after the handler set they contain, so publishing writes
  rows and deploys nothing: a set is deployed the first time a delivery needs
  it, and a changed set is a new script rather than an overwrite of a running
  one. Publish responses keep the shape they had before the runtime existed.
- Invocations reach a worker only through the events dispatcher, which has no
  zone route and requires the internal secret; the backend's own deploy path is
  the rehydrate route the dispatcher calls on a namespace miss. Locally there is
  no dispatcher, so the controller hands the service an in-process transport
  that deploys on miss itself.

The SDK stops allowlisting `puter` as a handler global — a handler that reaches
for an ambient SDK is now refused at publish time, naming `user` instead, rather
than passing the scan and failing on its first delivery.

Requires `events.workerNamespace`, `events.dispatcherUrl` and
`events.internalSecret`; without them nothing is addressable and background
deliveries stay retriable, as they did with the runtime off.

* fix: a handler's delivery token gets through the read routes

An events handler acts as the subscriber through the access token its
invocation carried, but every FS read route refused scoped access tokens
outright, so `user.fs.stat(event.path)` — the design's own example — answered
403 inside the worker. The read-side routes now admit them; the ACL each
handler already runs intersects the token's grant with its issuer's, which is
the check that keeps a token to what it was minted for. The end-to-end suite
asserts the stat from inside the isolate.

* fix: shorthand-method handlers publish as functions

`{ ingest({ event }) { … } }` stringifies without the `function` keyword, so
its source is not an expression and the events worker baked it as a broken
stub — every delivery a retriable 500 until the subscription suspended, with
nothing at publish time to say why. The SDK now gives a shorthand method the
keyword before hashing and sending; getters, setters and computed names are
left for the server-side check to refuse.

* feat: an app's events worker is listable and destroyable

An app with published handlers has an events worker, and hosted deployments
bill it monthly per app, so its owner needs to see it and be able to take it
down. The core announces the lifecycle on the bus — `events.worker.create`
when an app's first handler is published, `events.worker.destroy` when its last
one goes — with the owner as the actor, so pricing can plug in from outside.
`GET /events/workers` lists the caller's workers (paginated, with the script
each set deploys as) and `POST /events/workers/destroy` removes every handler
of an app under the same owner scoping as the handler routes, suspending the
subscriptions bound to them. `puter.events.workers.list/destroy` in the SDK,
a docs page, and a 5 MB cap on an app's combined handler source
(`events_worker_too_large`) so a set that publishes can always deploy.

* fix: harden the events worker runtime for production

- A 4xx is terminal only when it carries the handled marker the runtime (and
  the dispatcher) stamp on every answer that came from a script; an unmarked
  4xx — an edge 404 for a wrong dispatcher hostname, a WAF page — stays
  retriable and is logged, once per script per minute, with the runtime's
  reason header.
- Script names are scoped to this backend's exposed API origin, so two
  backends sharing a namespace never resolve one script with the wrong
  endpoint binding or key. Shape unchanged.
- Each handler is validated in the exact context it is emitted into and the
  whole generated file is compiled once; a source that would break the script
  marks every handler broken instead of deploying a SyntaxError.
- Locally, events scripts live under their own registry key: the public local
  worker host cannot reach them and an ordinary worker cannot take their name.
- A suspended or deleted app owner stops invocations; deploys are throttled
  per app per hour; in-flight deploys are keyed by app and script; the
  upstream deploy call times out; the generated source is size-capped with a
  margin over the publish cap; boot fails when the runtime is on but its
  preamble is not built. Byte-length secret compare, appUid shape check,
  dispatcher URL prefix preserved, wider connection pool.

* feat: background workers are listed in the sessions manager

A user paying for an app's events worker needs somewhere to see it and take it
down. The sessions manager gets a section listing the apps that run event
handlers in the background, with a Destroy action that removes their published
handlers.
2026-09-04 17:32:57 -07:00
dependabot[bot] dfab3d50cd chore(deps-dev): bump browserslist from 4.28.5 to 4.28.9 (#3759)
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.28.5 to 4.28.9.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.28.5...4.28.9)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-version: 4.28.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:20:29 +00:00
dependabot[bot] 9daaec4ec4 chore(deps): bump qs from 6.15.3 to 6.16.0 (#3757)
Bumps [qs](https://github.com/ljharb/qs) from 6.15.3 to 6.16.0.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.15.3...v6.16.0)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.16.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 16:17:36 -07:00
dependabot[bot] b7effe1e01 chore(deps): bump fast-uri from 3.1.0 to 3.1.7 in /src/puter-js (#3756)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.0 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.0...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 16:17:08 -07:00
Daniel Salazar 2a68e7aafb ci: add Dependabot config and patch auto-merge workflow (#3760)
Weekly npm/docker/github-actions updates plus vulnerability PRs;
patch-level Dependabot PRs auto-merge, minor/major stay manual.
2026-09-04 16:16:04 -07:00
Daniel Salazar 3355190a17 feat: add astra to models (#3755) 2026-09-04 15:56:53 -07:00
Nariman Jelveh 6ebbc6ae0c feat(gui): let users reposition and zoom a new profile picture (#3754)
* feat(gui): let users reposition and zoom a new profile picture

Picking a photo in the dashboard's Account tab used to stretch the whole
image into a 150x150 square, so anything that was not already square came
out distorted and off-center. The pick now opens an adjust step: a
dashboard-style modal (centered card on desktop, bottom sheet on phones)
where the user drags to reposition and zooms with the slider, pinch, or
wheel before saving. The saved result is the same 150x150 PNG as before.

Geometry lives in profilePictureCrop.js with unit tests; the modal owns the
DOM and pointer handling.

* fix(gui): stop double-encoding the crop modal's hint and frame label

i18n() already HTML-encodes its output, so wrapping it in html_encode()
again turned any apostrophe or ampersand in a translation into a literal
"&apos;" / "&amp;" on screen. Also puts the new profile_picture_* keys
in alphabetical order.

* fix(gui): let the crop frame take focus on click so arrow keys work after a drag

pointerdown's preventDefault() also cancels the click-to-focus that a
mousedown would have done, so after dragging the photo the arrow keys and
+/- went to the dialog container and did nothing. The frame now focuses
itself on pointerdown. Focus that arrives by pointer draws no ring; the
first key press lifts that so keyboard users still see where they are.

Also ignores secondary mouse buttons and treats a lost pointer capture as
a release so a pointer can't stay stuck in the gesture map.

* fix(gui): return focus to the avatar when the crop modal closes

The modal remembered document.activeElement to restore focus later, but
at that moment focus sits inside the file picker, which closes right
after -- so on Save, Cancel or Escape focus fell to <body>. The Account
tab now names its avatar as the place focus returns to, and the avatar
becomes a real button (role, tabindex, label, Enter/Space) so it can
hold that focus and be reached from the keyboard at all.

* fix(gui): announce the crop zoom as a magnification, not a 0-100 slider value

Screen readers read the range input's raw value, which maps to nothing a
user can picture. aria-valuetext now carries the zoom factor (1.0x-4.0x)
and follows every zoom source: slider, buttons, keys, wheel, pinch.
2026-09-04 14:38:55 -07:00
404oopsandClaude Opus 5 7248951410 fix(ai): make chat fallback reach streamed Claude calls and rank Azure explicitly (#3743)
* fix(ai): make chat fallback reach streamed Claude calls and rank Azure explicitly

- ClaudeProvider opens the upstream stream and awaits its connection
  before returning the populator, so an overloaded or rate-limited route
  throws from complete() and reaches the driver's fallback loop instead
  of surfacing as an error frame on a 200
- the OpenAI-compatible chat and completions routes only pin a provider
  when the caller sent one, so they get the same preferred healthy route
  puter.js callers do and unhealthy-route skipping applies to their
  first attempt
- Azure is ranked ahead of the vendors it fronts by an explicit tier in
  modelRouting rather than a price tie plus registration order
- drop Together's synthetic always-failing model-fallback-test-1 entry
- test that a 4xx leaves a route in rotation while a 503 marks it

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai): surface swallowed Claude stream errors, keep compat-route defaults

Review follow-ups on the fallback work.

- The pre-created event iterator only receives an error if a reader is
  already waiting on it, so a failure landing between the connect and the
  populator's first pull ended the stream cleanly — truncated content
  billed and reported as a success. Rethrow when the stream is errored.
- A refused stream deleted its Anthropic uploads but left the caller's
  message parts pointing at those file ids, so the fallback route was
  handed handles it cannot resolve. processPuterPathUploads now returns a
  restore() that both failure paths call.
- The OpenAI-compat routes keep pinning OpenAI when the caller sends no
  model at all, so the default model stays put instead of moving to
  Azure's.
- Say why /openai/v1/responses and /anthropic/v1/messages stay pinned:
  each translates one provider's native shape by hand.
- PREFERRED_PROVIDERS is unexported and its doc now states the rank is
  unconditional; the duplicated hidden-model list is one constant.

Co-Authored-By: Claude Opus 5 <[email protected]>

* fix(ai): undo the puter_path rewrite by field instead of snapshotting the part

Copying the content part kept whatever the caller sent on it — a large
inline `source` or `text` alongside `puter_path` — reachable until the
request ended, where overwriting the field used to make it garbage right
away. The only fields this function writes are `type`/`source` on success
and `type`/`text` on failure, and the fallback uploader keys off
`puter_path` alone, so restore undoes those three by name.

Co-Authored-By: Claude Opus 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-04 11:10:05 -07:00
Reynaldi Chernando 2966cee6b0 explicit mention of kv and fs being user scope and workers for backend (#3751) 2026-09-05 01:02:22 +07:00
Reynaldi Chernando d8735d950e fix kv set value type (#3750) 2026-09-05 00:30:17 +07:00
Reynaldi Chernando 48abf73d1a kv add array example (#3748) 2026-09-05 00:22:46 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 11c5a7ce52 dynamic workers docs (#3747)
* dynamic workers docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-04 22:57:11 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI b6876c50bf PUT-1477 Add fs commands to cli (#3746)
* Add fs commands to cli

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-04 21:31:33 +07:00
Daniel Salazar 6d7be8bd6a docs: kv key layout is the ACL boundary (PUT-1689) (#3693) 2026-09-03 19:43:28 -07:00
Daniel Salazar b451d05d10 feat: app-minted kv share handles (PUT-1688) (#3692) 2026-09-03 19:43:27 -07:00
Daniel Salazar f139588f89 feat: revocation settles kv handles (PUT-1687) (#3691) 2026-09-03 19:43:27 -07:00
Daniel Salazar ac5446f877 feat: cross-user KV share grants and handles (PUT-1686) (#3690) 2026-09-03 19:43:27 -07:00
Daniel Salazar adde6c3a29 feat: notification GUI rides the events surface (PUT-1685) (#3689)
Widen the notif: match filter and fetch scope for a session's own
generic developer/app-user subscribe: today it pins ref to the
session's own uuid, so a row naming an app (handler-suspension
notices, app-bound worker deploys) never matches live and never
replays on reconnect, even though the audience predicate already
grants the holder every such row it owns. The predicate is the
authority and already reruns per row/page after the match, so
widening the filter to it (account is unaffected — it never names an
app) adds no exposure.
2026-09-03 15:39:22 -07:00
Daniel Salazar fe5f81e52e feat: scoped app access to notifications (PUT-1684) (#3688)
An actor holding an app reads the `app-user` rows naming that app, plus its
`developer` rows when the holder owns it. `account` rows reach no app, and a
slice an actor may not see comes back empty rather than refused. The audience
predicate becomes the enforced read path in the same change that lifts the
blanket app-actor 403, layered behind an audience/app_uid SQL scope; two-segment
`notif:` subjects expand server-side from the actor's own app, so an app can
never name another app's uid.

No feature flag: `audience` defaults to 'account', so every pre-registry row is
default-denied to app actors and the backfill can only narrow.
2026-09-03 15:39:22 -07:00
Daniel Salazar 796683a133 feat: missed-event fetch and notification fold-in (PUT-1681) (#3687) 2026-09-03 15:39:22 -07:00
Daniel Salazar eb8f497e9f feat: presence and cross-region event forwarding (PUT-1679) (#3686)
* feat: presence and cross-region event forwarding (PUT-1679)

* fix: fan cache bumps to sibling nodes and stop the forward shed cascading (PUT-1679)

`outer.events.generationBumped` and `outer.events.presenceBumped` rode
`outer.*`, which the broadcast service only webhooks to peer regions;
only `outer.pubsub.*` also fans over Redis to a region's other nodes.
Both caches are per-process maps with no expiry, so a bump landing on
one node left its siblings stale until that user's next transition.
Renamed onto `outer.pubsub.events.*`; the listeners already accept the
`from_outside` copy the Redis re-emit carries.

`PeerForwardQueue.push` called `onOverflow` synchronously and the
handler pushed markers straight back, each of which re-tripped the
bound and shed the next item: one item over a 5000 bound recursed ~2200
deep, threw a RangeError, and turned ~2200 queued deliveries into gap
markers. It also re-summed `bytes` over the whole queue per drop. The
handler now returns its markers and the queue appends them past the
bound check, sheds deliveries before markers, keeps one pending marker
per (peer, subscription), and subtracts bytes per dropped item.
2026-09-03 15:39:21 -07:00
Daniel Salazar 32a838d2e2 fix: one ** per match pattern and one * per segment (#3736)
`compileMatch` turned every `*` into an unbounded `[^/]*`, so a subject
like `fs:~/x/*a*a*a*a*a*a*a*a*a*a*z` tested against a 24-char filename
with no `z` — both named by the same subscriber — cost the regex engine
C(34,10) splits before failing: 1.0 s per event at ten stars, 3.7 s at
eleven, on the fs-write dispatch path, inside the 256-character cap.

With one `*` per segment the delimiters pin each star and a wrong split
dies in one step; the single `**` is the only choice point left, so the
worst allowed shape is O(depth × length): 0.4 ms at depth 60 of 200-char
segments. Every documented pattern (`*.png`, `**/build.log`, `**/*.png`,
`dir/**`, `report-?.csv`) stays valid; `*a*`, `a**b` and `**/x/**` are
refused with `invalid_subject_pattern`. The rule is stated where the
syntax is introduced, on the limits page, and in onLocal's error table.
2026-09-03 15:08:49 -07:00
404oops 1b777cd760 Merge pull request #3730 from HeyPuter/filipkujundzic/put-1620-video-generation-timeouts-return-500-and-page-on-call
fix: stop video generation timeouts and failures from paging (PUT-1620)
2026-09-03 22:54:58 +02:00
Daniel Salazar 799fc4ac3c feat: send app icons as a subdomain URL plus an API fallback (#3734)
App payloads carried only the /app-icon endpoint URL, which 302s to the
icons hosting subdomain. Networks that mangle that redirect render no icon
at all, and every icon load pays a round trip for the hop.

Ship the direct subdomain URL as `iconCdnUrl` alongside it (taskbar items,
installedApps, recent/recommended launch apps, suggested apps), and have the
GUI load that first with the endpoint URL as a one-shot retry - desktop
taskbar, start menu, dashboard app grid and recents. Only rows whose `icon`
column is already an http(s) URL get one: a data: column means the resize
pipeline has not written anything to the subdomain yet.

Also folds the four copies of the generated-size list into one exported
APP_ICON_SIZES.
2026-09-03 13:32:55 -07:00
Juan Fernando Castro d855315db8 Merge pull request #3714 from HeyPuter/juancastro/put-1708-26-teamcontroller-and-audit-writes
🏗️ PUT-1708 + PUT-1709 + PUT-1743: TeamController, teams_enabled, and the isolation suite
2026-09-03 16:21:02 -04:00
Juan Castro 9bc2cd741c feat: add TeamController, the teams_enabled flag, and the isolation suite
Covers PUT-1708, PUT-1709 and PUT-1743.

Twelve routes, every one setting requireUserActor -- that option is what
installs requireAuthGate, requireVerifiedAccount and requireNonAccessTokenGate,
because server.ts derives `needsAuth` from the route options. Reads need it as
much as writes: without an auth option a route gets no suspension check and
admits access tokens, so a just-disabled member could still read the roster and
a scoped third-party token could read the audit log.

Authority is checked before anything observable. Validating the body first made
POST /members answer 400 before 403, and resolving :username first turned the
member routes into a global username-existence oracle.

Provisioning applies the same username and email rules as signup rather than
its own -- USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and
validator.isEmail, now exported from AuthController. Without them a workspace
could mint accounts signup would refuse, claim unregistered reserved names, and
mail arbitrary unvalidated addresses.

Handle problems are 400 or 409 rather than a bare Error, which the server turns
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable drops sessions through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key; without that a
disabled member kept authenticating from cache for the session TTL, which is
exactly the "takes effect on the next request, not after a cache TTL" property
disable is supposed to have. Revoking also preserves last_ip/last_user_agent,
which the member-facing audit view reads.

Audit writes live in TeamService at the point of each action rather than in the
route, so a caller reaching the service directly cannot skip them, and the SQL
lives in TeamStore. Audit reads map internal user ids to usernames, and remain
readable by the owner after the workspace is soft-deleted -- otherwise the
delete_team entry was written and immediately unreachable.

teams_enabled gates route registration through an optional isEnabled() the
server honours, so with it off the paths do not exist rather than existing and
refusing. It does not gate DDL.

TeamIsolation.http.test.ts asserts the negative the feature rests on: the
workspace manages accounts and cannot read them, including through a
full-access token and after the member is disabled. It asserts outcomes rather
than the absence of an implicator.
2026-09-03 15:57:23 -04:00
Juan Castro 673f7fe75f feat: provision org accounts from a workspace
Covers PUT-1705. The master account supplies { username, email }; the account
is created with no password, gets the default filesystem tree, joins with
org_owned = 1, and receives a one-shot activation link.

Activation reuses password recovery rather than new token machinery: the same
pass_recovery_token, the same one-hour purpose-scoped JWT, the same
/action/set-new-password link. No team_activation table, no new token type,
and no unauthenticated endpoint on the team surface. Activation state needs no
column either -- an unactivated account is one with no password.

Applies the same username and email rules as signup rather than its own:
USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and validator.isEmail,
now exported from AuthController. Without them a workspace could mint accounts
signup would refuse -- the username becomes the /username home-directory
segment -- claim unregistered reserved names, and send activation mail to
arbitrary unvalidated addresses at the route's daily limit.

Usernames come from Puter's global pool, so a taken one is refused with free
alternatives rather than silently modified: a suffixed name would appear in
every share dialog that person ever sees, and they never agreed to it. The
check runs before any write, so a rejected provision leaves no orphaned user
row -- asserted by a test on the workspace's member count.

The new account carries requires_email_confirmation, since the address came
from the administrator rather than its holder.

Adds a team_account_activation email template stating what the workspace can
and cannot do -- including that it can reset the password, which the design
requires be said rather than only claiming files are private.

free_storage stamping and the billing event are phase 3.
2026-09-03 15:57:23 -04:00
Juan Castro da2b42d7fe feat: add TeamService workspace lifecycle and disable/re-enable
Covers PUT-1704 and PUT-1707: creating a workspace, admitting the master
account, and the whole of offboarding.

`createWorkspace` admits the creator with org_owned = 0, which is what makes
the master pay for itself and stay an invalid target of every member route.
`checkOwnerInvariant` asserts the rule no dialect can express -- the owner is
a member with org_owned = 0 and the only such member -- and a test breaks it
deliberately, since the schema cannot refuse a second one.

Three authority checks: 404 to a stranger so the endpoint is not an existence
oracle, 403 to a member who is not the master, and the master refused as a
target of member routes.

Handle problems surface as 400 (unusable) or 409 (taken), including the
unique-index race. `TeamStore` throws a bare Error, which the server would turn
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable writes `user.suspended` as well as suspended_at and suspended_reason.
PUT-1707 named only the latter two, but those are siblings added by 0061 and
0063 -- `userProtected` rejects on `if (user.suspended)` and reads neither.
Setting only the timestamp and reason would have recorded a disable that never
took effect, and disable is the whole of offboarding here.

Sessions are dropped through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key with its double-delete
pattern; without that a disabled member keeps authenticating from cache for the
session TTL, which is exactly the "next request, not after a cache TTL"
property disable is supposed to have. Revoking also preserves last_ip and
last_user_agent, which the member-facing audit view reads.

Files are untouched and re-enable restores the account.

Adds team_not_found, not_the_master_account and not_an_org_account to the
HttpError legacy codes, which the controller also needs.

Billing events, invalidateActorSubscription, audit rows and the GUI push are
deliberately not here -- they belong to phase 3 and PUT-1708.
2026-09-03 15:57:23 -04:00
Juan Castro 8b4e259421 feat: add team membership reads and writes to TeamStore
Membership management for workspaces: addMember, removeMember,
getMembership, isMember, listMembers and listTeamsForUser. The permission
scan is untouched -- readUserGroupPerms already joins jct_user_group and
resolves group grants; this is the management side.

Resolves the ticket's "do not leave two writers" by splitting domains and
enforcing the split in SQL rather than by convention. Every existing caller
of GroupStore targets a seeded system group -- ADMIN_GROUP_UID,
default_user_group, default_temp_group -- never a team, so the two stores
were already disjoint in practice. GroupStore.addUsers/removeUsers now carry
`AND kind IS NULL`, making a team uid a no-op there, which costs no extra
query because it folds into the existing subquery and matches how addUsers
already treats an unknown username. TeamStore's writes select group_id from
a kind-filtered subquery, so neither store can reach the other's rows.

org_owned is written here but never accepted from a request; TeamService
sets it at provisioning and workspace creation only.

listMembers is keyset-paginated on id per doc/pagination.md, using the
shared cursor and limit helpers and fetching one row past the limit to
decide whether a cursor is warranted.

Passes 1/0 for org_owned rather than db.booleanValue, which yields a real
boolean on postgres and is rejected by the smallint column there -- sqlite
accepted it silently.
2026-09-03 15:57:16 -04:00
Juan Castro 37cb11df8c feat: add TeamStore for workspace CRUD
`GroupStore` has only addUsers/removeUsers; nothing creates, reads back or
lists a group at runtime. `TeamStore` is that missing half, scoped to rows
with `kind = 'team'`.

A workspace is addressed by `uid`, which `group` has carried as NOT NULL
UNIQUE since 0015. `handle` is a mutable display label with no addressing
role, so a rename invalidates nothing and a stale reference can never
resolve to a different workspace.

Soft delete releases the handle and keeps `name`. Nothing points at a
handle, so the name returns to the pool instead of being reserved forever
by a global unique index that cannot exclude dead rows -- mysql has no
partial indexes, so that exclusion was never available.

Handles validate to ^[a-z0-9]+(-[a-z0-9]+)*$, 3-64 chars, against a
reserved list. The charset is deliberately narrower than the column so the
engines' collations cannot disagree: mysql's utf8mb4_unicode_ci also folds
accents and eszett, which sqlite's NOCASE and postgres's lower() do not.

Every read filters `kind = 'team' AND deleted_at IS NULL`, which is what
makes the seeded admin/system groups unreachable rather than merely absent.
Handle lookups compare lower(handle) on postgres, where the index is on
that expression rather than the column.
2026-09-03 15:20:07 -04:00
Juan Fernando Castro ad70e8763b Merge pull request #3708 from HeyPuter/juancastro/put-1701-13-add-audit_team_membership-and-shareholder_group_id
🏗️ PUT-1701: Add audit_team_membership and share.holder_group_id
2026-09-03 15:17:34 -04:00
Juan Castro 64db3fa050 feat: add audit_team_membership and share.holder_group_id
Insert-only record of what a workspace administrator did to an account,
shaped like \`audit_user_to_group_permissions\` after 0019: nullable FK
beside a NOT NULL \`_keep\` column. The FKs are ON DELETE SET NULL, never
CASCADE, so hard-deleting an account cannot erase the record of the
resets performed on it.

Two indexes rather than one. The member's own view is the only place a
reset becomes visible to the account it was performed on, so
(user_id_keep, id) is a read path, not an optimisation.

\`share.holder_group_id\` mirrors \`holder_user_id\` from 0067. The existing
unique index does not constrain team shares at all -- it leads with
\`holder_user_id\`, which is NULL on every team share, and NULLs are
distinct -- so the group-scoped unique index is what prevents duplicates.

Drops the \`role\` column from the specified DDL: it contradicted the
settled single-administrator model.
2026-09-03 15:07:53 -04:00
Juan Fernando Castro 43f62cf7d0 Merge pull request #3705 from HeyPuter/juancastro/put-1700-12-deduplicate-jct_user_group-before-adding-the-unique-pair
🏗️  PUT-1700: Deduplicate jct_user_group and make membership writes unique
2026-09-03 15:07:49 -04:00
Juan Castro dc7b456afa fix: run the membership dedup once instead of on every boot
Review follow-ups on the dedup migration.

mysql and postgres track no per-file applied state and re-execute every
migration on each boot, so the unguarded DELETE self-joined the whole table at
every process start, forever. Both now sit behind the same index-existence
check that guards the ALTER, which also stops a rolling deploy deleting on one
instance while another adds the index.

The dedup test was a false green. `targetVersion: 67` never applies 0071 -- the
loop breaks on `threshold + 1 >= targetVersion` but stamps the target anyway --
so the fixture asserted the current schema version on a database missing a
migration. It now replays the real 0072 file against a fully migrated database,
and a second test pins the off-by-one so nobody builds a fixture on it again.
2026-09-03 14:33:46 -04:00
Juan Castro c1c1588939 fix: deduplicate jct_user_group and make membership writes unique
\`jct_user_group\` had no unique constraint on (user_id, group_id) and
\`GroupStore.addUsers\` had no conflict clause, so re-adding a member
inserted a second row. \`readUserGroupPerms\` joins the junction table on
group_id alone, so each duplicate returned another copy of every group
permission the user holds.

Deduplicate keeping the lowest id, add the unique pair index, and make
\`addUsers\` ignore conflicts via the existing \`insertIgnoreInto\` helpers --
without that last part the index turns a re-add into a raised error, which
five call sites would log as a failed signup step.

mysql cannot delete from a table it reads in a subquery (error 1093), so
it uses a self-join with the same lowest-id-wins semantics.
2026-09-03 14:32:30 -04:00
Juan Fernando Castro abb426a717 Merge pull request #3704 from HeyPuter/juancastro/put-1699-11-add-team-columns-to-group-and-jct_user_group-three
🏗️ PUT-1699: Add team columns to group and jct_user_group
2026-09-03 14:26:58 -04:00
Juan Castro eae25459f4 fix: make handle lookups case-insensitive and pin the reset flag
Review follow-ups on the team columns.

NOCASE moves onto the `handle` column itself, not just the index. Index-only
NOCASE makes uniqueness case-insensitive while leaving `WHERE handle = ?`
case-sensitive, so the same lookup would match on mysql (utf8mb4_unicode_ci)
and miss on sqlite. Postgres still needs lower(handle) at the call site.

`requires_password_change` becomes NOT NULL DEFAULT 0 on all three dialects,
matching the three sibling `requires_*` flags. Left nullable, any query written
as `= 0` would silently exclude every pre-existing user.
2026-09-03 12:10:48 -04:00
Juan Castro 2a5c089dbe feat: add team columns to group and jct_user_group
Schema foundations for teams. Ships dark — nothing reads these columns yet.

group:          kind, name, handle, plan_id, deleted_at
jct_user_group: org_owned
user:           requires_password_change

`kind = 'team'` marks a workspace; the seeded system groups keep it NULL so a
team query can never surface them. `org_owned` distinguishes accounts the
workspace created from the master account, deciding who pays rather than who
may read. `requires_password_change` is a fourth `requires_*` flag for
assertVerifiedAccount, needed by the phase 5 reset flow but shipped here to
avoid a second three-dialect migration.

`handle` uniqueness is case-insensitive, and the dialects disagree by default:
mysql gets it from utf8mb4_unicode_ci, sqlite needs COLLATE NOCASE (as
0055 does for usernames), and postgres indexes lower(handle). Without this the
same migration would accept `Design-Team` next to `design-team` on sqlite and
postgres while mysql rejected it. Postgres handle lookups must therefore
compare lower(handle) to use the index.

idx_group_owner is sqlite-only: mysql and postgres already index that column.
idx_jct_user_group_group is composite, unlike the existing single-column keys.
2026-09-03 12:10:40 -04:00
404oops 55fa5cadcb Merge pull request #3732 from HeyPuter/gemini-38-flash
gemini 3.8 flash direct integration
2026-09-03 17:35:22 +02:00
Reynaldi Chernando 644f59f9dd gemini 3.8 flash direct integration 2026-09-03 22:18:02 +07:00
404oopsandClaude Fable 5.1 8b01077d44 docs: describe the real txt2vid error body and per-provider wait windows
The Errors section claimed a `{ message, code }` body and a single
ten-minute wait. The body also carries `error`, `errorCode`, `provider`,
`upstreamCode` and, for pre-job rejections, `upstreamStatus`, and the
Sora provider still waits five minutes and folds refusals into
`upstream_failed`. Document the fields and the exceptions.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-03 16:57:36 +02:00
404oopsandClaude Fable 5.1 627a5b2d5e fix: stop video generation timeouts and failures from paging (PUT-1620)
A video job that outlived its poll window, an SDK request that timed out,
or a Veo operation that finished with an error all reached the HTTP error
handler as plain Errors. Each became an unhandled 500 with critical
severity and paged on-call for what is the provider's pace or the
provider's fault.

Video providers now share one poll loop that gives up with a 504
`upstream_timeout`, treats a transient poll failure (timeout, dropped
connection, 408/429/5xx) as a missed poll rather than a failed job, and
stops polling with a 400 `client_aborted` when the caller disconnects, so
nothing is metered for a clip nobody will receive. The driver controller
exposes the disconnect as an `abortSignal` on the request context. The
window is ten minutes for every provider; Together and BytePlus move up
from five.

Failed jobs are classified: content-filter refusals become a 400
`bad_request` with `errorCode: moderation_flagged`, rejected parameters a
400 `upstream_bad_request`, and anything else a 502 `upstream_failed`,
each carrying the provider's own code. Veo's filtered output keeps
`disallowed_value` and gains the same `errorCode`. The sanitizer and
content-filter pattern move from the Replicate provider into a shared
util so image and video agree.

Status-less SDK connection timeouts are translated to a 504
`upstream_timeout` at the driver boundary, and the chat driver records
them per attempt so an all-timeout chain is a 504 and a mixed chain is
`upstream_failed` instead of an `internal_error` 500. The Together chat
client gets the same ten-minute request timeout as the other providers.

The OpenAI video provider is left alone beyond an import path: its API is
scheduled to shut down on 2026-09-24.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-03 16:47:16 +02:00
Daniel Salazar 72011fa6ed feat: events metering and quotas (PUT-1683) (#3685)
* feat: events metering and quotas (PUT-1683)

* fix: drop the standing subscription charge and price single deliveries at 100 µ¢

An idle durable row costs nothing worth billing; the plan quotas bound how
many an account holds. Removing the daily line also removes the global
day-claim, the whole-table scan and the timer it rode on.

* test: stop asserting on documentation pages

The limits test read rate-limits-and-quotas.md and grepped it for numbers,
so every rewording of the page failed the backend suite. Docs are kept in
step by the PR and checked in review; AGENTS.md now says so.
2026-09-03 01:39:07 -07:00
Daniel Salazar 41b837b398 feat: background delivery seam, retries, and consent (PUT-1682) (#3684) 2026-09-03 01:39:06 -07:00
Daniel Salazar 04c00385b9 feat: event handlers, context, and suspension machine (PUT-1680) (#3683) 2026-09-03 01:39:06 -07:00
Daniel Salazar 9626ab9c71 feat: KV change events and KV subjects (PUT-1678) (#3682) 2026-09-03 01:39:06 -07:00
Reynaldi Chernando 3de6eeb474 init docs recipes (#3728) 2026-09-03 10:04:01 +07:00
Neal Shah af27115f68 Attempt 1: Fix PUT-1625 (#3727) 2026-09-02 21:36:04 -04:00
Daniel Salazar d8ebfc8023 feat: delivery re-check cache, revocation and anchor settle (PUT-1677) (#3681)
* feat: delivery re-check cache, revocation and anchor settle (PUT-1677)

* fix: authorize re-anchors, settle each row once, purge revoked backlog (PUT-1677)

- A path-form row whose anchor is deleted only climbs to an ancestor its
  holder may still watch under the mode it subscribed with; otherwise it ends
  with `anchor_deleted`. It used to land on any surviving ancestor (a guest's
  row on the owner's home), where the re-check denied every delivery but the
  row still held an anchor slot and a filter evaluation there.
- After a climb the new anchor is re-verified and the climb repeated if a
  recursive delete took that level too, instead of leaving the row on a dead uid.
- suspend() is one conditional write per row and reports which rows it was the
  one to suspend; concurrent settles of the same grant (an unshare revokes
  several strings) no longer each purge, forget and notify the same rows.
- One "subscriptions ended" notification per holder and app, carrying the count
  and subjects, instead of one per row.
- A revoke that removed nothing no longer announces; the sweeper purges (not
  defers) the backlog of a permission_revoked row; the reap purges pending
  entries with the row.
- The delivery auth cache indexes entries by subscription so forget() is not a
  scan of the whole cache.
2026-09-02 16:00:14 -07:00
Daniel Salazar 41d4850ab4 feat: pending event deliveries and delivery-class invariants (PUT-1676) (#3680)
* feat: pending event deliveries and delivery-class invariants (PUT-1676)

* fix: make pending delivery claims and drains atomic, keep the region under its ceiling (PUT-1676)

- claim() and the drain-time reindex run as Lua over the subscription's own
  {subId}-tagged keys. Two claimers can no longer both lease the head, and a
  drain that finds the queue empty deletes it in the same step it checks, so a
  concurrent enqueue is never wiped between the two.
- An append writes the entry and its queue position in one MULTI (same slot),
  with the index seeded before it and corrected after, so an entry is never
  visible without its position and never left out of the sweeper's index.
- Pipelines no longer mix slots (index/counter vs. per-subscription keys), so
  the store works on a multi-shard cluster, not only a single-shard one.
- Region shedding counts the marker it leaves behind; it used to stop one over
  the ceiling and convert a real event into a marker on every enqueue after.
- A claimed or suspended subscription moves to the back of the sweeper's index,
  so a delivery nobody settles cannot hold the head against every other backlog.
- `single` rows must carry a `worker` target: with sockets exhausted and no
  handler, an unacknowledged delivery would sit at the head forever.
- A gap marker for a row with no socket target is dropped rather than counted
  as a delivery of nothing.
- Backlog keys carry a 7-day TTL, refreshed by every claim, as a backstop for
  keys a purge/enqueue race left unindexed.
2026-09-02 16:00:14 -07:00
Daniel Salazar 0dfbceb047 feat: durable event subscriptions store, cache, and routes (PUT-1673) (#3679)
* feat: durable event subscriptions store, cache, and routes (PUT-1673)

* fix: durable subscription hardening (PUT-1673)

- Expired rows stop delivering at dispatch time and no longer count toward
  the per-account cap, instead of waiting for the sweep.
- The expiry sweep runs hourly with a jittered first pass shortly after boot;
  a 24 h interval never fired on a fleet that redeploys more often than that.
- Only a durable generation bump marks peer regions cold. A session
  subscribe/unsubscribe in one region used to force a primary read in every
  other region on its next dispatch.
- `subject`/`anchor_path` widen to varchar(4096) to match `fsentries.path`,
  and subjects longer than that are refused with `invalid_subject` rather than
  failing the insert on MySQL/Postgres.
- The dispatch and durable integration suites wait for the specific delivery
  they expect and assert only within their own folder; the old any-delivery
  `settle()` let a late event from a previous test satisfy or pollute the
  next one under CI load.
2026-09-02 16:00:13 -07:00
Daniel Salazar cab5253923 feat: puter.events SDK module (PUT-1675) (#3678)
* feat: puter.events SDK module (PUT-1675)

* fix: end event subscriptions on a server-side disconnect and retry budget refusals (PUT-1675)

- A disconnect socket.io will not retry (`io server disconnect`) now fails
  every subscription with `events_connection_failed` and drops the socket, so
  the next onLocal() starts a fresh connection instead of reusing a dead one.
- The connection closes when the last subscription lapses on re-subscribe.
- A re-subscribe refused with `too_many_requests` is retried after 10 s
  rather than ended for good.
- The API suite no longer passes when it cannot connect; every runtime it
  runs on (node, browser, workerd) carries the socket.
2026-09-02 16:00:12 -07:00
404oopsandClaude Fable 5.1 d89dc347a2 fix: keep the credit-bounded max_tokens on the context-length retry (PUT-1628) (#3718)
OpenRouter and Together reject a request whose prompt plus max_tokens
overflows the model's context window. Both providers retried by deleting
max_tokens, which threw away the output cap the credit gate had sized to
the caller's remaining balance and let the retry run to the model's full
output limit with no second gate and no new hold.

The retry now goes through a shared helper that sizes a new cap from the
window and input count the rejection reports, falling back to the model's
declared context and a doubled prompt estimate, and never exceeds the cap
the gate set. When no window can be determined or no output fits, the
original rejection is rethrown instead of retrying uncapped. The rejected
params are copied rather than mutated, so the first attempt's record is
not rewritten after the fact.

Also corrects the estimator's own comment, which described the mean of
two approximations as a deliberate halving.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-02 13:28:36 -07:00
Daniel Salazar 66dc0a6396 feat: subscribe ACL and app scoping for event subscriptions (PUT-1672) (#3677) 2026-09-02 13:08:23 -07:00
Daniel Salazar fb8e2976dc fix: don't redirect puter.com (#3723) 2026-09-02 11:57:30 -07:00
404oopsandClaude Fable 5.1 fd7b517448 fix: classify Replicate prediction failures instead of 500ing (PUT-1608) (#3719)
A Replicate prediction that ran and ended `failed` reaches the provider as a
plain Error with no HTTP status, so the driver-boundary translator could not
classify it and it surfaced as an unhandled 500, a critical alarm, and an
on-call page. Most of these are the model's content filter refusing the
user's prompt.

Wrap the run call and classify the failure: content-filter refusals become
a 400 with `errorCode: moderation_flagged` (the code chat refusals already
use); anything else becomes a 502 `upstream_failed`, which the alarm gate
skips. Status-bearing SDK errors pass through untouched so the boundary
translator keeps handling them. Upstream messages are stripped of markup and
bounded so an HTML error page can no longer ride into a response body or an
alarm signature.

Documents the codes callers can now act on in the txt2img reference.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-02 11:51:08 -07:00
Neal Shah 262f1dc5c5 allow concept of "home regions" (#3699)
* allow concept of "home regions"

* remove extraneous config value not applicable to repo
2026-09-02 14:11:55 -04:00
Daniel Salazar 7ec674d33b feat: app sockets join their own events room (PUT-1671) (#3676) 2026-09-02 09:01:22 -07:00
Daniel Salazar b07d2e109f feat: session event subscriptions and dispatch hot path (PUT-1666) (#3675) 2026-09-02 09:01:22 -07:00
Daniel Salazar 92e1b40361 feat: events subject registry, parser, anchors, filters (PUT-1660) (#3674) 2026-09-02 09:01:21 -07:00
Daniel Salazar d0ee19a2c2 chore: remove dead notification surfaces (PUT-1669) (#3673) 2026-09-01 22:04:33 -07:00
Daniel Salazar 82076ea767 feat: notification retention sweep (PUT-1668) (#3672) 2026-09-01 22:04:32 -07:00
Daniel Salazar 5f37817a5c feat: notification type registry and audience rules (PUT-1665) (#3671) 2026-09-01 22:04:32 -07:00
Daniel Salazar 148b930182 feat: notification scope columns and backfill (PUT-1659) (#3670) 2026-09-01 22:04:31 -07:00
Daniel Salazar 0385072e06 feat: better verification step handling (#3717) 2026-09-01 21:17:19 -07:00
Daniel Salazar 1736d23ee7 Ds/put 1674 (#3715)
* feat: app-scoped share listing and revoke surface (PUT-1670)

* fix: address review on app-scoped share surface

- Scope the uid-addressed revoke to the named row: only that row's
  issuer's grant is withdrawn, and only that one invite cancelled —
  an app or owner addressing one row no longer takes another issuer's
  grant on the same (item, recipient) pair with it.
- Delete a pending row directly on uid-addressed revoke, so an invite
  whose address registered but never claimed can still be withdrawn.
- Read the legacy `issuerAppUid` data key in the SQL app filter and
  grouping, alongside the unified `issuedByApp`.
- Refuse malformed `appUid` input (duplicated param, empty string)
  instead of silently listing everything, and refuse app-listing
  cursors that decode but name no appUid.
- Derive the acting app from `effectiveApp` alone, per the actor
  contract; drop the second derivation site.
- Pin the attribution semantics with tests: one row records one
  issuance, so re-sharing the same pair re-attributes it to whoever
  issued last, in both directions.
- Soften the uniform-404 docblocks to what the gates actually answer.

* feat: readable grant audit trail (PUT-1674)

* fix: cover the apps summary's no-app-group first page (PUT-1670)

listOutboundApps sorts the no-app group first via an empty-string
sentinel. Add a regression test pinning that a first page (no cursor)
actually returns it, and that the cursor it hands back resumes past it
into the app-keyed groups rather than skipping or repeating.
2026-09-01 14:07:15 -07:00
Daniel Salazar 66a975f659 feat: global outbound share listing (PUT-1664) (#3694)
* feat: global outbound share listing (PUT-1664)

* fix: address review on outbound share listing

- Check share-row liveness per (holder, entry, issuer) so a grant
  withdrawn outside unshare doesn't stay listed while another issuer
  still reaches the same holder; batch the permission reads across the
  whole page instead of per holder.
- Retire a revoked issuer's unclaimed invites in the revoke cascade,
  and hide invites whose issuer lost their authority at read time.
- Unify the pending/active app-attribution key on `issuedByApp` and
  dual-read the legacy `issuerAppUid` spelling.
- Add the missing share issuer index (sqlite, postgres) and correct
  the listOutbound plan comment.
- Refuse cursors that decode but name no id instead of silently
  restarting from page one.
- Consolidate the five hand-built ResolvedShare literals and the two
  listing endpoints' parse/shape code.
- Ship the SDK surface: puter.fs.listSharedByMe() with docs, types,
  suite coverage, and the rate-limit page entry.
2026-09-01 13:52:37 -07:00
404oopsandClaude Fable 5.1 6478a8f47d Add Claude Fable 5.1 (#3711)
Register claude-fable-5-1 in the Claude catalog and gate it in the
provider the same way as Fable 5: no sampling params, effort via
output_config, adaptive thinking with summarized display. The bare
claude-fable / claude-fable-latest aliases move to 5.1, matching how
the Opus aliases moved when Opus 5 landed.

Fable 5.1 keeps Fable 5's $10/$50 per MTok, 1M context and 128K output,
but bills cache reads at 0.025x input instead of the 0.1x every other
Claude model uses, so the catalog row carries its own rate and a test
pins it.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-01 15:34:49 -04:00
Nariman Jelveh fdd86ceb03 Stop server-rendered landings flashing for signed-in users (#3707)
The shell renders its anonymous markup — the marketing homepage, an
`/app/<name>` landing — off the session cookie alone, and that cookie is
set with no maxAge, so a browser drops it on quit while the GUI's
localStorage token lives on. A returning user is served the anonymous
page and the GUI only tears it down once `whoami` answers, a network
round-trip after first paint. That teardown is the flash.

Gate it before the paint instead. The shell now emits, as the first thing
in <head>, a rule hiding `.hide-if-logged-in` under an <html> class that
an inline script adds iff `auth_token_v2` is in localStorage. The rule is
already in the cascade when the markup is parsed, so a browser holding a
token never paints it at all.

`initgui` settles the guess the token represents: `whoami` confirming the
session removes the nodes outright (replacing the old `#appLanding`
removal), and no session — none stored, or one `whoami` rejected — drops
the class so the markup comes back. The gate carries its own 12s failsafe
so a bundle that never boots can't strand a blank page.

SEO is unaffected: the HTML is byte-identical for every client, nothing
branches on user-agent, and a crawler has no stored token so it never
adds the class. Unreadable storage fails open the same way.

Anonymous markup opts in with `class="hide-if-logged-in"`, which
`home.html` already carried.
2026-09-01 09:19:00 -07:00
Daniel Salazar 7584dffc4b fix: metering for ai (#3703) 2026-09-01 09:11:04 -07:00
jelveh b400e06473 Remove login landing fade-out overlay
Drop the delayed fade-out when a logged-in user reaches the GUI and remove the landing overlay immediately instead. This avoids the overlay sitting above the email/phone/card verification gates because of its z-index.
2026-08-31 20:13:58 -07:00
jelveh 22c3316109 Hide app landing after session restore
Remove the /app landing overlay once `whoami` confirms the user is authenticated, even if the server rendered the page as anonymous because the session cookie was missing. This prevents the overlay from covering the email, phone, and card verification gates during localStorage-based session restores.
2026-08-31 20:07:47 -07:00
404oops a27852de0a Merge pull request #3655 from HeyPuter/FK/normalize-openai-format
Comprehensive Normalization.
2026-08-31 22:24:34 +02:00
jelveh f80a0d72b3 fix: dashboard 'Add Existing Account' dialog buried on phones/tablets
On device-phone/device-tablet, style.css pins every .window to
z-index 9999999 !important — including the fullpage dashboard window.
The Add Existing Account login dialog opens with backdrop: true, and
the .window-backdrop wrapper is not a .window, so it kept its small
inline z-index and rendered entirely behind the dashboard.

Pass stay_on_top: true like every other backdropped dashboard dialog,
and make the Forgot-password child dialog inherit the flag so it does
not open buried under the now stay-on-top login window's backdrop.

Verified with Playwright on iPhone 13 emulation and a desktop
viewport: login and recover-password dialogs stack above the
dashboard in both.
2026-08-30 21:45:43 -07:00
Nariman JelvehandClaude Fable 5 3e8b2735bb Cap /app/<name> meta descriptions at 150 characters (#3698)
The app shell fed the row's full description into <meta name=
"description">, og:description and twitter:description. Long
descriptions now collapse whitespace and cut at a word boundary with an
ellipsis, never exceeding 150 characters.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-30 20:17:04 -07:00
Daniel Salazar 5065bcbf0a fix: document allowedAppIdsGate's real non-app-gate behavior (PUT-1667) (#3669)
The JSDoc claimed non-app actors are rejected; they are passed through, and
the check reads `actor.app` rather than `effectiveApp`, so an app in the
token chain is invisible to it. Callers depend on the pass-through — routes
combining `adminOnly` + `allowedAppIds` are reached with an admin's browser
session, and the dev-account surface is called from the desktop's own
session — so the docs are corrected instead of the behavior, on the gate and
on the `allowedAppIds` route option. Tests cover the gate per actor shape
(user session, worker, full-access token, app-issued access token,
app-under-user allowed/disallowed) and the admin composition.
2026-08-30 12:55:39 -07:00
404oopsandClaude Fable 5 7e1cdefd98 feat(ai): backfill provider catalogs from live /models listings
A live audit of every keyed provider's /models endpoint against the hardcoded
catalogs found no stale entries but large gaps. This backfills them under four
rules: nothing vendor-deprecated, nothing without a price confirmed on the
vendor's official pricing page (each entry's source was recorded during
review), nothing absent from the live /models listing, and nothing that fails
a live routing probe.

Added: 46 Alibaba entries (qwen3/3.5/3.7/3.8 families, VL/omni/MT lines, and
Model Studio's hosted GLM/DeepSeek/Kimi third-party models) plus 9 dated
aliases; OpenAI chat-latest and gpt-4o-2024-11-20 plus 16 snapshot aliases;
Gemini gemma-4-31b-it and gemma-4-26b-a4b-it (vendor-documented free tier)
plus rolling -latest aliases; Mistral-hosted zai-glm-5-2 and a
mistral-medium-3.5 alias; deepseek-v4-flash-vision-exp; glm-5.3-flash.

Culled by the rules: 15 vendor-deprecated OpenAI entries (the 3.5/4/4-turbo
legacy line, gpt-4o-2024-05-13, o1-pro, four chat-latest predecessors, the
5.x codex line — deprecations page, most shut down 2026-10-23) and dated
aliases onto the deprecated o1/o3-mini/o4-mini; qwen3-vl-flash-2025-10-15
(live routing probe returned upstream 400 twice). Tiered Alibaba prices are
encoded at the base tier and busy-hour rates where time-of-day priced, noted
in comments.

Every surviving addition was verified end-to-end through a local deployment:
58/58 answered a live prompt, including all 46 Alibaba entries and every
spot-checked alias.

Not changed, flagged for maintainers: pre-existing o1, o3-mini and o4-mini
entries are now vendor-deprecated (shutdown 2026-10-23); the pre-existing
deepseek-v4-flash/-pro prices no longer match DeepSeek's current pricing
page; gemma-4 emits its own <thought> markup inline in content.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-30 20:50:46 +02:00
404oopsandClaude Fable 5 4199a2506d feat(ai): forward Mistral prompt_mode so magistral can emit separable thinking
Live-probing magistral-small-latest showed the model inlines its reasoning as
answer prose in a flat string — no ThinkChunk content, no markers, nothing a
client can separate. Mistral's chunked thinking shape is requested via
`prompt_mode: 'reasoning'`, which the provider previously dropped on the
floor: there was no way to even ask for it.

`custom.prompt_mode` now forwards to the SDK's `promptMode`, following the
BytePlus custom-params precedent. Opt-in rather than a default because the
API rejects the mode where the account/model lacks it ('Reasoning prompt
mode is not enabled for this model', code 3051) — verified end-to-end: the
3051 travels back through the stack, which also proves the parameter is
delivered. The moment Mistral enables the mode, the ThinkChunk content flows
into the existing splitter and comes out as `message.reasoning` and
`reasoning` stream chunks with no further changes.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-30 20:50:46 +02:00
404oopsandClaude Fable 5 4aa66c7c9b feat(ai): make puter.ai.normalize tri-state — unset means the release-date policy
The SDK-wide flag now mirrors the per-call option's tri-state. Left unset
(the default), the release-date policy applies: models released on or after
2026-09-01 are normalized to the OpenAI shape, older models keep their
vendor-native shape, and nothing rides the wire — the server's policy
resolution decides. Setting `true` force-normalizes every chat() call
regardless of release date; `false` disables normalization for every call;
either explicit value is sent on each call that doesn't set its own.
A per-call `normalize` overrides the flag in both directions, and assigning
`undefined` restores the policy.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-30 20:50:15 +02:00
Daniel Salazar afe46a17a0 fix: alaem cascade (#3668) 2026-08-29 03:41:23 -07:00
Juan Fernando Castro 81d700d146 fix: publishing a directory takes ownership, not write (PUT-1619) (#3654)
Creating a hosted subdomain gated `root_dir` on `write`, and hosting serves
everything under that directory with the ACL deliberately bypassed. So a
recipient of a `write` share could point a `*.puter.site` subdomain at the
owner's folder and make the subtree world-readable — continuously, covering
files the owner added later, with the row under the recipient's account where
nothing the owner can list would show it. `update` had the same gate for a
changed `root_dir`.

`#checkPublishAccess` now decides both: the actor's own tree still takes
`write`, anyone else's takes `manage` — "Can edit & share", the level that
delegates the decision.

Keyed on who owns the entry rather than asking for `manage` outright, which is
what the ticket proposed. `manage`'s is-owner implicator declines to answer for
app actors, so a flat `manage` would refuse every app publishing a directory
its user handed it, with no way for the app to obtain the grant. The write
check still runs first — it is what masks a directory the caller cannot see as
a 404 — and `manage` satisfies every lower mode, so the order costs a
manage-holder nothing.

The GUI's Publish As Website item reuses the own-it-or-`manage` answer it
already computes for sharing, so it is not offered where this would refuse.
Docs state the rule on `hosting.create()` and in `share()`'s level list.

Regression tests fail without the driver change: a write-share recipient is
refused on create and on repointing an existing subdomain, while `manage` and
the actor's own directory are accepted.
2026-08-29 01:46:14 -07:00
LEANDER ROSHAN L d5e9c14b0d fix: update sharing indicator #3649 (#3665) 2026-08-29 01:43:40 -07:00
Juan Fernando Castro c17613669c fix: open read-only shared files instead of forwarding write_url=undefined (#3667)
Opening a file you hold read-but-not-write access on (a read-only share)
failed silently. For such a file the backend correctly omits write_url from
the /open_item signature, but launchApp appended it to the app iframe URL
unconditionally. URLSearchParams coerces undefined to the string \"undefined\",
so the app received puter.item.write_url=\"undefined\" — truthy, so the editor
believed the file was writable, and an invalid URL, so it broke on open. Only
read-only shares hit this; files you can write carry a real write_url.

Extract the puter.item.* param building into append_signed_item_params and
guard the write_url append so it is only added when present. launchApp.js is
too coupled to UIWindow/jQuery/window globals to unit-test directly, so the
pure helper carries the logic and its own test, matching the helpers/ pattern.

Regression test pins both directions: a read-only signature omits write_url
entirely (no \"undefined\"), and a writable signature still forwards it.
2026-08-29 01:42:46 -07:00
Juan Fernando Castro cdfd38bd4b fix: stop an embedded fs: in a path from escalating to full FS access (#3666)
The fs-path-to-uid permission rewriter split permission strings on the raw
\`fs:\` substring instead of parsing on component boundaries. A path can
itself contain \`fs:\` — a home dir named \`…fs\`, or \`fs\` in the mode
position — and the raw split mistook that for the mode delimiter.

Crafting \`fs:/<victim>fs:junk:read\` made the split do two things at once:
it dropped the \`junk:read\` mode, and it consumed the trailing \`fs\`, turning
the harmless-looking nonexistent path shown in the consent dialog into the
victim's real home path. The rewriter then stored a bare \`fs:<home_uuid>\`,
which subsumes every mode via parent-permission matching — full
read+write+delete from a request the user saw as \`junk:read\`.

Parse with PermissionUtil.split() in both matches() and rewrite(), matching
every other permission parser, and locate the \`fs\` component by position
(after an optional \`manage\` prefix). The path component is now exactly what
sits between \`fs\` and the next unescaped colon, and all trailing components
are preserved as the mode — so an embedded \`fs:\` is data, never a delimiter.
The crafted input now addresses the nonexistent \`/<victim>fs\` and 404s.

Regression tests pin both halves: the exact PoC must 404, and an \`fs:\` in
the mode position must survive the rewrite instead of collapsing to a bare
permission. Both fail on the old rewriter and pass on the fix.
2026-08-29 01:41:40 -07:00
Juan Fernando Castro 693e86d763 Merge pull request #3657 from HeyPuter/juancastro/put-1616-shareblocks-is-missing-requireuseractor-any-authorized-app
🔧 PUT-1616: Close /share/blocks to app tokens
2026-08-28 16:46:04 -04:00
Daniel Salazar 6d10ef2821 fix: stop a URL parameter putting puter.js into app mode (#3660)
`env = 'app'` was decided by the presence of a `puter.app_instance_id` query
parameter and nothing else, so a crafted link put any page that loads the SDK
into app mode — and app mode is what makes the URL's `puter.api_origin`
authoritative for every credentialed call.

App mode now also requires the document to be framed. The GUI only ever
launches an app into an iframe, so this costs a real app nothing while a
top-level document carrying the parameters is treated as the third-party site
it is. It is not an attestation that the framing document is the GUI — a
cross-origin ancestor's identity is not readable — so the token paths carry the
rest:

- The `web` boot branch adopted a stored token without consulting the origin it
  was bound to, which is what completed the fixation: one link plants a token
  bound to an attacker's origin, and every later visit adopted it. It now
  applies the same binding rule the app branch does, and drops a token that
  fails it rather than leaving it to be re-read.
- `signIn()` had no env guard, and in app mode delivered a real token to
  whatever `puter.api_origin` the launching URL named. Apps get their token
  from the session that launched them, so it now rejects there with
  `not_available_in_app`. Nothing internal reaches it in app mode:
  `authenticateWithPuter` and both implicit-auth call sites already gate on
  `env === 'web'`.
- The cross-origin-isolated branch polled `${this.APIOrigin}/login/wait` and
  adopted whatever came back. Pinned to `defaultAPIOrigin`, the same way the
  popup and its message handler already pin `defaultGUIOrigin`.

Backward compatibility: no signature, response field or existing error code
changes. The only behaviour a caller can observe is the new `signIn()`
rejection, which replaces a call that could not have worked correctly.

Covers the SDK side of the parameter PUT-1395 and PUT-1427 closed on the GUI.
2026-08-28 11:16:10 -07:00
Daniel Salazar 9fab4742c9 fix: complete the malformed-input guards, including the two paths still live (#3663)
Review of the previous change found three of its claims unmet.

The image-generation crash it reported fixed is still reachable. The assert was
scattered across three helpers, and Gemini and OpenAI call `isHttpUrl` directly
on `input_images` without going through any of them — so two of seven providers
still 500 on a non-string. `isHttpUrl` now refuses a non-string itself, and the
shape is settled once in `ImageGenerationDriver.generate`, where the driver call
arrives, rather than per helper. That also covers `input_images` that isn't an
array, which produced a different crash per provider.

The sixth case in the ticket, previously unlocated, is
`Messages.js` reading `tool_call.function.name` with no guard — reachable with
`{"messages":[{"role":"assistant","tool_calls":[{"id":"x"}]}]}`. Guarded, along
with the same shape in `make_claude_tools`: a TypeError there carries no status,
so the retry loop reads it as a provider failure and marks the route unhealthy
for every caller.

`#hardExpiryFromExpiresIn` returning null for a bad type moved the failure past
the session INSERT, leaving an orphaned non-expiring row and still answering
500. Reverted; the controller guard is the fix, now covering fractions,
negatives and unparseable durations rather than only wrong types.

Also: the batch write handlers check that the body is an array but not what is
in it, so a null element 500s the same way; `#requireObjectBody` accepted an
array despite its name; `handleCreateAccessToken` destructured a body that may
be absent; and two AGPL notices had been rewrapped with a Markdown link.
2026-08-28 11:14:30 -07:00
Daniel Salazar f3a46a9be4 fix: stop sockets outliving the session that authenticated them (#3658)
A socket was checked once at handshake and never again. Nothing in the backend
disconnected one, so logout-everywhere, password reset, session revoke and
suspension all left every connection streaming legacy FS entries, upload paths
and notification bodies — up to 400 per account — on a credential that had
already been revoked.

Three gaps, three fixes:

- The handshake skipped the suspension and pending-verification checks every
  authenticated HTTP route gets. `decideSocketAuth` now applies both.
- `revokeCascade` reports which rows it revoked, AuthService announces that as
  `auth.sessions.revoked`, and SocketService drops the account's room. The
  whole room goes, not just the revoked session: narrowing it would need
  `fetchSockets`, which the adapter builds on `serverCount()` — and that calls
  node-redis's `send_command`, which ioredis does not implement. A connection
  whose session survived reconnects on its own and re-authenticates.
- A bulk suspension writes `user.suspended` without touching `sessions`, so no
  revoke fires. A five-minute sweep re-verifies each live socket's token and
  drops the ones that no longer authenticate. De-duplicated by token, since a
  browser's tabs share one.
2026-08-28 11:12:27 -07:00
Daniel Salazar e460e9034c fix: answer 400 instead of 500 on four malformed-input paths (#3662)
Each of these read a field off caller input that wasn't the shape the code
assumed, threw a TypeError, and was served as a 500 with a critical page.

- POST /fs/write and /startWrite: a request whose body never parsed left
  `req.body` undefined, and the first read of `fileMetadata` threw.
- POST /drivers/call, image generation: `input_image` / `input_images` entries
  are documented as strings but nothing checked, so a number or an object
  reached `.startsWith`. Type confusion on caller input, so reachable on
  demand rather than by accident.
- POST /auth/create-access-token: `expiresIn` went to the expiry parser
  unvalidated, where anything but a string or a number has no `.trim`.
- POST /login/wait: destructuring `session` out of an absent body threw.
  Same class as the POST /login body ticket; that one covers /login itself.
2026-08-28 10:22:51 -07:00
Daniel Salazar 5c8defb940 fix: bound failed WebDAV Basic-auth attempts per account and per address (#3659)
The only thing in front of the bcrypt compare on the DAV host was the 600/min
request ceiling, keyed on a fingerprint that rotates with client-controlled
headers. /login guards the same credential with a captcha and two much tighter
buckets; DAV had neither, which left password and TOTP guessing viable from a
host that answers any origin.

The request ceiling can't double as a credential ceiling — a working DAV client
resends its credentials on every request — so the new buckets count only
verifications that failed: 10 per account and 50 per address per 15 minutes,
sized like /login's. They're read before the compare, so an exhausted bucket
costs no bcrypt round, and successful requests never draw them down. `-token`
attempts are held by the address bucket alone; bucketing them per account would
let bad tokens lock out good ones.

Reading a bucket without spending from it is new, hence `peekRateLimit` and the
matching `peek` on all three backends.

Also documents the DAV limits, which were undisclosed.
2026-08-28 10:21:34 -07:00
Daniel Salazar 88b851a423 fix: derive the signup client IP from req.ip instead of x-forwarded-for (#3661)
Both signup paths read the x-forwarded-for header directly for the
puter.signup.validate event, the puter.signup.success event and the
signup_ip_forwarded column. Behind an appending proxy a client can prefix that
header with anything it likes, which gives every per-IP abuse signal a fresh
bucket per request.

req.ip is the value `trust proxy` resolves, so it is the one the server can
stand behind; server.ts already uses it for the ip.validate gate for exactly
this reason. All three now share one derivation, so a per-IP counter is written
and read under the same key. The raw chain is still recorded, but only as
audit_metadata.ip_fwd, where nothing keys on it.

Rows written before this hold whatever the proxy appended, so per-IP velocity
over the trailing window is undercounted until they age out.
2026-08-28 10:19:24 -07:00
Juan Castro 94d798ccce fix: close /share/blocks to app tokens
All three block routes set only `subdomain`, `requireVerified` and a rate
limit, and the controller's actor check accepts anything carrying a user, so an
app-under-user token passed. Unlike `share` and `revoke`, nothing else bounded
them: an app the user authorized once could read exactly whom they had blocked,
then clear the list — blanket switch and per-username — and reopen the channel.

A block list is a personal safety control, not something an app can otherwise
reach, so it falls outside the "an app can share what it can already reach"
carve-out the other share routes rely on. `requireUserActor` on all three, and
deliberately without `allowFullAccessToken`: this is security management, which
stays closed to every access token, personal ones included.

The only caller is the desktop's Blocked Senders window, which sends the user's
own session token.

Tests fail without the gate: an app token minted through
/auth/get-user-app-token is refused on all three routes while the user's own
session still succeeds, and a metadata check pins the gate on every block route
so a fourth cannot quietly ship without it.
2026-08-28 12:32:17 -04:00
404oopsandClaude Fable 5 07998c32ed test(ai): point the provider-level replay test at the real driver fallback test
The comment claimed the fallback loop cannot be driven from a test — false
since ChatCompletionDriver.test.ts gained a two-attempt fallback test that
drives the actual loop. Say where that test lives instead.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 04d12d9524 fix(ai): route Mistral streamed thinking to the reasoning channel, unconditionally
Third triple-check round. Ten findings were put through independent skeptics
first; six did not survive — pre-existing on main, inert, or resting on a false
premise — and are not acted on here.

Mistral streamed thinking now goes to `reasoning` on every path. The previous
commit gated the split, which made this the only place in the repo where
chain-of-thought reached the visible text channel, and made Mistral the only
provider whose streamed chunk *types* depend on a response-format flag. Every
other reasoning path routes thinking to `reasoning` unconditionally —
ClaudeProvider's thinking_delta, the DeepSeek/OpenRouter rename, and this
branch's own Responses summary-delta handler. Removing the gate restores that
uniformity and makes the documented promise that streaming is unaffected by
normalization true again; the two opposing tests collapse into one that runs the
same fixture with and without `normalize` and asserts identical event streams.

Docs stop claiming older models are unchanged. Four reasoning fields were made
consistent across all models, ungated, and one of them removes a field: on
non-streaming responses `message.reasoning_content` is now `message.reasoning`.
chat.md gains a table naming all four so a caller reading `reasoning_content`
learns why it disappeared, instead of reading that nothing changed for them.

Adds the driver-level fallback test. Writing it surfaced that the invariant it
was meant to assert is false and always was: the driver rewrites string
`content` into text blocks in place on the caller's own messages
(`normalize_single_message`, pre-existing) before any provider runs. The test
now asserts what is true and load-bearing — both attempts receive the same array
reference, and the reasoning artifacts survive attempt 1 so attempt 2 can still
replay them.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oops 875aac5db0 Revert "docs: record AI-normalization review rules in CLAUDE.md"
This reverts commit f220276724fdc1a16ae56f9c95e41773fe4231ed.
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 216e8d68ed fix(ai): scope the normalized flag, gate the Mistral stream split, correct four doc claims
Acts on a second triple-check audit.

`normalized` means one thing. The driver stamped it on both the OpenAI-shape
path and the legacy `response.normalize` path, which converts toward Anthropic
blocks — so a caller could get `normalized: true` alongside array content, and
the flag told them nothing they could branch on. The legacy branch no longer
sets it. That branch is reachable only by a direct driver call with
`response.normalize` and no `normalize`; the four wire routes pin
`normalize: false`, which skips it.

Mistral streaming, split by concern. Flattening a reasoning model's chunked
`delta.content` to a string is a correctness floor and stays ungated — the
shared handler passes the value straight to `addText`, so an array reaches the
caller as stringified objects. Splitting the thinking text out into a
`reasoning` delta is the dialect change and now sits behind the policy gate
like the non-streaming remap. On the native path the thinking text is kept
inline rather than dropped.

Mistral `finishReason` is deleted only once its value carried over. The delete
ran unconditionally, so a non-string `finishReason` with no `finish_reason`
left the choice with no finish reason at all.

Four doc claims corrected against the code paths they cover: `content` is
string-or-null on normalized responses (tool-only turns carry no text, and the
`// always a string` example comment was wrong); `reasoning_details` is not
scoped to normalized responses, since Responses models emit it either way; and
the release-date rule depends on the serving provider's own dates — OpenRouter
derives them from its live API, so models newly listed there from 2026-09-01
normalize by default.

Adds the test the copy-on-write fix was actually for: one messages array sent
through two sequential calls, asserting the caller's array is untouched and
both attempts carried the thinking signature. That is the fallback hazard; the
harness wires one provider per model, so the fallback loop itself cannot be
driven from a provider test.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 c1b420e480 fix(ai): gate the Mistral remap, unify the reasoning join, stop mutating caller messages
Acts on a triple-check audit of this branch.

Gate the Mistral dialect remap. The camelCase→snake_case rewrite and the
chunked-content flattening were firing for every Mistral call regardless of
`normalize` or the cutoff, deleting `finishReason` and `message.toolCalls` out
from under any caller reading them. Both now sit behind the policy resolution
the driver already used, extracted as `shouldPresentAsOpenAI` so the provider
and the driver cannot drift. The streaming chunk-array split stays ungated:
handing an array to `addText` is a plain bug, and streamed chunks are
provider-uniform by design. The conformance matrix now passes `normalize: true`,
which is the contract it was always testing.

Unify the reasoning join. Three code paths produced two separators while one
doc sentence described them all: the coercer joined thinking segments with '',
the Responses handler and Mistral with '\n\n'. The coercer now matches, and
chatresponse.md's claim is true for every path it covers. Text blocks still
join with '' — Anthropic splits prose mid-sentence across them.

finish_reason is an open set. chat.md's normalize bullet and the SDK
ChatMessage typedef still declared a closed four-value set, contradicting the
documented pass-through of unmapped vendor reasons and the coercer that
implements it.

Stop mutating caller messages. Both reasoning-replay input paths deleted
output-only fields from the caller's own message objects, which the driver
reuses across fallback attempts. Both strip a copy now; tests pass a frozen
message through each.

Drop three dead things the type cleanup left: the no-op ChatProvider
checkModeration stub (no subclasses, no callers — its removal restores a
pre-existing baselined TS2420), the redundant second normalizeReasoningContent
call in BytePlus and ZAI, and the coercer's bare-string branch that no provider
reaches. A bare string now passes through by reference instead of being
coerced.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 fbb9038651 revert: restore the 31 typecheck-baseline entries this branch removed
Commit 6b4f5b01f dropped 31 entries (49 error instances) from
tools/typecheck-baseline.json as a side effect of fixing the AI-code type
errors they suppressed. The file is a ledger the maintainer owns, and shrinking
it inside a feature PR ships a CI-gate change nobody asked for.

Restoring is free: tools/typecheck.mjs fails only on *regressions*
(count > baseline), so stale entries are non-fatal — the gate prints "49
baselined error(s) fixed. Run npm run typecheck:update to lock that in." and
exits 0. Nothing else in the repo reads the file, vitest included, and
tsconfig.build.json's noCheck:true means emit is unaffected. The errors stay
fixed in the source either way; only the ledger's own bookkeeping is deferred
to whenever the maintainer chooses to regenerate it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 74640d4e83 fix(ai): flatten Mistral reasoning models' chunked content
`magistral-*` returns `message.content` as a ContentChunk[] rather than a
string, with the thinking text nested one level deeper inside `thinking`
chunks. The camelCase remap did not touch it, so a non-streamed magistral
response reached the caller as an array with no `reasoning` — the one case
left where a provider did not produce the equalized shape this branch
promises. Streaming had the matching bug: the chunk array was handed to
addText, which would have stringified it into the text stream.

Both paths now split chunked content into a string `content` plus a
`reasoning` string, joining multiple thinking chunks with a blank line as the
Responses handler and the Anthropic coercer do. The streaming fix rides the
existing Mistral-only `chunk_but_like_actually` hook, so no new deviation is
introduced.

The conformance matrix had no Mistral reasoning fixture, which is why it
missed this; it now has one carrying chunked content, verified to fail
without the flattening.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 3b8de6a90b docs: record AI-normalization review rules in CLAUDE.md
Rules from the post-normalization review: don't silently change what existing
models return, verify doc claims against every code path they cover, record
self-disclosed defects in the PR draft, finish with a fresh build + suites, and
check git stash before concluding an edit was lost. Plus the standing
typecheck-baseline no-go.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 a6830f341a fix(ai): round-trip reasoning artifacts and pass unmapped stop reasons through
Closes the reasoning gaps left open by the normalization work.

Reasoning replay. The coercer dropped Anthropic thinking-block signatures and
the Responses handler dropped reasoning item ids/encrypted_content, so a
normalized reasoning turn could not be replayed — Anthropic rejects an
extended-thinking tool-use continuation whose thinking blocks lost their
signature. Both now ride `message.reasoning_details` verbatim, and both input
paths accept them back: ClaudeProvider splices the blocks ahead of the content
(Anthropic requires them to lead), and the Responses input processor expands
them into standalone `reasoning` items. Output-only fields a replayed message
carries (`reasoning`, `refusal`, `normalized`) are stripped on both paths,
since neither upstream accepts them. The docs caveat recommending
`normalize: false` for agentic Claude loops is gone; it is no longer true.

Unmapped stop reasons. chatresponse.md promised a vendor `finish_reason` with
no OpenAI analog "passes through unchanged" — true for the Mistral remap, false
for the Anthropic coercer, which discarded it. Anthropic's `pause_turn` means
"continue this turn", so flattening it to `stop` destroyed the signal. The
coercer now passes unmapped values through verbatim, matching both the doc and
the Mistral path, and the docs gain the full Anthropic stop-reason table.

Reasoning summaries. Multi-part summaries joined with '' instead of a blank
line, and the streaming Responses path emitted no reasoning at all;
`response.reasoning_summary_text.delta` now feeds the same `reasoning` stream
channel the chat-completions handler uses.

Types. `text?: string & { verbosity?: ... }` was an uninhabitable intersection
(providers read `text?.verbosity` as an object), and the verbosity enum was
`'concise' | 'detailed'` where OpenAI accepts `'low' | 'medium' | 'high'`.
Adds `reasoning`, `reasoning_details`, and `refusal` to the SDK ChatMessage
typedef.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Fable 5 548d26f884 test(ai): cover Hoonify in the provider-consistency matrix
Adds the upstream Hoonify provider (feat #3499, merged after this
branch was cut) to the cross-provider output conformance suite. It
speaks the OpenAI chat-completions dialect and conforms out of the box.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-28 01:08:48 +02:00
404oops 9b67a2b1dd Add AI response normalization controls
Introduces a new `normalize` option for chat completions, plus release-date based default normalization (post-2026-09-01) to coerce provider-native outputs into a consistent OpenAI-style shape. Adds shared normalization utilities, extensive driver/provider consistency tests, and controller safeguards that pin provider-native output where route-specific translators are used. Also wires the option through puter.js (`chat` options and `puter.ai.normalize` default), updates AI/chat response docs and examples, and resolves related TypeScript typing issues reflected in the typecheck baseline.
2026-08-28 01:08:47 +02:00
Juan Fernando Castro f952b5006b Merge pull request #3652 from HeyPuter/juancastro/put-1594-permission-requests-should-auto-return-true-and-resolve
🔧 PUT-1594: Settle a permission request from what is already granted
2026-08-27 19:02:15 -04:00
Nariman Jelveh 56b4ef7f2b Leave worker notifications out of the dashboard entirely 2026-08-27 15:59:25 -07:00
Nariman Jelveh 1d3963539f Keep worker notifications out of the dashboard toasts 2026-08-27 15:47:57 -07:00
Juan Castro c4be7fabac fix: settle a permission request from what is already granted
`puter.perms.request()` already pooled a permission read and prompted only
for what was missing. The raw `puter.ui.requestPermission()` did not, so
every caller still on it re-asked the user on each launch — including
`perms.requestAppData()`, whose own docs promise the opposite, and the
driver-denial retry.

- puter.js: `ui.requestPermission()` reads what is held before prompting and
  resolves true when the whole request is covered. Only in env=app and
  env=web, the environments that raise a prompt; elsewhere the method still
  answers false without asking anyone. A check that cannot be made — no
  token, an unreadable request shape, a failed read, or one that outlasts its
  timeout — falls through to the prompt rather than standing in for an
  answer. Public signature unchanged.

- GUI: the request-permission popup asks the same question as the app, using
  the user-app token its own exchange already mints, and skips the dialog
  when the access is held. This is the one case the SDK cannot settle for
  itself: a signed-out site holds no token to check with. An origin the
  browser does not vouch for never reaches the check, since the exchange
  fails first.

Both checks are time-boxed, because each one stands in front of something
that is waiting: the popup's gates the dialog, so a stalled read would leave
the prompt unshown and the opener pending, and the SDK's spends the browser's
transient activation, which a slow read would cost the popup.

Note that driver, service and feature scopes are implicitly granted to every
app (backend/data/hardcoded-permissions.js), so requests for those now
settle silently — the dialog was asking about access the app already had.
Consent scopes (email, fs, apps, subdomains, app-data, app-root-dir) are
unaffected and still prompt until granted.

Fixes a bug this method already had on the way past: `pollDecision` read an
undeclared `permission`, so every attempt threw a ReferenceError into its
network-failure catch and the COOP-severed-opener recovery burned its full
five-minute timeout before answering false. It polls `requested` now, and
requires the whole list.

Tests: the e2e suite drove its dialogs with an implicitly-held driver
permission, so the fixture now asks for a driver nothing implies, fresh per
page load, which also removes the cross-test grant carry-over the old
revokes worked around. The reconciliation tests ask for the held scope plus
an unheld one, since a fully-held request no longer reaches a dialog. Adds a
backend contract test for check-permissions under an app-under-user actor,
which is what the two new client paths rest on.
2026-08-27 16:35:06 -04:00
Nariman Jelveh 167e281cc8 Keep read notifications in the dashboard panel
The panel listed only what the user hadn't dismissed, so acting on a
notification made it vanish. It now shows the 30 most recent regardless
of state, with read/unread as a visual flag: unread rows carry a dot and
a check to mark them read; reading one (check, click-through, toast
close, mark-all, or an ack from another tab) leaves it listed but
quieter. The badge, tab title, and header count follow the unread count.
2026-08-27 13:31:39 -07:00
Nariman Jelveh c6ecc0c786 Soften dashboard notifications empty-state title 2026-08-27 13:14:05 -07:00
Nariman Jelveh f38dec07ea Remove empty-state hint from dashboard notifications 2026-08-27 12:59:01 -07:00
Nariman Jelveh 7332993ff3 Show share notifications in the dashboard (#3653)
* Show share notifications in the dashboard

The dashboard opens its own socket but never listened for notif.* events,
so a file shared with a user on the dashboard was announced to nobody.

Add a notification center to the dashboard: a bell in the sidebar with the
unread count (a dot when collapsed, and on the mobile hamburger), a panel
anchored to it — a bottom sheet on phones — listing what the server still
holds as unacknowledged, and toasts for what arrives live. Clicking a share
lands on Files › Shared with the item selected; dismissing, and "Mark all as
read", acknowledge on the server so other tabs clear too. A toast timing out
is not a dismissal: the entry stays unread in the panel.

Arrivals fold in by uid (a regrouped share rewrites its row and toast in
place), the panel refreshes on open, reconnect and tab focus so shares the
backend folds in silently past the sender's budget still surface, and a
burst on connect is capped at three toasts plus a summary.

Backend: notif.unreads carries created_at so delivered-on-connect items can
be dated, and listings break same-second ties by id so order is stable.

The desktop's mark-ack calls move to a shared helper; UINotification gains an
optional auto-hide timeout that pauses on hover/focus.

* fix: look up notification glyphs and icons by own key only

A notification whose `source` or `icon` was a prototype key such as
`constructor` made the glyph lookup return a function, and the toast
icon builder then threw inside the socket handler — which dropped the
whole burst of toasts it was part of. The list rendered the same entry
as "[object Object]".

* fix: show dashboard toasts above open app windows

App windows opened from the dashboard are stay-on-top and stack in the
99999999+ band, so the toast container at 10000000 sat underneath them:
any notification arriving while an app was open — the usual state of a
dashboard session — was drawn behind the app and never seen.

* fix: reveal the dashboard when a toast is clicked over an open app

Clicking a share toast switches the dashboard to Shared, and the "N new
notifications" toast opens the panel — both inside the dashboard window.
With an app maximized over it, all of that happened out of sight and the
click appeared to do nothing. Minimize the covering app windows first,
the way the minimize controls do, landing the URL on the dashboard's own
route, and act once the history pop has settled so the tab switch is not
traversed over.

* fix: keep keyboard focus in the notifications panel across re-renders

Every change to the list rebuilds its rows, which dropped focus to the
document body: dismissing an entry with Enter threw the keyboard out of
the dialog (the row it had just been moved to was replaced), and so did
any arrival while an entry was focused. Put focus back on the same entry
after a render, or on the one that took a dismissed entry's place, and
let Shift+Tab from the panel itself wrap to the last control instead of
leaving the dialog.

* fix: keep the notifications panel inside short viewports

The anchored panel is bottom-aligned with the bell and grows upward, but
its height was capped only by the viewport (560px or 100vh - 24px), not
by the room above the bell — on a short window the header ran off the
top of the page. Cap it to that room, so the list scrolls instead.

* fix: make notifications with nowhere to go inert in the panel

An entry with no target (a worker deploy result) showed a default cursor
yet, when clicked, was dismissed and took the panel with it — nothing
opened, the entry was gone, and the list had closed. Render such entries
as text with the ✕ as their only action; only share entries remain
buttons. The ✕ now names its notification for screen readers, since on
an inert entry it is the only focusable control.

* fix: stop a listing in flight from resurrecting dismissed notifications

The list is refreshed on open, on reconnect, and when the tab comes back
into view; a dismissal landing while that listing is in flight was undone
when it resolved, since the server's snapshot predates the ack. When the
listing also resolved after the server's own ack event, nothing removed
the entry again and it stayed until the next refresh. Leave out of the
reconciliation whatever was acknowledged — here or in another tab —
since the listing was requested.
2026-08-27 12:49:20 -07:00
Daniel Salazar 909949c68d fix: failed email alarms (#3651) 2026-08-26 23:22:14 -07:00
Juan Fernando Castro 5a6a1020a1 Merge pull request #3644 from HeyPuter/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat
🔧 PUT-1585: Share flag in readdir shared users in stat
2026-08-26 18:51:30 -04:00
Nariman Jelveh 7472002cf3 feat: add Hoonify AI chat provider (#3499) 2026-08-26 15:39:29 -07:00
Juan Fernando Castro 7bd6a2d89b Merge pull request #3646 from HeyPuter/juancastro/put-1590-sharing-shared-files-are-not-different-than-regular-files
🔧 PUT-1590: Mark shared items in the file listings
2026-08-26 17:55:19 -04:00
Juan Castro 3f7dac63b9 Merge remote-tracking branch 'origin/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat' into juancastro/put-1590-sharing-shared-files-are-not-different-than-regular-files
# Conflicts:
#	src/gui/src/UI/Dashboard/UIShareModal.js
#	src/gui/src/UI/UIWindowShare.js
2026-08-26 17:06:25 -04:00
Juan Fernando Castro 3d28b2bb99 Merge pull request #3647 from HeyPuter/juancastro/put-1586-1599-share-dialog-correctness
🐛 PUT-1586 + PUT-1599: Share dialog says what it can grant, and what it did
2026-08-26 16:58:36 -04:00
Juan Castro 63688f81b4 Merge branch 'main' into juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat 2026-08-26 16:55:51 -04:00
Juan Fernando Castro 14ac6542e3 Merge pull request #3645 from HeyPuter/juancastro/put-1589-shared-file-fs-events-seem-to-be-lost
🐛 PUT-1589: Shared file fs events seem to be lost
2026-08-26 16:54:09 -04:00
Juan Castro 5739f4e59f fix: stop pushing file metadata to revoked recipients (PUT-1611)
The realtime fan-out resolved its audience straight from the `share`
index, which has no live-grant check. `/auth/revoke-user-user` deletes
the permission and leaves the index row, so a revoked recipient's socket
kept receiving name, size, masked path and mtime for every write and
move under the folder, with no expiry.

The service already solves this elsewhere — `#reachingHolders` returns
exactly the holder/entry pairs whose grant still stands, and
`listSharedWithMe` was moved onto it for the same reason. The realtime
path never got the same treatment; it does now.

Free on the unshared path: with no share rows reaching the entry there
are no holders to check, so the write path every user takes is unchanged.
Pinned by a test that counts permission reads.
2026-08-26 16:35:38 -04:00
Juan Castro cae51ce67a fix: trashing a top-level share is a removal, not a move
A grant on the entry itself is keyed on uuid, so it follows the entry
into the owner's Trash. Both ends of the move then resolved, and the
recipient was told the shared item had moved — to the GUID name Trash
gave it. Their own copy got renamed to a GUID and stayed on screen.

`shared-with-me` has always omitted trashed entries, so the listing and
the event disagreed; only the event was wrong. Trashing now reports
item.removed at the path the recipient knew, which is also what the
desktop's data-path selector needs to find the row. Restoring out of
Trash reports item.added.

A move that leaves the recipient's masked address unchanged now stays
quiet — a share masks its own root, so the owner shuffling it around
their tree is invisible to the recipient and the event carried nothing.
2026-08-26 16:25:02 -04:00
Juan Castro 8e622189ca fix: tell every recipient when a shared file disappears
Both ways a file can vanish were silent for anyone holding the folder
above it, so a third party's window kept showing a file that was gone
and 404'd on click.

A delete built its audience from the permission rows it removed, and a
file inside a shared folder has no grant of its own — only the folder
does — so the audience was empty. A move resolved its audience from the
entry's new path, and the GUI's Delete is a move to the owner's Trash,
where no recipient has a share.

Resolve the audience from where the entry was rather than from the
grants that went with it:

- Deletes also fan out to holders reaching the entry through an
  ancestor, coalesced by parent so a subtree stays a couple of queries.
  A holder covered by both passes is told once.
- Moves resolve both ends. Reaching both is item.moved, only the
  destination item.added, only the origin item.removed.

Recipients are named by the path they knew, masked through their own
share rather than the owner's tree.
2026-08-26 14:54:06 -04:00
Daniel Salazar c1c830fc32 docs: add puter site config docs (#3648) 2026-08-26 11:11:25 -07:00
Juan Castro cf00ebb9d4 Merge branch 'main' into juancastro/put-1589-shared-file-fs-events-seem-to-be-lost 2026-08-26 13:33:24 -04:00
Juan Fernando Castro b05ee63cb2 Merge pull request #3641 from HeyPuter/juancastro/put-1568-rename-srcguisrchelpers-files-to-camelcase
🧹 PUT-1568: Rename GUI helper files to camelCase
2026-08-26 13:06:58 -04:00
jelveh 40667bc811 Open Document Picture-in-Picture windows on behalf of apps
Browsers only allow documentPictureInPicture.requestWindow() from a
top-level document, and an app lives in an iframe, so an app calling it
gets NotAllowedError ("only allowed from a top-level browsing context").
The `document-picture-in-picture` token in the iframe's `allow` list does
nothing — it is not a policy feature the browser knows. Video PiP
(video.requestPictureInPicture) already works inside apps.

The GUI is the top-level document, so a new PictureInPictureService opens
the window for the app and fills it with an iframe of a page the app names,
which must come from the app's own origin (checked against the message's
origin, now carried on the IPC caller context). One window per app
instance; it closes with the app's window, and the app hears about a close
it didn't ask for. The window's opener is the GUI, so the page inside it
can reach its app's frame through parent.opener.frames and share objects
directly — a MediaStream included, which postMessage cannot carry (tracks
are not transferable between windows in Chromium).

puter.js gains puter.ui.requestPictureInPicture({ url, width, height,
onClose }) and puter.ui.exitPictureInPicture(), with docs.
2026-08-26 09:10:02 -07:00
Juan Castro 4b3d1112d1 Say when a share changed nothing
Sharing a file with someone who already had it answered "Shared with
X", the same as a first share, so the dialog claimed to have done
something it had not.

The service already knew — it computes isNew to decide whether to
notify the recipient — but the flag stopped at the controller. It now
travels on share results only; a listing describes standing access and
says nothing about it.

Which of the three things happened is settled client-side, since the
mode each recipient holds is already on screen: created access, raised
or lowered it, or changed nothing. That keeps the previous mode off the
wire, and an older backend that omits isNew still reads as a share,
which is what these dialogs said before.
2026-08-26 11:57:05 -04:00
Juan Castro 70edef561f Stop offering an access level a delegate cannot grant
A recipient given "can edit & share" could not pass that level on: the
dialog offered it, the server refused it, and the refusal was a bare
403 Forbidden that reads as a bug. Handing out manage needs authority
over manage, which only the owner has — the refusal is right, the dead
end and the silence were not.

The dropdown now withholds it from anyone who does not own the item; a
row already set to it keeps it, so opening the dialog cannot downgrade
the owner's own grant, and a mixed selection follows its strictest item.

The server says why, and only to someone who can already share the item
— a stranger still gets the ACL's own safe error, which does not admit
the node exists.

Verified against a running server: a delegate grants read and write as
before, and manage now answers cannot_delegate_manage with a sentence
naming the owner as the one who can.
2026-08-26 11:48:54 -04:00
jelveh e430ade7bd Reach Shared and Trash from the dashboard Files tab on phones
Below 480px the Files tab hides its directories sidebar, which was the only
entry point to Shared (a query, not a directory) and Trash (filtered out of
the Home listing). Home now carries a row for each; CSS shows them only at
the widths where the sidebar is hidden, using the complementary breakpoint so
the two never both show or both disappear.

The rows borrow the item markup for layout but aren't `.item`, so sorted
insert, selection restore, share-link selection and socket updates ignore
them. The footer count and keyboard select-all exclude them explicitly, and
the placeholder-removal sweeps leave them in place. They navigate on tap and
offer the same menu as their sidebar entry via the ⋯ button or long-press;
the Trash icon tracks empty/full alongside the sidebar's.

Also guard the `window.user.directories` lookup in renderDirectory: it is
undefined for some sessions, and since `puter://shared` doesn't look like a
path it always reached that branch, threw outside the try, and left the
spinner up with navigation stuck — from the desktop sidebar as well.
2026-08-26 08:18:13 -07:00
Juan Castro 519b132355 Inset the shared marker onto the icon glyph
It sat on the icon box's corner, which is 5px outside the artwork on
every side, so the dot read as clipped — half of it hanging over empty
padding with the icon's own drop-shadow falling across it. Nudged in on
both axes: 7px on the desktop, 2px on the dashboard rows.
2026-08-26 11:16:49 -04:00
Juan Castro b08dc70654 Move the shared marker onto the icon's corner and shrink it
Review feedback: too big, and in the wrong place. It sat in the badge
cluster, which the dashboard pins to the row's top-left corner rather
than to the icon — and at 12px it dominated a 24px row icon.

Now a dot on the icon itself, lower-right: 9px on the desktop's 45px
icons, 7px on the dashboard's 24px rows. Anchoring to .item-icon rather
than to the badge cluster is what keeps it on the corner at both sizes.

Dropping the people glyph with it — unreadable at either size, and
colour was the signal the ticket asked for.
2026-08-26 11:07:48 -04:00
Juan Castro a1d027e4a1 Mark shared items in the file listings
A shared file looked exactly like a regular one. The data to tell them
apart arrived with the readdir/stat share flag; nothing rendered it.

Adds the badge to both listings — UIItem (desktop, explorer windows,
file dialogs) and the dashboard's Files rows — fed from is_shared, and
keeps it in step with the share dialogs: both funnel every grant, mode
change and revoke through one render, so the badge follows without
waiting for a re-listing.

Inherited access is deliberately not badged. It is a state of the folder
that was shared, so marking every file inside would repeat one fact on
hundreds of items; the backend flag is direct-only for the same reason.

The icon (owner-shared.svg) and the strings (item_shared_by_you, in 40
locales) were already in the tree, unused — only the wiring was missing.
The blue ring is doing the work: list view shrinks badges to 8px, where
a glyph is illegible and the white circle the sibling badges use
disappears into the row.
2026-08-26 10:56:10 -04:00
jelveh defa37c106 Fix dashboard files tab mobile scrolling
Replace the Files tab's viewport-height sizing with flex-based layout sizing through the dashboard content chain. This keeps the internal file list within the visible content area on mobile browsers, preventing the bottom rows from being clipped when browser toolbars are shown.
2026-08-26 07:42:44 -07:00
Juan Castro 1514ab3cb5 Ignore fs events replayed by replication
Raised in review: could an event from another node re-trigger the
fan-out? Not today — broadcast carries outer.* and pubsub.* only, so
fs.* never crosses a node boundary, and the emitted outer.gui.* is
consumed on the peer by SocketService while ShareService listens to
fs.* alone, so nothing re-enters.

That safety is a property of what broadcast happens to replicate, which
is not this service's to rely on. The handlers now skip anything tagged
from_outside: the node that did the write has already told the audience,
and a second fan-out would only duplicate it.
2026-08-26 09:22:49 -04:00
roar_zx ca25915a38 docs: link Dutch and Odia README translations (#3635) 2026-08-26 00:59:16 -07:00
Juan Castro ee2b5adcf9 Tell share recipients about creates and renames
A recipient's client keeps its cache fresh from fs events pushed over
their socket, and ShareService fans those out to holders — but only for
write, move and delete. A new entry emits fs.create.<flavor>, not
fs.write.file, and an in-place rename emits fs.rename; neither had a
listener, so a recipient watching a shared folder never learned that a
file appeared in it or was renamed. Part of why: those keys and
outer.gui.item.renamed were missing from the typed event map, so a
listener for them did not compile.

Delivering the event is only half of it. Paths were masked against the
entry itself, so item.added named a parent no cached listing was keyed
on, and the payload carried no dirpath, which is how the desktop finds
the container to render into — the event would have arrived and changed
nothing. Paths are now masked at the share the holder reached the entry
through, which is the address their own reads returned, and from_path on
a move and old_path on a rename travel the same way (dropped when the
move started outside the share, self-masked when the share is on the
entry itself, where the root already carries the new path).

Creates fire per entry, so an upload would have cost one share lookup
per file; they are coalesced by parent folder the way subtree deletes
already are. Measured on a 25-file burst into one folder: 25 lookups
before, 1 after. A holder with a share on both a folder and something
inside it is told once, by the nearer of the two.
2026-08-25 19:38:38 -04:00
Juan Castro 2ef5d00d9b Type is_shared and say who the flag counts
stat() and readdir() return FSItemRead, so the is_shared the docs lead
with typechecks for TypeScript consumers rather than erroring on FSItem.

The docs said "you have shared", but the query has no issuer predicate:
a manage delegate's re-share sets the owner's flag too, which is the
useful answer and matches getShares().
2026-08-25 18:15:15 -04:00
Juan Castro bac3dde945 Forget cached entries when sharing changes
Nothing invalidated the SDK entry cache on share or unshare — the socket
handlers only fire on item mutations — so is_shared, which now rides in
the cached entry, stayed stale for every consistency: 'eventual' read.
That includes the GUI's own listing refresh, which defaults to it, so a
badge would not have appeared until an unrelated write flushed the cache.
2026-08-25 18:15:05 -04:00
Juan Castro 1600dc38a5 Retire the share index row with the grant it records
revoke-user-user withdraws a grant without touching the share index, so
the row outlived the access — invisible until now, because listSharesOf
filters against live grants, but the new flag reads the index and would
report a file as shared to nobody, permanently.

Drop the row where the grant goes. The alternative, filtering liveness on
the read side, is the per-entry work the flag exists to avoid.
2026-08-25 18:14:52 -04:00
Juan Castro 0c8dd8b6e4 Document the share flag and returnShares 2026-08-25 17:45:15 -04:00
Juan Castro 3217533594 Give the puter.js API suites rate-limit headroom
Around 500 tests run through one seeded free-tier account, so the
per-tier windows were throttling the suite rather than anything it
tests — adding a single readdir was enough to trip fs:readdir-burst.
Resolve the seeded users to the unlimited policy, alongside the
subscription gate the harness already turns off for the same reason.
Test config only; no published limit changes.
2026-08-25 17:44:56 -04:00
Juan Castro 0591f13cc2 Expose is_shared and returnShares in puter.js
Carry the flag into the v1 entry shape and add returnShares to stat(),
whose shares are mapped into the same Share objects getShares() returns.
returnShares joins the dedupe key and bypasses the entry cache in both
directions — a result carrying share data is never written to it, so a
later plain stat cannot serve other people's names from cache.
2026-08-25 17:44:50 -04:00
Juan Castro ee2f14576b Report sharing in stat and readdir
isShared on /fs/stat and /fs/readdir, is_shared on their legacy
counterparts, and return_shares now fills the shares array the legacy
stat has been stubbing with []. Null for entries the caller does not own,
so a share recipient is never told who else can reach the owner's files.
The share-to-wire mapper moves out of ShareController so both controllers
publish one shape.
2026-08-25 17:44:41 -04:00
Juan Castro 5115fdd48c Answer which entries are shared in one indexed query
Directory listings need a shared/not-shared flag per entry. Ask the share
index once per batch, chunked, rather than a query per entry, and answer
only for entries the caller owns so listing a foreign directory never
queries at all. Direct shares only — inherited access stays with
listSharesOf, which already reports it.
2026-08-25 17:44:25 -04:00
jelvehandClaude Fable 5 e006b19773 puter.peer: room names, relayed guest grants, servers that come back
A peer server could only be reached by the invite code the signaller
minted for it, which is unknowable ahead of time and dies with the
server's socket. Any app that wanted a link to share in advance or reuse
had to run a registry of its own that maps a stable code to whatever
invite is live — and keep it live, since the signaller socket can drop
without the library saying a word (`onclose` was an empty function),
leaving a host serving an invite nobody can dial.

`serve({ name })` serves under a room name of the caller's choosing;
`connect(name)` dials it. A name is held by whoever serves it and free
once they stop; serving a held name from another identity rejects with
`name_in_use`, from the same identity it takes over. A connection to a
room nobody serves fails with `no_host`, so a lobby has a definite answer
to poll on instead of guessing between "not yet" and "gone".

`serve({ guestGrant })` and `server.setGuestGrant()` leave a guest grant
with the signaller, which hands it to every anonymous guest in the connect
reply; the connection redeems it before making its offer, so a guest with
no session gets relays without the app publishing the grant anywhere.

A server whose signaller socket drops now re-registers on its own with
backoff — under the same name, or a fresh code announced by the new
`reconnect` event — and pings the socket to keep idle proxies from cutting
it. It fires `close` (`replaced`, `name_in_use`) when it has to stand
down for good. Existing connections are peer-to-peer and never affected.

Signaller refusals reach the connection's `error` event as an Error with
a `code` next to the message. Keepalive replies and stray frames no longer
throw in the message handlers.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-25 13:24:45 -07:00
Juan Castro 07e73efa40 Merge remote-tracking branch 'origin/main' into juancastro/put-1568-rename-srcguisrchelpers-files-to-camelcase 2026-08-25 15:10:29 -04:00
404oops 2a45a10854 Merge pull request #3630 from HeyPuter/FK/dedupe
refactor: dedupe model catalog aliases and stop mutating shared catalogs
2026-08-25 18:07:27 +02:00
Nariman Jelveh e052aec7fb Carry a share link through the OIDC round trip (#3642)
A share email lands on `/?shared=…`, and its recipient is signed out or
in a temporary session more often than not. Password login reloads the
same URL, so the link came back with it — but an OIDC login leaves the
origin and returns to a URL the backend builds from a whitelist of bare
paths. `return_to` was a pathname only, and `/` wasn't on the list, so
the recipient came back signed in on the Dashboard's Apps tab with
nothing to say what had just been shared with them.

`shared` now travels with the path. The GUI sends it along with the
return path; the backend accepts it as the one parameter a `return_to`
may carry, and only values shaped like the masked share path the mail was
built from — everything else rejects the whole value, the path is still
matched as a raw string so a protocol-relative one can't smuggle an
origin through, and the query is rebuilt rather than reflected. The items
ride onto the error page too, so a failed attempt retries on the link
instead of losing it.

The root joins the whitelist for this, but only when it names something:
on its own it is where the flow already lands.
2026-08-25 08:42:47 -07:00
Daniel Salazar cf4d743854 fix: testing for peer (#3643) 2026-08-25 08:31:45 -07:00
Daniel Salazar 3c866c645b feat: guest turn for peer (#3640) 2026-08-25 07:51:02 -07:00
Juan Castro 4a5d689af7 refactor: rename GUI helper files to camelCase
src/gui/src/helpers was mostly snake_case, disagreeing with itself and
with the camelCase file convention in AGENTS.md. Rename the 45 offenders
with git mv and repoint every import specifier at the new paths.

Files only — exported symbols are unchanged. applyItemAddedToContainers.js
still exports apply_item_added_to_containers and no call site is edited
beyond its import line; those identifiers cross module boundaries and some
hang off window, so renaming them is a separate, riskier change.
2026-08-25 10:49:59 -04:00
jelveh 6f1548e26a Move Share above Download in file context menus 2026-08-25 07:28:53 -07:00
Daniel Salazar 764ce2f8e2 feat: rep score middleware gateway (#3639) 2026-08-25 07:11:41 -07:00
Daniel Salazar 717c098503 fix: metering buffer improvements for manual editing (#3633) 2026-08-24 18:36:43 -07:00
Nariman Jelveh 2c9a5c4f7e Land share-email links in the Dashboard's Shared view (#3634)
* Land share-email links in the Dashboard's Shared view

A share email's links opened the item on the desktop, and "Open Puter"
went to the bare origin — the recipient arrived on Home with nothing to
say what had just been shared. Every link now lands in the Dashboard's
Files tab, on Shared, with the shared items selected.

Each named file still links to itself. "Open Puter" carries every item
in the mail (?shared=a&shared=b), so the whole batch lands highlighted;
the digest records now keep each item's masked path for that. The
button's href is rendered raw like the item links, so `=` and `&` read
the same in the html and text parts, and the unsubscribe link is built
from a separate origin value.

On the GUI side `/?shared=` boots the dashboard rather than the desktop;
`/desktop?shared=` keeps opening the item in place. The Files tab
selects the rows by uid, which survives a rename. A share only reaches a
real account, so a share link never mints a temp user and an existing
temp session is asked to sign in first, as the desktop already did. The
address-bar cleanup both shells need moves into a helper.

* Keep the share email's button link within what mail clients tolerate

The "Open Puter" link names every item in the mail, capped at twenty. But
a single item's parameter is ~150 characters once the owner, uuid and an
encoded name are in it, so twenty of them run to several kilobytes — past
the ~2000 characters where older mail clients cut a URL off or stop making
it clickable — and the count alone couldn't do what its comment promised
for the mail's primary button.

Add a length budget beside the count: items go in, in digest order, only
while the whole link stays under it, so the button always works and the
first items are the ones highlighted. A single item always fits.

* Keep a share link's own item however long its name runs

`shareDeepLink` now builds through `sharedViewLink`, whose length budget
applied to the first item too: a parameter that alone overran it was
dropped, and the link came out as a bare `?shared=`. A name of a few
hundred characters — the GUI allows five hundred, and encoding triples
every non-ASCII one — was enough, so that item's own link in the mail
landed on Shared with nothing picked out, where it used to open the item.

The first item now always goes in; the budget only decides how many more
join it. One long link is still the item the mail is about, and it is no
worse than a bare origin for the clients that truncate it.
2026-08-24 17:01:55 -07:00
jelveh 34af8d18d0 Share a whole selection from the Dashboard's Files tab
Sharing was one item at a time: the only way to give someone four files
was to open the dialog four times. A multi-selection now offers Share in
its context menu and in the mobile selection bar, and UIShareModal takes
a list of items.

With several items the access list folds into one row per person rather
than repeating per file, so a grant, a mode change or a revoke is one
decision about a person. A row says what it can't otherwise show: how
much of the selection the person reaches ("On 2 of 4 items", with an
"Add to all" that extends them), a mode select that rests on a
placeholder when their grants disagree rather than presenting one item's
mode as the batch's, and inherited grants left uncontrollable where they
belong. Changing a mode touches only the items the person already holds
— nothing here widens access without saying so.

Requests are chunked to the documented 50-items-per-request cap, and a
selection past it skips the per-item listing instead of firing one
request per file on every refresh.

The eligibility rule the Share entry already used moves into a shared
can_share() helper, since the multi-select menu needs the same answer
for every row.
2026-08-24 14:08:37 -07:00
Juan Fernando Castro 0527bcd98e Merge pull request #3632 from HeyPuter/juancastro/put-1560-file-sharing-deeplink-in-emailnotifications
🔧 PUT-1560: Deep-link the shared item from email and notifications
2026-08-24 12:30:07 -04:00
Juan Castro ea530e604c Merge branch 'main' into juancastro/put-1560-file-sharing-deeplink-in-emailnotifications 2026-08-24 12:21:21 -04:00
Juan Fernando Castro 97d137b94b Merge pull request #3623 from HeyPuter/juancastro/put-1564-perms-docs-drop-the-app-root-dir-raw-permission-string-wire
🛠️ PUT-1564: Perms docs — drop the app-root-dir raw string, wire the playground examples
2026-08-24 10:59:56 -04:00
jelveh 1ceffbe291 Add sudoku to recommended apps
This change adds Sudoku to the curated recommended apps list so it appears alongside the other bundled games in the app recommendations flow.
2026-08-23 21:59:52 -07:00
Daniel Salazar 684d6752f9 fix: provide fallback for signup verification (#3631) 2026-08-23 21:49:35 -07:00
jelveh 76cea02513 Treat .md like .txt; remove markus suggestion
Merge the .md extension case into the plain-text branch so it suggests ['editor', 'code'] instead of ['markus', 'editor', 'code']. Update the related test to use 'viewer'/'png' as the built-in guard example since 'markus'/'md' no longer applies.
2026-08-23 21:17:28 -07:00
404oopsandClaude Fable 5 308b78e96c refactor: dedupe model catalog aliases and stop mutating shared catalogs
- Add modelLookupNames() and route every provider's list() through it so
  repeated or self-referential aliases can't double-list a model.
- Stop the chat/image/video drivers' #buildModelMap from writing through
  to the module-level catalog constants (ids/aliases accumulated across
  instantiations); bucket copies preserve the models() wire shape.
- Remove self-aliases and exact duplicates from the claude, deepseek,
  groq, mistral, and openai catalogs; fold the duplicated
  gemini-3.7-flash entry — this corrects the live cached-token price
  from 8 to 7.5 and the knowledge date to 2026-03 (the previously
  effective first entry carried stale values).
- ClaudeProvider: derive default max_tokens from the resolved entry
  instead of a second name-based lookup that capped dated ids at 4096.
- Add cross-provider catalog invariants (modelCatalogs.test.ts) with a
  disk-scan registration guard, plus regression tests for catalog
  non-mutation, alias-dedup routing, wire shape, and max_tokens ceilings.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-24 03:42:47 +02:00
nevrkla 2d787c3fdf feat(i18n): add Czech GUI localization (#3617)
* feat(i18n): add Czech GUI localization

* feat(i18n): add Czech GUI localization

* fix(i18n): polish Czech GUI translation
2026-08-23 17:22:44 -07:00
Daniel Salazar 5918e3f4e2 fix: kv max number value (#3629) 2026-08-23 06:31:08 -07:00
Nariman Jelveh 2c9e818e5f Docs/remove discord reddit links (#3628)
* docs: remove Discord and Reddit links

Drops both links from the docs README (header row and Support list) and
from the footer generated on every built docs page.

* docs: remove Mastodon link
2026-08-22 22:14:56 -07:00
Nariman Jelveh 8826a9fc51 Remove Discord and Reddit links from README (#3627)
Removed links to Discord and Reddit from the README.
2026-08-22 13:07:33 -07:00
Daniel Salazar 8307e8c9f0 chore: bump up deps (#3626) 2026-08-21 21:03:54 -07:00
Daniel Salazar 732786a029 fix: clean up email validation (#3622) 2026-08-21 20:27:05 -07:00
Juan Castro 202c43f46e feat(share): deep-link the shared item from email and notifications
A share email named the file but had nowhere to go: the only link was "Open
Puter", and finding what someone shared meant hunting for it under Shared. Each
named file in the digest now links to itself, and a notification covering a
single item points at that item.

The link carries one parameter, the masked path a recipient is already given:

    https://puter.com/?shared=%2Falice%2F<uuid>%2Freport.txt

Its second segment is the uuid, so there is no second copy of it to disagree
with the path, and the GUI can still find the entry when a rename has left the
name segment stale - it stats the path, then falls back to the uuid.

Built from the owner, uuid and name rather than from `ResolvedShare.path`.
That path is masked for whoever made the request, and the issuer owns the
entry, so it comes back as the owner's *real* path - mailing it would tell the
recipient which folders the owner keeps things in, which is the one thing
masking exists to prevent. A test asserts the real path never reaches the mail.

`digestLines` now returns `lead`/`items`/`trail` beside `what`, so the template
can put an anchor around each name while Handlebars keeps escaping the names
themselves; the URL is machine-built from the configured origin and one encoded
path, so it stays literal. Concatenating the parts reproduces `what` exactly,
which a test pins - the linked and sentence forms must not describe different
shares.

Notifications carry the masked path rather than a URL: the recipient is already
in the GUI, which opens the item in place instead of reloading. Only a
single-item notification gets a target; folding into a group drops it rather
than picking one of five.

In the GUI, `?shared=` joins `?download=` and `?app=` as a param that keeps the
desktop booting at `/`, and the handler reuses the `/@user` public-folder flow -
extracted to `open_path_target`, which carried a TODO asking for exactly this -
so a file opens in its associated app and a folder in an explorer window. The
param is stripped from the address bar first, so a reload lands on the desktop
rather than opening the item twice.

Invites are deliberately not linked: there is no account to route to yet, and
the invite's own call to action is to create one.

Rolling-deploy safe: a digest entry queued before this has `names` and no
`items`, and still flushes - without links. New entries write both, so a node
on the previous build can flush them too.
2026-08-21 16:08:39 -04:00
Juan Castro e7be457a56 docs: redirect the retired perms pages, add the missing playground examples
Both were left as follow-ups on the previous commit.

Thirteen `puter.perms` method pages went when the surface collapsed onto
`request(resource, details)`, and `requestAppData` moved to `/Perms/appData`.
All fourteen had shipped, so an external link or a bookmark landed on nothing.
The build already generates meta-refresh redirects from a map; they are entries
in it now. Pages that only ever existed on the branch that removed them are
deliberately absent — nobody can hold a link to a URL that was never published.

Six documented APIs annotated a code block with a playground id but had no
example file behind it, so the build warned on every one and the Try-it link
resolved to nothing: `fs.share`, `fs.unshare`, `fs.listShared`, `fs.getShares`,
`ai.txt2speech` with Speechify, and `ui.showFeedbackDialog`. The files are the
documented blocks themselves, extracted rather than rewritten, so the example
and the page it appears on cannot drift. Each is indexed in `examples.js` beside
its siblings.

The build now runs clean: 12 warnings to none.
2026-08-21 11:26:32 -04:00
Juan Castro 9bf49b7574 docs(perms): drop the app-root-dir raw string, wire the perms playground
`app-root-dir:` was listed among the raw permission strings, but it reads as
nothing during a permission check - the rewriter yields the sentinel outside a
grant, so `check()` reports it ungranted even once it is and `request()` prompts
every call. Both contradict what the same page promises. The `'appRootDir'`
resource is the supported path and asks the server instead; the list now says so
rather than implying the string behaves like its neighbours.

The playground listed no perms examples at all: six files existed but nothing
referenced them in examples.js, so a freshly-reshaped API shipped with nothing
runnable. Adds a Perms group after UI, matching the sidebar's order, and two
examples for the features that had none - one prompt covering several resources,
and check() deciding whether to prompt.
2026-08-21 10:20:05 -04:00
f0cd251626 🛠️ PUT-1521: Cleanup puter js permissions api + backend routes (#3607)
* refactor(puter-js): collapse puter.perms request* to resource + access

Fifteen request methods differed only by a folder name or an access level, so
every new resource meant another method. Replace them with requestFolder,
requestApps, requestSubdomains and requestAppRootDir, each taking the access
level as an argument.

The old names stay as @deprecated aliases: puter.js ships unpinned from
js.puter.com/v2, so removing them would break live apps. They remain in the
generated declarations because stripInternal has no effect on declarations
emitted from JavaScript, and hand-omitting them would break TypeScript callers
the runtime still serves.

Also drops the user-to-user and user-to-group grant wrappers (groups.js and the
grantUser/grantGroup half of grants.js) plus the req_ shim, none of which were
documented or called. The app, origin and dev-app grants stay: the dashboard
uses puter.perms.revokeApp() to clear grants on app uninstall.

* docs(perms): document the collapsed puter.perms surface

Replace the twelve one-method-per-task pages with requestFolder, requestApps
and requestSubdomains, and rewrite the Perms overview around the seven public
methods. The deprecated aliases keep working but are no longer documented.

Boy Scout: drops the long-dead commented-out grantUser/revokeOrigin sidebar
block for pages that were never published.

* refactor(perms): drop unused user-to-user and user-to-group permission routes

Filesystem access is shared through /share, which records the grant so the
owner can see and revoke it. The older direct-grant paths were left behind with
no caller anywhere - not the GUI, not a doc, not an app: grant-user-user
(already a 501 stub), revoke-user-user, grant/revoke-user-group, and the five
/group/* CRUD routes.

Removing them orphans PermissionService.grant/revokeUserGroupPermission and its
group-members cache bump, the three PermissionStore group writers, and six
GroupStore methods, so those go too.

What stays, and why:
- grant/revokeUserUserPermission - ACLService and ShareService power fs.share
  through them.
- The group permission read path (#scanUserGroup, readUserGroupPerms) - a
  migration seeds the admin group unrestricted driver access, so it is
  load-bearing.
- GroupStore getByUid/addUsers/removeUsers - signup, save_account, OIDC and the
  self-hosted default user assign group membership.

No schema change: user_to_user_permissions, user_to_group_permissions and their
audit tables are untouched. Group rows now come only from migrations, so tests
that need one seed it with SQL the way a migration does.

* refactor(perms): reduce GroupStore to membership writes

With the /group/* routes gone, `getByUid` had no production caller left — the
routes were the only thing that read a group back. Removing it takes the row
decoder and the GroupRow type with it, since they exist only to shape its
result.

What remains is `addUsers`/`removeUsers`: signup, save_account, OIDC and the
self-hosted admin bootstrap all assign group membership. Permissions attached to
a group are read through PermissionStore, which joins the junction table itself
and never needed the store.

Tests that wanted a group id now select it, which is all `getByUid` was doing
for them.

* chore(perms): drop the three by-hand groups no code reads

freeai, experimental and dangerous exist in prod but in no migration — they
were added by hand when hardcoded permissions were keyed by group name. That
map is now a flat per-user floor (`default_user_permissions`), so a group
nothing looks up grants nothing.

Guarded rather than unconditional, because both tables the delete can reach
cascade: dropping a group that still carries permissions or members would
silently revoke them from every member. Only a group with neither goes. One that
survives has dependents and needs a deliberate decision — query
user_to_group_permissions by group_id to see what it holds.

system, admin, user and temp are untouched: config names two of them and code
names the others.

Matches on `extra.name`, not `metadata.name` — `metadata` carries the display
title and colour, and `critical: true` is set on all of these including freeai,
so it does not discriminate.

* feat(puter-js): collapse puter.perms onto request(resource, details) + check()

One method per task meant a new method, doc page and sidebar entry for every
resource. `request` now takes the resource and a payload whose accepted fields
depend on it, and `check` answers the same question without prompting.

    request('folder', { name: 'Documents', access: 'write' })  -> path
    request('apps', { access: 'read' })                        -> boolean
    request('email')                                           -> address
    check('folder', { name: 'Documents', access: 'write' })     -> boolean

Returns stay per-resource: a folder gives its path, email the address, the rest
a boolean, and anything denied is falsy so one `if` covers both.

An array asks for several at once. Everything already held is settled first, so
the prompt covers only what is missing and does not appear when the whole set is
held - the user answers once for the lot. `check` answers per entry, in order,
so a caller can tell which parts are missing rather than only that some are.

Each resource declares four things in one registry entry: how to ask for it
alone, whether it is held, the strings a batch pools into a prompt, and the
value once held. The strings themselves are defined once in
lib/permissionStrings.js, so a request and its check cannot name them
differently. `check` is built on /auth/check-permissions, already live and
already used by UI.js, and it throws rather than answering false when the check
cannot run: a caller that cannot tell "denied" from "never ran" would prompt
someone who had already granted it.

Backward compatibility: all 22 older methods stay callable and typed, marked
@deprecated with the call that replaces them. A lone string still routes to the
raw-permission path - no resource name contains a `:` and every permission
string does, so the two forms cannot collide. The grant/revoke app methods are
untouched; the consent dialog and the dashboard's uninstall path use them.

Also drops three copies of the access-level assertion onto one shared
validator, and gives `appRootDir` a non-prompting server probe, since
`app-root-dir:` only resolves while a grant is being written and a permission
check on it always answers false.

* docs(perms): document request() and check() as the perms surface

Five per-method pages became one `request()` page carrying the resource table,
the batch form and the raw-string escape hatch, plus a `check()` page. The
overview is rewritten around the two methods.

requestAppData's page is re-homed as /Perms/appData rather than deleted - its
scope table, private-entry guidance and lifetime notes are not signature
documentation and have nowhere else to live. Inbound links from KV/set.md and
Objects/app.md follow it.

Playground examples move to the new call form. They are not wired into
examples.js, but an example demonstrating a deprecated method is worse than one
nobody loads.

* fix(perms): keep /auth/revoke-user-user as a deprecated route

Dropping this route with the rest of the unused user-to-user plumbing went too
far. The grant side is retired and stays retired - puter.fs.share() is the only
way in - but access those grants left behind has to remain withdrawable, and a
caller reaching the endpoint over HTTP directly had no replacement. Revoking can
only ever narrow what someone can reach, so keeping it carries no risk.

revokeUserUserPermission never left the permission service; it is load-bearing
for puter.fs.share(). This only re-wires the handler to it, with the gates it
always had.

Nothing in this repo calls the route, which makes it exactly what a later
cleanup reads as dead, so a test pins the registration and its gate alongside
the restored 400 and grant/revoke round-trip cases.

The 501 stub at grant-user-user and the never-called /group/* routes stay
deleted, as does puter.perms.revokeUser - puter.fs.unshare() replaces it and
falls back to live grants when no share row exists.

* fix(perms): let a write grant satisfy a read check on apps and subdomains

`apps-of-user:<uuid>:write` covers managing the user's apps, which includes
reading them, but nothing said so to the permission system. Prefix implication
only widens the other way — an `apps-of-user:<uuid>` grant covers both modes —
so a scan for `:read` missed a `:write` grant, and `puter.perms.check('apps')`
reported an app holding write as holding nothing. A batched request would then
prompt again for access already granted.

Adds the read-from-write exploder for both namespaces, mirroring
`fs-access-levels`. The widening runs one way only, and does not cross into
another user's namespace; both are covered by tests.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): answer the app-root-dir check without provisioning it

`/auth/request-app-root-dir` conflates two questions: may the caller claim its
root directory, and where is it. The second provisions `AppData/<uid>` on first
ask, so a caller that only wanted the first — `puter.perms.check('appRootDir')`
— created a directory by asking about it.

Adds `check: true`, which runs the same actor guard and stops at the answer.
A caller that may not claim it still gets the 403, so the flag can't widen
anything. Existing callers are unaffected: without it the route behaves exactly
as before.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): put request() and check() on one path

`request` dispatched to the old per-task methods while `check` asked the
permission tables, so the two answered different questions about the same
access. Concretely, before this: `request('folder', { access: 'write' })`,
`'apps'`, `'subdomains'`, `'appData'` and `'permission'` prompted every time,
whether or not the access was held — which the docs said they wouldn't;
`check('folder')` reported false for a folder the app could read through an ACL
grant that no `fs:` string names, so a batch prompted for it needlessly; a
batch entry for `'appRootDir'` skipped the post-grant retry the single call
does, resolving `undefined` after a grant that had in fact succeeded; and an
N-entry batch made N permission reads plus 2N `whoami` calls.

Both now run the same pipeline — resolve the permission strings, read what is
held once, prompt for the remainder, resolve each entry — with per-resource
hooks for the parts only that resource can answer. So a batch costs one
permission read and one `whoami`, a check reports exactly what a request would
skip the prompt for, and `'folder'` uses the same stat-or-permission reading in
both.

Also:

- A resource is looked up as an own property, so `request('constructor')` is
  the permission string it always was rather than a TypeError.
- A permission read that fails no longer decides anything: `request` falls
  through to the prompt it would have raised anyway, `check` throws. Before,
  `check('appRootDir')` folded a failed check into "not granted", which is what
  the documentation says must not happen.
- Drops `requestFolder`, `requestApps`, `requestSubdomains` and
  `requestAppRootDir`. They were added in this branch and immediately deprecated
  — never shipped, and `request()` no longer needs to route through them. The 22
  methods that did ship keep their exact behaviour, prompting without consulting
  what is held, which the suite now asserts alongside the new behaviour.
- Documents `'appRootDir'`, which was a supported resource in every overload and
  in `PermsResource` but named in none of the docs.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): act on review of the three preceding commits

- `src/puter-js/test/perms.test.js` still called `requestApps`,
  `requestFolder`, `requestSubdomains` and `requestAppRootDir`, which the
  previous commit removed. Four cases in the interactive browser harness threw.
  Pointed at `request(...)` instead.

- `request('appRootDir', …)` made two round trips where the shipped method
  makes one: a read-only probe, then the call that names the directory. A
  request is going to claim it either way, so the claim is now the check, and
  the entry it returns carries through to the result. `check` keeps the
  read-only mode, which is the reason that mode exists. Matters because the
  route sits on the FS_SIGN bucket, shared with signed-URL minting.

- `requestPermission` is one of the shipped methods, and the previous commit's
  message was wrong to say all 22 keep their exact behaviour: it forwards to
  `request`, so it now settles a permission the caller already holds instead of
  prompting for it. The value can differ, not just the prompt count — a user who
  would have clicked Deny on a re-prompt used to get `false`. It is the more
  honest answer (the app does hold the access, and denying a re-prompt never
  took it away), but it is a change, and the suite assertion had been switched
  to an unheld permission, which hid it. Asserted both ways instead, in the unit
  tests and the API suite.

- An entry that names no permission no longer rides a grant given for the other
  entries in the same call. Unreachable today — every resource either names one
  or reports itself held — but nothing pinned it.

- Reverted three type-union reformats in `LegacyFSController.ts` that a
  formatter had folded into the app-root-dir commit. That file was not
  prettier-clean to begin with; reformatting it is somebody else's change.

- Docs and types: `Perms.md`'s `appRootDir` row now matches `request.md`'s,
  `check.md` says that a `true` is per entry and a batch still prompts if any
  one entry is missing, and `types.js` no longer names `requestFolder` /
  `requestAppData` in prose that ships in the generated declarations.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-20 23:48:08 -07:00
ZHOU7821 943adae622 docs: use absolute link to router page in workers create docs (#3615) 2026-08-21 13:30:30 +07:00
Daniel Salazar 65572dcf23 fix: nicer sms card fallback (#3621) 2026-08-20 23:12:09 -07:00
ZHOU7821andZHOU7821 b3923f86da docs: fix typo in workers description (#3614)
Co-authored-by: ZHOU7821 <[email protected]>
2026-08-21 12:26:20 +07:00
Nariman Jelveh 729114bd09 Prioritize app-center in recommendations
Reordered the recommended app list so app-center appears earlier and dev-center is kept at the end. This keeps the default app suggestions aligned with the expected homepage/app-center placement without changing behavior beyond ordering.
2026-08-20 19:16:03 -07:00
Nariman Jelveh da1c85d3c5 Add Meetings to recommended app defaults
Include `meetings` in `RECOMMENDED_APP_NAMES` so it appears in the default recommended apps set alongside other core productivity tools.
2026-08-20 17:16:39 -07:00
Nariman Jelveh 66410d7f1f Revert "Let a browser send the device fingerprint header it is offered"
This reverts commit fef1337b2b.
2026-08-20 17:10:55 -07:00
Nariman Jelveh 76bd05f228 Dashboard: let the trial be a detail line, not an alarm
The Usage card gives each of its three columns a meter and two lines. The
plan column had a boxed, tinted, bordered paragraph instead — and at the
~230px a third of that card comes to, thirty words of it wrapped into a
five-line ribbon standing twice as tall as the storage and resources meters
beside it. The box is the shape we use for the dunning banner. A trial is a
date you should know about, not a fault, and it was reading as the loudest
thing on the card.

So it takes the slot its neighbours use for "n% of 100 GB": a quiet hint-
coloured line under the badge. All three columns now measure the same.

The copy stopped repeating itself while it was there. The badge said "ends
Sep 20" and the sentence beneath it said "ends on September 20, 2026" — two
elements, one fact, and neither of them answering the question someone
actually has, which is how much room is left to decide. The badge now
carries "31 days left" and the line under it carries the consequence:
"First charge Sep 20 unless you cancel." Billing keeps the full sentence and
the opt-out, and Manage is directly below.
2026-08-20 16:59:07 -07:00
Nariman Jelveh fef1337b2b Let a browser send the device fingerprint header it is offered
`fingerprint.ts` reads the device fingerprint from a request body or, for
authenticated requests with no body to carry it, from
`x-puter-device-fingerprint`. That header is documented there as "the
header the GUI may send the device fingerprint on for non-signup
requests", and the middleware has always honoured it.

A browser could never actually send it. The CORS allowlist in
`#installCors` does not include it, so the preflight comes back without
it in `Access-Control-Allow-Headers`, the browser abandons the request,
and `fetch` rejects before anything reaches the server. Nothing in tree
sends the header today, which is why this went unnoticed: the first
client to try it sees every call fail with a network error rather than a
readable status.

Adds the header to the allowlist, and a preflight test asserting the
browser is allowed to ask for it.
2026-08-20 12:40:48 -07:00
404oops 58794f4f76 feat: add MetaProvider for Muse Spark models and integrate with ChatCompletionDriver (#3616)
- Implemented MetaProvider to interface with Meta's Muse Spark models.
- Integrated MetaProvider into ChatCompletionDriver, allowing for dynamic model selection.
- Added unit and integration tests for MetaProvider to ensure functionality and reliability.
- Updated documentation to include Meta as a supported vendor for AI models.
2026-08-20 15:09:56 -04:00
Nariman Jelveh 9809e428b5 Update shared label in email template
Adjust the share-notification email copy to reference the current "Shared" files section instead of "Shared with me," in both HTML and plain-text variants. This keeps onboarding messaging aligned with the UI.
2026-08-19 20:05:56 -07:00
Nariman Jelveh eaddd1d44e redesign: share notification emails (#3612)
* redesign: share notification emails

The share digests were an unstyled fragment: a 520px div with a couple
of paragraphs, no preview line, no small-screen or dark-mode handling,
and an html-only body that left every text-only client reading a machine
down-conversion of the markup.

Both digests (`file_shared_with_you` and `file_shared_invite`) now render
through one layout: a 600px table column that goes full width under
600px, every color written inline so a client that drops `<style>` still
gets the design, with the stylesheet carrying only what inline CSS can't
express — the small-screen and dark-mode overrides. The shares are a
tinted panel of hairline-separated rows rather than loose text, the call
to action is a cell-padded button Outlook can draw, and the reason for
receiving plus the unsubscribe sit below the card where they belong. No
images at all, so nothing depends on an asset a client may refuse.

Templates may now carry a `text` part, and both digests do: it goes out
beside the HTML as multipart/alternative, which is what plain-text
clients, screen readers and spam scoring prefer.

Copy: a preview line that adds to the subject instead of repeating it, a
heading before the list, and an honest unsubscribe label — the link
silences notification mail account-wide, not just shares. The invite now
says what to do with the address and that an existing account can claim
the shares by confirming it.

Also drops `share_by_username` and `share_by_email`, unreferenced since
the digests replaced them.

* fix: widen the template map to the interface before compiling

`Object.entries` over the literal map yields a union with a distinct
type per template, and only the share digests carry a `text` part —
reading it off the union is an error the build tsconfig hides
(`noCheck: true`) and `npm run typecheck` catches.
2026-08-19 18:49:11 -07:00
Reynaldi Chernando 53b792c52c connect to worker's kv with cli (#3613) 2026-08-20 08:36:46 +07:00
Daniel Salazar 2dd7073034 fix: vscode webdav (#3610) 2026-08-19 14:04:56 -07:00
Daniel Salazar a552ce0b87 fix: better metrics for cache hit rates (#3609)
* fix: better metrics for cache hit rates

* chore: type + docs
2026-08-19 11:38:14 -07:00
Neal Shah c04827fd3f add dynamic worker create event (#3608) 2026-08-19 14:20:15 -04:00
Daniel Salazar 9debdffc5d fix: email send frequency + ai ordering (#3604) 2026-08-18 18:13:55 -07:00
Nariman JelvehandJuan Castro 57b4cbab66 feat(dashboard): rewrite Files-tab sharing as a from-scratch modal (#3598)
* feat(dashboard): rewrite Files-tab sharing as a from-scratch modal

The Files tab's Share… action opened UIWindowShare, a desktop UIWindow,
which doesn't fit the dashboard. Replace it with UIShareModal, built on
the same overlay pattern as UIItemPropertiesModal: a centered card on
desktop, a bottom sheet on mobile, styled with the dashboard's design
tokens (dark mode included).

Wiring follows the onShowProperties precedent: generate_file_context_menu
gains an optional onShare hook that TabFiles supplies; every other caller
falls back to the desktop share window unchanged.

Feature parity with UIWindowShare — grant by email/username with an
access level, list who has access (owner, direct, and inherited-from-
ancestor grants), change a grant's mode, revoke — plus:

- revoke confirmation inlined into the row instead of a stacked UIAlert
- real form semantics: Enter submits, button disabled until input,
  spinner while in flight, API errors keep the typed name for correction
- dialog a11y: focus moves in on open and back on close, Tab is trapped,
  Escape closes (first Escape cancels an open revoke confirmation),
  status messages announce via aria-live
- backdrop close keyed on where the press started, so a text-selection
  drag out of the input can't dismiss the dialog
- name-derived avatar colors and a loading state for the first fetch

Also fixes a double-encoding bug (shared by UIWindowShare) where mode
labels ran through html_encode twice, rendering "Can edit &amp; share".

* fix(dashboard): keep keyboard focus inside the share modal across re-renders

Changing a grant's mode, revoking, or Escape-canceling the inline
confirmation re-renders the access list (or disables the focused
control), which dropped focus onto <body> - outside the dialog's Tab
trap and invisible to screen readers. Focus is now restored explicitly
after each of those actions: to the same holder's control when it
survives, else to the dialog itself (tabindex=-1), and the Tab trap
steps back into the cycle when focus rests on the dialog container.
A failed grant likewise returns focus to the recipient input.

* fix(dashboard): give share modal controls 44px touch targets

On touch devices the close (32x32) and revoke (30x30) buttons and the
26px-tall per-row mode select were below the 44px minimum. On coarse
pointers the icon buttons now hit-test at 44x44 through a centered
pseudo-element (visuals unchanged, and the row spacing absorbs the
overhang without overlapping neighbors), and the row select gets
taller padding. Fine-pointer rendering is untouched.

* fix(dashboard): clear the iOS home indicator in the share bottom sheet

The app opts into viewport-fit=cover, so on notched phones the
bottom-docked sheet extends to the physical screen edge and its last
row sat under the home indicator. The sheet body's bottom padding now
adds env(safe-area-inset-bottom), matching how the codebase pads other
bottom-docked surfaces.

* fix(share): say 'Updated access' when changing a grant's mode

Changing someone's access level reported 'Shared with X', the same
message as a fresh grant, which reads as if a new share happened.
Both share dialogs now confirm mode changes with a dedicated
'Updated access for X' message. The UIWindowShare call also drops the
html_encode() around the recipient - i18n() already encodes
interpolations, so the wrapper double-encoded.

* fix(dashboard): align the share dialog's accessible name with the properties modal

The dialog's aria-label read 'name - Share' with an em dash while the
sibling properties modal uses plain 'name Properties'; screen readers
announce the dash as noise. Use the same name-then-noun pattern.

* fix(dashboard): ellipsize the share input's placeholder when it overflows

At phone widths the 'Add people by email or username' placeholder was
clipped mid-letter; text-overflow: ellipsis truncates it cleanly.

* fix(dashboard): raise the share selects' chevron contrast in light mode

The hardcoded slate-400 stroke measured ~2.6:1 against light surfaces,
below the 3:1 minimum for non-text indicators. Light mode now uses
slate-500 (~4.7:1); dark mode keeps slate-400, which already clears
5:1 there.

* fix(dashboard): stop Files-tab shortcuts from firing behind the share modal

The document-level keydown.tabfiles handler kept running while the
share (or item-properties) modal was open. With focus on any of the
modal's buttons or selects, Enter and Space were preventDefault-ed
before they could activate the control, arrows could not drive the
mode selects, and letter typeahead was hijacked into row typesearch —
while Enter opened the selected row behind the overlay, Delete moved
it to Trash, and Cmd+A/C/X/V acted on the hidden list. Yield the
keyboard to the modal for as long as one is up; its own handlers
already cover Escape and Tab.

* fix(dashboard): keep the on-screen keyboard down when the share sheet opens

Autofocusing the recipient input popped the keyboard over the bottom
sheet the moment it opened, hiding the access list before the user had
chosen what to do — the same reason the revoke flow already focuses
the dialog instead of the input. On touch-primary devices give the
dialog container initial focus (which also anchors the Tab trap);
desktop keeps the input autofocus.

* fix(dashboard): give the share sheet's confirm and submit buttons 44px touch height

The inline revoke confirmation's Cancel/Remove pair rendered ~29px
tall on touch — small targets 6px apart where one of the two is
destructive — and the submit button was fixed at 38px. Grow both to
44px under pointer: coarse, matching the standard the modal's icon
buttons already meet. The touch block moves below the confirm-button
base rules it now overrides, since a media query adds no specificity
and source order decides.

* fix(dashboard): name the person in the share rows' accessible labels

Every grant row's mode select announced as bare 'Access level' and
every revoke button as 'Remove access', so a screen reader user
tabbing the list could not tell whose grant a control changes. Carry
the holder in the aria-label ('Access level for alice' / 'Remove
access for alice'); the visible UI and the revoke tooltip stay as
they were. i18n() encodes the interpolated string as a whole, quotes
included, so the labels stay attribute-safe for any holder name.

* fix(dashboard): keep the share status region in the accessibility tree

* fix(dashboard): give the share modal's add-row controls 44px touch height

* refactor(gui): extract the share dialogs' pure logic into tested modules

* feat(dashboard): adopt the shared mode helpers and handle pending invitations

---------

Co-authored-by: Juan Castro <[email protected]>
2026-08-18 17:34:40 -07:00
Juan Fernando Castro 5e1645d40e 🔧 PUT-1524: Sanity check file sharing api (#3603)
* fix: sharing answers "who can reach this" from the grants, not the index

Two ways a share listing could name access that was no longer there.

A recipient's listing paired the flat permission read with the list of
permissions it asked for by array index. That read drops misses and
dedupes its keys, so the two are not positional: one entry's live grant
vouched for another entry that had none, and listShared() kept
publishing a withdrawn item's name, size and signed thumbnail URL to
someone who could no longer open it. Read the permission off the value
instead.

getShares() had the same gap from the owner's side, with no liveness
check at all — a grant withdrawn through /auth/revoke-user-user or an
ACL mode change left the index row behind, and the owner was told
someone could reach a file they could not. Checked against the grants
now, one batched read per distinct holder. Pending invites are not
subject to it: they have no grant yet, which is the point of them.

Also covers the access-token actor, which reaches the same reach bound
as an app through a different arm of the ACL check. No behavior change
there — it was correct and untested.

* test: pin that a revoke crosses regions

A revoke reaches a peer region as a replicated SQL delete plus three
invalidation events, one per cache the region owns: the u2u row cache,
the flat view and the scan generation. The delete alone changes nothing
there, and each cache has a different consequence if its event is lost
— 20s for the generation, 5 minutes for the row cache, and forever for
the flat view, whose grant-path entries carry no expiry.

Nothing covered the whole path end to end. The store tests prove each
event is emitted and applied; this proves the result, which is that the
recipient stops being able to read. It asserts the read still succeeds
after the SQL delete alone, so the test also records why replication is
not sufficient on its own.

* test: pin that a leaked uuid buys no access

A masked share path hides which folder an item sits in; it was never the
thing deciding who may open it. Nothing checked that at the route level,
so the guarantee rested on unit tests of the resolver alone.

Reads one shared file through its masked path, then tries the sibling
four ways: the shared uuid with the sibling's name, the sibling's own
uuid, a `..` back out of the root, and the owner's real path.

Worth knowing about this one: it's mutation-checked. Removing the head !== root.name guard in sharePathMask.ts fails it with reachable: /testuser/55dd54c0…/share-http-1df8933e.txt. I verified that specifically because two tests I wrote earlier in this chunk passed with their guards broken — both were vacuous, and I deleted them rather than commit false assurance.

* docs: say what listShared's total actually counts

`total` counts the shares recorded for you; items are filtered after the
page is read, so a withdrawn grant leaves the count higher than anything
paging will yield. The page description already explained the short-page
behaviour, but the field read as an exact count and the example printed
it as one.

The test pins the gap it describes: two shares, one withdrawn outside
the index, one item listed and a total of two.

* fix: keep an undelivered broadcast event instead of dropping it

The outbound queue was cleared before the send, so a peer that timed out
took its events with it. Most were survivable — a lost cache invalidation
heals when the entry's TTL lapses. A revoke's flat-perm invalidation is
not: grant-path entries carry no expiry, so a peer went on serving a
withdrawn grant until something else wrote that key.

Failed sends now go back on the queue, which the existing flush timer
retries. Anything queued since wins over the retry, and the queue is
bounded at 10,000 with the oldest dropped first, so a peer that stays
down cannot grow it without limit. Each retry is signed at send time, so
it is not rejected against the replay window.

* fix: stop plus-addressing from deciding who a share reaches
2026-08-18 15:51:23 -07:00
b795b219a8 ✨ PUT-1497: Share file link sharing and notifications (#3595)
* refactor(share): move share notifications into their own service

* feat(share): invite an address with no account, and email it

* feat(share): surface pending invites in the SDK and share dialog

* fix(share): unreachable revoke confirmation, and double-encoded labels

* feat(share): budget share announcements, group them, and let people block senders

Sharing had one defence against noise: a 15-minute quiet window per (sender,
recipient) pair, which dropped the second share rather than folding it in.
Twenty senders each under their own window could still bury someone, and there
was no way to make one of them stop.

Announcements are now budgeted on two axes through the existing sliding-window
limiter: 1 per 15 minutes and 20 per day from one sender, and 10 per hour /
50 per day to one recipient from anyone. Over budget the share still succeeds
and the recipient's notification is still brought up to date — only the
interruption is dropped. Invite email to an address with no account is budgeted
the same way, keyed on a hash of the canonical address.

Notifications now fold across senders: a new share rewrites the notification
the recipient hasn't dismissed, so "alice and bob shared 5 items with you"
replaces a stack of five. The record is written even when suppressed, so the
count is right whenever they next look.

Blocking is a new `user_block` table with enforcement in ShareService: a blocked
sender's share is refused with `recipient_not_accepting_shares`, spends no
quota, and writes no row, and their unclaimed invite is dropped when the address
is confirmed. Existing access is untouched — that is what revoke is for.
Managed from a Blocked people card in the dashboard's Security tab.

Also publishes the sharing limits, including the ones already on this branch
that were never documented.

* fix(share): name the item in share email, instead of 'an item'

* fix(share): make the invite lifecycle canonical, authorized, and race-safe

* refactor(email): drop EmailClient.isConfigured; callers read config.email

* feat(share): batch share email into a per-recipient digest, durably

* docs(share): document the share error codes; steady the disk migration tests

* fix(share): log why a digest wasn't sent, and recover orphaned ones

* feat(share): email recipients about shares by default, with a way to decline

Share email was off unless a deployment opted in, which meant an account
holder was told about a share in the app only. It is now on unless
`share_email_notifications` is set to false.

The reason it defaulted off was that nobody could decline. So this also
honors `user.unsubscribed` — the account-wide opt-out the /unsubscribe page
already writes and app feedback already respects, which share email ignored —
and the digest carries that link. Sharing and the in-app notification are
unaffected by it; only the mail stops.

The link is composed in the template around an interpolated uuid rather than
passed pre-built: Handlebars escapes interpolated values, so a whole URL came
out as `user_uuid&#x3D;…`, which browsers decode but link scanners and older
mail clients need not.

* fix(share): count every shared file in the digest, not just the first

* feat(share): let a recipient refuse shares from everyone

Blocking answered "not from this person" but had no answer to "not from
anyone", so the only way to stop a stream of unwanted shares was to name
each sender after they had already reached you.

Stored as a key in the user row's existing `metadata` blob rather than a
column: the share path already holds the recipient's row by the time it
asks, so reading it costs nothing, and a one-bit preference doesn't earn
a migration per dialect. `updateMetadata` merges and refreshes the cached
row, so the switch bites on the very next share.

Refusing everyone reports the same code as refusing one person — which of
the two it is is the recipient's business, not the sender's. Enforced at
both moments the per-sender block is: when the share is issued, and when
a pending invite is claimed. The per-sender list is untouched while the
blanket switch is on, so turning it off restores what it hid.

`GET /share/blocks` now carries `all`; `POST`/`DELETE` take `{ all: true }`
beside the existing `{ username }`. Managed from the same Blocked people
card in the dashboard's Security tab.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(share): keep the digest sweep off a window that still has a timer

The sweep treated an entry as orphaned the moment its window closed, which
is also the moment the node that armed it fires. Claiming an entry is only
exclusive among flushers that can see each other's deletes, so the two
could each claim a share of the same digest and both send. It now waits
out a grace period first, which costs a genuinely stranded digest that
much delay and nothing else.

Both digest listings were capped at 200 with no word when they hit it — a
truncated flush sends a digest that undercounts and reads as complete.
The cap is named and logged.

Also: `#emailHolder` still described share email as off by default, which
it stopped being; the config doc said the batch window defaults to 60s
when it is 90; and the two tests that need several calls inside one window
were racing a 50ms window across four sequential round trips, so they
failed under full-suite load rather than on the behaviour they cover.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(share): stop double-encoding the recipient in two dialog messages

`i18n()` encodes what it returns, replacements included, so encoding the
recipient first showed the entities to anyone whose address or username
contains one. Same pattern already fixed two lines above.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(notification): widen the mysql shown/acknowledged columns

Both arrived from the v1 schema as `tinyint(1)`, where they were flags.
The backend rework changed the writes to a unix second; sqlite (`INTEGER`)
and postgres (`bigint`) took it and mysql did not, so on mysql every
`markShown` and `markAcknowledged` has failed with
ER_WARN_DATA_OUT_OF_RANGE and left the column NULL. Dismissing a
notification never stuck — the unacknowledged count never moved and one
already delivered came back on every reconnect.

No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a
legacy `1` keeps meaning "yes" once widened. Guarded on the current type,
because changing a column type copies the table and this directory
replays on every boot.

Not reachable from the test suite — it runs against sqlite and postgres,
both of which already have the right type. Verified by hand against mysql:
`/notif/mark-read` and `/notif/mark-ack` now persist, and a dismissed
share notification is no longer the one a later share folds into.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-18 09:36:33 -07:00
Reynaldi Chernando 401b6066b3 Use gpt 5.6 luna as default for docs (#3602) 2026-08-18 09:36:23 -07:00
Daniel Salazar ec3c33fea6 fix: misc hardening fixes (#3600) 2026-08-18 00:06:58 -07:00
Anany Singh 95d797b047 feat(workers): expose app_uid in puter.workers.get and puter.workers.list (#3596) 2026-08-18 00:05:17 -07:00
Daniel Salazar bd06e88185 feat: bring back referrals PUT-1463 (#3599) 2026-08-17 23:49:09 -07:00
Nariman Jelveh cabedc30b0 Require login on app landing pages
Prevent first-time visitors from being treated as temporary users when they open an app landing page such as `/app/<name>` or `/desktop/app/<name>`. These routes now always go through the real login/signup flow.
2026-08-17 20:11:29 -07:00
Daniel Salazar f81b76c1d1 fix: webdav connectivity fixes for macos (#3597) 2026-08-17 19:02:55 -07:00
404oops 9604cc3f23 feat(ai-chat): overhaul provider model catalogs to match current APIs (#3586) 2026-08-17 14:06:44 -04:00
2c852bf6b3 ✨ PUT-1412 File sharing backend api (#3553)
* refactor(permissions): drop hardcoded group permission map for a flat default

* test(drivers): assert credential-gate intent instead of a 403 proxy

* fix(permissions): report whether a revoke removed anything and persist the linked grant row before the flat view

* feat(permissions): replicate permission invalidations across regions

* feat(share): extend the share table into an index of active shares

* feat(share): query and maintain active shares in ShareStore

* feat(users): add a batched lookup by email

* fix(cache): apply cache updates broadcast from peer regions

* fix(permissions): scope a revoke to the issuer that granted it

* feat(share): add ShareService with a per-day share limit

A share is two writes that belong together: the permission grant, which
authorizes access, and a share row, which makes it listable and ties it to an
fsentry so it dies with the file. Nothing else grants fs:* to a user.

Authorization reuses canManagePermission — an owner satisfies it through the
is-owner implicator, a delegate through an explicit manage:fs:<uid> grant. An
owner may clear any issuer's share of their node; anyone else only the ones
they issued, or their own access. Self-revoke skips the manage gate but still
requires `see`, so it cannot be used to probe for files.

The per-day limit counts shares created rather than live rows, so revoking and
re-sharing cannot recycle a slot, and changing an existing share's mode is not
new reach and does not spend budget. Tunable via share_daily_limit.

* feat(share): expose sharing over HTTP

POST /share, POST /share/revoke, GET /share/shared-with-me, GET /share/shares.
The controller was registered but entirely commented out.

Recipients × items fan out concurrently — every pair is a distinct
(holder, entry) key, so none of them contend — bounded by
runWithConcurrencyLimitSettled, which returns results index-aligned with the
input for the per-pair outcome list. Responses carry usernames only, never
internal ids, and the 404-not-403 rule is preserved so a failed call cannot
confirm a file the caller could not otherwise see. Notifications are fired off
the response path; a share must not fail over its own notification.

Per-request caps on recipients and items bound one call's fan-out; the daily
limit bounds the total.

* feat(share): keep recipients consistent when a shared item changes

* fix(fs): stop listing issuer homes at the filesystem root

* fix(acl): serialize concurrent mode changes on one node and pin app containment on shared paths

* fix(fs): expire signed URLs over entries the signer doesn't own

signFile defaults to a ~317k-year TTL and verifySignature checks only uid,
expires and signature — never the ACL. A recipient who ever signed a shared
file therefore held a permanent, revocation-proof URL to its bytes: revoking
the share did nothing to it.

signEntry now takes the acting user and drops to NON_OWNER_SIGNATURE_TTL_SECONDS
(1 hour) when the signer is not the entry's owner. Owners keep the permanent
default, so no existing client changes behavior.

The signature-authenticated directory listing bounds its children
unconditionally: that route has no session actor, and a signature proves
possession rather than ownership, so a recipient holding a short-lived
directory signature could otherwise mint permanent URLs for every child.

A bounded window is not revocation — the durable fix is a per-entry signature
epoch folded into the HMAC and bumped on any permission change.

* refactor(permissions): drop the unused permission-issuer lookup

listUserPermissionIssuers and its store method listUserPermissionIssuerIds
existed to synthesize the filesystem root from the home directories of everyone
who had granted the caller a permission. That listing is gone — it advertised
folders readdir then refused to open — and the share index answers "who shared
with me" directly, so nothing wants them back.

One removed test only asserted that the call returned an array; the other
covered readLinkedUserUserPerms round-tripping and is kept, rewritten without
the issuer lookup.

* fix(share): return the created share, not just an acknowledgement

* feat(puter.js): add file sharing to puter.fs

share(), unshare(), listShared() and getShares() on puter.fs, following the
existing FS operation shape: positional and options-object forms through
defineOperation, JSDoc overloads as the published signature, relative paths
resolved against the app's root directory.

A bare recipient string is read as an email when it contains @ and as a
username otherwise. Sharing an item with someone who already has it replaces
their access rather than stacking a second grant, so raising read to write is
one more call.

Adds a sharing suite to the API runner, which passes unchanged on node,
browser and workerd. Documents all four methods with runnable examples, and
corrects the FS overview callout that told readers one user cannot read
another's files — true before this, not after.

* feat(gui): add a Shared folder for items others shared with you

A sidebar entry listing everything other users have shared with you, backed by
puter.fs.listShared().

The path is the sentinel `puter://shared` rather than /<user>/Shared: this is a
query, not a directory, and a path-shaped value could collide with a folder
someone actually creates. refresh_item_container and update_window_path both
branch on it to skip the stat there is no fsentry for, and the listing swaps
readdir for listShared.

Entries render at their real paths under their owners' directories — the item
container already preferred an explicit fsentry.path over joining onto the
container, so nothing else had to change. Each carries who shared it and at
what level, which the context menu reads next.

* feat(gui): share items from the context menu

A sharing dialog shaped like its neighbours — options object, HTML-string
template, jQuery wiring, delegating to UIWindow() — with a recipient field, a
read/edit/share dropdown, and the current access list with revoke buttons.

Reached from a new "Share…" context menu entry, which is hidden on items shared
*with* you: re-sharing needs manage, so the dialog would only surface an error.

Those items get "Remove from Shared" in place of Delete. Delete moves an item
to *your* trash, which for someone else's file means moving their data out of
their tree — FSService refuses it, and the user saw a bare 403. Removing your
own access is what the action was reaching for, so that is what it now does.

* fix(share): withdraw what a removed recipient re-shared

* feat(share): report access inherited from a parent folder

* fix(gui): load the puter.js bundle the server configured

* refactor(gui): extract the action icon set into a helper

* feat(gui): surface Shared in the file browser

* feat(gui): manage access from the share dialog

* test(share): cover access inherited from a parent folder

* fix(share): keep downstream access from surviving a delegate who leaves

* fix(gui): name the real owner in the share dialog

* fix(gui): page through every shared item instead of the first 50

* feat(share): return item metadata with a share

* fix(share): invalidate a holder's cache when the entry is deleted

* fix(gui): treat items inside a shared folder as someone else's

* feat(permissions): let manage inherit down the filesystem tree

Access already reached descendants through the ancestor chain while authority did not, so someone trusted to manage a shared folder could re-share the folder but nothing inside it, and could not see who had access to a file within it.

A manage-inherits-from-ancestor implicator resolves it in the permission layer, beside is-owner, so every caller agrees rather than just ShareService. It consults only the immediate parent — resolving that re-enters one level up, making a chain of depth d cost d checks rather than d².

That makes two cascade gaps reachable, both fixed here. A revoke now walks the subtree, since a grant on a descendant can rest on authority held at the folder. And it stops at a delegate whose authority survives another issuer, because what they granted was never theirs to lose.

Also pins that manage is not transitive: granting it needs manage:manage:fs:<uid>, which only the owner holds, so delegation is one level deep by construction.

* fix(gui): offer sharing inside a folder you manage

The menus encoded "manage does not inherit" and would now hide an action that works. The Shared listing records each root's mode; the menus resolve a child's by longest matching ancestor, loading on demand so a deep link or restored window works too.

* fix(share): make the daily share limit hold under concurrency

* test(share): cover concurrency, measure cost, and name cases for what they verify

* fix(gui): import the ownership helpers the item menu calls

The single-item context menu handler calls is_owned_by_me and
shared_mode_for, but the imports were only ever added to
generate_file_context_menu.js — so every right-click on an item threw a
ReferenceError before the menu could build, and the non-owner Delete
gating never ran.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): authorize before resolving the recipient

share() looked up the recipient in parallel with the entry, before the
manage check — and the two failures carried different error codes. Any
verified user with a real entry uid could probe arbitrary emails and
usernames for account existence, at no quota cost. Resolve the entry,
authorize, and only then resolve the recipient: an unauthorized caller
now sees the identical safe 404 whether or not the recipient exists.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): broadcast permission row-cache invalidations to peer regions

Every publishCacheKeys call for the u2u, u2a, and access-token row
caches omitted broadcast, so a revoke only cleared the mutating
region's Redis. A peer region applied the replicated generation bump,
re-scanned, read the deleted row from its own still-warm 5-minute row
cache, and re-warmed the flat view from it — revoked access outlived
the revoke by the row-cache TTL instead of the intended 60-second
bound. CacheReplicationService already consumes these events; the
emits were just never sent.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(fs): refuse to rename an entry owned by another user

remove and move both refuse to act on an entry the caller does not
own, even when the ACL allows the write — rename had no such guard, so
a write-mode share recipient could rename the owner's file, or the
shared folder itself, rewriting the owner's whole subtree's paths.
rename now takes the acting user and applies the same policy.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): decide a flat delete from the primary, not a lagging replica

revokeUserUserPermission deletes the SQL grant, then only drops the
flat KV entry once no issuer still grants the permission. That
remaining-check read through the row cache the delete had just
invalidated, straight to a replica — under any lag the deleted row
reappeared, the flat delete was skipped, and the stale rows were
re-cached for another five minutes. Grant-path flat entries carry no
TTL, so the holder kept working access with zero SQL rows behind it,
invisible to every listing. The check now reads the primary and
re-warms the cache with what it actually saw.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): keep a failed remote flat-invalidation from crashing the process

The outer.permission.flatInvalidated applier was fire-and-forget with
no catch, and it awaits a KV delete — one transient KV error while
applying a peer region's revoke became an unhandled rejection, which
is process-fatal under default Node. Its sibling appliers were already
guarded; this one now logs and moves on, leaving the entry to the next
invalidation or its TTL, same as a lost event.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): revoke every requested item and recipient, not just the first

revokeShare destructured only the first recipient and first item while
the parsers accept arrays up to the request caps — unshare({items:
[a, b, c]}) returned success having revoked only a, leaving access the
caller believes is gone. Revoke now fans out over every (recipient,
item) pair exactly like POST /share, reports per-pair outcomes, and
sums the revoked count; the response stays backward compatible.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): only a confirmed email designates a recipient

Recipient resolution by email accepted unconfirmed accounts, so
pre-registering someone else's address (unconfirmed) was enough to
receive shares meant for them once no confirmed account held it.
An email now only resolves to an account that has confirmed it;
username shares are unaffected.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): accept tilde-rooted paths like the FS routes do

The SDK resolves relative paths to ~/..., but the share routes never
expanded the tilde — a ~-prefixed string was read as a uid and every
relative-path call 404'd. Item parsing now treats ~ as path-shaped and
expands it to the actor's home with the same helper the legacy FS
routes use, on share, revoke, and the shares listing.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): walk a directory revoke by parent linkage, not path prefix

listByFsentrySubtree matched descendants with fsentry_id = ? OR path
LIKE ?, which has two problems: fsentries.path is lazily backfilled
and NULL on old rows, so those descendants' shares silently survived a
directory revoke, and the OR'd predicates forced a scan of every
active share. A recursive CTE over parent_id — the same shape the
lineage resolver already uses — covers every descendant and runs on
idx_parentId_name.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(gui): give each item its own share dialog

single_instance keyed the dialog on the app id alone, so opening
Share… on a second file focused the first file's dialog — typing a
recipient there granted access to the wrong file, with only the title
hinting at it. The dialog is now instanced per path: same item
refocuses, different item opens fresh. Also stops pre-encoding the
title, which UIWindow encodes again.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(permissions): let manage answer a write check

A manage grant let its holder re-share a folder but not work in it: the
ACL mode family stops at write, and the fs exploder had no rule for the
narrowest mode, so `manage:fs:<uid>` never satisfied `fs:<uid>:write`.

Fold manage into the candidate list for every non-manage mode, in both
the access-token branch and the scan branch, and give `write` an (empty)
exploder rule so the manage arm is emitted for it too.

* fix(fs): authorize restructuring by write on the parent

* fix(fs): let a share recipient work inside a shared folder

rename, remove and move refused outright when the entry belonged to
someone else, so a recipient with write could neither delete nor rename
anything inside a folder shared with them. The GUI compounded it by
hiding Delete for any item it did not own.

Authorize the three by ACL write on the entry's parent. For an owner
that is the same answer; for a recipient it grants the inside of a
shared folder and withholds the folder itself, whose parent is the
owner's private tree.

Deleting sends the item to its owner's trash rather than the deleter's,
so it leaves the recipient's view without leaving the owner's account
and without changing hands. A move may not otherwise carry someone
else's entry out of their tree.

* fix(fs): give a new entry to the owner of the folder it lands in

A file a share recipient added to a shared folder was recorded as
theirs while living in the owner's tree, so a subtree could hold rows
belonging to several people — and the storage it consumed was checked
against the writer while being counted against the owner.

Take the owner from the parent row at every insert, charge the
allowance to that owner, and hand a moved entry over to the tree it
moves into. An entry now always belongs to whoever owns the directory
holding it.

* feat(fs): address shared entries as ~/share/<uid>

A recipient could read the owner's whole path off any shared entry —
where they keep the file and what sits beside it, neither of which the
share is about.

Give shares their own namespace. `~/share/<entry-uid>/rel/path` resolves
to the real path on the way in, and outgoing paths are rewritten to it
on the way out. Entries the actor owns pass through untouched, so no
existing client contract moves.

* revert(fs): mask only the directory bar, not the addressing

fe535d598 made `~/share/<uid>` the actual address for every shared
entry. That reached far past the intent: item names became uuids, the
Shared views rendered uuids instead of filenames, and navigation
addressed entries through a namespace nothing else understood.

Put real paths back everywhere — responses, the share listing, and
request handling — and do the masking where it was wanted, in the
window's directory bar. A recipient sees `Shared › Contents › sub`
while every crumb keeps the real path it navigates to.

The share listing now carries the entry's name, content type, owner and
a signed thumbnail. A share row has no fsentry behind it for a client
to stat, and the stored thumbnail is an `s3://bucket/key` URI that no
client can render and none should see.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix: path obfuscation, webdav + small ui stuff

* test(share): assert the masked share path by its exact shape

The substring check tripped on the scratch files' own names, which start
with `sharing-`; the exact-equality assertion on `/<owner>/<uid>/<name>`
already proves nothing above the share leaks.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): rename policy, shared-view guards, webdav share parent, quota/bucket invariants

- Rename: a directly-shared FILE renames with write on it; a shared
  folder root stays fixed (its name is the owner's tree structure).
  GUI can_rename mirrors the backend, guards all editor entry points.
- Up from a share root goes to the Shared view on both surfaces; the
  Shared view gets a single crumb, a disabled Up button, and refuses
  drops, New/Paste, uploads and ctrl+V everywhere (it is a query, not
  a directory).
- WebDAV: PROPFIND on /owner/uuid answers as a virtual collection
  holding the share root (ACL-gated; 404 for strangers).
- Storage allowance override no longer crosses user boundaries: a
  recipient's plan cannot raise the owner's cap.
- Overwrites stay in the bucket the entry already lives in instead of
  repointing to the handling server's bucket and stranding the old
  object.
- manage mode documented as implying write (matches enforcement);
  share dialog label now "Can edit & share". Documented that fs socket
  events are owner-only.
- Sidebar: saved orders gain the Shared entry once the user has shares.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): close review findings, and retire /auth/grant-user-user

Security
- Recipient writes no longer echo the owner's real path back. The pending-write
  event and the upload-progress meta both go to the *acting* user, so a write
  into a shared folder handed out the layout above the share root.
- `maskerFor` failed open: the first caller fixed the actor, and one that ran
  before the request knew who was acting pinned `undefined` — after which every
  path published unmasked. It now adopts the first real actor and rebuilds for
  a different one.
- `/auth/grant-user-user` returns 501. It wrote user-to-user grants straight to
  the permission tables with no share row, so nothing could list or cascade a
  revoke over them — and the new `manage` mode meant a delegate could reach it
  for the owner's files. Filesystem access goes through `/share`; nothing else
  is meant to pass between two users. Undocumented (its docs page was never
  written) and no callers. Revoking is untouched, so grants made before this
  can still be withdrawn.
- `#assertCanManage` asks about the permission actually being granted rather
  than a fixed `read`, so sharing at `manage` is refused at the share layer
  instead of by `grantUserUserPermission` two levels down.
- `listSharedWithMe` checks the grants, not just the index: withdrawing access
  any other way left the row publishing name, size and a signed thumbnail URL.
- Moving an item into another tree now retires its shares. Grants are keyed on
  uuid, so they followed it and left the new owner with recipients they never
  agreed to.

Correctness
- `acl.check` gets the real path again, not the masked one. ACL matches on the
  path string, and a mask hides the `AppData/<appUid>` shape it needs.
- "Leave this share" works for a grant that predates the index; the fallback
  scoped the delete to the caller as issuer, which can never match.
- `move`/`copy` reject a name with a slash or a `.`/`..` segment, as `rename`
  already did. It matters more here: a move into the owner's Trash skips the
  destination write check.
- Descendant walks scope by path, not by owner, so rows predating the
  one-owner-per-subtree invariant aren't orphaned by their parent's deletion.

Performance
- Index `user_to_user_permissions(permission)`. Retiring a node's grants is a
  prefix match, but the primary key is (issuer, holder, permission), so it was
  a full scan.
- Retirement is coalesced and chunked. `remove()` emits one event per
  descendant, so deleting a directory fired one unindexed lookup per file, all
  at once and unawaited.
- `listReaching` is a plain `fsentry_id IN (...)` on `idx_share_fsentry`. It
  runs behind every file write, and the join-plus-OR it replaced was
  unindexable.

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-08-17 09:08:23 -04:00
Ayush tiwari fc861da89f feat: add Gemini 3.7 Flash (#3587) 2026-08-16 23:51:37 -07:00
Parman Mohammadalizadeh facb844747 fix: derive express subdomain offset from the configured root domain (#3575)
Express derives `req.subdomains` by dropping a fixed number of labels
from the right of the hostname, defaulting to 2. A deployment whose
`domain` has more than two labels therefore reads its own root domain as
an active subdomain, so the user-site redirect sends the root origin to
the static hosting domain, which sends it back. Self-hosting docs
recommend exactly that shape (`puter.example.com`).

Set the offset from the label count of `config.domain`. Two-label
domains keep the express default, so existing deployments are unchanged.

Fixes #3561
2026-08-16 23:48:20 -07:00
Yashwarhade8010 3b791bb334 docs: update web search examples to gpt-5.6-luna (#3589) 2026-08-16 23:45:02 -07:00
Daniel Salazar a64a444a17 feat: batch delete for flush + metering fixes (#3593) 2026-08-16 15:26:33 -07:00
Daniel Salazar d3c9828207 fix: metering gui maths (#3592) 2026-08-16 14:57:29 -07:00
Daniel Salazar 87180b1081 fix: metering (#3591) 2026-08-16 14:19:05 -07:00
Daniel Salazar d22b656df8 fix: metering issues with addons (#3588) 2026-08-15 22:06:51 -07:00
Nariman Jelveh 19cd5f4476 feat: add BytePlus ModelArk providers (chat, image, video) (#3498)
* feat: add BytePlus ModelArk chat provider

Adds BytePlus ModelArk as a provider for the puter-chat-completion
driver, following the MiniMax/ZAI providers as reference per
doc/contributing-apis.md.

- OpenAI-compatible endpoint at ark.ap-southeast.bytepluses.com/api/v3
  (apiBaseUrl config selects the region)
- Static catalog of 16 chat models (Seed 2.x/1.x incl. vision, GLM,
  DeepSeek, GPT-OSS) with limits and per-token pricing from the
  official docs
- Passes Ark's thinking/response_format/stop params through custom;
  normalizes reasoning_content to reasoning
- Bare deepseek-v4-* names stay with the first-party DeepSeek
  provider; BytePlus only claims prefixed aliases
- Offline unit tests (mocked SDK against a real test server) plus an
  env-gated integration test

* feat: add BytePlus image and video providers

Extends the BytePlus ModelArk integration to the puter-image-generation
and puter-video-generation drivers, reusing the same services.byteplus
API key and regional apiBaseUrl as the chat provider.

Image (Seedream/SeedEdit via OpenAI-compatible /images/generations):
- dola-seedream-5-0-pro (pixel-tier pricing + billed input images from
  the 2nd on), seedream-5-0-lite, 4-5, 4-0, and seededit-3-0-i2i
- quality tiers 1K/1.5K/2K; aspect ratios resolve to Ark's documented
  pixel sizes; explicit WxH passes through with Ark's bounds enforced

Video (Seedance via Ark's async /contents/generations/tasks + polling):
- Seedance 2.0 / 2.0 Fast / 2.0 Mini / 1.5 Pro / 1.0 Pro / 1.0 Pro Fast
  (2.5 is priced but its API isn't live yet, so it's excluded)
- per-video-token billing from usage.completion_tokens, with per-second
  estimates feeding the credit cap; audio vs silent rates for 1.5 Pro
- first/last frame and reference-image inputs; generate_audio param
  added to IGenerateVideoParams

Pricing and capabilities hardcoded from the official docs (ModelArk
pages 1544106, 1330310, 1520757, 1521309, 1541523). Offline unit tests
mock the SDK / global fetch; integration tests are env-gated on
PUTER_TEST_AI_BYTEPLUS_API_KEY.

* fix: correct BytePlus catalogs and validation against the live API

Verified the three BytePlus providers against ModelArk with a real key;
these are the mismatches that surfaced.

- Drop seededit-3-0-i2i-250628. Ark reports it as Shutdown and every
  request 404s. Its now-unreachable image-to-image branches in the
  provider go with it.
- seedream-4-5 and the 5.0 series enforce a 3,686,400 pixel minimum, so
  they only accept the 2K tier. Mark them 2k-only and snap an
  unsupported tier up to the nearest allowed one, which also keeps the
  aspect-ratio table from mapping to a sub-minimum size.
- glm-4-7 has a 204,800 token context, not 256K.
- Guard the actor in the image provider like the video provider does.
- Round a sub-minimum video duration up to the shortest supported clip
  instead of reporting it as insufficient funds.
- Gate video resolution on the model's own dimensions; the dims table is
  shared across a family and accepts more than any one model does.

* Tighten BytePlus AI provider handling

Extract shared reasoning-content normalization for OpenAI-style chat providers, and harden BytePlus image/video behavior. This updates image tier and size validation, normalizes aspect ratios and input image refs, prevents mismatched BytePlus key/base URL fallback config, makes video resolution matching case-insensitive, and rejects excess reference images instead of silently truncating them. Tests were expanded to cover the new BytePlus request and validation paths.
2026-08-15 21:08:17 -07:00
404oops fc56075115 Merge pull request #3584 from rupesh0001-tech/rupesh
feat(ai-chat): add grok-4.6 model to xai provider
2026-08-15 21:22:01 +02:00
404oops e1998e39ab Increase max_tokens limit to 500,000 2026-08-15 21:20:45 +02:00
Daniel Salazar fb7968a1c7 fix: metering hardening; handle burst of unfinished ai requests (#3585) 2026-08-15 10:52:29 -07:00
rupesh0001-tech c88eeb3868 feat(ai-chat): add grok-4.6 model to xai provider
- Add grok-4.6 model definition with pricing and context specs
- Add aliases x-ai/grok-4.6 and grok-4.6-latest
- Add unit tests for alias resolution and metering calculation

Closes #3562
2026-08-15 20:32:22 +05:30
Reynaldi ChernandoandCopilot Autofix powered by AI 9e25ce1401 [PUT-1478] Add KV commands to CLI (#3579)
* Add KV commands to CLI

* docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-08-15 11:43:30 +07:00
Daniel Salazar f15d835eeb fix: restrict openai and anthropic compatible endpoints to be subscription (#3583) 2026-08-14 18:55:15 -07:00
Daniel SalazarandClaude Fable 5 27c4506e02 feat: usage rendered as credits on a 50c base, free allowance doubled (#3569)
* feat: usage rendered as credits on a 50c base, free allowance doubled

Usage is now displayed in credits: a configurable creditsPerDollar rate
(default 2,000, making the free tier's 50c allowance an even
1,000-credit base) ships with the metering usage response, and the
dashboard's usage cards and per-API table show raw credit numbers
through one shared formatter. The dashboard budget math is also fixed —
capacity is spend plus server-netted remainder, so held purchased
credit can never render as negative usage. The registered-user free
allowance doubles to 50c, tier display names move to Basic/Plus/Pro,
and dead referral-promise strings leave the English dictionary.

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: the credits display rate exists only in deployment config

No default in code and no rate on any wire the plan surfaces read: a
deployment that doesn't configure creditsPerDollar renders usage in
dollars, exactly as before credits existed. The usage endpoint sends
the rate only when configured, and nothing in code or comments states
what any deployment's rate is.

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: metering reports pre-scaled credits, never raw amounts

The usage, per-app usage, and cost-catalogue endpoints multiply every
monetary field by the config multiplier (renamed creditMultiplier)
before responding, and flag the unit; the multiplier itself never
ships. Counts, units, and byte figures pass through untouched. With no
multiplier configured the endpoints report raw amounts and clients
render dollars, as before credits existed.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: absent addon fields stay absent when scaling to credits

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 16:29:01 -07:00
Neal Shah 6a7c09f54e blacklist dynamic workers (#3582) 2026-08-14 18:35:34 -04:00
Reynaldi Chernando 6254714484 deprecate gemini 2 flash and lite (#3581)
* deprecate gemini 2 flash and lite

* fix test
2026-08-14 17:27:48 -04:00
Neal Shah efa2cb4441 add more extensibility to workers driver (#3565) 2026-08-14 17:24:36 -04:00
Nariman Jelveh 4bc969b99e Update RecommendedAppsService.ts 2026-08-14 14:05:44 -07:00
Reynaldi Chernando 65f6d38432 update gpt 5.6 luna and terra pricing (#3580) 2026-08-14 16:38:13 -04:00
Daniel SalazarandClaude Fable 5 d641f6f7a1 tests: count only this test's dynamo reads in the block-window assertion (#3576)
The spy sits on the test server's shared dynamo client, so a raw call
count also picks up background work and the async tail of earlier tests
in the file — which made the block-window test flakily report a third
read. Each test runs in its own random namespace, so filtering the
spy's calls to that namespace removes the cross-talk without loosening
the assertion.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 11:05:25 -07:00
Daniel SalazarandClaude Fable 5 00828b9998 fix: concurrency limiter heals legacy INCR keys instead of failing open (#3574)
The zset rework of the redis concurrent backend (#3529) kept the same
key names the old INCR counter used. Against a leftover string key every
zset command in the acquire MULTI fails WRONGTYPE while the trailing
EXPIRE still succeeds — so steady traffic refreshes the stale key
forever, and the failed zcard result turned into NaN, which admitted
every caller unbounded. Release then errored WRONGTYPE on each request
(the '[concurrent] release failed' log storm).

Surface per-command MULTI errors instead of coercing them to NaN, and on
WRONGTYPE delete the legacy key and re-acquire against a clean zset.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 01:40:05 -07:00
Daniel SalazarandClaude Fable 5 126ec09556 docs: publish rate limits and quotas, and make limit changes move the docs (#3568)
Developer-facing reference for every rate limit, concurrency cap,
quota, and error shape — an advanced page, since puter.js already turns
the common failures into prompts. AGENTS.md and CONTRIBUTING.md now
carry the rule that a PR moving any of these numbers updates the page
in the same PR: an undisclosed limit is one developers discover as a
service failure.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 00:41:01 -07:00
jelveh 25a5799088 Update RecommendedAppsService.ts 2026-08-14 00:38:05 -07:00
Daniel SalazarandClaude Fable 5 ea2f9967a6 fix: over-quota batch uploads surface the storage prompt instead of failing quietly (#3566)
* fix: over-quota batch uploads surface the storage prompt instead of failing quietly

Partial batch failures now carry each item's code/status, and when every
failed item failed the same way the shared code/status is hoisted onto
the rejection itself — so an upload that exceeds the storage quota
rejects with storage_limit_reached/413 and the SDK's upload handler
shows the free-up-space prompt. A partial failure is also no longer
misread as the signed-batch endpoint being unavailable.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: out-of-storage copies prompt to upgrade the same way uploads do

Any fs operation the server refuses with 413 storage_limit_reached now
surfaces the upgrade prompt — the check that lived inline in upload's
error handler moves to a shared helper wired into the operation
scaffold's reject path, so copy/move/mkdir/rename get it too. The
desktop's copy/paste suppresses its generic alert for that code, since
the SDK dialog already explains the refusal and carries the fix.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 00:26:07 -07:00
Daniel Salazar ba9b885a5a Revert "feat: dashboard app search continues into the App Center (#3567)" (#3572)
This reverts commit 6cc241ac31.
2026-08-13 21:00:02 -07:00
Daniel SalazarandClaude Fable 5 6cc241ac31 feat: dashboard app search continues into the App Center (#3567)
A search that matches none of the user's own apps now puts the same
query to the App Center catalogue and renders the hits under the empty
state — debounced, filtered to apps the user doesn't already have, and
launchable in place.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-13 20:48:03 -07:00
Nariman Jelveh 7f8e24f06d fix: keep "Add an app" in the grid while searching
Typing in the Apps tab's search dropped the tail tile — deliberately, on the
grounds that a query asks which apps you HAVE and a tile matching nothing has
no business in the answer. That reads the search too narrowly: looking for an
app you don't have is the same gesture as looking for one you do, and the
moment the search comes up short is exactly when the way to GET an app is
worth having on screen.

So the tile is pushed unconditionally now, and a query that matches nothing no
longer replaces the grid with a bare "No apps match your search" — the line
moves above a grid holding just that tile. Keeping it a real grid, rather than
parking a tile inside the empty state, is also what keeps it reachable: arrows
and Enter only walk tiles inside .myapps-page, and the tail tile carries
tabindex -1 for the pager's roving focus.

The no-apps-at-all state is untouched: with nothing installed and nothing
typed, the empty state still stands in for the whole grid.
2026-08-13 18:53:59 -07:00
Nariman Jelveh 9b6627b1ea fix: an app's background helper must stay private to it
`launchApp(name, args, { background: true })` hid the child's window and
nothing else, so the instance was hidden but not private: it kept its
taskbar item, lit the running dot on its dashboard tile, and — because
the dashboard enforces one instance per app — WAS what clicking that tile
opened. Minimize the parent, click the child's tile, and Puter handed the
user the window the parent was mid-conversation with; the app, which only
ever meant that instance for its caller, said so on screen. Showing it
also dropped the never-shown marker, so from then on the helper outlived
its launcher and owned the tile.

Hidden now means unreachable. `user_facing_windows` is the one rule: of
an app's windows, the ones that are the user's. Everything the user
drives filters through it — the tile's reopen and its running dot and
Quit item, the taskbar item's click, window list and show/hide/close-all,
the file row's switcher and open-file dot, and the dashboard's
Back/Forward — so a helper is never focused, restored, listed, or counted
as running. With none of the user's own left, the tile simply launches a
fresh instance and the helper keeps serving whoever launched it. Paths
that act on the APP rather than on a window it happens to have open —
uninstall, and a launcher's close taking its helpers down with it —
deliberately keep the unfiltered list.

The taskbar item is now what a hidden launch EARNS by becoming visible,
rather than something it advertises from the start; makeWindowVisible
adds it (and fires the dots event) before showing, since focusWindow
marks an item active and needs it to exist. Windows stamp data-in_taskbar
when they take their place in data-open-windows, so the close path only
decrements a count it actually joined — otherwise a helper closing beside
the user's own window would zero the count and strip the item off a
window still open. For the same reason a helper's close no longer pops
the dashboard URL: it never claimed an entry (the push only happens for a
window created visible), and consuming one would minimize the user's
window of the same app.

is_unseen_background_window gains the other half of its own definition:
launched hidden AND launched BY another app. An app that is always
windowless (`background` on the app record) starts hidden too, but when
the user opens it there is nobody it is serving — it is the instance
their tile has to find, or every click would start another one they
cannot see.
2026-08-13 18:38:13 -07:00
Nariman Jelveh 7576e0b03a fix: route dashboard file menus by input modality, not touch capability
A mouse right-click on a file row in the dashboard Files tab opened the
mobile context-menu sheet (centered over the row, nowhere near the
cursor) on any touch-capable laptop or desktop, because phone/tablet
detection used navigator.maxTouchPoints > 0. The same check applied the
.touch-device class, degrading mouse selection and drag on those
machines.

Route by how the menu was invoked (the row's tracked pointer type) plus
isTouchPrimaryDevice() — coarse pointer and no hover — which is false on
touch-capable laptops but true on iPads whose UA claims macOS. Also let
sidebar-folder and background menus accept taphold on such iPads, where
they were previously unreachable (taphold dismissed, and iOS never fires
native contextmenu).
2026-08-13 17:46:58 -07:00
Nariman Jelveh d980a9f590 fix: don't invert full-color icons in the dashboard context menu 2026-08-13 17:33:14 -07:00
Nariman Jelveh d18ea1adb5 fix: a background app must not outlive the app that launched it
An app launched with `background: true` gets a real window from the moment it
starts, just hidden. Nothing ever took it down: when the app that launched it
closed, the child kept running with no way to reach it and no reason to be
there. On the dashboard the only sign was a running dot on a tile the user had
never lit up, and clicking that tile did nothing at all.

Three things were wrong, and all three had to go:

A window nobody has seen now dies with its launcher. UIWindow's close path
closes hidden children the closing app launched, keyed on a marker stamped at
creation. makeWindowVisible drops that marker the first time the window becomes
visible — showing itself with `puter.ui.showWindow()`, or the user showing it —
because from then on the window is the user's, and keeps running.

The dashboard tile is a real handle again. focusExistingAppWindow only routed
MINIMIZED windows through showWindow(); a hidden one fell through to
focusWindow(), which leaves it invisible while handing it the keyboard. With no
taskbar in dashboard mode the tile is the only handle on a background app, so
that click had nowhere else to go. It now asks whether the window is on screen
at all. The Files tab's row-click had the same one-line defect.

And a background instance can't take a tile from the user's own session: an
on-screen window wins, then a window the user has seen, then a hidden one.

Removing the child then hit a crash of its own: ExecService's `remove` handler
dereferenced the launcher's iframe to say goodbye, and the launcher was already
gone. Throwing there aborts jQuery's remove() itself, so the window stayed in
the DOM — running dot and all. That one also hit anyone closing a background app
from the taskbar after its launcher had closed.

The predicates behind all of this live in one helper, window_visibility.js, with
showWindow() reading the same `hidden, not minimized` rule it spelled out inline
before.
2026-08-13 14:36:08 -07:00
Nariman Jelveh 08f075d774 Update RecommendedAppsService.ts 2026-08-13 11:45:28 -07:00
Nariman Jelveh 664de0016b fix: a hidden window must not hold the keyboard
Launching an app to serve another one took focus away from the app the user
was working in, and they had to click it to get it back. Typing a guest name
into Calendar's event editor — which launches Contacts to read the address
book — dropped the user out of the field mid-word.

Four things conspired, all of them variations on "a window nobody can see is
treated as the window in front":

`window-active` was hardcoded into every window's markup, including windows
created hidden. focusWindow() — the thing that normally grants that class,
and strips it from everyone else — is already skipped for a hidden window, so
two windows ended up claiming it. It now follows the same rule as the
window_stack push a few lines above: visible windows only.

Both "focus the window once its IPC attaches" sites read that class to decide,
so a hidden child window pulled focus a beat after launching. Neither focuses
a window with data-is_visible="0" any more.

makeWindowInvisible left `window-active` on the window it had just hidden.
Since focusWindow() disables pointer events on every other app's iframe, an
app calling puter.ui.hideWindow() on itself left whoever launched it both
unfocused AND unclickable until the next click. It now hands activation back
to the top of the window stack, the way closing a window does, and drops
itself out of the activation order until it is shown again.

showWindow only ever un-minimized: for a window hidden by hideWindow() (which
keeps its geometry and writes no data-orig-*) every branch left it hidden and
stamped NaN geometry on it from the missing attributes. It now routes such a
window to makeWindowVisible. That is what makes the taskbar item a real
handle on a background app — clicking the item calls showWindow on the group.
2026-08-13 09:59:37 -07:00
Nariman Jelveh e273431f14 feat: let an app launch another app in the background
`puter.ui.launchApp(name, args)` had no way to say "I need this app's API,
not its window". That matters because we create and show an app's window
before the app's own code runs, so an app launched purely to serve another
one cannot avoid appearing on screen: the best it can do is call
`puter.ui.hideWindow()` once it boots, which reads as a window flashing
open and shut. In dashboard mode it was worse than a flash — the child
maximized into the tab and minimized its parent behind it, so asking a
service app a question took the user's app away from them.

So `launchApp` now accepts `background: true`, and the window starts
hidden. The app is otherwise entirely normal: it keeps its taskbar item,
so a user can see that it is running, show it, or close it, and it can
show itself with `puter.ui.showWindow()` whenever it has something to say.
Only a literal `true` counts, since the flag arrives over IPC from another
app.

The decision now lives in one predicate, `starts_hidden(app_info, options)`,
which folds this together with the existing app-level `background` flag and
is used everywhere the old flag was read — including the dashboard's
minimize-the-parent branch. `show_in_taskbar` deliberately still keys on the
app-level flag alone: an app that is always windowless has nothing to put in
the taskbar, while a background *launch* should stay visible there.

Existing callers are unaffected: with `background` unset, both paths
evaluate exactly as they did.
2026-08-13 09:47:35 -07:00
Daniel Salazar 8ed8feed4b fix: don't let an aborted upload take down the process (#3557) 2026-08-13 01:02:17 -07:00
Daniel Salazar 22f5bf5429 fix: duplicate emails (#3556) 2026-08-12 22:00:48 -07:00
Nariman Jelveh 198184f986 fix: drop the feedback dialog's rate limit entirely
Reopening still failed intermittently: any call the backoff refused looks
exactly like the user closing the dialog, so an app's "Send feedback"
button just did nothing, with nothing to see anywhere.

Rate limiting this dialog is not worth that. The other modals an app can
open with no user gesture — requestPermission, alert, prompt — have no
rate limit either, so one dialog at a time is the only rule left, and a
refusal now says so in the console.
2026-08-12 17:46:05 -07:00
Nariman Jelveh 026ae46530 fix: let an app reopen the feedback dialog after the user closes it
The abuse guard counted every dismissal that sent nothing as a strike
against the app, so closing the dialog blocked the app's next
showFeedbackDialog() for 10s, then 60s, then for the rest of the page's
life. The key is the app uuid, so relaunching the app did not clear it,
and a blocked call replies sent:false — the app just sees nothing happen.

Key the guard off how fast the app comes back instead. A reopen within a
second of the app's last dialog activity is machine-paced and gets the
same escalating backoff; attempts made while backed off count as activity
too, so a loop cannot wait out a tier and start from zero strikes. Any
human-paced reopen clears the app's record, so the tiers are only reached
by an app that reopens at machine speed several times running.

One dialog at a time, and a successful send clearing the record, are
unchanged.
2026-08-12 17:35:54 -07:00
Nariman Jelveh 0615c3aa3c feat: add "Show hidden" to the dashboard's folder context menu
The dashboard's files tab filtered out dot-files unconditionally, so
there was no way to reach them the way the desktop Explorer allows.
Add the same "Show hidden" toggle next to Refresh, driven by the
shared show_hidden_files preference so the two stay in sync, and dim
revealed rows with item-revealed like the Explorer does.

The socket-driven item.added path applies the same rule, so a hidden
file created elsewhere can't appear in a view that filtered its
siblings out.
2026-08-12 17:26:04 -07:00
Nariman Jelveh 748461aabb fix: show the new folder's row in the dashboard before mkdir answers (#3555)
* fix: show the new folder's row in the dashboard before mkdir answers

Both New Folder paths in the dashboard Files tab awaited mkdir before
drawing anything, so on a slow connection the button appeared to do
nothing for seconds. The toolbar path then did a full renderDirectory on
top of that -- a second round-trip, and with the default eventual
consistency the new folder could miss the listing entirely, leaving the
rename editor unopened.

Both now go through one createFolderInstant(), which renders the row
first: a locally predicted name (mirroring the backend's " (N)" dedupe
convention), a temporary uid, and the name editor already open. mkdir is
still asked for the plain "New Folder" path with rename: true, so the
server keeps owning deduplication; the prediction is only what we draw in
the meantime, and the row corrects itself if the two disagree. On failure
the row is withdrawn and the error is shown rather than swallowed.

Renaming is the next thing the user does and it needs the real uid, so
rename() awaits the create promise parked on the row. Everything else
that acts on the item -- open, menus, drag -- sits out while the row is
pending. mkdir's item.added comes back to the originating client (the
event carries no original_client_socket_id) and cannot match a temporary
uid, so _creatingItem still suppresses it, now as a counter: as a flag,
one create finishing uncovered another still in flight.

Two things found on the way:

- directory_depth_limit_exceeded had no translation key, so that alert
  (including the pre-existing desktop create_folder path) rendered the
  raw slug.
- The empty-directory notice is now restorable without refetching, since
  a withdrawn create has to put it back. It carries its own class because
  the loading overlay shares the same container.

Verified end to end against a dev backend with mkdir slowed to 3s:
toolbar and context-menu paths, rename and Escape before the response,
two overlapping creates, and a rejected mkdir.

* fix: don't draw the instant new-folder row while a listing is rebuilding

renderDirectory() clears .files before its readdir resolves, so a row
drawn during a load survives the clear and is left behind in whatever
directory the load lands on. Clicking the new-folder button while a
folder is still opening produced a row for `<old dir>/New Folder` sitting
in the new directory's listing, with the rename editor open on it —
clicking it navigated elsewhere and renaming it renamed a folder the user
wasn't looking at.

Drop the click while a rebuild is in flight, the same way renderDirectory
already drops navigation clicks that arrive while it renders.

* fix: stop drops onto a not-yet-created folder row from mangling files

A row drawn ahead of its mkdir carries a predicted path, but it was still
registered as a live drop target. Dropping a file on it called
move_items() with a destination that does not exist yet, and move treats
a non-existent destination as a rename target — so the dragged file was
silently renamed to "New Folder" instead of moving into the folder.
Reproduced against a slow mkdir: victim.txt became a 5-byte file named
"New Folder".

Sit the row out of the jQuery UI droppable (drop and the spring-load
hover) until the real fsentry lands, matching the guards already on
opening it, its menus and dragging it. The native-file dragster drop gets
the same guard: it uploads into the same predicted path.
2026-08-12 17:16:44 -07:00
Daniel Salazar 78cfe0c0f6 metrics: add metering buffer store coutns (#3554) 2026-08-12 15:55:03 -07:00
Nariman Jelveh 81f9f00fc0 fix: stop app feedback dialog clicks from focusing the iframe beneath
Clicking inside the feedback overlay (e.g. the textarea) reached
initgui's global mousedown -> focusWindow path: mouseover_window is
computed geometrically and is blind to the overlay, so focusWindow
focused the app window's iframe underneath — stealing keyboard focus
from the textarea and forwarding the click into the app.

Suppress it the established way (same fix as the dashboard app-group
panel): set window.mouseover_window = undefined in the overlay's own
mousedown handler, which runs before the document-level one; undefined
is the only value initgui's guard skips.
2026-08-12 15:00:32 -07:00
Nariman Jelveh 556380cf16 fix: keep app feedback send button blue on hover
The generic .app-feedback-btn:hover:not(:disabled) rule out-ranked the
.app-feedback-btn-primary:hover override (the :disabled argument inside
:not() counts toward specificity), so hovering Send repainted it with
the light --afb-hover background under white text. Strengthen the
primary hover selector so the blue gradient wins.
2026-08-12 13:42:11 -07:00
Daniel Salazar 908ec23eda fix: sanitize user data (#3552) 2026-08-12 12:08:54 -07:00
Juan Fernando Castro d5ae5a0049 🔧 PUR-1072: Flatten driver permissions to hardcoded values (#3545)
* refactor(permissions): drop hardcoded group permission map for a flat default

* test(drivers): assert credential-gate intent instead of a 403 proxy
2026-08-12 12:08:38 -07:00
Daniel Salazar 419d0aaa89 doc: update bounty doc 2 (#3551) 2026-08-12 09:56:34 -07:00
Daniel Salazar c3a0c28705 docs: update bug bountu (#3550) 2026-08-12 09:55:13 -07:00
Nariman Jelveh 7ceb2090b7 feat: app user feedback system (#3546)
* feat: app user feedback system

Add puter.ui.showFeedbackDialog(), letting users send feedback to an
app's developer. In the app environment the Puter desktop renders the
dialog; on a third-party website a puter.com popup hosts it. The message
is stored in a new app_feedback table and emailed to the app owner's
confirmed email — it never passes through the app's own code.

Feedback is strictly opt-in per app via a new apps.feedback_enabled
column (a real column, not an app-metadata key, so Dev Center's
whole-blob metadata saves can't silently erase it), settable through the
existing puter.apps.update path (feedbackEnabled).

Backend follows the layered stack: AppFeedbackStore (durable count
queries) -> AppFeedbackService (opt-in check, message normalization,
abuse caps, best-effort owner email) -> AppFeedbackController
(POST /app-feedback, GET /app-feedback/target). New app-user-feedback
email template uses the escaping-safe nl2br triple-stash.

Defensive by design:
- requireUserActor blocks app tokens, so feedback can't be submitted
  programmatically; guiOriginOnly keeps cross-origin pages out.
- App identity comes only from the validated IPC sender (desktop) or the
  browser-attested opener origin (popup), never from message contents.
- The send-feedback popup action is in NON_AUTH_POPUP_ACTIONS, so it
  never delivers a token to the opener.
- Layered limits: route rate limits, plus DB-count caps that fail closed
  when the limiter backend is down, plus a per-app daily owner-email cap.
- Owner email is fully best-effort: an unconfigured transport,
  unconfirmed/unsubscribed/suspended owner, or send failure never fails
  the request or blocks storage.
- The dialog and SDK method are resolve-only and always settle, so a
  caller is never left hanging.

Migrations for sqlite/mysql/postgres, puter.js types, docs, backend
tests (sqlite + postgres), and a Playwright e2e spec are included.

* feat: add feedback control to the dashboard app-drawer

Surface the feedback dialog directly from the app window's chrome in
dashboard mode: apps that opt in (apps.feedbackEnabled) get a "Send
Feedback" button in the dashboard app-drawer, next to minimize/close.
It opens the same UIWindowAppFeedback dialog, targeting this app by uid.

The control is only rendered when the app opted in — feedback_enabled is
threaded from the launched app's metadata into the window options — and
the dialog still re-checks opt-in server-side, so a stale flag can't send
anywhere. Reuses the existing .dashboard-app-drawer-btn styling and the
app_feedback_title i18n string, so no new CSS or strings.

Adds e2e coverage: the control appears and opens the dialog for an
opted-in app, and is absent for an app that hasn't opted in.

* feat: enable app feedback by default in Dev Center

New apps created in Dev Center now have feedbackEnabled set on creation,
so users can send the developer feedback without any extra setup. A "User
Feedback" toggle in the app's settings lets developers turn it off (and
back on); it's wired into the save payload, the dirty-state tracking, and
the reset-to-original path like the neighboring toggles.

The Save update omits feedbackEnabled unless the toggle is present, and
the backend leaves an omitted field untouched, so the default survives
the create-then-save flow Dev Center runs. Add an SDK apps-suite guard
covering that round-trip (create-on -> unrelated update keeps it -> can
be turned off).

* fix: feedback modal polish + share sender email

Address four issues with the app feedback UI:

- Dashboard app-drawer: the extra "feedback" control pushed the close
  button past the drawer's derived width and clipped it. A `has-feedback`
  modifier widens the surface by one button + gap so all three controls
  fit. The control's glyph is now a message bubble with text lines, which
  reads more clearly at 14px than the previous bare speech bubble.

- The feedback dialog is no longer a UIWindow. It's a from-scratch
  overlay modal in the spirit of the dashboard modals (uninstall,
  add-app): a fixed scrim + centered card with self-contained,
  theme-aware color tokens (light default + dark override), a bottom-sheet
  layout on narrow screens, backdrop/Escape close, and an entrance
  transition. This renders consistently across the three contexts it's
  opened from (desktop app-IPC, dashboard drawer, standalone popup), so
  the callers no longer pass UIWindow-specific window_options.

- Feedback now shares the sender's email (not just their username) with
  the developer so they can respond: the owner email sets Reply-To to the
  sender and shows the address in the body — but only when the sender's
  email is verified (an unverified address could be anyone's, so it's
  never used as a reply target). EmailClient.send gains an optional
  replyTo. The dialog note now says the email will be shared.

Tests: e2e updated for the new modal (7 pass); backend feedback suite
covers the verified/unverified sender-email split (sqlite + postgres);
EmailClient + GUI unit suites pass; type-check clean.

* fix: resolve 'app-'-prefixed app names in feedback target lookup

APP_NAME_REGEX allows names beginning with "app-" (e.g. the seeded
app-center), but resolveTargetApp's startsWith('app-') heuristic sent
those to a uid-only lookup with no name fallback, so feedback for such
apps 403'd even when enabled. Use AppStore.resolveApp (uid, then name)
like the rest of the codebase.

* fix: make feedback daily caps fail closed under concurrent submissions

The per-user and per-user-per-app caps were check-then-insert and the
per-app email cap was count-then-send, so parallel requests (or multiple
nodes, or the route limiter failing open) could all read a stale
under-cap count and push past every limit — the exact scenario the
DB-backed caps exist to stop.

Now the user caps recount after the insert (own row included) and roll
the row back with 429 if a burst breached them, and the email cap claims
its slot (email_sent=1) before sending, recounts, and releases the slot
if over cap or if the send fails.

* docs: disclose the Dev Center's feedback-on-by-default in SDK docs/types

The Dev Center deliberately creates apps with feedbackEnabled (see
32c950de5), but the SDK docs, apps.d.ts, and AppFeedbackService's class
doc all described feedback as strictly opt-in / default-false with no
qualification — so a Dev Center developer reading them would wrongly
conclude feedback is off for their app. State the Dev Center behavior
alongside the API default, and correct the update-path docs: an omitted
feedbackEnabled leaves the current value unchanged rather than
defaulting to false.

* fix: don't mint a user-app token as a side effect of the feedback popup

Every embedded_in_popup boot ran the user-app token exchange, and the
exchange is a write: /auth/get-user-app-token bootstraps an app row for
the opener origin, grants flag:app-is-authenticated (what makes the
site count as connected to the account), and creates its AppData dir.
So merely opening — or immediately cancelling — a send-feedback popup
recorded a user<->site relationship the read-only feedback flow never
needs: the server resolves the feedback target from the attested origin
without any of it.

Gate the exchange behind runsUserAppTokenExchange(action) in all three
popup paths that mint (main postAuthActions exchange, temp-user signup
success, manual signup fallback). request-permission keeps the exchange
since grants are written against the app row it bootstraps.

* fix: refuse feedback when the deployment cannot deliver it

With no email transport configured (the common self-hosted default),
submissions were stored in app_feedback — a table with no read path
beyond the abuse-cap COUNTs — the owner email was silently skipped, and
the sender was still shown 'Feedback sent. Thank you!'. The developer
never learns the feedback exists while the user believes it was
delivered.

Gate acceptsFeedback on clients.email.isConfigured so the pre-flight
reports enabled:false (the dialog shows its 'not accepting feedback'
pane) and submit returns 403 instead of swallowing messages. Owner-level
store-without-email cases (unconfirmed owner email, per-app email cap
overflow) keep their existing deliberate semantics.

* fix: settle showFeedbackDialog instead of hanging on older host GUIs

In the app environment showFeedbackDialog awaited an IPC reply with no
capability check. A host GUI that predates this feature (self-hosted
Puter running the live js.puter.com SDK) has no handler for the message
and never replies, so the promise documented as 'never rejects' also
never resolved.

The GUI now advertises the IPC dialogs it can answer via a
puter.gui_features param on the app iframe URL, and the SDK resolves
false when 'feedback-dialog' isn't listed. A reply timeout could not
substitute: legitimate replies only arrive when the user closes the
dialog, so any timeout would false-negative while the user is typing.
Older SDKs ignore the extra param.

* fix: guard app-triggered feedback dialog against desktop-lockout loops

The showFeedbackDialog IPC handler had no re-entry or abuse guard, and
the dialog it opens is a full-viewport overlay above the taskbar and
every window — so 'while (true) await puter.ui.showFeedbackDialog()'
kept the desktop permanently covered (for signed-out users, the same
loop spams the full-page signup window instead). Every dismissal just
settled the promise and let the app immediately reopen it.

Allow one dialog at a time, and back off reopens per app after each
dismissal that sent nothing: 10s, then 60s, then blocked until page
reload. A successful send resets the backoff, and user-initiated paths
(dashboard drawer) are unaffected since they don't go through IPC.

* fix: carry feedback_enabled in suggested/recommended app summaries

launch_app uses options.app_obj verbatim when provided, and the
suggested-apps launch paths (open_item.js, UIWindowSearch.js,
UIDesktop.js) pass summaries from toAppSummary — which omitted
feedback_enabled. So an opted-in editor launched by opening a .txt file
showed no Send Feedback control in the dashboard drawer, while the same
app launched from the Apps tab (full puter.apps.get object) did.

* fix: gate feedback dialog Cancel/X on an in-flight submit

Escape and backdrop clicks were already ignored while the POST was
pending, but the X, Close, and Cancel buttons weren't — clicking one
mid-send settled the promise false and tore down the overlay while the
submission still landed server-side: the developer got the email, the
app was told sent=false, and a user who resubmitted 'the failed one'
sent a duplicate and burned a daily-cap slot.

Apply the same !sending gate to the buttons and disable them visually
while the send is in flight.

* fix: reject feedback origins longer than the source_origin column

readTargetParam accepted values up to 3000 chars but the raw origin is
stored verbatim into source_origin VARCHAR(2048) (MySQL/Postgres), so a
2049-3000 char origin passed every validation and then blew up the
INSERT with an HTTP 500 on Postgres/strict MySQL — or was silently
truncated on non-strict MySQL, corrupting the abuse-forensics value the
column exists for. Cap the param at the column size.

* fix: make the feedback dialog's privacy note match what is shared

The note unconditionally said 'Your email address will be shared with
the developer so they can respond', but AppFeedbackService shares the
username always and the email only when it exists and is verified — an
unverified or temp-user sender was promised a reply path that never
materializes, and nobody was told about the username.

Show 'username and email' when the signed-in user's email is verified,
and 'username' otherwise.

* fix: stop HTML-escaping email subject lines

Subjects were compiled with default Handlebars escaping, so the
app-user-feedback subject rendered a title like "Bob's App & Games" as
"Bob&#x27;s App &amp; Games" — literal entities in the recipient's mail
client. Subjects are plain-text headers, not HTML; compile them with
noEscape. Header safety is unaffected: the transport encodes newlines
and free-form values collapse whitespace upstream.

* docs: state exactly what showFeedbackDialog shares with the developer

The doc claimed the dialog 'tells the user their username will be
shared', while the dialog's note talked only about the email address and
the implementation shares the username always plus the email (as
Reply-To) only when verified. Describe the actual disclosure: username
always, email when verified.

* docs: document the COOP false-resolve limitation of showFeedbackDialog

Under COOP the popup's opener link is severed, so the SDK deliberately
resolves false while the popup stays open and the user can still submit
(a feedback submission has no server read-back the way a permission
grant does). The documented contract ('resolves to true if the user
submitted feedback') was silently wrong on cross-origin-isolated pages —
state the limitation in the doc and the SDK jsdoc: false means 'not
confirmed', not 'not sent'.

* feat: point the feedback email footer at the Dev Center

The footer told developers to turn feedback off via a puter.apps.update
one-liner, but the toggle lives in the Dev Center app settings — and
Dev Center is where apps get feedback enabled by default in the first
place. Reword it to 'manage it in the Dev Center' with a link built
from config.origin (like app_link) so it holds on self-hosted
deployments.

* test: cover the feedback service and store layers directly

The service owns every feedback business rule — target resolution,
eligibility, message normalization, the durable caps, and the owner-email
preconditions — but was only reachable through the controller's tests. Give
it and the store their own suites so a regression names the layer it broke.

Service coverage adds the branches the route tests could not reach: a
blocked origin resolving to null rather than surfacing a 403, owners who are
suspended or unsubscribed, length measured after normalization, the 24h cap
window boundary, subject-header injection via the app title, and the email
links being rooted at config.origin.

The two describes already labelled `AppFeedbackService ...` move out of the
controller test, which keeps only the caller-facing promise that a failed
send still returns success.

* refactor: prep the feedback cap error once instead of via a factory

Both throw sites are in one call and only one can ever run, so a plain
const reads the same and drops a function that existed only to defer a
constructor.

* Require verified users for app feedback API

Add `requireVerified: true` to both app feedback routes (`GET /target` and `POST /`) in `AppFeedbackController`. This tightens access control so only verified user accounts can fetch feedback targets or submit app feedback.
2026-08-12 09:49:24 -07:00
jelveh ea63c3093e feat(dashboard): add Publish as website and Properties to folder menu
The Files tab's folder-background context menu (empty listing area,
sidebar folders, breadcrumb segments) now offers Publish as website and
Properties for the folder the menu was opened on, matching the desktop's
folder menus. Both are omitted at the filesystem root and in Trash, and
Publish is omitted inside trashed folders.

The properties modal can now resolve an entry by path when the caller
has no uid, which the folder-background menu doesn't.

The publish gate (account creation + email confirmation) was duplicated
in three places, so it moves to a publish_as_website helper that all
call sites now share.
2026-08-12 08:50:03 -07:00
Daniel Salazar e8236a100c chore: bump versions (#3548) 2026-08-12 01:18:17 -07:00
Daniel Salazar 20b3b88e39 metering: big fixes to metering + jsdoc types (#3547)
Changes are:
- global egress metering
- remove file egress cost
- introduce file op cost for the per request cost s3 has
- enforce fs read/download etc to through 402 when out of usage; allow for subdomains
- enforce kv metering when out of usage through 402; allow for workers
- jsdoc as source of truth for puter.js types
- kv driver caching for get and batchget operations with decreased costs
2026-08-12 01:06:04 -07:00
Chen, Ting-An cbae39a08f fix(i18n): add Traditional Chinese app management strings (#3541)
Translate the app discovery, app request, desktop shortcut, and app folder controls that currently fall back to English in the Traditional Chinese interface. Preserve the positional placeholders used by accessibility labels.
2026-08-11 21:18:37 -07:00
Daniel Salazar afad3e37bf fix: server health tolerance for primary read (#3542) 2026-08-11 01:19:09 -07:00
Daniel Salazar a6b2161b2c fix: misc sec fixes (#3540) 2026-08-10 22:24:44 -07:00
Neal Shah 9e864e9574 PUT-1432 (#3539) 2026-08-10 23:33:20 -04:00
Daniel Salazar 79d4201f12 fix: rate limits, AI routing, and a type-check gate (#3529)
- declare rate + concurrency limits on every route and driver that lacked one
- add acquireConcurrent for websocket connections and the DAV mount
- bucket AI models by identity key only; keep resold duplicates of any vendor
- skip recently-failed provider routes; cap the fallback chain at 3 attempts
- let full-access access tokens bind a worker to an app their own user owns
- cache resolved subscriptions so tiered limits don't add a round trip
2026-08-10 19:09:47 -07:00
Daniel Salazar 0c2d7dfa34 feat: add user created timestamp on whoami (#3538) 2026-08-10 19:08:50 -07:00
Neal Shah 2a6d2fea32 allow max-tokenless models on openrouter (#3536) 2026-08-10 18:40:10 -04:00
Juan Fernando Castro 1dec5e90c4 docs: add requestAppData() to Perms sidebar (#3534) 2026-08-10 11:38:09 -07:00
Neal Shah 566298d89c make context length on togetherAI more lenient (#3533) 2026-08-10 13:14:33 -04:00
Daniel Salazar 6bb5eacf61 fix: metering kv spamc (#3532) 2026-08-10 08:36:56 -07:00
Reynaldi Chernando 390a83c860 Fix puter.js docs and types drift 10 aug (#3531) 2026-08-10 12:47:18 +07:00
Nariman Jelveh 8f23bf7b75 fix(gui): hover the add-an-app options with the accent wash, not grey
The rows already answer hover with a --select-ring border; filling them
with the grey --dashboard-hover under that accent ring read as two
systems disagreeing. The weak select tint keeps the hover in one voice
and stays under the icon chip's stronger tint in both themes.
2026-08-09 18:23:07 -07:00
Nariman Jelveh aa85dddd38 Refresh Add App AI description copy
Update the English `add_app_ai_desc` translation string in the GUI to a more direct, enthusiastic message: “Describe the app you want and AI will build it!”.
2026-08-09 18:21:31 -07:00
Nariman Jelveh 8284f35b93 fix(gui): honor godmode file_paths launches now that /apps returns booleans
The godmode gate in ExecService.launchApp checked `godmode === 1`, but
GET /apps/:name serializes the flag as a boolean, so the check always
failed and file_paths launches (e.g. Dev Center opening a worker's
source in the editor, #2218) silently dropped the file. Accept both
shapes, matching the existing guard in launch_app.js.

Also fix the two issues hiding behind that gate:
- puter.apps.get returns `uid`, not `uuid`, so the /sign call for the
  target app sent app_uid undefined — no write grant and no user-app
  token for the launched app. Use `uid` with a `uuid` fallback.
- The closeApp IPC handler had the same `godmode === 1` comparison,
  preventing godmode apps from closing other apps' windows.
2026-08-09 18:06:58 -07:00
Nariman Jelveh ccec3ce39b fix(gui): open add-an-app choices through the deep-link intro, not a + tile morph
60686643b made App Center and the AI builder morph out of the + tile the
user clicked, which answers where the window came from but not the
question these two options exist to answer: where the APP went. The grid
never showed it arriving, so the user still ended a launch with nothing
to find, no tile for minimize to fly back to, and no lesson learned.

Choosing either option now plays exactly what an /app/<name> landing
plays (beginDeepLinkLaunch): when the app already has a tile the intro
travels to it, plays the click flourish, and the window grows out of the
app's own slot; when it doesn't, the tile is INSTALLED first — the slot
opens, the progress stroke draws, the icon springs in — and only then
does the window morph out of the tile that just arrived. Same beats,
same exposure decay, same interruptibility, same duplicate-launch claim.

The app info is prefetched in parallel and handed to launch_app, the
same contract as initgui's landing path, so the intro (which may need to
draw the arriving tile from it) never costs a second round-trip.

This also reverts 60686643b's anchor plumbing (UIWindow's
dashboard_tile_el option and the dashboard_tile_in_view export): the
intro installs a real tile for the by-app-name morph to find, so no
stand-in anchor is needed.
2026-08-09 18:02:39 -07:00
Nariman Jelveh 60686643b3 fix(gui): launch from the add-an-app modal with the tile morph, not a fade
Choosing App Center or the AI builder opened the window with the plain
fade while every other launch in the Apps tab grows out of the icon that
was clicked — the one launch that most needs to say where it came from
read as the app appearing from nowhere.

The morph resolved its anchor by app name, which cannot work here: the
whole point of these two options is installing an app you do not have,
and an app with no tile has no name to find. So the launcher may name its
anchor element (dashboard_tile_el), and TabApps resolves it once before
launching so the click flourish and the window's half can never disagree
about which tile the launch came from: the app's own tile when it already
has one on screen — minimize flies the window back there, so that is
where opening should come from — and otherwise the + tile itself.

dashboard_tile_in_view's visibility test is now a named helper it shares
with the new path, so an anchor on an offscreen pager page is still
rejected and still falls through to the fade.
2026-08-09 17:49:50 -07:00
Nariman Jelveh 61bfcbdc83 feat(gui): add an add-an-app tile at the end of the Apps grid
The Apps grid showed what you have and offered no way to get more. A
dashed plus tile now rides at the tail of every page of it: after every
app, on the last page, and nowhere else. It wears the same tile skeleton
as an app so hover, focus, and arrow keys treat it identically, but
carries no app name or group id — which is what keeps it out of the
saved order, the running dots, and the deep-link tile lookups. It cannot
be picked up, is never a drop target (so no app can land past it), has
no context menu, and sits still while the apps jiggle in reorder mode.
Search results leave it out: a query asks which apps you have.

Clicking it asks how you would like to get an app: find one in the App
Center, have one built by the AI builder, or request one that does not
exist yet. The request is a form in the same card rather than the stock
Contact Us window — a titled pane one step deeper, whose Back arrow
returns to the options with the draft intact. It posts the same
/contactUs request that window did; a failure is said inside the form,
where the request is still there to retry, rather than in an alert
window the next click on the dashboard would bury.
2026-08-09 17:37:09 -07:00
Nariman Jelveh c8985b621d Add copy-link support for app tiles
Extract app tile URL generation into a shared helper and use it for a new dashboard context menu action that copies the tile's destination. This keeps copied links aligned with the existing open-in-new-tab behavior and adds regression coverage for app, external, encoded, and folder tiles.
2026-08-09 16:54:18 -07:00
Nariman Jelveh 42299a64ed Update recommended apps list
Add 'blockarena' and remove 'pretty-tiles', 'galaxy-troops', and 'blend-fruits' from the recommended apps list.
2026-08-09 11:20:40 -07:00
jelveh 344db5cb0b Remove vault from recommended apps
Drop `vault` from the recommended apps list so it no longer appears in app recommendations.
2026-08-08 11:54:56 -07:00
jelveh 66824f607d feat(gui): install an app onto the grid when its landing is what installs it
A /app/<name> landing for an app the dashboard didn't list opened with
a plain fade and left the app NOWHERE: the grid had been fetched before
the launch granted the permission that installedApps reports, so Back
found no tile to minimize into and the user landed on a dashboard
without the app they were just inside. The round trip that teaches
"windows are inflated tiles" broke on exactly the landing where a new
user is forming that model.

Now the landing SAYS what it does. The app joins the grid at the tail
(where new apps land) with a tile drawn from the landing's own app-info
prefetch -- only a confirmed-real app ever materializes, with its real
icon -- held invisible until the intro has revealed the grid and
travelled to the tile's page, and then it is INSTALLED, in the grammar
every app store taught: the slot opens (the folder well's own surface)
with the icon dim inside it, a progress stroke draws clockwise around
the slot, and on completion the icon springs to full color and size as
the label names it. Only then does the flourish play and the window
grow out of the tile. The arrival is the one beat that never decays:
per-app news delivered at most once per app, not a repeated lesson.
Minimize now has a target, and the app is simply there afterward.

The regression surface is guarded on every edge: the parked-invisible
state is re-applied by renderApps itself so mid-intro re-renders can't
strip it, and settleDeepLinkLaunch -- which the landing's finally
always reaches -- force-reveals it, so no path leaves an invisible
tile. A _pendingInstalls overlay (the mirror of _removedLocal) keeps a
refresh fetched before the grant lands from evicting the tile, and
retires itself once installedApps confirms the app or the user
uninstalls it again. The order is never saved on the splice, so the
default-position append can't freeze a custom order the user never
made. Failed or absent app info adds nothing; installed-app landings
are untouched; animations-off and reduced-motion users skip the
choreography but still get the tile, silently. The progress ring is a
transient element (the tile's ::before is the well, its ::after the
running dot) drawn as an SVG dashoffset sweep -- no Houdini dependency
-- and the app-info prefetch now asks for the 128px icons every other
tile already uses.
2026-08-08 11:25:48 -07:00
jelveh 66dedf3082 feat(gui): let the app drawer's intro flash retire once learned
The control drawer on headless dashboard apps opened itself on EVERY
window open, forever -- 2.6 seconds of tray over the app's own top
edge to teach a lesson (the controls live in this tongue) that is
learned in a few exposures. Same doctrine as the dashboard's
deep-link intro: pedagogy that knows when to step aside.

A per-account kv counter records deliveries, once per window: a flash
played while the user could see it, or the user opening the drawer
themselves (hover, tap, focus) -- the stronger proof, since the
drawer is a headless app's only chrome and must never decay out of a
user's awareness. After three, new windows keep the bare tongue and
the flash stops; hover, tap, and focus expansion never decay.

The attach is synchronous and the stored count is not: the first
drawer of a session races the read against a short cap and defaults
to teaching -- a hung read must not hold the intro hostage, and every
failure mode (unparseable value, failed read, timed-out read, failed
write) errs toward teaching once more, never toward never teaching.
The resolved count is cached and kept current in memory, so later
windows decide synchronously and a session's own not-yet-written
increments still count. kv.incr keeps two devices from losing an
increment, and the pre-read caps the key once the lesson is learned.

If the user beats a still-deciding flash to the drawer, the flash is
skipped rather than played over their hand -- its auto-collapse would
shut a drawer they are actively using. A flash in a hidden tab still
defers to first reveal (it rides requestAnimationFrame, as before)
and an unseen one is never counted.
2026-08-08 10:25:26 -07:00
Daniel Salazar 95da802b13 fix: metering hardening (#3527) 2026-08-08 06:10:01 -07:00
Daniel Salazar 0491d5fea7 fix: mcp better uploads (#3528) 2026-08-08 06:09:49 -07:00
Juan Fernando CastroandDaniel Salazar d202be10a9 feat: let apps use another app's data with user consent (#3516)
* feat(perms): add cross-app app-data permission vocabulary

* feat(perms): sweep app grants by permission prefix

* feat(perms): resolve and withdraw cross-app data grants

* feat(kv): support an authorized namespace override and per-key privacy

* feat(kv): gate cross-app KV access behind app-data grants

* feat(fs): allow cross-app AppData access and require a scope to delete

* feat(auth): accept permission lists and gate app-data grants

* feat(perms): add requestAppData to the puter.js SDK

* feat(gui): carry permission lists through the IPC and popup transports

* feat(gui): describe cross-app data requests in the consent dialog

* docs: document requestAppData and per-entry KV privacy

* perf(perms): sweep cross-app grants only for origin-bootstrapped apps

* fix(gui): stop double-encoding cross-app consent text

* fix(perms): close three gaps in cross-app grant enforcement

* fix(kv): meter and batch the per-entry privacy probe

* fix(perms): resolve app identifiers and scopes more strictly in the SDK

* test(perms): cover the cross-app consent flow end to end

* fix: small missing token resolution for app

also adds the same exclusion for the batchPut api, small change

* fix: make resolved actor optional

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-08-08 04:04:07 -07:00
jelveh 2c17476c07 fix(gui): folder pages, live drag-out, and one merge pulse in My Apps
Three fixes to the My Apps folders:

A folder big enough to need it now paginates like the grid outside
(iOS folders page too) instead of growing a scrollbar: pages of
cols x 3 tiles in a scroll-snap scroller with dots below, wheel and
swipe to flip, and a drag held at the card's edge flips folder pages
so reordering works across them. The card's height stays constant
across pages, and a deep-link landing on an app that lives on a later
folder page travels there before the launch morph.

Dragging an app out of the open folder no longer drops it blind:
after a beat held outside the card, the app leaves the folder, the
card closes behind it, and the SAME drag carries on over the grid --
placeholder shuffle, edge page flips, even a drop into another
folder -- so the app lands where the user watched themselves put it.
A quick drop during the beat still lands beside the folder, and
swinging back inside within the beat cancels cleanly.

The folder well's fill played twice on nearly every merge: the rest
countdown anchored at first contact with the target, so the hand
decelerating INTO the tile always read as movement when the dwell
elapsed, and the deliberate refill-from-empty replayed the fill. The
countdown now re-anchors on the move events themselves -- rest is
only visible at its edges -- so the fill that completes is the one
started by the last movement, played exactly once.
2026-08-08 01:28:09 -07:00
Nariman Jelveh 06d94534e8 feat(gui): iOS-style app folders in the My Apps grid (#3525)
* feat(gui): iOS-style app folders in the My Apps grid

Drag one app onto another and let it settle: a well opens under the
target and the drop makes a folder of the two. Dropping onto an existing
folder joins it. A folder opens by growing out of its own icon into a
card over a blurred grid, where its apps can be launched, rearranged,
renamed, or carried back out.

Hovering a tile mid-drag means two things — "push over, I'm passing
through" and "swallow me" — and the tile is barely bigger than its icon,
so pixels can't separate them; motion does. The shuffle is held while
the folder offer stands and fires when the icon leaves the tile, or at
the drop, so a quick drop onto a neighbour still reorders exactly as it
did. The offer itself re-arms rather than cancelling on movement: the
last events of a drag are the ones carrying the icon onto the target and
nothing is dispatched while it rests, so a cancel-on-movement dwell
could never fire at all.

Folders are stored in their own kv key; the grid's ORDER stays entirely
in the existing saved app order, with a folder occupying its first
member's slot and its members contiguous. Every saved order therefore
stays valid with no migration, and an app whose installedApps page
failed to load keeps both its folder and its position. Folders never
nest, one that drops below two apps dissolves, and a corrupt kv value
degrades to "no folders" rather than a broken tab.

Elsewhere:
- Search looks THROUGH folders — a match the user then has to hunt for
  inside one is not an answer.
- Minimize morphs into the FOLDER when an app lives in one, and a
  /app/<name> landing opens the folder so the launch grows out of the
  icon where the app actually is.
- The uninstall FLIP keyed surviving tiles by app name, which a folder
  tile doesn't have; it now keys by identity.

New pure model in appGroups.js with tests; verified end to end in the
running dashboard (create, join, open, rename, reorder, eject, ungroup,
launch-from-folder) alongside plain reorder, search, and uninstall.

* fix(gui): keep the folder name field from inheriting input[type=text] sizing

style.css styles every input[type=text] with `width: 100%` and grows it to
`padding: 7px; border: 2px` on focus. `input.myapps-group-name` matches at the
same specificity, so it only wins the properties it actually declares — width
was never one of them, and the focus rule declared neither padding nor border
width. The name field therefore spanned the entire folder card (so its hover
and focus chip read as a full-width bar rather than the name) and grew 8px
taller the moment it was clicked, shoving the folder's app grid down.

Spell the three out, in both the resting and the focus rule — the same trap
.myapps-search already documents next door.

* fix(gui): size folder icon ghosts to the icon they stand on

Border-box only reaches a folder's icon through `.dashboard * { box-sizing }`,
and every ghost cloned from one is appended to <body>, outside that rule: the
drag ghost, the click-time launch flourish, and the open/minimize morph ghosts
all fall back to content-box, where .myapps-group-icon's 5px padding is added
to the 56px slot. Each ghost rendered 66px square and 5px off, so it visibly
popped at exactly the moment it was supposed to sit flush on the real icon.

State box-sizing on the rule itself so a clone carries it wherever it lands.

* fix(gui): stop a closing folder from swallowing the next click

_closeGroup drops the open class and leaves the overlay in place for
GROUP_PANEL_CLOSE_MS so the card can recede into its tile. The scrim is
`position: fixed; inset: 0` and still hit-testable for that whole quarter
second, so a click on the grid during it landed on the outgoing overlay — whose
handler only re-runs _closeGroup, now a no-op. Shutting a folder and reaching
straight for an app did nothing.

Take the outgoing overlay out of hit-testing; it has no interactive job left.

* fix(gui): keep a folder name typed right up to the moment it closes

The name box commits on blur, and every exit that goes through a pointer blurs
it while the folder is still open — so clicking outside, or launching an app
from inside, keeps what was typed. Escape does not: _closeGroup clears
_openGroupId first and only then moves focus to the tile below (or removes the
card outright), so the blur arrives with no open folder to rename and
_renameGroup drops it. A brand-new folder opens with its name selected for
exactly this edit, so "type Games, press Escape" — the obvious way to dismiss
a dialog — was the path most likely to lose it.

Commit the pending name on the way out, before the folder id is gone.

* fix(gui): close an open folder when the Apps tab is re-entered

The dashboard hides an inactive section and calls onActivate on the way back
in; there is no deactivate hook, so a folder left open survives the round trip
and greets the user still open over a grid they walked away from. Worse,
onActivate's focusSearch then lands the caret in the search box behind the
folder's scrim, and typing filters the grid the card is covering — a modal with
the keyboard pointed outside it.

Shut the folder as the tab comes back: returning to the tab is returning to
the grid.

* fix(gui): move focus into a folder when it opens

Opening a folder called .focus({ preventScroll: true }) on a jQuery
object. jQuery's .focus() shorthand reads a lone non-function argument
as event DATA and binds a handler with it, so it never moved focus:
the folder opened modal over the grid with focus still on the tile
behind its scrim, where Tab walked away through the inert grid instead
of cycling inside the dialog — and the object it bound as a handler
threw a TypeError on that tile's every subsequent focus.

Focus the DOM node instead, as every other focus call in this file
already does.

* fix(gui): stop renaming a folder from swallowing the click that commits it

The folder's name box commits on blur, and blur fires on the PRESS —
before the click that press belongs to. Committing re-rendered, and the
re-render replaced every tile in the open folder, so by the time the
click was dispatched the tile under the pointer was detached and the
delegated handler never saw it. Typing a name and then tapping an app
in the folder — the path a brand-new folder puts the user on — renamed
the folder and did nothing else; the app only opened on a second click.

Rebuild the folder's contents only when they actually differ from what
is on screen. A rename doesn't change them, so nothing is detached, and
a background refresh no longer throws away hover/focus either. Tiles
that survive get their drag resting-rects cleared, since the card can
have moved under them since the rects were taken.

* fix(gui): keep Enter in a folder's name box from leaving the folder

Committing the name with Enter blurred the box, which left focus on
<body> — outside a dialog that is marked aria-modal and that traps Tab
on its own subtree. The next Tab therefore walked off through the inert
grid the folder is covering, exactly what the trap exists to prevent.

Step out onto the folder's first app instead; the same blur still
commits the name. The keystroke is stopped at the box because the
grid's document-level key handler reads Enter on a focused tile as
"launch it", and would otherwise have taken the focus move as its cue
to open an app the user never asked for.

* fix(gui): stop an open folder clipping its own uninstall badges

The folder's grid scrolls, so it clips anything outside its padding box
— and reorder mode's uninstall badge deliberately overhangs the top-left
corner of every tile. The top row's badges therefore rendered as flat
tabs rather than circles, on the one surface where they are a touch
user's only way to uninstall an app they have filed away.

Give the scroller 9px of top padding for the overhang to sit in and take
it straight back off as margin, so the card and everything in it stays
exactly where it was.

* fix(gui): hold page edge-flips while a folder merge is being offered

A tile in the pager's last column sits inside the 60px edge-flip zone, so
resting a dragged icon on it — the folder-making gesture — armed the edge
dwell alongside the merge dwell, and the page flipped out from under the
very folder the user was watching form (the merge target scrolls away
mid-offer, and the drop then resolves against its stale resting rect).
Worst on phones, where 72px tiles overlap the zone across the whole last
column.

A live merge offer now holds the edge flip: entering the zone arms no
dwell while an offer stands, and an offer that arrives during a running
dwell is re-checked at the flip (a resting pointer fires no event that
could clear the timer). Carrying the icon off the tile withdraws the offer
and the hold with it, so deliberate flips — resting in the bare edge
gutter — behave as before.

* fix(gui): make Escape cancel a folder rename instead of saving it

Escape pressed mid-edit fell through to the folder's close handler, and
closing commits whatever the name box holds — so the one key every inline
rename uses for "never mind" stored the abandoned half-typed name. Escape
in the box now puts the stored name back and steps out to the folder's
tiles, exactly the cancel Finder/Explorer taught; with nothing left to
cancel (name untouched) it falls through and closes the folder as before,
so a second press still exits.

* fix(gui): refill the folder well when the merge countdown restarts

The merge dwell pins its stillness anchor at first contact with the tile,
and a hand decelerating INTO a target routinely covers more than the
7px allowance between that moment and the first tick — so the countdown
quietly restarts. The well's fill, tuned to the same 460ms, had already
completed by then and just sat there half-open: a drop it seemed to
promise a folder for actually reordered. Restart the fill with the
countdown, so what the well shows is always the countdown that is
actually running.

* fix(gui): keep keyboard focus inside an open folder across its edits

The folder card is a modal dialog, but three of its flows stranded focus
on <body>, where Tab walks the inert grid behind the scrim:

- Remove from Folder rebuilds the card's grid after the context menu has
  dropped focus, so nothing inside the dialog holds it. The rebuild now
  hands focus back to the same app's tile (or the first) whenever it finds
  focus on <body> — never stealing from an uninstall modal, which holds it
  legitimately.
- The same eject dissolving the folder re-renders the grid right after
  _closeGroup's focus hand-back, replacing the very tile it chose. The
  ejected app's own tile — where the user is looking — takes focus instead.
- A click on the card's empty space focused nothing at all. The card now
  carries tabindex=-1 so such clicks land on it (no visible ring; a focus
  ring around the whole card would misread plumbing as selection), and the
  Tab trap wraps from there instead of stepping off through the scrim.

* fix(gui): size the folder name box to the name it holds

The box sat at the browser's default ~20ch regardless of content: a short
name floated in a hover pill far wider than the word, and a 40-character
one clipped while the card had room to spare. field-sizing: content hugs
the text between a 90px floor and the card's width, iOS-style; engines
without the property keep the default box exactly as before.
2026-08-08 00:28:03 -07:00
Neal Shah aea828ff12 add worker create event and new query methods (#3524) 2026-08-07 17:51:37 -04:00
Daniel Salazar c24f0fe08b fix: mcp fs copy, move, rename (#3517)
* fix: mcp fs copy, move, rename

* feat: more mcp tools for kv
2026-08-07 17:14:40 -04:00
jelveh 085ac116c7 fix(gui): keep app landings through signup, and unstack the session picker
Signing up from a direct /app/<name> landing redirected to '/' on
success, silently dropping the app the URL asked for — the deep link's
launch (and its dashboard intro) never happened. Signup now mirrors
login's pathname-preserving redirect, but only for app-landing routes
(/app/<name>, /desktop/app/<name>): every other route keeps the
historical '/', notably /action/signup, where returning to the same
path would just show the signup form again. The query string stays
dropped, matching login's credential-leak hygiene. This covers all
three signup entry points reachable from a landing: the login cover's
"Sign up", the session picker's "Create Account", and the
must_login_or_signup fallback.

The session picker (UIWindowSessionList) also left itself on top of
the cover windows its two links open, hiding their username fields:
"Create Account" tried to close the picker via the LOGIN window's
c2a selector (which matches nothing in the picker), and "Log Into
Another Account" never closed it at all. Both now close the picker —
in the reload flows only: the no-reload (popup) flows keep it open,
where it doubles as the fallback UI when the login/signup window is
abandoned mid-flow.

Picking an account was already correct (location.reload() keeps the
landing URL); with these fixes all three picker paths, plain login,
and signup all return to the app landing, where the boot replays the
launch and its intro.
2026-08-07 11:51:44 -07:00
jelveh 5538a5d24b feat(gui): let the deep-link intro step aside for users who know it
The intro exists to teach the dashboard's spatial model; once learned it
would only tax every bookmarked landing. Two mechanisms remove that tax:

- Interruptible: any real user input (pointer/key/wheel, isTrusted only)
  during the intro skips the remaining choreography and launches at
  once — it wakes the in-flight beat sleep, so the skip is immediate.
  Input never cancels the launch itself and is never swallowed;
  whatever it was doing (typing a search, clicking a tile) still
  happens.

- Exposure decay: after 3 delivered — or deliberately skipped — intros
  the beats collapse and the sequence plays in one breath, exactly like
  a warm tile click. Counted per account in kv (kv.incr, atomic across
  devices; capped at the threshold so the key stops changing) rather
  than per device: the lesson lives in the user's head and the account
  follows them, while localStorage would also bleed between accounts on
  a shared browser. Only real exposures count: a delivered flourish or
  an active skip with the grid on screen — hidden tabs, timeouts, and
  no-tile landings teach nothing and don't count. The animated page
  flip is exempt from decay: it isn't a repeated lesson, it's live
  wayfinding to where this app lives, and its settle is needed anyway
  to put the tile in view for the morph. Every failure mode (slow or
  failed kv read) errs toward teaching once more, never toward never
  teaching.

Also bounds the intro's wait on the app-list load by the same deadline
as the tile wait — fetch has no timeout of its own, so a stalled
installedApps request used to hold the deep-link launch hostage
indefinitely.
2026-08-07 11:23:47 -07:00
jelveh 14c14fb5c5 feat(gui): animate the pager flip when a deep-linked app lives on a later page
The deep-link intro used to flip to the tile's page instantly behind the
grid's load-fade, so landings on off-page apps woke up on page N with
nothing but the pager dots hinting a move happened. Now the grid always
reveals on its first page, holds the grid beat, and then visibly travels
to the tile's page before the tile's flourish — the journey shows where
the app lives, which is also where minimize will put it back.

Smooth scrollTo has no reliable completion event across engines, so the
travel uses a settle allowance (DEEP_LINK_INTRO_FLIP_SETTLE_MS, same
pattern as the drag code's DRAG_FLIP_SETTLE_MS): the scroll's ~450ms
plus a rest so the landing reads before the tile pops. First-page apps
skip the flip entirely and are unaffected; the hidden-page bail-out is
re-checked after the flip settles.
2026-08-07 11:12:03 -07:00
jelveh 41fc8d890b feat(gui): play the tile click→morph→open intro on /app/<name> landings
A direct landing on /app/<name> now plays the same sequence a real
Apps-tab tile click does — grid appears, a beat, the tile's icon ghost
pops out of its slot, a beat, and the window morphs out of the tile —
so the landing shows the user what is being opened and where minimize
puts it back.

TabApps.beginDeepLinkLaunch waits (3s cap) for the tile to be genuinely
visible (list loaded, render done, pager flipped to the tile's page,
load-fade revealed, icon painted), paces the beats, and claims the app
in _launchingApps so a click mid-intro can't spawn a duplicate. The
launch's app-info fetch is prefetched in parallel so the intro never
delays the app's own round-trip. No tile, animations off, reduced
motion, or a background tab (hidden pages throttle timers and defer
rendering) all skip straight to the immediate plain-fade launch.

Also fixes the tab title sticking as the app's name after closing a
deep-linked app: the landing's replaceState('/') committed the
dashboard's history entry while the page still carried the server's
app-name title, and Chrome shows an entry's stored title when close/
Back traverses onto it — document.title is now reset before the
replaceState so the entry is stamped with the dashboard's own title.
2026-08-07 10:34:21 -07:00
Reynaldi Chernando 45bdc46df0 Remove deprecated models 7 aug (#3522) 2026-08-07 23:20:50 +07:00
jelveh 82ca390bf4 fix(gui): create-in-folder via sidebar right-click renders in target dir
Right-clicking a sidebar/breadcrumb folder in the dashboard Files tab and
choosing New > Folder (or any file type) created the item in the right-
clicked folder correctly, but the UI inserted the new row and started the
inline rename in whatever directory was currently open — so the item
appeared to be created in the wrong place until a refresh.

Now, when the creation target isn't the directory on screen, navigate to
the target and run the select + rename flow there; same-directory
creation keeps the incremental insert.
2026-08-07 08:49:13 -07:00
Neal Shah edfd67a83c fix PUT-1444 (#3515) 2026-08-06 17:11:04 -04:00
404oops ff16ec8900 Add Neuralwatt support (#3509) 2026-08-06 17:01:33 -04:00
Daniel Salazar c7edfc0c74 perf: improve metering perf + add extra compat for monthly charges (#3513)
closes PUT-1445 and PUT-1446
2026-08-06 10:26:47 -07:00
velzie 09f30d693c Peer: allow anontoken in place of authentication (#3501) 2026-08-06 02:03:53 -04:00
Reynaldi Chernando a9ec15d121 Update NPM readme (#3510) 2026-08-05 17:07:02 -07:00
Daniel Salazar eb53c842dd fix: oidc issues with cache (#3512)
closes #3497
closes #3502
2026-08-05 17:05:31 -07:00
Daniel Salazar 1be5ce45f7 feat: support dekstop app linking again (#3511) 2026-08-05 14:16:02 -07:00
Neal Shah 294055e055 PUT-1436, PUT-1435, PUT-1371 (#3507) 2026-08-05 17:11:50 -04:00
jelveh 036008d8a7 fix: keep uninstalled recommended apps from resurrecting in the Apps tab
Uninstall only revokes permissions, but the recommended launch list is a
global hardcoded set that knows nothing about per-user revokes — so an
uninstalled recommended app's tile came back on every reload. Persist
uninstalled app names in kv (dashboard_removed_apps) and filter only the
recommended merge against them; installedApps is never filtered, so a
genuinely (re)installed app always shows.
2026-08-05 11:01:14 -07:00
Daniel Salazar 6df292f495 fix: PUT-1429 (#3508) 2026-08-05 01:17:34 -07:00
Daniel Salazar 40c8d09f05 fix: PUT-1401 (#3504) 2026-08-04 16:37:33 -07:00
meridah7 4a211b9092 docs: document the provider option and refresh the vendor list in chat() (#3505) 2026-08-04 16:22:03 -07:00
meridah7 24fc0b0e5f fix(ai-chat): infron default model is dropped by the aggregator alias rule (#3506)
* fix(ai-chat): use a resolvable default model for the infron provider

* test(ai-chat): assert the infron default model is in the served catalog
2026-08-04 16:21:55 -07:00
Nariman Jelveh f58772ec33 feat: swap browser favicon to the focused app's icon in dashboard 2026-08-04 14:16:32 -07:00
Neal Shah eb830799c0 add limit bypass (#3500)
* add limit bypass

* update test
2026-08-03 19:43:59 -04:00
jelveh 8df72ef343 fix: dashboard app drawer icon popping in late
The drawer resolved its own icon URL (a differently-sized variant under a
different URL than the dashboard tiles use), so its <img> fetched cold and
the icon popped in mid-intro. Reuse the bitmap an Apps-tab tile or Home-tab
recent already decoded — instant from the image cache; when nothing is
rendered yet (deep links), fade the icon in on load instead of popping, and
fall back to the generic app icon on a failed fetch.
2026-08-03 11:50:43 -07:00
jelveh 63b2fc34d9 feat: highlight pasted items in dashboard files tab
Pasted (copied or cut) items now land selected, the same treatment
uploads get. copy_clipboard_items resolves with the created items'
paths instead of firing and forgetting, moveClipboardItems returns
each move response's authoritative final path (with Keep Both the
landed name differs from the source's), and both dashboard paste
entry points refresh with strong consistency and hand the new paths
to selectUploadedRows. Pastes into a folder that isn't rendered match
no rows and the highlight is a no-op.
2026-08-03 06:52:40 -07:00
Nariman Jelveh 33d7324a24 feat: 'Keep Both' option in name-conflict dialogs (#3496)
Pasting or moving onto an existing name only offered Replace or
Cancel. Add a macOS-style middle ground: Keep Both retries the
operation with dedupe_name, landing the item as "name (1).ext" and
leaving the existing item untouched.

- puter-js move.js now forwards dedupeName to the wire (copy already
  did; move dropped it).
- All four conflict dialogs offer the new button: copy_clipboard_items,
  copy_items and move_items in helpers.js, and the dashboard's
  moveClipboardItems. Multi-item selections show
  Replace / Replace all / Keep Both / Skip.
- New keep_both translation key (other locales fall back to English).
2026-08-03 06:39:18 -07:00
Nariman Jelveh 62be61642d fix: phantom name conflict when pasting onto a just-renamed path (#3495)
Renaming (or moving) an entry left the OLD path's cache key serving
the pre-update entry for the full 60s TTL: updateEntry invalidated
only keys derived from the updated row, whose path is already the new
one. Pasting an item under the freed name then hit the stale cache in
the collision check and reported a conflict for a file that no longer
exists — and accepting the Replace it offered deleted the renamed file,
since the stale entry carries its uuid.

Read the pre-update entry when the patch changes the path and
invalidate its keys alongside the new ones. The move flow was shielded
by the outer.gui.item.moved cache-invalidation handler at the
controller layer; rename had no such band-aid, and fixing the store
covers every caller regardless of which events fire.
2026-08-02 23:53:59 -07:00
Nariman Jelveh 8cebbcf3be fix: thumbnails breaking after copy — duplicate the S3 object per copy (#3494)
A copied fsentry kept its source's thumbnail pointer verbatim, so both
rows shared one S3 thumbnail object. fs.remove.node deletes the
pointed-to object, so the first removal among the sharers — e.g. the
remove performed by a replace-on-copy — broke every other sharer's
thumbnail. FSService already emits fs.copy.node for exactly this
reason (its doc comment describes the duplication), but the thumbnails
extension never subscribed to it.

Add the missing handler: S3-copy the thumbnail to a freshly minted key
and repoint the copied row, under the same only-keys-we-minted trust
rule as the read/remove paths. If the shared object is already gone
(pre-fix damage), null the pointer instead of leaving the row
advertising a thumbnail it doesn't have.
2026-08-02 23:22:24 -07:00
Nariman Jelveh 5c2a423658 fix: progress window covering paste conflict dialog; ghost row after Replace (#3493)
* fix: progress window covering paste conflict dialog; ghost row after Replace

Pasting a copied file onto a name collision buried the Replace/Cancel
dialog under a stuck 'Preparing...' progress window, and answering
Replace left a stale duplicate row in every client.

- helpers.js: copy_clipboard_items armed its delayed progress window
  with a 0ms timer (its siblings use 2s), so the window opened
  instantly over the dialog. Use 2s, and in all three of
  copy_clipboard_items / copy_items / move_items pause the timer while
  a conflict dialog (or the own-location / trash-deny alerts) is
  waiting for input, re-arming it after. A window that opens
  mid-operation now shows the current file instead of a stuck
  'Preparing...', and the trash-deny bail no longer leaks a timer that
  opened an orphan window after the operation ended.
- LegacyFSController: /copy and /move dropped the legacy 'overwritten'
  response field and never emitted item.removed for the entry an
  overwrite deleted, so clients kept a ghost row until re-listing.
  Resolve the entry before the operation, return it, and emit
  item.removed on success.
- helpers.js: copy_items read resp[0].overwritten but removed
  resp.overwritten (always undefined), and the data-uid cleanup
  selectors were unquoted — invalid CSS when a UUID starts with a
  digit. Fixed all three sites.

* test: pin the v1 overwrite/collision wire contract for move and copy

The collision tests asserted only statusCode 409, which is how the
item_with_same_name_exists code regressed to 'conflict' unnoticed and
broke every replace/skip prompt in the GUI. Assert the legacy code and
entry_name explicitly, and add controller tests for the overwrite path:
the replaced entry must ride along as 'overwritten' in the /copy and
/move responses and be announced via outer.gui.item.removed so clients
drop its row.
2026-08-02 22:17:10 -07:00
Nariman Jelveh 1a9f025db6 fix: surface name collisions when pasting instead of failing silently (#3492)
Cut+paste onto a folder containing an item with the same name failed
with no feedback: the dashboard's moveClipboardItems swallowed the
error into console.error and cleared the clipboard, and the rewritten
backend broke the v1 wire contract the GUI's conflict prompts key on.

- FSService: move() and copy() collisions again return
  item_with_same_name_exists + entry_name (the contract the write path
  already preserves) instead of a generic 'conflict' code. This also
  restores the desktop's existing Replace/Skip dialogs, which were
  silently broken against the new backend.
- Dashboard: moveClipboardItems now mirrors the desktop's move_items
  conflict flow (Replace / Replace all / Skip / Cancel, retry with
  overwrite) and surfaces other move errors in an alert.
- keyboard.js: the desktop's global Ctrl+V handler threw an uncaught
  TypeError on every paste in dashboard mode (its file container has
  no data-path); guard it — which also prevents a double-paste if
  dashboard containers ever gain one.
2026-08-02 21:33:01 -07:00
jelveh a4667073f5 Update RecommendedAppsService.ts 2026-08-02 20:55:33 -07:00
jelveh 23d251e482 fix: dashboard files list blanking during same-directory refresh
Re-rendering the directory already on screen (after an upload, sort
change, undo, etc.) used to clear the list and show a spinner before
the readdir round-trip, blanking the pane for the whole fetch. Keep
the current rows visible until the fresh listing arrives, then swap
the DOM in one pass and restore the scroll position. Navigation to a
different directory still clears immediately.
2026-08-02 20:30:20 -07:00
jelveh 0c9dfb03e2 Show action bar only in mobile select mode 2026-08-02 19:49:14 -07:00
Nariman Jelveh 6dc928e0c3 fix: file picker dialogs going behind parent window when dragged by titlebar 2026-08-02 17:38:25 -07:00
Daniel Salazar 84582ee33d fix: alarm channels (#3491) 2026-08-01 15:55:06 -07:00
Daniel Salazar 116d6e6663 tests: big test push for better coverage (#3490) 2026-08-01 14:29:59 -07:00
Daniel Salazar 7d0d44aef9 feat: slack alarms (#3489) 2026-08-01 13:12:41 -07:00
Daniel Salazar 3ac6c8532e fix: open ai cost (#3488) 2026-08-01 12:34:36 -07:00
Nariman Jelveh f6a50c4fb4 Update recommended apps list order
Reorders and updates the recommended apps list: removes 'butler', 'code', and 'traffic-tap-puzzle'; adds 'contacts', 'diagram', and 'basketball-tap' (relocated from end); adjusts overall ordering of several apps.
2026-08-01 12:26:44 -07:00
Daniel Salazar c8113d7514 fix: auth message popups (#3487)
* fix: auth message popups

* remove v1 auth
2026-08-01 10:45:16 -07:00
jelveh 6159bc9ae1 Polish dashboard settings button styling
Update `.dashboard-settings-card .button` to use dashboard theme tokens for text, background, and borders, and add clearer hover/focus/active states with subtle elevation and transitions. Also override disabled styles in this context so dark mode no longer shows hard-coded light greys from the base `.button:disabled` rule.
2026-08-01 00:27:27 -07:00
Daniel Salazar 3a11d0d1f1 feat: sandboxed app workers (#3486) 2026-08-01 00:02:06 -07:00
jelveh 535cf5753e Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-31 14:14:11 -07:00
jelveh cee4a6ccce Improve app reorder mode on touch devices 2026-07-31 14:14:04 -07:00
Daniel Salazar 1f1f95c2f8 fix: auth me for local dev (#3484) 2026-07-31 14:04:59 -07:00
jelveh f600e857b0 GUI: touch reorder mode for the dashboard Apps tab
Long-press-to-drag can't be made reliable on touch: touch-action is
consulted at gesture start, so the pager's pan-x claims the finger
before a drag can begin (worst on iOS). Replace it with an explicit
edit mode - a cog button (touch-primary devices only) enters it, tiles
jiggle and drag on first movement, iOS-style x badges uninstall, and
Done exits. Drops still persist immediately, same as desktop.

Also raise the drag ghost above the window z-index bands; it was
rendering invisibly behind the fullpage dashboard window.
2026-07-31 13:46:17 -07:00
Neal ShahandDaniel Salazar 08d1708378 change cors auth path (#3464)
* change cors auth path

* undo oidc ref changes

* scary OIDC state changes

* fix: bad cors signin

* puterjs changes

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-31 01:55:01 -04:00
Nariman Jelveh f1bc065730 GUI: fix dev-server port-retry crash under Express 5 (#3482)
Express 5's app.listen wraps the listen callback in once() and also
invokes it on 'error', with the error as the first argument — at which
point server.address() is null, so the startup log threw a TypeError
and killed the process before the EADDRINUSE handler could try the
next port. Bail out of the callback when it receives an error and let
the 'error' listener own the retry.

Verified: with 4000 occupied the server now logs the retry and comes
up on 4001; with the port free it binds 4000 as before.
2026-07-30 18:11:12 -07:00
Nariman Jelveh b55782f066 GUI: add npm start --server=<domain> to run the GUI against a remote backend (#3481)
- npm start --server=puter.com (or -- --server=...) skips the local backend
  and serves the bundled GUI locally, pointed at the remote server's API.
  Bare domains resolve to https://api.<domain>; full origins are used
  verbatim. gui_origin points at the remote origin so /whoarewe, login,
  anti-csrf, socket.io, and builtin apps hit the real backend (CORS-open).
- --extensions=<dir>[;<dir>...] bundles out-of-tree GUI extension
  directories (sugar for PUTER_GUI_EXTENSION_PATHS). Their imports resolve
  as if the files lived in src/gui/src/extensions, with the extension's own
  files taking precedence, and bare imports fall back to the repo-root
  node_modules.
- Fix the bit-rotted dev-server: Express 5 wildcard routes, pass gui()
  params (previously called with none), inject the service_script shim,
  load bundle.min.js + bundle.min.css in prod mode, serve /sdk, and
  properly await the webpack build (it previously resolved immediately).
2026-07-30 16:29:54 -07:00
jelvehandClaude Fable 5 2c9ae3489e Suggested apps: rank registered apps above the editor fallback
For extensions with no intentional built-in mapping (doc, docx, and
every other unmapped type), suggestionsForExtension fell back to
['editor'], and #resolveForExtension always placed built-ins ahead of
apps from app_filetype_association. Since suggested[0] drives the GUI's
double-click open path and /open_item, a .docx defaulted to opening as
plain text in editor even when a word processor explicitly registered
the extension.

Tag the unknown-extension result as a fallback and order third-party
filetype-association apps ahead of it. Intentional mappings (code, txt,
md, images, pdf, media) keep built-ins in the head slot as before, and
the editor guess still appears as a last-resort option.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-30 13:51:01 -07:00
Daniel Salazar b28b41c90b otel: better telemetry numbers (#3480) 2026-07-30 13:23:16 -07:00
Nariman Jelveh f34c4dc335 Apps: normalize filetype associations to bare lowercase extensions (#3479)
* Apps: normalize filetype associations to bare lowercase extensions

Suggested-apps lookups match app_filetype_association rows against the
bare lowercase extension ('docx'), but writes stored whatever the
developer typed. Rows like '.docx' never matched, so those apps
silently dropped out of Open With suggestions.

AppStore now canonicalizes on write (trim, lowercase, strip leading
dots, dedupe, drop empties) and tolerates the dotted legacy form on
read: getAppsByFiletype normalizes the requested extension, matches
both 'docx' and '.docx', and dedupes apps associated under both forms.
Cache invalidation keys are normalized the same way. Existing dotted
rows work without a data migration.

* Update apps tests for extension canonicalization

Adjust apps API tests to match current normalization behavior for `filetypeAssociations`: extension values are stored as lowercase bare extensions (e.g. `.txt` -> `txt`), while MIME types remain unchanged. Added inline comments in both test suites to document this expected remap.
2026-07-30 13:13:14 -07:00
Daniel Salazar 5a157197b6 fix: PUT-1398 (#3478) 2026-07-30 01:42:40 -07:00
Daniel Salazar 8a711e0254 driver controller change (#3477)
* fix: launch app

* driver controller change
2026-07-30 00:41:29 -07:00
jelveh a22321daa2 Dashboard: closing a launched app goes home, not into its launcher
An app launched by another app minimizes its launcher (iOS-style
takeover). Closing the child then popped onto the launcher's history
entry, and the popstate handler restored it — so quitting an app threw
the user into a different full-screen app they had not asked for, zoom
animation and all. Closing an app should go home.

Mark the launcher when it is minimized FOR a child
(data-minimized_for_child). If it still carries that mark when the child
closes, the close's single history hop lands on the dashboard instead:
the entry is rewritten to the dashboard's route and the launcher is left
minimized, still running behind its tile's running dot. No extra history
traversal — the joint session history an app's iframe shares is exactly
what pop_dashboard_app_url's watchdog exists to survive, so the hop
count is unchanged.

Back is untouched and still returns to the launcher — that is the
navigation gesture, whereas close dismisses. Any restore (Back, tile
click, Forward) clears the mark, so quitting a child after the user has
brought the launcher back leaves the launcher alone.
2026-07-29 22:04:10 -07:00
jelveh fbf1bd18b1 Dashboard: an app launched by another app takes over the tab
In dashboard mode apps are full-tab experiences, but an app launched by
another app (puter.ui.launchApp) opened as a floating titlebar window
over its full-tab parent: the parent stayed on screen around its edges,
couldn't be raised (focusWindow never raises stay_on_top windows), and a
child with no Apps-tab tile had no switcher to come back to once
minimized.

Give child launches the same treatment as tile launches — maximized, so
they get the headless chrome and control drawer — and minimize the
parent behind them, iOS-style. State only: the parent's /app/<name>
history entry already sits beneath the child's, so Back from the child
(and the child's close, which consumes its own entry) lands on the
parent's entry and the existing popstate handler restores it. The parent
keeps running while hidden, so parent/child IPC is unaffected.

Explorer keeps its windowed form, background apps don't minimize their
parent, and an explicit `maximized` option still wins. Desktop mode is
untouched: both changes are gated on is_dashboard_mode, so an app
launching an app there still gets a floating child over a visible
parent.
2026-07-29 21:36:47 -07:00
jelveh dc371fc4eb Windows: keep dialogs and their apps in the stay-on-top z band
focusWindow re-raised a focused window's parent (and children) with the
bare z counter. For a file dialog owned by a fullpage/dashboard app that
DEMOTED the app out of the 99999999+ stay-on-top band it was created in,
burying app and dialog under every other open app window — opening a
file picker from an app stacked over another (e.g. an app launched from
Dev Center) made both vanish beneath the window below. The dialog itself
had the same flaw: raised into the plain counter band, under every app.

Raise each window within its own stacking band instead: stay-on-top
windows (and windows hanging off one through parent_uuid, walked up the
chain) get 99999999 + counter; everything else keeps the bare counter —
so desktop stacking arithmetic is unchanged. Same demotion family as the
showWindow restore fix in #3427.
2026-07-29 21:11:21 -07:00
jelveh 97e4e78df4 Dashboard files: drop the root crumb from the breadcrumb
Every path rendered as "> Puter > user > ...", but the root crumb is
noise on anything beneath it. Render it only at the root itself, where
it's all there is to show (the Up button can still reach /, and an
empty breadcrumb bar there would strand the user). Each crumb keeps
its leading caret, including the first.
2026-07-29 20:12:22 -07:00
jelveh 007649296b Keyboard: stop swallowing Enter when nothing claims it
The global Enter handler preventDefault'ed unconditionally, then
returned without acting unless a launch-menu item, context-menu item,
or selected item existed. That suppressed the browser's native Enter
activation on whatever was focused — a focused link or button (e.g. a
dashboard sidebar item, a dialog button) did nothing on Enter.

Move preventDefault/stopPropagation into the branches that actually
handle the key; otherwise fall through so native activation runs. The
handled paths are unchanged: Enter still opens the selected item,
launch-menu entry, and context-menu entry.
2026-07-29 19:40:16 -07:00
jelveh 9392aed402 Dashboard: restyle keyboard focus ring on sidebar controls
Sidebar items (and the user/collapse/close buttons) had no
:focus-visible style, so keyboard focus painted the UA's default blue
ring — hard-edged and bleeding outside the item. Use the dashboard's
own selection ring instead, tucked inside the control so it hugs the
rounded corners. :focus-visible only, so mouse clicks never paint it.
2026-07-29 19:40:06 -07:00
jelveh 32e417fe79 Dashboard files: clicking an open file returns to its window
In dashboard mode there is no taskbar, so a minimized app window's only
switcher was its Apps-tab tile — invisible from the Files tab. Opening a
file, minimizing, and clicking the file again launched a second instance
of the app, stranding the first (and any unsaved edits) somewhere
unreachable.

Make the file row itself the switcher:

- launch_app stamps the opened file's uid on the window (data-file_uid;
  the signature's uid wins so shortcuts resolve to their target)
- open_item (dashboard mode) restores/focuses an existing window that
  has the file open instead of launching a duplicate, mirroring the
  Apps-tab tile's single-instance behavior — keyed by file, not app, so
  opening a different file still gets its own instance
- re-clicks while a launch's fetches are still in flight are swallowed
  (same idea as TabApps._launchingApps, keyed by file uid, TTL'd so a
  failed launch can't swallow clicks forever)
- dashboard file opens now default to maximized, so they get the same
  headless full-tab chrome + control drawer as tile launches instead of
  a floating titlebar window
- rows show a dot while their file is open in a (possibly minimized)
  window — under the name in grid view, inline after it in list view —
  driven by the existing dashboard-app-windows-changed event

Desktop mode is untouched: the reuse branch and maximized default are
gated on is_dashboard_mode, and opening the same file twice there still
creates two windows as before.
2026-07-29 19:03:33 -07:00
Daniel Salazar 0e1f617401 fix: errors for bad keys (#3475) 2026-07-29 15:58:53 -07:00
Felix-Ayush bd6efaaa4e fix(puter-js): ignore peer offers for unknown connections (#3473)
createAnswer ran outside the connection null-check, so stale or unknown
offer ids threw on the signalling socket. Guard the whole offer path and
return the #message promise from onmessage so callers can observe errors.
2026-07-29 15:58:39 -07:00
Nariman Jelveh 2b6dc528ce Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-29 15:48:21 -07:00
Nariman Jelveh 9368ad57b6 Preserve UI state across tab visibility refresh 2026-07-29 15:47:59 -07:00
Felix-Ayush e97d338f3a fix(ai-chat): preserve max_tokens 0 for Alibaba, DeepSeek, and Claude (#3472)
These providers used `max_tokens || default`, so an explicit zero was
replaced by the fallback. Use nullish coalescing so 0 is forwarded.
2026-07-29 18:38:31 -04:00
Nariman Jelveh a87a517b82 Anchor file row menu to button edge
File row overflow menus in the dashboard now open anchored to the ⋯ button (below it with right-edge alignment) instead of pointer position, improving placement consistency. UIContextMenu now supports `position.right` for right-edge pinning, and the ⋯ trigger keeps an active visual state while its menu is open so hover styling doesn’t drop when the cursor moves onto the menu.
2026-07-29 15:25:51 -07:00
Nariman Jelveh dbb52171c8 Refine dashboard file row hover styling 2026-07-29 15:05:21 -07:00
Daniel Salazar 6c020318f9 perf: traces and perf improvements (#3471) 2026-07-29 14:35:15 -07:00
Nariman Jelveh e16994db3b Fix I-beam cursor on dashboard file rows in Safari
Safari ignores user-select: none for cursor styling, showing a text I-beam over file rows. Setting cursor: default explicitly fixes this and is inherited by child elements.
2026-07-29 14:18:50 -07:00
Daniel Salazar 8491a0b55d fix: limit api pointing to apps only (#3470) 2026-07-29 14:03:27 -07:00
Felix-Ayush 710e4da4c0 fix(puter-js): size img2txt data URIs by decoded bytes (#3457)
Match speech2txt/speech2speech and reject on decoded payload size, not
data-URI string length. Base64 expands ~4/3, so string-length checks
falsely rejected valid images under the 10MB limit.
2026-07-29 12:21:21 -07:00
Felix-Ayush fd1f4e2ac2 fix(puter-js): forward temperature 0 and max_tokens 0 in ai.chat (#3455)
Truthy checks dropped legitimate zero values, so deterministic sampling
never reached the driver. Match the stream option and use !== undefined.
2026-07-29 12:21:15 -07:00
Kobi Hikri 70bf03a6e7 ci: attach provenance and SBOM attestations to the published image (#3463) 2026-07-29 12:21:02 -07:00
Felix-Ayush 6f90d7a199 fix(ai-chat): forward temperature 0 and max_tokens 0 to providers (#3456)
Completions-family providers used truthy spreads, so temperature: 0 and
max_tokens: 0 were stripped before the SDK call. Align with Responses
providers that already use !== undefined.
2026-07-29 11:49:36 -07:00
Daniel Salazar f0b1947c54 feat: make uninstall also revoke + dashboard css changes (#3469)
* feat: make uninstall also revoke

* fix: wording for free trial
2026-07-29 10:40:43 -07:00
Daniel Salazar 92193ff11f chore: kill collector (#3466) 2026-07-28 19:02:49 -07:00
Daniel Salazar f9396b7283 fix: thumbnail res (#3465) 2026-07-28 17:31:01 -07:00
Sauhbagya PrasadandDaniel Salazar 7cf3e03433 Replaced Nginx with Caddy (#3378)
* Replace nginx with Caddy

* updated caddy location

* seprate logic for local and with domain caddy setup

* updated install

* updated install scripts

new ps1 script with caddy, and removed instances of nginx

* fix Caddy Host routing, restore TLS/SELinux/healthcheck, update docs

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-28 14:52:13 -07:00
Neal Shah 0a61b4e78b add fixes 1 (#3458)
* add fixes 1

* change socketio test

* clarify that success/error callbacks are legacy

* fix wisp issues
2026-07-28 17:34:12 -04:00
Reynaldi Chernando 1db4324c02 remove deprecated ai models (#3444)
* remove deprecated ai models

* test

* fix tests
2026-07-28 12:41:04 -07:00
Daniel Salazar 69529ee0db perf: try to improve app opens speed (#3462)
* perf: try to improve app opens speed

* make /rao non blocking to allow faster conn swap
2026-07-28 11:12:44 -07:00
Daniel Salazar 7a7ff47442 chore: bump version (#3461) 2026-07-27 23:59:51 -07:00
Daniel Salazar 148771a82b fix: performance degradations in app fetching + puter-js module parent class (#3460)
* fix: centralize module methods in parent class

* fix: performance degradations in app fetching
2026-07-27 23:41:51 -07:00
Daniel Salazar 5bad6a972f fix: puter-js cleanup around docs and ai resolution (#3459)
* fix: puter-js cleanup around docs and ai resolution

* more jsdoc stuff

* fix: docs

* fix: doc types + misc hardening

* fix: harden auth
2026-07-27 22:31:05 -07:00
Luke Oliff 8bb7d59527 feat: add Speechify TTS driver (puter-tts) (#3453)
* feat: add Speechify TTS driver

Adds a SpeechifyTTSProvider under the ai-tts driver (mirrors the xAI/
ElevenLabs REST-provider shape), registered in TTSDriver alongside the
existing providers. Wires puter-js txt2speech with provider: "speechify"
support, default voice/model, and the speechify-tts driver alias.

Every outbound request sets Speechify-Caller: puter; base URL is
https://api.speechify.ai only; default model is simba-3.2.

* fix: replace placeholder voice IDs with real simba-3.2/simba-english voices

DEFAULT_VOICE and the starter voice catalog used invalid IDs (henry,
cliff, kristy, george, aria) that the real API rejects with 400. Swapped
in confirmed-real voices from a live GET /v1/voices call: geffen_32,
dominic_32, harper_32, hugh_32, imogen_32, and alec for the
simba-english model-override test.

* fix: purge remaining placeholder voice IDs from client, docs, and types

The previous fix only covered the backend provider — the client-side
default in tts.js, its test, the txt2speech docs page, and the ai.d.ts
type comment all still referenced the invalid henry/cliff/kristy/george/
aria set. All replaced with the live-verified voices (geffen_32 default).
Also corrects the docs link to docs.speechify.ai.
2026-07-27 17:04:11 -07:00
Neal Shah 8732494442 dead code cleanup (#3441) 2026-07-27 17:51:15 -04:00
Daniel Salazar cade75ac9b fix: PUT-1385 PUT-1382 PUT-1381 PUT-1380 PUT-1379 (#3454) 2026-07-27 11:42:40 -07:00
meridah7 9abc9d1a19 feat: add Infron AI chat provider (#3433)
Adds Infron (https://infron.ai) as an aggregator provider for the
puter-chat-completion driver, following the OpenRouter provider as
reference per doc/contributing-apis.md.

- OpenAI-compatible gateway at llm.onerouter.pro/v1
- Dynamic model catalog (kv-cached 15 min, authenticated fetch),
  filtered to chat-capable LLM entries
- Bills the gateway-reported authoritative cost when present
  (usage: {include: true}); falls back to catalog per-token pricing
- Registered as an aggregator so its aliases never shadow
  first-party providers
- Offline unit tests (mocked SDK/axios against a real test server)
  plus an env-gated integration test
2026-07-27 08:38:04 -07:00
Nariman Jelveh f3fd8a30da Rework permission requests: new dialog, working popup flow for websites (#3447)
* Rework permission requests: new dialog, working popup flow for websites

- Replace the UIWindow-based permission prompt with a standalone top-layer
  <dialog> (responsive, light/dark, app/site identity, input protection)
- Implement puter.ui.requestPermission for env=web: opens the GUI's
  /action/request-permission popup with pinned origin/source/msg_id,
  popup-closed detection, and a check-permissions polling fallback for
  crossOriginIsolated openers
- Move the GUI's request-permission action into postAuthActions so
  signed-out users sign in first; identify the app by opener origin,
  correlate responses with original_msg_id, close the popup after answering
- Always respond from the IPC handler so the SDK promise can't hang;
  normalize the result to a strict boolean (a failed grant no longer
  resolves truthy)
- Accept origin in /auth/grant-user-app and /auth/revoke-user-app,
  mirroring grant-dev-app (fixes puter.perms.grantOrigin/revokeOrigin)
- Add Playwright e2e coverage for both the desktop and popup flows;
  update e2e harness for the auth_token_v2 localStorage key
- Update types and docs (perms request methods now work on websites)

* Harden permission request flows

- Grace period before treating popup close as denial: the GUI posts the
  decision then closes the popup, and postMessage delivery is not ordered
  relative to `closed` becoming true, so a grant could race to a false
- Unique popup window name per request so window.open name-reuse can't
  hijack a still-pending request's popup
- request-permission action always answers the requester and closes the
  popup, even when app resolution or the dialog throws
- Permission dialog: refuse unidentifiable requesters, allowlist icon URL
  schemes, and time out the grant request into the retryable error path
- Validate app_uid/origin/permission types and length in grant-user-app
  and revoke-user-app
- Tests: revoke-by-origin and input-validation backend tests; e2e for
  dialog dedup, unsupported permissions, and the consent-dialog path

* Sign in first in the requestPermission fixture's email flow on the web

In env=web the site has no auth token, so whoami threw 401 immediately
and the email button appeared to do nothing. Sign in via popup first,
matching the real third-party flow; env=app already has a token and is
unaffected.

* Point the requestPermission fixture at the real api subdomain

The SDK sends credentialed CORS requests; the GUI host doesn't answer
with Access-Control-Allow-Credentials, so whoami (and any authed call)
from the fixture origin failed CORS and looped through retries.

* Fix /auth/list-permissions schema mismatches

The endpoint's queries referenced columns that don't exist:
user_to_app_permissions stores a numeric app_id FK (not app_uid), and
user_to_user_permissions uses holder_user_id (not target_user_id) —
every call 500'd. Join apps to expose the app's uid and use the real
column names.

Replace the catch-either-branch test (which documented the breakage
instead of failing on it) with real assertions covering all three
sections of the response.

* Identify permission requesters by origin only

The request-permission action took `app_uid` straight from the query
string and used it as the grant target whenever the origin was absent or
unresolvable. Now that /auth/grant-user-app accepts `origin` and prefers
`app_uid` when both arrive, the displayed identity and the grant target
could diverge; with only `app_uid` in the URL the dialog rendered with an
empty name, so a link could produce a bare "Allow" prompt for an unnamed
requester. Resolve the uid from the origin alone, and let the server
resolve it from that same origin when the client lookup fails.

Also give the no-gesture consent popup a unique window name. UI.js does
this on the direct path because window.open() reuses a window with a
matching name, but the PuterDialog fallback opened under the default
'Puter' — the same name sign-in uses, so a consent click could navigate
an in-progress sign-in popup away.

And guard the IPC responder: an app that closes its own window while the
dialog is up leaves target_iframe.contentWindow null.

* Keep the permission popup from signing the site in

The popup loads the GUI with embedded_in_popup=true, so it ran the
sign-in token exchange and posted puter.token to the opener before the
user answered the prompt. A site that called requestPermission() walked
away holding a user-app token for the account even when the user pressed
"Don't Allow" — and because the SDK's global puter.token handler feeds
event.data.token into setAuthToken() without looking at `success`, a
failed exchange posted token: null and wiped a token the site already
had. Keep running the exchange (it bootstraps the app row the grant
needs and caches host_app_uid) but leave the token in the popup. A site
that wants credentials still has to call signIn().

Escape on the SDK's consent dialog left the caller pending forever.
PuterDialog wired its Cancel and close buttons but not the <dialog>'s
native cancel event, so the browser dismissed the dialog and nothing
reported it: no dialog, no popup, no answer. Route cancel to the same
handler. Programmatic close() fires only `close`, so launching the popup
— which closes this dialog — is unaffected, and the implicit-auth flow
stops hanging on Escape too.

Serialize the permission dialogs. showModal() makes the whole document
inert rather than just the requesting app's window (which is what the
UIWindow it replaced did), and the dedup map only coalesced identical
requests, so an app asking for permissions in a loop stacked one modal
per request and walled the user off from the desktop — including from
the app doing it. Prompts now queue and open one at a time, and each
caller still gets its own decision.

Identify apps by more than their title. `title` is free-form text the
author picks and is not unique, so it was the whole identity of a prompt
an app titled "Puter Settings" could raise; the registered `name` is
unique and format-restricted, so show it underneath. Give the name line
the unicode-bidi isolation the origin line already had, since escaping
leaves bidi overrides intact.

Stop the dialog from answering over its own in-flight grant: a dismissal
while the POST was outstanding resolved false for a permission the server
was committing. Ignore dismissals while granting, and time-box the
request with AbortController (AbortSignal.timeout isn't everywhere) so a
hung network can't leave a modal no one can close. Fail closed on the
remaining paths that could reject or prompt uselessly — showModal()
throwing under <iframe sandbox>, and a requester known only by app_name,
whose Allow the server would always reject. Pass the error string to
.text() unencoded so translations containing an apostrophe don't render
&#39;.

The e2e suite covers all of it; each new test fails without its fix.

* Restrict the permission popup flow to third-party websites

requestPermission's new web path ran in every environment with a window,
including env='gui' — so a permission_denied driver retry inside the
Puter GUI would open a popup to the Puter origin from the desktop itself
and try to grant the permission to a phantom app for Puter's own origin.
Resolve false everywhere except env='web', the previous behavior.

* Settle the permission dialog when a failed grant has no dialog left

The cancel handler is preventDefault'd, but close requests can't be
suppressed forever: Chrome's close watcher lets a repeated Esc skip
cancel and force-close the dialog while the grant POST is in flight.
The close handler defers to that grant on purpose — but if the grant
then failed, fail_grant re-enabled buttons on a closed dialog and the
promise never settled, leaving the requesting app waiting forever.
Settle as a denial when the dialog is no longer open.

Also add regression tests for this and for the popup-flow env guard
(routing the CDN SDK URL to the local build, since the prod-built GUI
loads its SDK from js.puter.com).

* Withhold the auth token on the popup's first-visit paths

Keeping the token inside the permission popup only covered the plain
token exchange. Two other popup paths mint a user-app token and posted
it to the opener unconditionally: first-visit temp-user creation, and
the manual signup shown when temp users are refused. Both sit on the
path a brand-new visitor takes — the audience the website popup flow
exists for — so a site that asked about one permission and was denied
still walked away holding a token, for a temp account or a real one.
The SDK's global puter.token handler feeds whatever arrives straight
into setAuthToken(), so posting it is the whole of it.

Move the rule into util/popupAuth.js and consult it at every site that
posts the token, so the next token path has one place to ask.

The first-visit path also left the prompt itself unreachable: it waits
on the spinner promise, which only resolves when the spinner was up for
under 2s. End that wait for any action that keeps the popup open;
sign-in still closes the window as before.

The e2e test fails without the fix — the site holds a token after the
user presses "Don't Allow".

* Poll for the decision when the popup's opener is severed

crossOriginIsolated was the test for "the popup can't message us back",
but being isolated also requires COEP. A site sending COOP: same-origin
on its own still has its opener relationship severed when it opens the
Puter popup, and took the watch-the-window path instead — where the
detached proxy reports closed === true on the first tick, so
requestPermission resolved false about a second after the popup opened,
while the user was still reading the dialog. Their "Allow" then had
nowhere to go. Treat an already-closed popup as severed and poll.

Pin the expected event.source before those early returns. popupWindow
was assigned after them, so for the whole consent-dialog wait — as long
as the user takes to click Continue — the handler accepted a decision
from any window on the GUI origin. A forged answer is only advisory
since the grant is written server-side, but the check may as well hold.

Settle instead of rejecting when the consent dialog can't be appended:
document.body is null in a <head> script, and the throw both rejected a
promise documented to resolve to a boolean and left the message listener
behind.

* Key the dialog dedup by the identity its gate accepts

The gate treats an empty app_uid as absent and falls through to the
origin; the dedup key used ?? and kept the empty string, so two requests
from different origins would collide on one key and share a single
decision. No caller can produce a blank uid today — server uids are
never empty and the IPC path's empty attribute is stopped by the gate —
but the two lines have to agree.

* Close the gaps the permission-request flow left open

Seven defects found reviewing the new permission flow end to end, each
reproduced against a running server before being fixed.

Security:

- `cross_origin_isolated=true` bypassed `deliversTokenToOpener` entirely.
  That branch is checked first, mints a user-app token, publishes it via
  `/login/set` and returns — so one query parameter on a
  request-permission URL skipped the prompt and handed the opener a token
  through the unauthenticated `/login/wait`. Gate it with the same rule.

- Grant/revoke by `origin` could land on an unrelated app. An origin with
  no app row synthesises `app-<uuidv5>`, and the permission services
  resolve their identifier as uid *or name* — and the uuid namespace is a
  source constant, so the string is computable offline and registrable as
  an app name. Resolve origins to a uid that names a real app row.

- A website's host was elided on the right, hiding the registrable domain
  that says who is asking. Elide it from the left, as the sibling rule
  already intended.

- A grant whose response was lost (client-side abort, dropped reply) left
  the row committed while the dialog reported a denial. Withdraw it when
  the user then answers "Don't Allow".

Correctness:

- `pollDecision` needs the site's own token, which a permission popup
  deliberately never delivers, so a signed-out cross-origin-isolated site
  burned the full five-minute timeout before answering. Answer at once
  when there is nothing to poll with.

- `getUserAppToken` reports failure by returning null, and three callers
  read `.app_uid` off it. Guard all three, keep the first-visit spinner
  promise settling on its failure paths, and dispatch the `login` event on
  the manual-signup path so `postAuthActions` runs at all — a user who
  signed up inside a permission popup got a blank window and the site got
  no answer.

- Time-box the lookups that run while a request holds the dialog queue's
  slot: they have no timeout of their own, and a stall (not a failure)
  wedged every later permission request in the page.

Also harden the grant/revoke input validation the PR introduced — it
skipped `extra`/`meta`, so a non-object faulted *after* the row was
written, and its length cap was 16x the column it lands in — stop a
non-URL `origin` from throwing past the answer-and-close, and drop the CSS
left behind by the deleted dialog.

* Close three gaps left in the permission-request flow

Each was reproduced first — the squatting grant against a running server,
the COOP timing in a real browser — and each fix was then confirmed by
reverting it and watching the new test fail.

Security: the dialog could name one site and grant to another.

The squatter guard added for grant/revoke by `origin` only ran when
`app_uid` was absent, and the dialog sends both — so `app_uid` won and the
guard never applied. `getAppUIDFromOrigin` returns the synthetic
`app-<uuidv5(origin)>` for any origin with no app row of its own, and the
grant endpoint resolves `app_uid` as uid *or name*, so the grant landed on
whoever registered an app under that computed name (the format allows it,
and the namespace is a source constant). A link like
`/action/request-permission?origin=https://a-site-you-trust.example` named
that site in the prompt while "Allow" handed the permission elsewhere.

Fixed on both sides of the wire. The action now sends the origin alone —
no uid resolved in the browser is safe to forward, whatever its source —
and a supplied `origin` now decides the target on the server even when an
`app_uid` travels beside it: the origin is what the prompt showed the
user, so it is what the grant has to follow.

Correctness: a COOP-only site was answered before the user decided.

7efc0c0b routed a severed opener to `pollDecision` by treating an
already-closed popup as severed, but that reads `closed` synchronously
after `window.open()` — before the navigation whose response headers cause
the severing has committed. Measured in Chromium: `closed` is false at
0ms and true by 200ms. So a site sending COOP: same-origin without COEP
still took the watch-the-window path, and `requestPermission` resolved
false 1.1s after the click while the dialog was still on screen. The
"Allow" that followed committed a grant the site had been told it did not
get. Tell the two apart by when the close lands, and keep the in-flight
message's grace period on both branches — an answer already on its way
outranks whatever the close is taken to mean.

Correctness: a grant that timed out was not withdrawn.

`grant_may_have_committed` was only set in the fetch's `catch`, which
cannot run until the timeout's timer callback returns — and that callback
already calls `fail_grant`, which settles the dialog as a denial outright
when the dialog was force-closed mid-grant. The reconciliation was
skipped in exactly the case it exists for. Record the unknown outcome in
the timer, where the timeout already means the request left the browser.

Also require a `token` from the user-app exchange rather than just a
non-null body: an HTTP failure (a blocked origin, a 5xx) returns the
parsed *error* body, which is truthy, so the guard added for this missed
it — handing the opener an `undefined` token, and prompting for a grant
whose app row was never bootstrapped.

Known limitation, now more reachable: a severed opener cannot signal a
denial at all, since nothing is written for one, so those sites wait out
the poll timeout before receiving false. A grant still resolves promptly.

* Deliver the uncertain-grant withdrawal from a closing popup

The permission dialog reconciles a denial after an uncertain grant by
firing a revoke in the background, but the popup flow posts the answer
and closes the window right after settling — and a plain fetch is
cancelled with its document, so the withdrawal never reached the server
and the user was told "denied" while the grant stayed live. Send it
with keepalive so the browser delivers it independently of the popup,
and cover the popup flow with a regression test (the existing
withdrawal tests only exercise the desktop flow, where the GUI
outlives the dialog).

Also make the popup boot's getAppUIDFromOrigin guard functional: the
helper reports failure by resolving to a null/undefined uid, not by
throwing, so the catch never engaged and a failed lookup clobbered
window.host_app_uid with undefined despite the comment claiming the
token exchange's value was kept.

* Keep the ai-chat model-map build inside the server lifecycle

onServerStart fired #buildModelMap without awaiting or tracking it, so
the network fetches it does (notably Ollama auto-discovery, which is
enabled by default and doomed on any machine without a local Ollama)
kept running after server.shutdown() resolved. In vitest that let the
provider's console.error land during worker teardown, which surfaces as
"Closing rpc while onUserConsoleLog was pending" — the unhandled error
that intermittently fails CI (last seen attributed to
WispController.test.ts). A rejection in the detached chain would also
have been an unhandled rejection.

Track the promise, catch and log rejections, and await it from
onServerShutdown so no provider I/O or logging outlives the server.
Also disable Ollama auto-discovery in setupTestServer's defaults —
every test server was firing a pointless model-list fetch at localhost.

* Settle requestPermission on the launch paths that could still throw

Three gaps left by the permission-request rework, each verified against a
live stack before and after the fix.

The IPC handler normalises a non-object `options` so it can always reply,
but `typeof null === 'object'` let null through the guard; reading
`.permission` off it threw out of the message listener before any reply,
so `puter.ui.requestPermission(null)` hung forever in env=app while
env=web answered false for the same input.

In the env=web branch only the consent-dialog path was wrapped, even
though its own catch reasons that this resolves to a boolean for every
other caller. A `window.open` refused by throwing rather than by
returning null escaped the launch branch and rejected instead.

The perms docs were flipped to platforms: [websites, apps], but every
entry point except `request()` reads the signed-in user's identity
first, so on a signed-out site they reject with Unauthorized and never
prompt — the permission popup deliberately does not sign the site in.
Document the sign-in precondition on those pages.

* Measure a permission's width after the rewrite that decides it

The new grant validation capped `permission` at 255 to match the column
it lands in, but it measured the caller's raw string. `fs:/path:mode` is
rewritten to `fs:<uuid>:mode` before storage, so what lands in the column
is ~44 characters however deep the path is. Granting access to a deeply
nested file therefore returned 400 even though the identical target
granted by uuid returned 200 and stored 44 characters — and because a 4xx
is read as an outright refusal, the permission dialog showed its
retryable error and could never succeed on retry.

Bound the request body only against absurd input, and enforce the column
width in the permission service on the rewritten string, before the app
is resolved so an oversized permission still refuses ahead of a missing
app. Covered both ways: a rewritten-short permission is accepted, and one
that no rewriter shortens is still refused.

* Require a registered app when a dev-app grant names an origin

The user-app handlers resolve a caller-supplied origin through
#registeredAppUidFromOrigin because appUidFromOrigin synthesises
app-<uuidv5(origin)> for an origin with no app row, and the permission
services resolve their identifier as uid-or-name — so the synthetic uid,
derived from a published namespace constant and computable offline,
lands on whoever registered an app under that literal name. The dev-app
handlers were left resolving the raw synthetic uid.

That leg matters at least as much: a dev-app grant is scanned with the
issuer's authority for anyone running as that app, so a squatted grant
hands over the granting user's permission. Verified against the store —
the grant landed on the squatter rather than rejecting.

Without a squatter the synthetic uid resolves to nothing and these
already 404, so the guard costs the legitimate case nothing; a test
covers a registered origin still resolving to its app.

Also assert that revoke accepts the same oversized-but-rewritten
permission grant does, since the dialog's withdrawal of an uncertain
grant depends on that symmetry.

* Revoke the row a user-app grant actually wrote

`app-root-dir:<uid>:<mode>` is a pseudo-permission: its rewriter resolves
it to a real `fs:<root_uid>:<mode>` only while a user-app permission row
is being written, and resolves to a match-nothing sentinel at all other
times so a scan can't match through the fs path.

Revoke shared that rewrite but not the flag, so it aimed the DELETE at
the sentinel: it removed nothing and reported success while the fs
permission stayed live. The permission dialog withdraws a grant whose
outcome it couldn't confirm through exactly this path, so a user who
answered "Don't Allow" after a dropped grant response kept the access
they had just refused.

Grant and revoke now share one rewrite helper. It also has to work for a
caller outside a request scope — an internal job, or a direct unit test —
where `Context.set` has nothing to set the flag on; an empty scope reads
the same as no scope, so it only makes the flag settable.

* Elide a long host from the left, as its own rule intends

The identity line is the only thing on the permission dialog naming the
requester, so a host too long for the dialog has to lose its front, not
its tail: the registrable domain is the part that says who is asking.

`direction: rtl` was there for that, but paired with
`unicode-bidi: plaintext` it does nothing — plaintext takes the base
direction from the content's own first strong character, which for any
Latin host is LTR, so the ellipsis went back on the right. Measured in
the real dialog, `account-security.paypal.com.verify-login.example`
rendered as `account-security.paypal.com.verify-l…`, reading as PayPal.

Isolating instead keeps the box anchored to the end of the text, and
still stops a bidi control character in the host from reordering
anything around it.

The test that covers this asserted the computed `direction` — the
property, not the outcome — so it passed throughout. It now measures
which characters are actually on screen, and that the host still reads
in source order.

* Tell a severed opener from a closed one by whether it can answer

The web popup flow decided which it was looking at by timing: a
`popup.closed` flip within 3s of `window.open()` was COOP severing the
opener, anything later was the user closing the window. Both halves
misfire, and both were reproduced against a real popup.

A COOP-only site whose popup navigation commits after the cutoff had its
severing read as a close, so the site was told "denied" about a second
later — while the prompt was still coming up. The Allow the user went on
to click then committed a grant the site had been told it did not get,
which is the failure the cutoff was introduced to prevent: how long a
navigation takes says nothing about whether the opener survived it.

The other way round, a signed-in site whose user dismissed the popup on
sight had that close read as severing, and fell back to polling for a
decision. A denial writes nothing to poll for, so the caller waited out
the full five-minute timeout instead of being answered.

So ask the question directly: the popup now announces itself to its
opener, which it can only do while the relationship is intact. Having
heard from it proves a later close is a real close and the answer is now;
never hearing from it means the prompt may be live in a window that
cannot answer, and the decision is read back from the server as before.
The announcement carries no token, and goes out before any sign-in gate —
a gate that delayed it would make abandoning sign-in look severed.

Measured: the popup announces itself ~290ms after opening, and a close
just after that is answered in ~1.3s rather than five minutes.

* Stop revoking a literal `*` after a dev-app revoke-all

The `*` arm of /auth/revoke-dev-app fell through: after
`revokeDevAppAll` it also ran `revokeDevAppPermission(…, '*')`, a
DELETE naming a row called literally `*` — which matches nothing —
plus a second `revoke` audit entry for the same action. The user-app
twin already if/elses its two arms; the dev-app handler now matches it,
and a test pins the parity: everything revoked, one audit row.

* Match the popup's messages against a canonical GUI origin

The web flow compared `event.origin` to `puter.defaultGUIOrigin` as raw
strings, but they are different kinds of value: the event carries the
browser's canonical origin serialization, while the configured origin is
whatever text was supplied — a trailing slash, an explicit default port,
or a stray path all name the same origin and all fail the comparison.

The mismatch doesn't read as a config error, it reads as the user's
answer: with every message from the popup dropped, the missing
`permissionPromptReady` makes the popup's close look like a severed
opener, and the missing decision leaves that path to answer on its own —
"denied", for a guest, moments after the user clicked Allow and the
grant committed.

Parse the configured origin once and compare canonical-to-canonical; the
popup URL is built from the same parsed origin, so a trailing slash no
longer yields a `//action/...` path either. A configured origin that
cannot parse could never have hosted the prompt, so it now denies up
front instead of opening a broken window. Pinned by an e2e test that
re-points the SDK at the same GUI through a trailing-slash origin and
expects the grant to be heard; it fails against the raw comparison.

* Take a permission popup's requester from the browser, not the link

`app_uid` was removed from the request-permission URL because the uid
names who receives the grant, so it has to come from the requesting
origin rather than from whoever built the link. Two other parameters
still carried exactly that identity, and the origin is the identity
twice over: it is the name the dialog attributes the request to, and it
is what the server resolves into the app the grant is written against.

`opener_origin` is believed on every popup boot, ahead of the referrer.
`origin` was the fallback in the request-permission block itself,
reached whenever there is no opener at all. Either one lets a bare link
raise a consent prompt in some other app's name — the token exchange
bootstraps an app row for whatever origin was typed, so the grant then
commits against it — while the user is looking at a domain the requester
does not control. That is the whole dialog defeated: a site can name
`docs.google.com` and have the Allow land on Google's app row.

So a permission popup now takes only an origin the browser vouches for:
`document.referrer`, or the opener's own reply to the `requestOrigin`
handshake. Neither can be forged to another origin. Nothing legitimate
relied on the parameters — the SDK sends neither, and the OIDC redirect
`opener_origin` exists for drops `action` too, so no permission flow can
arrive through one. The `origin` fallback only ever fired when there was
no opener, which is to say when there was no requester either; that path
now reports its denial as usual instead of prompting.

The referrer is safe here precisely because `action` is only
`request-permission` on the popup's first load, so it is always the
opener's. Restoring the action across an OIDC hop would break that — the
returning navigation's referrer is the identity provider — which is why
the round trip is withheld rather than repaired:

A popup that signs in through OIDC comes back to a redirect URI the
server hard-codes to `/action/sign-in`, so it returns believing it is a
plain sign-in popup. It posts `puter.token`, which the SDK's global
listener feeds straight into `setAuthToken()`, and never runs the action
it was opened for. For a permission prompt that is the exact outcome
`deliversTokenToOpener` exists to prevent: the site is signed in without
ever asking, the user is never shown the permission they were brought
there to decide, and the request resolves as a denial. Nothing in the
returned URL says what the popup was for, so it cannot recover on its
own. Until the redirect can carry the action back — and the opener
origin can be re-established from the handshake on return — a popup
whose purpose cannot survive the hop does not offer the hop. Email
sign-in stays in the window and is unaffected.

Pinned by e2e tests that spoof each parameter and expect the prompt to
name the real opener, or not to appear at all. The long-hostname test
drove the dialog through `origin=`, which no longer produces one, so it
now asserts the CSS elision contract directly while the popup test
asserts the real flow applies the `perm-dialog-entity-host` class that
contract keys on.

* Refuse an origin the grant could never have named

The identity line elides a long host from the left, keeping the
registrable domain visible, which it does with `direction: rtl`. That is
sound for a host — every character in one resolves left-to-right, so the
string reads in source order — but it is not sound for arbitrary text,
whose neutral and RTL runs can render in an order they were not written
in. On the one line of this dialog whose whole job is saying who is
asking, that is the wrong thing to be lenient about.

The entity resolver reached that state through its own fallback: when
`new URL(origin)` threw it displayed the unparsed string and still marked
it as a host. An origin the server cannot parse cannot name a grant
target either — `AuthService#originFromUrl` rejects it, and rejects
non-http(s) schemes with it — so there was never anything to prompt
about. Deny at the gate, next to the existing "requester the grant can't
name" check, on the same test the server applies.

* Give the decision poll a deadline it can actually reach

`pollDecision` bounds itself with a five-minute budget, but it only reads
the clock between iterations and its `fetch` had no timeout of its own. A
request that never settles — a stalled connection, a proxy that accepts
and never answers — parks that `await` forever: the loop never comes
back round to check, `settle` is never called, and the caller's promise
stays pending for the life of the page with the message listener and
interval still attached. It is the only unconditional hang left in the
flow, and the least recoverable one, because the popup is already closed
on this branch and nothing the user does can rescue it. Each attempt now
gets ten seconds — long enough that a slow-but-working connection is
still heard, short enough that the deadline means something.

The request id changes for a related reason. It was `#messageID++`, a
small integer restarting at 1 on every page load, and `event.source` is
not pinned for as long as the no-gesture consent dialog waits for its
Continue click — a stretch of time the user paces. A permission popup
left open from before a reload posts this exact message shape to its
opener on the way out, and its counter value collides with a fresh
request's, settling it with the decision the user made about a different
permission. A random suffix makes the two impossible to confuse; the GUI
echoes the value back verbatim, which the loose comparison still handles.

* Answer the app when the signup gate throws

The requestPermission branch grew a `respond` helper so every exit tells
the app something and its promise settles instead of hanging. One exit
still doesn't: `await UIWindowSignup(...)` is outside the guarded region,
and `ipc_listener` has no outer catch, so a throw from the signup window
escapes the listener entirely — before any reply — and leaves the app
waiting forever. That is the precise failure the helper was added to rule
out. Treat it as the refusal it amounts to.

* Revert "Keep the ai-chat model-map build inside the server lifecycle"

This reverts commit 1b2482dcb8.
2026-07-27 08:37:41 -07:00
Daniel Salazar b2324cd212 chore: cleanup puter-js driver to match rest of changes (#3452) 2026-07-27 08:25:44 -07:00
jelveh a1c8ba4cdd Fix stuck grab cursor on aborted drag
When dragging is aborted by returning false from the sortable 'start' handler, the cursor plugin never fires its cleanup, leaving the body cursor stuck on 'grabbing'. Reset the body cursor explicitly on each early-return path.
2026-07-26 23:29:33 -07:00
jelveh 30461e7d95 Hide Trash from file explorer listing
Trash is accessible via the sidebar and should not appear as a row in the file explorer. Filters it out from directory listings and prevents socket events from re-adding it to the view.
2026-07-26 22:32:32 -07:00
jelveh 46253fb80a Disable hover during rubberband file select
Add a `rubberband-selecting` state on the dashboard while marquee selection is active in `TabFiles`, and remove it when drag ends. CSS now disables pointer events on surrounding files chrome (sidebar, directory tree, and header) during that state so hover highlights/tooltips don’t trigger when the cursor leaves the clamped selection area.
2026-07-26 22:23:54 -07:00
jelveh 29f2a4d9c9 Default file picker and sidebar to Home
Make Home consistently visible and selected in file navigation. The file tab now shows the Home directory entry and auto-selects Home (instead of Desktop) when no initial path is provided. In `UIWindow`, saved sidebar layouts are backfilled to include Home if older preferences are missing that entry, preventing users from losing quick access after upgrades.
2026-07-26 22:14:17 -07:00
jelveh e00e1e57da Centralize trash icon update logic
Extract duplicated trash icon update code into two shared helpers:

- `window.update_trash_icons(is_empty)` updates all trash icon surfaces (taskbar, desktop items, shortcuts, open window headers, Dashboard sidebar) in one place.
- `window.refresh_trash_state()` stats the trash, emits the socket event, and calls `update_trash_icons`.

Replace all scattered inline stat+emit+icon-update blocks across UIItem, UIWindow, UIDesktop, UIDashboard, keyboard, helpers, and generate_file_context_menu with calls to these helpers. Also syncs the Dashboard sidebar trash icon on load and after every destructive operation.
2026-07-26 21:51:07 -07:00
Nariman Jelveh 6668d778d2 feat: highlight uploaded items after upload
Uploads now land selected wherever they finish: the desktop, explorer
windows, file dialogs, and the dashboard Files tab. Selection is applied
from both completion channels — the batch response selects elements
already in the DOM, and items whose item.added socket event arrives
later are remembered (with a TTL) and selected on creation. The explorer
footer count and the open-file dialog's Open button are updated to
match, and disabled items (e.g. filtered out by a dialog's file-type
filter) are never selected.
2026-07-26 21:14:26 -07:00
Nariman Jelveh 7f3691aea3 Auth: land back on the /app/<name> the user came for after login/signup (#3435)
Landing on /app/<name> logged-out and authenticating used to dump the
user at the root dashboard on several paths, losing the app they came
for:

- OIDC login/signup only sent return_to for /desktop and /dashboard
  (and the backend whitelist only accepted those two), so OIDC from an
  app landing redirected to /.
- UIWindowSignup defaulted its post-success redirect to /, so password
  signup reached via the session list, ?action=signup, or in-app signup
  prompts (IPC.js) lost the app.
- OIDC error redirects always went to /?action=..., so a recovered
  attempt (e.g. account-not-found bounced to signup) also lost the app.

New helpers in src/gui/src/helpers/auth_redirect.js:
- get_auth_redirect_url(): stay on the page auth started from;
  /action/* pages go to /; strips action/auth_error/message/
  request_code so the reload doesn't re-open the auth window or pass
  auth params through to the app as launch args.
- get_oidc_return_to(): pathname when whitelistable, now including
  /app/<name> (trailing slash normalized).

Backend (OIDCController):
- Shared isWhitelistedReturnPath() accepts /desktop, /dashboard, and
  /app/<name> (charset mirrors APP_NAME_REGEX — no open redirect).
- buildErrorRedirectUrl() lands on the whitelisted originating page
  from the signed state's redirect_uri instead of always /.

Tested: 47/47 OIDC controller tests pass (4 new: return_to accepted/
rejected on start, success redirect to /app/<name>, error redirect
keeping /app/<name>); verified live on local dev for first-visit temp
user, password signup (incl. email-confirmation gate), and password
login — all land on /app/camera with the app open.
2026-07-26 20:33:46 -07:00
Nariman Jelveh 1e280601c6 fix: showSaveFilePicker from external websites (#3450)
Saving a file via puter.ui.showSaveFilePicker from a third-party website
(popup flow) consistently failed with a DataCloneError alert after
clicking Save, and saving over an existing filename showed a raw error
instead of the Replace/Cancel prompt.

Two bugs:

1. privacy_aware_path is a curried factory (world => fspath => ...), and
   initgui.js is the only module that imports it directly — so the popup
   save handler's privacy_aware_path(res.path) returned the inner
   function, which postMessage cannot structured-clone. Every other call
   site resolves the bare name to the correctly bound
   window.privacy_aware_path global, which is why only the external-site
   popup flow was broken. Use the global at the call site and import the
   factory under a distinct name so a bare call can't silently resolve
   to it again.

2. The v2 backend returns `conflict` for a same-name write, but the v1
   wire contract is `item_with_same_name_exists` + `entry_name`, which
   the GUI's save dialogs key on to offer the overwrite prompt. Restore
   the legacy code/field on the write-conflict error and carry
   HttpError.fields through the /batch per-op error serializer.

Verified end-to-end against a local backend: fresh save resolves the
caller's promise with the signed saved_file and closes the popup;
saving an existing name shows Replace/Cancel and Replace overwrites.
Backend suite shows no new failures.
2026-07-26 11:28:38 -07:00
Daniel Salazar 2262975785 fix: readdir response type (#3449) 2026-07-25 18:06:50 -07:00
Daniel Salazar e6e6e3ba9a chore: cleanup API driver calls PUT-1324 (#3448) 2026-07-25 18:05:36 -07:00
Daniel Salazar e85cd9d53d feat: readdir with depth (#3446) 2026-07-25 14:04:21 -07:00
Nariman Jelveh 828b5d9430 Fix popup file pickers rendering an empty explorer
The cover-page rule added for the auth windows forced `width: 100%` on
every `.window-cover-page .window-body`, including the popup file,
directory, and save pickers. That body is an item-container with
`overflow: auto`, so it establishes its own formatting context and may
not overlap the floated `.window-sidebar`. Unable to shrink to fit, it
dropped below the sidebar, pushing the file list and the Cancel/Open bar
outside the 700x400 popup and leaving the page background showing
through as grey.

`.window-body-filedialog { width: auto !important }` was meant to cover
this but loses on specificity, one class against two. Scope the auth
rule with `:not(.window-filedialog)` instead; that class is only set for
the three dialog kinds, so no auth or session cover page changes.
2026-07-25 13:02:15 -07:00
jelveh 3b1ba72ddd Share item-added updates across GUI shells
Extracted desktop `item.added` container update logic into a shared `apply_item_added_to_containers` helper and wired it into both `UIDesktop` and `UIDashboard`. The helper handles overwrite updates, new item insertion, icon refresh, and resorting for matching item containers. This fixes dashboard-hosted explorer/file dialogs not reflecting uploads until navigation.
2026-07-24 23:01:10 -07:00
jelveh 4245ac04ac Isolate upload dialog to prevent double uploads
Replace reuse of `#upload-file-dialog` with a dedicated hidden file input owned by `init_upload_using_dialog`. This avoids collisions with the dashboard Files tab’s `onchange` property handler, which could trigger duplicate concurrent uploads and "Entry already exists" races. The new flow stores the current target path once per dialog open, clears input value before click (so re-selecting the same file still fires `change`), and snapshots `FileList` to an array before clearing to keep async uploads reliable.
2026-07-24 18:26:43 -07:00
Daniel Salazar b8559c1221 chore: cleanup other puter-js modules to match new structure (#3440) 2026-07-24 17:42:06 -07:00
Daniel Salazar 6c4fa629a9 fix: allow root token to also call ai drivers (#3442) 2026-07-24 17:41:38 -07:00
jelveh e6531d4357 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-24 15:38:38 -07:00
jelveh 0c27571030 Dashboard files: compact drag ghost + spring-loaded nav controls
- Replace the full-tile drag ghost with a compact icon+name chip:
  stacked sheets and a count badge for multi-selections, dimmed
  source rows, same design in list and grid views.
- Holding a dragged item over back/forward/up or a breadcrumb
  segment now spring-navigates there (700ms dwell); cancelling
  the drag returns to the original directory.
2026-07-24 15:37:33 -07:00
Neal Shah 6e6d9d2869 buildXhr, dedup, retry (#3432)
* buildXhr, dedup, retry

* change retry intervals
2026-07-24 18:37:09 -04:00
Nariman Jelveh 34529b2e79 Dashboard files: replace solid-blue selection with tinted accent wash
Selected tiles/rows now use a subtle accent tint plus hairline ring
instead of a solid --select-color fill, so icons and labels keep their
natural colors in both themes. Also refines the grid tile's hover
'...' button (inset, hairline border, hover state), fixes the
light-mode selected-row text color that referenced a dark-mode-only
variable, and drops the opaque icon chip visible on tinted list rows.
2026-07-24 13:09:31 -07:00
Nariman Jelveh 7e4ac6542e Update TabApps.js 2026-07-24 12:42:17 -07:00
Daniel Salazar 3331dbd464 chore: make fs upload cleaner in puter.js (#3436) 2026-07-24 12:18:19 -07:00
Daniel Salazar 391b175a68 fix: remove deprecated claude models (#3438) 2026-07-24 10:54:01 -07:00
Neal Shah 19f35b4200 add opus 5 (#3437) 2026-07-24 13:49:20 -04:00
jelveh 9877bb79d0 Polish mobile dashboard sidebar drawer UX
Refactors the dashboard’s mobile sidebar into a modal-style drawer with a coordinated state helper, scrim overlay, and dedicated close button. Adds accessibility improvements (`aria-label`, `aria-expanded`) and stronger dismissal behavior (scrim tap, close button, Escape, and swipe-left with velocity/threshold handling). Updates mobile CSS for safe-area-aware positioning, improved touch targets/pressed states, frosted toggle styling, drawer shadows/transitions, and drag-state handling for smoother interactions.
2026-07-23 18:56:43 -07:00
Nariman Jelveh 2982e894b1 Update dashboard.css 2026-07-23 18:14:47 -07:00
Nariman Jelveh 10bdf701e0 Dashboard: start the minimize morph immediately, bookkeep the URL behind it
The minimize controls consumed the app's /app/<name> history entry
first and let the popstate handler do the visible hide. When the app's
iframe had stacked joint session-history entries, the pop only settles
at pop_dashboard_app_url's 400ms watchdog — so clicking minimize sat
still for ~half a second before the morph began.

Hide the window in the click frame instead (minimize_window, shared by
the head button, the context menu, and the control drawer) and demote
the pop to pure URL bookkeeping. The eager hide is safe against both
ways the pop settles: every hideWindow branch marks data-is_minimized
synchronously, and the popstate handler and the watchdog both already
skip minimized windows. The browser's Back button still minimizes
through the popstate handler as before.
2026-07-23 17:50:32 -07:00
Nariman Jelveh e764a10ee4 Dashboard: recover minimize/close when history.back() misses the app entry
Minimize (and close's URL cleanup) on headless dashboard app windows
work by consuming the app's /app/<name> history entry with
history.back() and letting the popstate handler do the actual hide.
But back() traverses the JOINT session history, which the app's iframe
shares: if the app navigated internally after load (SPA router,
redirects — and browsers differ on which iframe navigations stack
joint entries), the back() consumes the IFRAME's entry instead. The
parent's URL never changes, no popstate arrives, and the
dashboard_url_pop_pending latch stays stuck — every later minimize and
close becomes a silent no-op, and the stranded /app/<name> entry makes
later traversals resurrect windows the user already dismissed.

Fix with a watchdog after each back(): a successful pop is a
same-document traversal of the parent (app entries are pushState over
the dashboard route), so its popstate lands well inside 400ms and the
watchdog no-ops. If the URL still names the app by then, do what the
popstate would have done — minimize through the same lookup and
minimized-guard — and replaceState the stranded entry back to the
dashboard's own route (captured at push time, tab hash included) so no
stale entry survives.
2026-07-23 17:23:03 -07:00
Neal Shah 7e461e14a2 remove unneeded write origin append (#3434) 2026-07-23 19:09:23 -04:00
Daniel Salazar 48cc706ad3 feat: paginated fetching for all (#3431)
* feat: paginated fetching for all

* fix: metering top up gui reporting
2026-07-23 15:07:58 -07:00
Daniel Salazar 20ea616b59 chore: cleanup kv module (#3430) 2026-07-23 10:05:19 -07:00
Nariman Jelveh 73a7a38ea8 Dashboard: redesign app control pill as a top-edge drawer (#3427)
* Dashboard: redesign app control pill as a top-edge drawer

The floating control capsule over headless dashboard apps consumed
noticeable space, especially on mobile. Replace it with a drawer: a
slim glass tray flush with the top edge (icon + title + minimize +
close) whose tongue-shaped handle hangs from its bottom center. At
rest only the tongue peeks in from the edge; hovering it (mouse),
tapping it, or keyboard-focusing the drawer slides the tray down.
The collapse is a single GPU transform instead of the pill's
max-width squeezes, and the open-then-retract intro now visibly
retracts into the tongue, teaching where the controls live.

Hover-expansion is gated on pointerType === 'mouse': touch taps
synthesize a pointerenter right before click, which made the old
pill's toggle see an already-open state and shut it again.

Also fixes two window-stacking bugs surfaced while testing:

- showWindow demoted stay_on_top windows (every fullpage/dashboard
  app window, created in the 99999999+ z band) to a plain counter
  z-index on restore. focusWindow deliberately never raises
  stay_on_top windows, so after minimize/restore a single dashboard
  focus buried the app permanently. Restore now re-raises into the
  stay-on-top band.

- Pressing the drawer now focuses its window, as the titlebar it
  replaces did. The document-level activation handler works off
  mouseover_window, which only mousemove refreshes — a tap with no
  intervening mousemove (touch, restored windows) would re-raise
  the dashboard over the app.

* Dashboard: restored windows come back with the drawer collapsed

The restore re-played the open intro (expand + auto-retract). A restore
should bring back the app, not the chrome — the user already met the
controls on open, and they just USED them to minimize. showWindow now
forces the drawer shut instead, which also covers the Back-button
minimize path, where the window hides without touching drawer state.

* Dashboard drawer: left-align the title against the icon

The tray is asymmetric — one icon on the left, two buttons on the
right — so a title centered in its fixed box landed off the tray's
true center and read as misaligned against the centered tongue below.
Left-aligned, icon + title cluster as the identity on the left and
the controls cluster on the right.

* Dashboard drawer: morph the tongue into the tray instead of sliding

The drawer is now ONE glass surface that changes shape, rather than a
tray that slides in from off-screen with a handle below it. At rest
it's the tongue; opened, the same surface swells into the tray while
the grabber bar rides the morph and settles into a slim strip along
the tray's bottom edge as the dismiss handle — same handle, both
states. Content is revealed from the center outward through an inner
clipping layer (the surface itself can't be overflow:hidden without
clipping the toggle's touch halo) and condenses in/out of focus with
an opacity + blur + scale crossfade. Opening gets a soft overshoot
and settles; closing is quicker and lands without bounce.

The open width is DERIVED in CSS from the same part variables that
lay out the controls row, so the surface always fits its content
exactly and breakpoints (touch sizes, hidden title under 500px) only
override parts.

* Refine dashboard drawer open-state handle

Update the headless dashboard app drawer so the grabber bar disappears when the drawer is open and only the bottom dismiss strip remains interactive. This adjusts tray/strip sizing on desktop and mobile, adds opacity timing for smooth dissolve/reappear transitions during open/close, and aligns in-code docs/comments with the new behavior.
2026-07-22 22:11:39 -07:00
jelveh 5c3fab5579 Revert "Dashboard: fix minimized apps getting permanently buried under the dashboard (#3429)"
This reverts commit bbc86e7b70.
2026-07-22 22:11:15 -07:00
jelveh 695862c61f Increase dashboard directory icon size
Adjusts the directory icon width in the dashboard files section from 28px to 35px for improved visibility.
2026-07-22 21:45:09 -07:00
Nariman Jelveh bbc86e7b70 Dashboard: fix minimized apps getting permanently buried under the dashboard (#3429)
* Dashboard: fix minimized apps getting permanently buried under the dashboard

Fullpage/dashboard app windows are created stay_on_top in the
99999999+ z band, but showWindow's restore path demoted them to a
plain counter z-index — and since focusWindow deliberately never
raises stay_on_top windows, the demotion stuck: one dashboard focus
after a minimize/restore buried the app permanently (URL and tab
title still naming an app the user could no longer see). Restore now
re-raises stay-on-top windows into their band.

Also make the control pill focus its window on mousedown, as the
titlebar it replaces did. The document-level activation handler works
off mouseover_window, which only mousemove refreshes — a tap with no
intervening mousemove (touch, restored windows) would activate the
stale window and could re-raise the dashboard over the app. Deferred
a tick so it runs after that handler and wins.

* Dashboard: restored windows come back with the pill collapsed

The restore re-played the open intro (expand + auto-retract). A restore
should bring back the app, not the chrome — the user already met the
controls on open, and they just USED them to minimize. showWindow now
forces the pill shut instead, which also covers the Back-button
minimize path, where the window hides without touching pill state.

* Dashboard pill: left-align the title against the icon

The pill is asymmetric — one icon on the left, two buttons on the
right — so a title centered in its fixed box landed off the pill's
true center and read as misaligned. Left-aligned, icon + title
cluster as the identity on the left and the controls cluster on
the right.
2026-07-22 21:40:04 -07:00
Daniel Salazar e35b57068b test: add extra tests (#3428) 2026-07-22 21:13:44 -07:00
Daniel Salazar 5faed55076 fix: autoclaim app when making a subdomain (#3426) 2026-07-22 21:12:38 -07:00
Daniel Salazar 928d5fec16 chore: cleanup AI module for puter-js (#3423) 2026-07-22 18:18:54 -07:00
velzie ebcc4bb597 Add loopback support to puter.peer (#3425)
* allow setting port instead of invite code in puter.peer

* fix grabbing fallbackice under wrong response
2026-07-22 20:13:35 -04:00
Neal Shah 51541d3739 initial pjs helper stuff (#3421)
* initial pjs helper stuff

* move resolveReauth

* move location of PuterClient and rename to networkUtils
2026-07-22 18:48:11 -04:00
Nariman Jelveh 43b9200205 Dashboard: headless full-tab apps with a floating control pill (#3424)
* Dashboard: headless full-tab apps with a floating control pill

Maximized app windows in dashboard mode no longer render a titlebar —
the app covers the full tab. The head's controls move to lighter-weight
affordances:

- A floating pill overlaid top-center of the app (parent DOM, above the
  iframe): expanded it shows the app icon, title, minimize, and close;
  it opens expanded so first-time and deep-link users see it, then
  collapses to a subtle capsule that re-expands on hover, tap, or
  keyboard focus. Minimize routes through the same URL-entry consumption
  as the Back button; close tears the window down normally. Dark
  translucent glass so it reads over any app content; safe-area aware;
  bigger targets on coarse pointers; title hidden under 500px; respects
  prefers-reduced-motion.
- Apps-tab tiles double as the app switcher: a macOS-dock-style running
  dot marks tiles with a live (visible or minimized) window, and the
  tile context menu gains a Quit item for running apps. UIWindow
  broadcasts window open/close so the dots stay current.
- puter.ui.setWindowTitle also updates the pill title.

Only maximized, URL-owning app windows in dashboard mode go headless:
dialogs, explorer, non-maximized child windows, and everything on the
desktop keep their titlebars. dashboard.css's 29px head-height
compensation is scoped to windows that still have heads, so headless
app bodies fill the window exactly.

* Fix pill buttons swallowed by their own hover halo

The pill's invisible hover halo (an absolutely-positioned ::after with
negative inset) painted above the pill's static children, so real
pointer clicks hit-tested to the halo and never reached the minimize/
close buttons — while element.click() (which skips hit-testing) worked,
hiding the bug from programmatic checks. Stack the halo at z-index -1
(the pill is its own stacking context, so the outside ring still sits
above the app's iframe) and raise the interactive children explicitly.

* Review round: pill/dashboard edge-case fixes

- Latch pop_dashboard_app_url while its history.back() is in flight: a
  double-click on minimize (or a close racing a minimize) issued two
  back()s, and the second popped the dashboard's own entry — navigating
  clean out of the page. Any popstate or push settles the latch.
- Close a running app's windows when it is uninstalled: the tile is a
  headless app's only switcher, so a minimized instance would otherwise
  be stranded — impossible to restore or quit.
- puter.ui.setWindowTitle in dashboard mode now also updates the
  browser-tab title (while that app owns the URL) and the window's
  data-name, so a later restore re-applies the app's current title
  instead of its launch-time one. Desktop behavior unchanged.
- Nudge collapsed-pill contrast (opacity 0.55 → 0.65, border alpha
  0.16 → 0.22) so the capsule stays findable over dark app content.

* Adjust dashboard tile spacing and label sizing

Tweaks My Apps tile layout constants to match the real icon/label geometry: tile height is increased, label top margin is widened, and the running-status dot is moved slightly lower to avoid overlap during icon hover scaling. It also raises dashboard app pill title line-height to prevent descenders from being clipped when ellipsis overflow is active.

* Pill and running-dot visual polish

- Fixed-width pill: the title gets a fixed 150px centered box instead of
  a cap, so every app's pill is identical and the minimize/close buttons
  sit at the same screen position in every app — they're operated by
  muscle memory, like any titlebar's controls.
- Give the pill title line-height 1.35: overflow:hidden (needed for the
  ellipsis) also clips vertically, and at line-height 1 it cut the
  descenders off g/p/y.
- More air around the tile running dot: label margin 6px → 10px and the
  dot 2px lower (tile row height 78px → 82px to keep the label line in
  budget — the pager layout reads the CSS var, so it follows). The
  icon's 1.08x hover growth no longer touches the dot.

* Size the pill icon to balance the capsule

16px read as undersized next to the 26px buttons in the 36px capsule —
bump the expanded icon to 20px with a proportional (25%) radius, and
object-fit: cover in case an app icon isn't square. Collapsed size is
unchanged.
2026-07-22 14:28:06 -07:00
jelveh d5e9417d2d Dashboard apps own the URL: /app/<name> entries with Back-to-minimize
- Opening an app in dashboard mode pushes a real history entry at
  /app/<name>; Back minimizes the app (it keeps running), Forward
  restores it — or relaunches it if it was closed
- The minimize/close buttons consume the entry via history.back() so
  the address bar never names an app that is no longer on screen
- Landing directly on /app/<name> now boots the dashboard with the app
  open maximized in-page instead of the desktop; embeds, popups, and
  explicit fullpage boots keep the desktop flow, and fullpage_on_landing
  no longer opts a landing out of the dashboard
- Desktop mode no longer rewrites the URL at all (no /app/<name> or
  folder-path replaceState on focus, no /desktop resets on close or
  minimize); tab titles still track the focused window
2026-07-22 10:50:10 -07:00
jelveh dec205d959 Lighten tile-to-window restore animation
Update `showWindow` to use a shorter, window-only morph when restoring a minimized app from a dashboard tile. `morph_window_from_tile` now accepts options for `icon_half` and `duration`, gates icon ghost creation accordingly, and derives transition/cleanup timing from the configured duration so un-minimizing feels distinct from a full app launch.
2026-07-22 09:57:24 -07:00
jelveh d87f3464fc Support Cmd/Ctrl+click and adaptive bg color
Update the dashboard app tile click handler so Ctrl/Cmd+click opens apps in a new tab (matching the context menu behavior) while keeping existing target-link/app-name routing. Also switch the default body background to white and add a dark-mode media query to use a dark background color.
2026-07-21 20:33:27 -07:00
jelveh 25410cac92 Allow tile morph animations on phones
Removes `isMobile.phone` checks that previously disabled dashboard tile morph transitions. Window open/close morph selection and tile-launch ghost setup now run on phone the same way they do elsewhere, while still honoring animation-disable and reduced-motion conditions.
2026-07-21 19:13:48 -07:00
jelveh 66a1baef24 Preplay dashboard tile launch morph animation
Add click-time launch feedback for dashboard app tiles so users get immediate visual response while app startup is still loading. TabApps now starts and settles tile launch state around `launch_app`, logs launch failures, and keeps duplicate-launch protection.

UIWindow now tracks in-flight tile feedback with a WeakMap, introduces `begin_dashboard_tile_launch`/`settle_dashboard_tile_launch`, and updates `morph_window_from_tile` to avoid replaying the icon half when it already ran at click time. Ghost/icon timing and cleanup were adjusted so fallback and failed launches do not leave stale tile state.
2026-07-21 19:09:32 -07:00
jelveh 5c4df9cbd7 Revert "feat(gui): start the tile open-morph on click, before the launch fetches"
This reverts commit f89b3a888a.
2026-07-21 18:25:39 -07:00
jelveh f89b3a888a feat(gui): start the tile open-morph on click, before the launch fetches
A fresh launch has a server round-trip (app info, signing) between the
click and the window's creation, so the open morph used to start late —
on a slow connection the click gave no feedback at all. The two halves
of the morph are now decoupled:

- On click, begin_dashboard_tile_launch starts the icon's half
  immediately: the ghost enlarges in place and dissolves exactly as
  before, and the real icon stays hidden — the empty slot reads as
  'launching' — while the fetches run.
- When the window opens, morph_window_from_tile claims the click-time
  ghost instead of spawning a second one, runs the window half alone,
  and restores the icon when it lands. On a fast launch the halves
  overlap into one motion; on a slow one the window zooms out of the
  empty slot.
- settle_dashboard_tile_launch (in the launch's finally) puts the icon
  back whenever the window never claims it: launch failed, app opened
  in background, or the morph fell back to the plain fade. The launch
  also gained a .catch, so a failed launch logs instead of leaking an
  unhandled rejection.

The un-minimize path has no fetch and is untouched: it still spawns its
own ghost inside the morph.
2026-07-21 17:58:05 -07:00
Nariman Jelveh bf545dda25 Dashboard: open apps maximized in-page, with minimize and single-instance behavior (#3419)
* feat(gui): open dashboard apps maximized in-page with minimize + single instance

Apps launched from the dashboard's Apps tab now open as maximized Puter
windows in the same page (via launch_app) instead of a new browser tab.
External website shortcuts still open in a new tab since arbitrary sites
can't reliably be iframed.

Dashboard app windows keep their minimize button (fullpage mode normally
hides it since there's no taskbar). With no taskbar item to animate
toward, hideWindow now hides the window in place and flags it with
data-minimized_in_place; showWindow un-hides it via that flag. An
explicit flag is used because data-orig-* can't distinguish the two
minimize paths - drag/maximize handlers set those attrs too.

Tiles are single-instance: clicking a tile un-hides a minimized
instance or focuses a visible one instead of launching a duplicate, and
launches in flight swallow repeat clicks.

Also fixes .window-body-app height in dashboard mode: fullpage mode
sizes it to 100% assuming no titlebar, which clipped the bottom 29px of
every app.

* feat(gui): iOS-style minimize-to-icon morph in dashboard mode

Minimizing a window in dashboard mode now zooms it into its app's tile
on the Apps tab — but only when the user can see that tile (Apps section
active AND the tile on the pager page currently in view); otherwise the
existing hide-in-place fade is kept, as on phones and with
prefers-reduced-motion.

The animation is a two-layer morph on one 450ms cubic-bezier(0.32,
0.72, 0, 1) path: the live window scales onto the tile's icon box
(transform + opacity only — no per-frame iframe reflow) while its corner
radius grows toward the icon squircle, and an enlarged clone of the icon
flies congruently with it, cross-fading in as the window fades out and
landing exactly on the real icon's slot (hidden until the handoff), so
the window reads as becoming the icon.

Window geometry is never touched: only inline transform/opacity/radius
are set and restored byte-identical when the animation ends, and
showWindow()'s existing minimized-in-place path restores as before —
including a reopen mid-animation.

* fix(gui): keep the minimize-morph ghost icon square (contain-fit)

The icon clone previously mapped rect-onto-rect over the window, which
stretched the square icon anamorphically. A covering square was tried
and jittered: it protruded past the card's short edge and the halo
popped in mid-crossfade.

Contain-fit resolves both: the ghost scales uniformly (always square),
sized to the window's short edge and centered — its side tracks the
card's short edge exactly for the whole flight (both interpolate
linearly between the same endpoints on the same curve), so the card's
rectangular flanks melt away around a steady, undistorted icon.

* feat(gui): add 'Open in new tab' to app tile context menu

Every tile's right-click menu now leads with Open in new tab, which
opens the app the way tiles did before in-page windows: /app/<name> in
a new browser tab (noopener), or the site link for external tiles.
Non-uninstallable apps previously had no menu at all; they now get this
single item, and uninstallable ones show it above a divider + Uninstall.

* feat(gui): reverse morph — dashboard tile icons zoom into their opening window

Opening an app in dashboard mode now runs the minimize morph backwards:
the window grows out of its tile's icon box (transform + opacity only,
radius relaxing from the 22% squircle to the resting value) while a clone
of the icon enlarges IN PLACE on its slot — capped at 2.5x, fully dissolved
by ~250% growth — as the window fades in underneath on the same 450ms
cubic-bezier(0.32, 0.72, 0, 1) path with the crossfade handoff mirrored
(window in at 80-240ms, ghost out at 120-320ms).

Both open paths get the morph, with the same visibility conditions as
minimize (Apps section active, tile on the pager page in view, not phone,
no reduced motion), falling back to the existing fades otherwise:

- fresh launch: TabApps passes window_options.morph_from_dashboard_tile,
  and UIWindow morphs instead of the 70ms opening fade. Since .window is
  display:none from the stylesheet until later in the pipeline, the hook
  shows it before measuring (nothing has painted yet, so neither the early
  show nor the hide-back on a declined morph can flash).
- un-minimize: showWindow's minimized-in-place branch morphs instead of
  fadeIn(150), keeping the z-index bump and 80ms focus timing.

A data-window_morphing flag guards against opposing morphs fighting over
the window's inline styles: minimize clicked mid-open-zoom (or a tile
click mid-minimize-zoom) falls back to the pre-morph fade behavior, and
each animation still restores every inline property byte-identical when
it ends.

* Update dashboard.css

* feat(gui): cap the open-morph ghost icon at 2x, dissolving in place

The tile icon no longer flies toward the window center nor grows to full
contain-fit (10x+ reads as a giant blurry sticker): it enlarges in place
on its own slot and its growth is capped so the fade-out completes right
around 200% of its size — the window alone carries the motion and the
rest of the growth.

* fix(gui): drop the tile :active press-shrink that stuttered the open morph

Clicking a tile launches the app and the open morph grows the icon in
place — the 0.95 press-shrink made every launch read as shrink-then-grow.
The icon now holds its hover scale through the click and enlarges
straight from there (the morph ghost is measured from the rendered rect,
so it picks up the hovered size seamlessly).
2026-07-21 17:41:45 -07:00
Neal Shah c706b4472d use max_tokens instead of hardcoded value 1000 (#3422) 2026-07-21 18:25:40 -04:00
Neal Shah 93b8041c3d don't allow prototype defined method to be called in drivers (#3413)
* don't allow prototype defined method to be called in drivers

* fix test
2026-07-21 18:01:52 -04:00
Daniel Salazar 89f9f9728f fix: PUT-1355 PUT-1351 PUT-1208 (#3420)
* fix: PUT-1355 PUT-1351 PUT-1208

* fix: dev center header
2026-07-21 14:17:33 -07:00
Daniel Salazar 645409eddb fix: misc hardening (#3418) 2026-07-21 13:55:00 -07:00
tamilselvanp57 58a763bcd1 Add Dev Center UI for viewing app users (#3349) (#3417)
Adds a 'View Users' link on the app Analytics tab that opens a modal
listing users who have authenticated into the app (username + email,
where the user has granted email:read to the app).

Consumes the existing puter.apps.get(...).users()/.getUsers() client
SDK method, backed by the app-telemetry driver's get_users - no new
backend work needed, per discussion on the issue.

Relates to #3349
2026-07-21 13:52:48 -07:00
Daniel Salazar fd91d72654 wip: better contributing docks (#3415) 2026-07-21 12:07:59 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 7fbf491e6f Update docs meta tags and description (#3416)
* Update docs meta tags and description

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-21 16:43:00 +07:00
Daniel Salazar a8833de9d5 fix: misc hardening (#3414) 2026-07-20 23:53:37 -07:00
Daniel Salazar 3a8b6394de feat: standardized api pagination (#3412) 2026-07-20 15:58:23 -07:00
Nariman Jelveh f5d656650f feat(email): accept custom message headers in sendRaw (#3411)
* feat(email): accept custom message headers in sendRaw

Lets callers set transport-level headers such as List-Unsubscribe /
List-Unsubscribe-Post. Also documents the puter-email driver's automatic
unsubscribe / report-abuse footer and the new `suppressed` result field
in the puter.js email module.

* feat(email): envelope override in sendRaw + suppressed in EmailSendResult type

Adds an optional `envelope` field to SendMailOptions so callers can set
transport recipients independently of the visible To/Cc headers (used by
the email-send driver for per-recipient deliveries), and adds the
`suppressed: string[]` field to the public EmailSendResult typing to
match the driver's documented result.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(email): document per-recipient best-effort delivery via `failed`

The email-send driver now attempts every recipient's private delivery
even when one fails, returning failed addresses in the result's
`failed` array instead of failing the whole call — so callers retry
only the failed subset and never re-mail delivered recipients. Adds
the field to the public EmailSendResult typing and module docs.
2026-07-20 11:55:46 -07:00
Daniel Salazar dd314da16d feat: require app or api tokens for ai api usages (#3407) 2026-07-20 08:34:44 -07:00
jelveh 47c1d4268f Fix overlapping context menu state on file rows 2026-07-19 20:08:00 -07:00
jelveh b9e87935cc Update dashboard.css 2026-07-19 20:00:18 -07:00
jelveh 73dd95bd41 Open Desktop by default in the Files tab 2026-07-19 19:53:52 -07:00
jelveh c5bc358ef2 Keep file rows visually active while their context menu is open 2026-07-19 19:51:38 -07:00
jelveh 57c9eb9e1a Fix grid badge drop shadow and hover overlap with item menu 2026-07-19 19:23:02 -07:00
jelveh a0c0544928 Remove extra top padding from files header 2026-07-19 19:15:51 -07:00
jelveh 6dcae81a56 Fix uninstall animation snapping after a prior FLIP slide 2026-07-19 18:57:04 -07:00
jelveh 09baee6381 Delay uninstall tile animation until after the modal closes 2026-07-19 18:48:06 -07:00
jelveh 68aab718a1 Add keyboard navigation to the Apps tab 2026-07-19 18:43:57 -07:00
jelveh 5d6b2f3265 Uninstall apps optimistically with a tile removal animation 2026-07-19 18:35:01 -07:00
jelveh 28730010c3 Fix inconsistent icon sizing in dashboard grid views 2026-07-19 12:24:21 -07:00
jelveh 817ece40ba Modernize folder icon yellows 2026-07-19 12:24:20 -07:00
jelveh 7c868e947a Fix Public folder icon grid and align glyph system 2026-07-19 12:07:10 -07:00
jelveh 66984cd3d6 Improve glyph sizing on system icons 2026-07-19 12:03:09 -07:00
jelveh 5fa508ba91 improve dashboard file list and grid styles 2026-07-19 11:48:46 -07:00
jelveh c1fca467bf Use bundled fallback for null app icons 2026-07-19 10:52:56 -07:00
jelveh e726c19311 Add compact grid option to Files view menu
Reworked the Files view toggle into a context menu with List, Compact Grid (`grid-sm`), and Grid modes, and refactored view handling into shared helpers (`isGridView`, `viewClass`, `setView`, `applyViewMode`) so grid behavior is applied consistently across rendering and drag previews. Added compact-grid CSS overrides to reduce tile, icon, text, and badge sizing while reusing existing grid layout behavior.
2026-07-19 09:48:31 -07:00
jelveh 6643642aa7 Fix item-more color for selected rows in dashboard
Ensure the item-more icon uses the correct sidebar item color when a dashboard file row is selected, including on hover.
2026-07-19 09:26:54 -07:00
jelveh 65dc3b5f4c Use thinner dashboard file nav icons
Switches the Files tab header navigation buttons to new thin arrow SVG assets and aligns icon styling with a lighter visual weight. It also updates several header action SVG paths (new folder, upload, list/grid, sort, select) and sets path action icon color to #5c5c5c so the controls look consistent.
2026-07-18 21:44:02 -07:00
jelveh c225fabbd4 Fix clipped app tiles and soften files header hover styling
Apps tab: tile rows only budget one label line, so two-line app names
made flexbox shrink the icon box into a landscape crop and clip the
label mid-glyph. Clamp labels to a single line and exempt the icon box
from flex shrinking.

Files tab: header hover states (nav arrows, breadcrumbs, action
buttons) now use a neutral #EEE chip and no longer recolor the glyph;
nav arrows also render at full opacity when enabled.
2026-07-18 21:27:24 -07:00
jelveh fdcc2e301b Fix My Apps tile icon and label overflow
Prevent app tile icons from shrinking when labels overflow, and clamp tile labels to a single line so they fit within the fixed tile height. This keeps the grid layout stable while preserving full app names in the existing tooltip.
2026-07-18 20:54:09 -07:00
Nariman Jelveh 183ab0780d fix: verified Dashboard bug fixes (routing, files, apps, usage, mobile menu, CSS) (#3406)
* fix: don't double-refresh Dashboard tabs on browser back/forward

A single back/forward navigation fires both popstate and hashchange, so
handleRouteChange ran twice, calling the tab's onActivate twice — every
back/forward to the Apps tab issued duplicate /installedApps and
/get-launch-apps requests and re-rendered the tab twice. Track the last
handled URL and skip the second event.

* fix: store raw values in Dashboard file-row data attributes

renderItem passed html_encode()d strings to setAttribute, which stores
them literally — a file named "Tom & Jerry.txt" got
data-path="...Tom &amp; Jerry.txt". Every flow that reads the attribute
back (keyboard cut/copy/paste, delete-to-trash, drag-and-drop, selection
actions) then hit the server with the mangled path and failed with
"Entry not found". The socket item.renamed/item.updated handlers had the
same problem via jQuery .attr(), and item.moved/item.removed matched rows
with html_encode()d selector needles that can't match raw attributes.

Store raw values everywhere (matching the desktop convention and the
raw readers throughout TabFiles), match paths by comparison instead of
selector interpolation, and repair the html_encode(x) ?? fallback
expressions that could never take their right-hand side. Verified live:
copy/paste of a name containing '&' now succeeds.

* fix: escape folder names in Dashboard nav-history menu (stored XSS)

The back/forward taphold menus interpolated path.basename(history_item)
straight into UIContextMenu item html, which renders it verbatim. A
folder named with an HTML/script payload executed when the user opened
the history menu after visiting it. The desktop file manager already
encodes this value; match it with html_encode(). Verified live: the
payload now renders as inert text and its onerror never fires.

* fix: harden Dashboard hash routing against malformed and unknown tabs

Three routing defects, all reachable from a shareable URL:

- A malformed percent-sequence (e.g. `#100%`) made parseDashboardRoute's
  decodeURIComponent throw at module load, blanking the entire GUI. Guard
  the decode and fall back to the raw hash. Verified: `#100%` now boots.

- A hash whose tab segment contained a quote (`#foo"bar`) was interpolated
  into a jQuery selector that throws, leaving dashboard event handlers
  unbound. Resolve the route tab against the known tab set (falling back to
  Apps) before it ever reaches a selector. This also fixes an unknown hash
  activating the Apps section without its `.dashboard-content.apps` styling.

- Re-clicking the already-active sidebar tab pushed duplicate history
  entries, so Back became a no-op until pressed repeatedly. Only pushState
  when the hash actually changes.

Verified live for all three.

* fix: keep Dashboard file list footer and size cells in sync on live updates

Incremental updates only touched hidden data attributes, so the visible
UI drifted from the data:

- Adding a file via the socket (item.added) or creating one inserted a
  row but never called updateFooterStats, so the "N items · size" footer
  stayed frozen at the old count. Removals and moves had the same gap.
- item.updated (and the in-place update in UIDashboardFileItem) rewrote
  data-size/data-modified but not the on-screen .item-size/.item-modified
  cells, so a remote overwrite left a stale size (e.g. "5 B" for a file
  that had grown to 2 KB).

Refresh the footer after add/remove/move and repaint the size/modified
cells on update. Verified live: add, overwrite, and delete now all
reflect immediately.

* fix: Dashboard Files shift-click selection after navigation and byte formatting

Shift-click selection:
- renderDirectory cleared the .selected class but left
  window.latest_selected_item pointing at a now-detached row from the
  previous directory. The shift handler set shift_clicked=true, found the
  anchor's index as -1, skipped the range select, and onclick then
  early-returned on shift_clicked — so the first shift-click in a new
  directory selected nothing. Reset the anchor when it detaches, only take
  the range path when the anchor is still in the list, and treat a
  shift-click with no valid anchor as a plain select that becomes the
  anchor (onclick otherwise ignores clicks while Shift is held).

formatFileSize:
- Returned "NaN undefined" for missing/invalid sizes and "1.5 undefined"
  for terabyte-plus files (sizes array stopped at GB). Guard non-finite
  and non-positive input, clamp the unit index, and add TB/PB.

Verified live: first shift-click selects the item, a second shift-click
extends the range, and formatFileSize returns 0 B / 1.5 TB / 2 PB.

* fix: show a message instead of a blank pane when a Dashboard folder fails to open

renderDirectory empties the file list before awaiting readdir, so a
readdir rejection (permission lost, folder deleted from another session,
network error) left a completely blank pane with only a console error.
Render a centered "This folder couldn't be opened." message on the catch
path. Verified live: a simulated readdir failure shows the message,
clears the spinner, and leaves navigation working (next folder loads).

* fix: Dashboard Usage/Home tab data freshness, errors, and dead controls

TabUsage:
- The Upgrade/Manage button was shown unconditionally and its click did
  `new window.UIUpgradeAccount()`, which only exists on hosted puter.com —
  on self-hosted it threw a TypeError from a dead button. Hide the button
  when UIUpgradeAccount is absent and guard the click.
- The tab had no onActivate and bound its refresh to a nonexistent
  element, so usage numbers were frozen at page-load for the whole
  session. Add onActivate to refetch (verified: reactivating refetches).
- Resource names rendered the backend's `_dot_` escaping literally
  ("gemini-2_dot_5-flash"); un-escape for display and html_encode the cell.
- The two loaders had no error handling; a failed fetch left the tab
  blank with an unhandled rejection. Catch each and show a fallback.
- Guard capacity 0 so storage no longer renders "NaN%".

TabHome:
- Plan button kept saying "Manage →" after a subscription lapsed; reset it
  to "Upgrade →" in the free branch.
- Guard capacity 0 ("NaN%") on the storage card.
- "Your Plan ›" card header had the arrow affordance but no target and did
  nothing on click; point it at the Usage tab like its siblings.
- Returning to the tab fired both focus and visibilitychange, and
  refreshAndBroadcast both called refresh() and dispatched the event its
  own listener handles — up to 4 duplicate reloads per focus. Drop the
  direct call and coalesce the focus/visibility pair.

TabHome + TabApps:
- window.open(externalAppUrl, '_blank') lacked noopener, exposing the
  dashboard tab to reverse tabnabbing from an external app site. Add
  noopener,noreferrer.

Verified live: Usage and Home render without NaN or console errors, the
upgrade button is hidden on self-hosted, and 'Your Plan' now navigates.

* fix: multiple Dashboard CSS defects (dead rules, contrast, responsive)

- The desktop rule that hides the row ⋮ button ended two selectors with a
  comma before an @media block, so the whole construct was invalid and
  discarded — the ⋮ showed on every list-view row on desktop instead of
  deferring to right-click. Verified: it's now hidden (display:none).
- Native-drop dark-mode styles keyed off .window[data-color-scheme="dark"],
  an attribute never set anywhere; converted to @media (prefers-color-scheme:
  dark) like every other dark rule so they actually apply.
- .myapps-tile-label set white-space:nowrap after a 2-line -webkit-line-clamp,
  defeating the clamp so long app names clipped mid-glyph on one line;
  removed it.
- .dashboard-sidebar-separator was defined a second time with a contradicting
  box model (background line + full-width margin), rendering a doubled,
  edge-to-edge divider; removed the duplicate so the inset border rule stands.
- Grid-view .item-icon had a hardcoded background:white and border-radius:2px
  overriding its own border-radius:8px (glaring white tiles in dark mode,
  square-ish corners); use var(--dashboard-background) and keep 8px.
- .files-footer used hardcoded #666/#CCC on theme-variable backgrounds
  (dim in dark, near-invisible separator in light); use the text/muted vars.
- Fixed the .ui-droppable-over typo (jQuery UI emits ui-droppable-hover).
- The context-menu backdrop's "desktop transparent" rule used min-width:768px
  while every mobile rule uses max-width:768px, overlapping at exactly 768px
  (a common tablet width) — a non-dimming, invisible modal shield; bumped to
  769px.
- At 481-768px the fixed hamburger toggle sat on top of the Files directories
  column's first folder because .dashboard-content.files padding wins over the
  media-query padding; add top clearance to the directories column there.

Verified light-mode desktop is unchanged (directories padding still 16px, ⋮
hidden, footer colors resolve).

* fix: only offer Uninstall for Dashboard apps where it actually sticks

Uninstall was suppressed via a hardcoded 8-name allowlist that had drifted
out of sync with the backend's ~26 recommended apps. For the ~18 unlisted
recommended apps (Calculator, Code, the games, …) the menu offered
Uninstall, revokeApp resolved, the tile vanished — then get-launch-apps
re-added it on the next load and it reappeared, so the uninstall silently
reverted.

Compute uninstallability from the actual lists: an app is uninstallable
only if it's in the user's installedApps AND not in the recommended list
(and not a protected core app). Recommended apps — installed or not —
resurrect on reload, so Uninstall is hidden for them. Verified live: a
synthetic installed-not-recommended app shows Uninstall; a recommended
app shows none.

* fix: guard Dashboard Apps loads against stale overwrites and drag/error clobbering

loadApps only checked for an in-progress drag before its await, and its
catch wiped whatever was on screen. Three concurrency issues followed:

- A slow, older load resolving after a newer one could overwrite the
  newer app list (and clobber a reorder the user saved while the stale
  fetch was in flight). Tag each load with an increasing id and skip
  applying one only when a strictly newer load has already applied —
  gating on "already applied" (not "latest started") so the first load to
  resolve still populates the list for the pager's ResizeObserver.
- A drag that began while a load was awaiting could have the grid rebuilt
  out from under it; re-check the drag after the await.
- A transient re-fetch error replaced a working grid with "Failed to load
  apps"; only show that placeholder when nothing has loaded yet.

Verified live: a slow older load no longer clobbers a newer one, and the
grid still renders normally on activation.

* fix: page through all installed apps in the Dashboard instead of capping at 100

The /installedApps endpoint clamps limit to 100 and paginates, but the
Apps tab fetched a single page — a user with more than 100 installed apps
silently lost the alphabetically-last ones from both the grid and search,
with no way to launch or uninstall them from the dashboard. Loop pages
until a short one comes back (the common <100-app case still makes a
single request). Verified: the request now carries &page=1 and stops
after one page for a small account; the loop pulls all pages otherwise.

* fix: make the mobile Dashboard context menu handle submenus, disabled items, and positioning

The touch context-menu modal (used whenever maxTouchPoints > 0) had three
defects:

- Items with a submenu and no onClick ("New", "Open With") rendered as
  plain buttons that did nothing and didn't even close the menu — the two
  submenu-bearing actions were simply unreachable on touch. Drill into the
  submenu on tap, with a Back row to return.
- disabled items were rendered as active buttons and executed their
  onClick, so a folder's disabled "Paste Into Folder"/"Publish as website"
  ran anyway. Render them inert (disabled attribute + dimmed class + a
  click-handler guard), mirroring the desktop UIContextMenu.
- Non-touch devices that still route here (touchscreen laptops) got the
  modal pinned at a hardcoded left:300px, far from the tapped item on a
  wide screen. Center it over the target and clamp to the viewport.

Verified live (with maxTouchPoints forced): "Open With" opens its submenu
and Back returns; disabled "Paste Into Folder" is inert; the modal is
positioned near the item.

* fix: repair Apps-tab regressions from the Dashboard bug-fix pass

- A non-array /installedApps response (an error payload) was read as
  end-of-pagination, silently rendering the grid without any installed
  apps; fail the load so the explicit error state shows instead.
- A loadApps result that resolved mid-drag was discarded with no retry,
  freezing the grid on stale data for the rest of the visit; stash it
  and apply it after the drag ends, reconciled against the drag's final
  order so it can't undo a reorder whose save is still in flight.
- Uninstall disappeared for recommended/recent apps, removing the only
  UI path that revokes their permissions; offer it again and instead set
  expectations in the confirm modal (the tile stays for apps that
  get-launch-apps re-adds).
- init and the initial-route onActivate both fired a full load on open;
  share the in-flight load instead of issuing a duplicate request trio.

* fix: repair Files/routing regressions from the Dashboard bug-fix pass

- The no-anchor shift-click fallback also fired when shift-clicking the
  current anchor, collapsing an existing multi-selection to one item,
  and it ignored Ctrl/Cmd; only run it when there is genuinely no
  anchor, and keep the existing selection when Ctrl/Cmd is held.
- Unknown hash values (a stale bookmark, an in-page anchor) were
  coerced to 'apps' and yanked the user off their current tab with a
  refetch and autofocus; ignore them outright, both at boot and on
  hashchange, while still keeping untrusted hashes out of selectors.
- The item.updated socket handler re-implemented the row-refresh from
  TabFiles and had already drifted (trashed items showed their raw UID
  instead of metadata.original_name); extract a shared row updater and
  delegate to it.

* fix: repair Usage-tab regressions from the Dashboard bug-fix pass

- Hiding the plan button whenever window.UIUpgradeAccount was missing
  at check time could hide it for the whole session on hosted
  deployments that attach the script after dashboard init; keep
  re-checking for a while before concluding the install is self-hosted.
- init and the initial-route onActivate both refreshed usage on a
  direct #usage open; share the in-flight refresh instead of issuing
  duplicate request pairs.

* fix: repair Dashboard CSS regressions from the bug-fix pass

- Repairing the malformed item-more rule put display:none into effect
  for the first time, removing the visible-and-used desktop row '⋯'
  button; drop the rule instead (it was never in effect on any release).
- The desktop context-menu backdrop breakpoint (min-width: 769px) left
  fractional viewport widths between 768px and 769px with the mobile
  dimming; use the exact complement of the mobile max-width: 768px
  rules instead.

* fix: make Dashboard app uninstall honest about recents and resilient mid-flight

- Recently-opened apps were classified as sticky-removable, but the
  recent list is built from app-open history that a revoke doesn't
  touch, so their tiles reappeared on the next load — the very bug the
  uninstall gating set out to fix. Treat recents like recommended.
- The confirm modal claimed every staying tile was 'provided by Puter',
  which is false for third-party recents; describe the actual reason.
- Uninstall now marks in-flight loads stale so a fetch that started
  before the revoke can't resurrect the removed tile.
- A pagination failure after the first page no longer discards the
  pages already fetched — one flaky request among N used to turn the
  whole grid into 'Failed to load apps'.

* fix: keep the Dashboard Usage tab stable through transient failures

- A failed refresh no longer wipes an already-rendered usage table;
  the unavailable note only shows when there is nothing on screen yet
  (mirrors the Apps grid's error handling).
- The plan-button check now re-checks indefinitely with backoff instead
  of giving up after ~10s, so an arbitrarily late UIUpgradeAccount
  attachment can't leave a subscriber without the button all session.

* fix: Dashboard Files footer sync and Home refresh resilience

- The shared row updater now refreshes the footer's item-count/total
  line, which is computed from the data-size attributes it just wrote;
  a remote overwrite no longer leaves the row and footer disagreeing.
- Size/Modified are only written when the update payload carries them,
  so a minimal event can't zero out correct values on screen.
- A failed readdir now resets the footer instead of keeping the
  previous directory's counts over an empty pane.
- Home's focus-refresh caps its wait on refresh_user_data, so a whoami
  that never settles can't leave the plan/usage cards stale forever.

* fix: contain partial Apps loads and keep the uninstall flag current

- A partial installed-apps list (a page beyond the first failed) now
  renders only when the grid is empty — it must never replace a
  complete grid already on screen — and saveOrder refuses to persist
  while the list is partial, since overwriting the saved order with a
  truncated one would drop the missing apps' positions for good.
- Opening an app from the grid now flips its tile's uninstallable flag:
  the open puts it in the server-side recent list, so a revoke from
  that moment on leaves the tile in place, and the modal must not act
  on the stale load-time snapshot.
- The in-flight-load invalidation moved into _invalidateInFlightLoads()
  so the seq protocol lives in one place.

* fix: Usage-tab error guard, dead spinner, and poll decay

- The transient-error guard now tracks 'a table has rendered' instead
  of data length, so an account with legitimately zero usage doesn't
  get its empty table replaced by the unavailable note on a flaky
  refresh.
- update_usage_details animated a spinner element that exists nowhere
  in this tab and held every refresh open for an artificial 1s minimum;
  both halves of the dead control are now gone.
- The plan-button recheck backs off to a 60s heartbeat instead of
  polling at 2s forever on installs where UIUpgradeAccount never
  appears.

* fix: guard minimal file updates; tighten selection and socket hot paths

- The shared row updater only writes the fields an update payload
  actually carries, so a minimal event can't blank a row's name/path —
  the same hazard the size/modified guards already covered.
- The all-rows lookup in the selection handler now happens only under
  Shift instead of on every click, and the single-select block shared
  by the no-anchor shift-click and drag-handle paths lives in one
  selectSingle helper.
- item.removed and item.moved narrow by data-uid (O(1) selector) before
  comparing paths instead of scanning every row per socket event.

* fix: replace partial-load and uninstall-flag patches with convergent designs

The previous round's containment patches each spawned new corner cases
(a persistently failing page froze all refreshes; the partial-load
saveOrder bail silently discarded reorders; the tile-open flag flip
lost races with in-flight loads, server recording, and popup blockers).
Replace both mechanisms:

- A partial installed-apps fetch (later page failed) now always applies,
  supplemented with the apps already known from the previous list — the
  grid and search can't shrink from one flaky request, refreshes never
  freeze, and an empty grid still gets the fetched pages.
- saveOrder carries over saved names missing from the current list
  instead of refusing to persist, so no app's saved position is ever
  dropped; stale names are harmless (reconcileAppOrder ignores them)
  and preserve the position of apps that return.
- The load-time uninstallable snapshot is no longer patched client-side
  on tile open. Uninstall now revokes, optimistically removes the tile
  when expected to stick, and refetches — the grid converges to
  server-side truth instead of guessing at recents state.

* fix: don't blank filenames on metadata-only updates; unify range selection

- A payload carrying metadata but no name resolved to an empty display
  name and blanked the row's visible filename — skip the write when the
  resolved name is empty.
- The shift-range branch now goes through the same applySelection
  helper as the no-anchor and drag-handle paths (it takes the row set),
  removing the last hand-rolled copy of the selection bookkeeping.

* fix: rank-preserving saved-order merge; keep the pager put after uninstall

- The saved-order carryover appended missing apps' names at the tail,
  permanently demoting their saved grid positions — the very thing it
  claimed to protect. Replace it with mergeSavedOrder in appOrder.js
  (beside its tested siblings, with unit tests): each missing name
  keeps its rank among surviving names, so a drag during a partial-load
  session neither drops hidden apps' positions nor teleports them
  behind the dragged tile.
- The post-uninstall convergence refetch rendered without preservePage,
  snapping the pager back to page 1 and replaying the load fade; thread
  render options through loadApps so the background sync keeps the
  user's page.

* fix: keep hidden apps' saved ranks through the mid-drag stash path

- _applyPendingLoad reconciled a stashed load against the visible-only
  on-screen order, tail-appending any app returning to the grid and
  contradicting the order the drag had just saved — the rank-demotion
  defect resurfacing through one more path. It now prefers
  _savedOrderNames, the canonical saved list that saveOrder keeps
  merged with hidden apps at their ranks, over the possibly pre-drag kv
  snapshot the stashed load fetched.
- The post-uninstall optimistic render now also skips the load fade;
  it was the one in-place rebuild still hiding the grid behind the
  opacity-0 icon gate.
- mergeSavedOrder's rank rescan collapsed to a single forward
  insertion pointer (verified output-identical; the 20 unit tests pin
  the behavior).

* fix: gate stale kv-order snapshots on the latest local save

A drag that started and ended while a load was in flight escaped the
stash path entirely: the load applied normally and replayed its
pre-drag kv snapshot, visibly reverting the just-saved reorder (and
permanently clobbering it after the next save). Conversely, the stash
path unconditionally preferred the local record, discarding a genuinely
fresher order fetched from another window when the drag committed
nothing.

Record the load-seq boundary at each saveOrder; a resolving load (both
apply paths, via _resolveOrderNames) replays its fetched kv order only
when it was issued after the latest local save — otherwise the
canonical in-memory saved list wins. Also rewrites the _applyPendingLoad
header, which described the pre-rank-merge implementation.

* Simplify app uninstall tile handling

Remove the uninstallability flag and modal warning logic, and treat uninstall as a local UI removal after permission revoke. This avoids an immediate reload that could re-add recommended/recent apps, while stale in-flight loads are still invalidated. Also simplifies app loading by dropping unused render options from `loadApps`/`_fetchAndRenderApps` and related merge comments.

* fix: match raw item paths in move_items cleanup and shortcut re-point

Item rows store data-path/data-shortcut_to_path unencoded, so the
html_encode()d attribute selectors missed names containing & < > " '
and the destination-row exclusion guard could remove a legitimate row
just created by a concurrent item.moved handler. Compare raw values
case-insensitively instead.

* fix: bound the Usage plan-button retry poll

The UIUpgradeAccount retry chain decayed to a 60s heartbeat but never
stopped, leaving self-hosted installs polling forever and pinning the
captured $el_window. Give up after ~30s of decaying retries; onActivate
re-checks on every return to the tab, so late script loads still get
the button.

* fix: drop the recent-opens list from the Apps tab grid

Recents are open history, not installs: they resurrected uninstalled
apps' tiles and showed merely-visited sites as if installed. The grid
is now recommended + installedApps; anything the user actually uses
still appears because opening an app grants it a permission. Recents
still power the Home tab.
2026-07-18 20:39:57 -07:00
jelveh 81c69f243b Align sign-up email button with OIDC styling 2026-07-18 09:35:13 -07:00
jelveh 6a3ecaff2a improve and unify OIDC button styling 2026-07-18 09:18:21 -07:00
jelveh 70f343d6fe Defer taskbar creation in fullpage mode 2026-07-17 14:38:06 -07:00
jelveh 172aa01e1c Trim file context menu actions 2026-07-17 14:23:52 -07:00
jelveh cba6ac8184 Clamp and normalize window progress values 2026-07-17 14:14:50 -07:00
jelveh 492f3d235a Add active label variant to context menu height rule 2026-07-17 13:54:12 -07:00
jelveh d9a01a3748 Add responsive item properties modal to Dashboard 2026-07-17 13:21:15 -07:00
Nariman Jelveh aab21ef91d don't record app open in iframe env 2026-07-17 11:42:48 -07:00
Daniel Salazar 2479e6a064 fix: FS permissions (#3404) 2026-07-17 11:01:34 -07:00
jelveh 7cba26bb39 Revert "fix(dashboard): make app drag-to-reorder work on touch (iOS)"
This reverts commit af0dc06f49.
2026-07-17 08:53:46 -07:00
jelveh af0dc06f49 fix(dashboard): make app drag-to-reorder work on touch (iOS)
On iOS/WebKit the pager's `touch-action: pan-x` made horizontal touchmove
non-cancelable, so the moment a long-press drag moved the finger the
browser started a native page-pan and fired pointercancel, aborting the
reorder before it visibly began. Preventing pointermove doesn't stop
native scrolling on iOS, and the reordering `touch-action: none` rule was
applied only after the drag began — too late, since touch-action is
latched at gesture start.

Drop the explicit pan-x (back to auto) so horizontal touchmove is
cancelable, and install a non-passive touchmove listener that
preventDefaults only once reordering is armed (post long-press). Before
arming it's a no-op, so a quick swipe still flips pages natively.
2026-07-17 08:47:58 -07:00
Nariman Jelveh 14900514b9 Dashboard: drag-to-reorder apps in the Apps tab (#3401)
* feat(dashboard): drag-to-reorder apps in the Apps tab

Let users arrange their My Apps tiles in any order via drag-and-drop.
The order is stored per-user in puter.kv (`dashboard_apps_order`) as a
list of app names and re-applied on load; apps installed since the order
was saved are appended, and uninstalled ones are dropped.

Interaction:
- Mouse/pen: drag past a small threshold to reorder; a plain click still
  opens the app.
- Touch: long-press to pick up a tile (a quick swipe still flips pages).
- Dragging to a scroller edge dwells, then flips to the adjacent page so
  tiles can move across pages; the floating drag image lives on <body> so
  the pager's overflow can't clip it, and displaced tiles reflow with a
  FLIP animation.
- Escape (or losing the window) cancels and reverts; reordering is
  disabled while a search filter is active; prefers-reduced-motion is
  honored.

Also fixes a latent pager bug where programmatic page changes (arrows,
dots, wheel) didn't update the active page indicator, because those
scrolls don't reliably emit 'scroll' events; goToPage now tracks the
page eagerly.

The order-reconciliation logic is extracted into a pure module
(appOrder.js) with unit tests.

* fix(dashboard): ignore stray pointers and no-op drags

- Filter drag pointermove/up/cancel by pointerId so a second finger can't
  hijack or prematurely end an in-progress reorder.
- Only persist a new order when the drop actually changed it, so an
  accidental long-press or drop-in-place doesn't freeze the default
  ordering into a custom one.

* fix(dashboard): address review of app drag-to-reorder

Correctness:
- Preserve the touch long-press → Uninstall path. A long-press now *arms*
  reordering instead of immediately grabbing the tile; moving begins the
  drag, while holding still lets the native context menu fire (which
  cancels the pending pickup). This restores the only touch route to
  Uninstall that the first cut had removed.
- Only swallow the post-drag click when the drop actually reordered
  something, so a small pointer drift or a drop-in-place still opens the
  app like a plain click.
- Guard the pre-start "intent" window: cancel a pending pickup before any
  re-render (ResizeObserver / loadApps), and bail out of _beginDrag if the
  pressed tile was detached, so a layout change mid-press can't reinsert a
  stale node and persist a corrupted order.

Cleanup:
- Reuse reconcileAppOrder for the drop path instead of a duplicate helper.
- Read the saved order in parallel with the app-list fetches.
- Cache the reduced-motion MediaQueryList instead of re-querying per move.
- Fold the touch-scroll cancel into _endDrag(false); drop _abortDragIntent.

* feat(dashboard): smoother, more tolerant app reordering

Make drag-to-reorder feel like the iOS home screen instead of snapping
and flickering:

- Slow the reflow to 320ms with a gentle spring easing.
- Hit-test against each tile's *resting* box, not its live
  getBoundingClientRect. Mid-animation a tile sits between slots, so
  testing its live box was dropping it under the pointer and swapping it
  straight back — the source of the back-and-forth jitter.
- Require the dragged icon's centre to be well inside a tile before it
  becomes the drop target (a deadzone around each tile), so hovering a
  boundary no longer thrashes.
- Probe with the dragged icon's centre rather than the fingertip, and
  batch the FLIP into a single reflow so large grids stay smooth.
2026-07-17 07:57:05 -07:00
Daniel Salazar 7faaef8cf0 tests: more tests for puter.js (#3402) 2026-07-16 22:35:15 -07:00
Nariman Jelveh c188257f85 Session Manager: compact redesign + responsive modal (no UIWindow) (#3400)
* redesign: compact, scannable Dashboard Session Manager

The session list rendered each entry as a bulky card with a 5-row
Created/Last active/Expires/IP/Client key-value table and a large filled-red
Revoke button, so only a couple of sessions fit on screen and destructive
actions dominated the layout.

Reworked it into an icon-led, compact list:
- One row per session: a device/kind icon tile (laptop / phone / globe by OS,
  bolt for workers, key for API tokens, the real app icon for app sessions),
  the title + badges, and two tight meta lines (client · IP, then a fainter
  last-active · created · expires with absolute-time tooltips).
- Destructive actions are calm: per-row Revoke and the rename pencil are quiet
  and reveal on hover (always shown on touch); "Revoke all other sessions" is a
  subtle ghost button instead of a filled block competing with search.
- Current session gets a green accent, tint, and pill, and has no revoke button.
- Toolbar search gains an inline icon + focus ring; added a session count line;
  restyled kind badges; nested child sessions get a proper tree line.

Styling uses the existing dashboard design tokens. All behavior is preserved:
search filtering, parent/child tree with expand/collapse, inline rename
(optimistic + rollback), per-session revoke (access-token vs session routing),
revoke-all, current-session guard, and focus/interval refresh.

Adds ui_session_count_one / ui_session_count_other to en.js (other locales
fall back to en).

* refactor: render Session Manager as a responsive modal, not a UIWindow

Replaces the draggable UIWindow shell with a self-contained DOM modal:
- Backdrop + centered card on desktop (max 680px / 90vh); full-screen sheet
  on phones (<=640px). Fade/scale in, respects prefers-reduced-motion.
- Adds a header bar (title + close), backdrop-click and Escape to close, and
  a scroll-contained body.
- Confirmations no longer use UIAlert (itself a UIWindow). Revoke, revoke-all,
  and the rename-error path now use in-modal confirm/alert sheets, so there is
  no cross-window z-index juggling — important because this can be opened from
  a stay_on_top window (UIWindowCopyToken).

Drops the UIWindow and UIAlert imports entirely. All behavior is preserved:
search, parent/child tree + expand/collapse, inline rename (optimistic +
rollback), per-session revoke (access-token vs session routing), revoke-all,
current-session guard, and the focus/60s-interval refresh (cleaned up on close).
2026-07-16 20:48:54 -07:00
Nariman Jelveh 4543a645f2 fix: verified Dashboard bugs (10 separate fixes) (#3399)
* fix: prevent crash rendering website files without a workers array in Dashboard Files

TabFiles.renderItem accessed `file.workers.length` unguarded when deciding
whether to show the website badge. Directory entries that have a published
website but no `workers` array (the common case) made this throw
`Cannot read properties of undefined (reading 'length')`, so the row failed
to render — the file silently disappeared from the list (swallowed by the
Promise.allSettled batch) or was never added on socket-driven updates.

The sibling `is_worker` derivation on the same entry already guards with
`file.workers?.length`, and the desktop reference (UIItem.js) normalizes
`workers` to an array and shows the badge via `!is_worker`. Reuse the safe
`is_worker` flag here instead of touching `file.workers.length` directly.

* fix: correct Dashboard usage that showed negative spend for users with credits

The Usage tab and Home usage card derived month-to-date spend as
`monthUsageAllowance - allowanceInfo.remaining`. But the metering service
computes `remaining = remainingAllowance + remainingPurchasedCredits`
(MeteringService.getAllowedUsage), so `remaining` exceeds the monthly
allowance whenever a user holds purchased credits — making the displayed
spend and percentage negative (e.g. "-$9.90 used of $0.25", "-3960%").

Use the actual reported spend (`res.usage.total`, already returned by the
same endpoint) and clamp the percentage/bar to 100%. For a free user with no
credits and no overage this yields the identical value as before.

* fix: make Dashboard Files Download button work for a single selection

In mobile select mode the floating action bar appears with one item selected
(minCountForActionBar is 1) and shows an enabled Download button, but the
click handler only acted when 2+ rows were selected — so downloading a single
file did nothing.

Gate on `> 0` instead. window.zipItems already normalizes a single item to an
array, so the one-file case works. On desktop the bar only appears at 2+
selections, so this changes nothing there.

* fix: auto-fit column width in Dashboard Files no longer scales with row count

Double-clicking a column resize handle is meant to fit the column to its
longest cell. The per-row loop did `maxWidth = Math.max(maxWidth + 10, textWidth)`,
which forces `maxWidth` to grow by at least 10px on every row regardless of
content — so a folder with N files produced a column ~10*N px wide and
persisted that runaway width to KV storage.

Compare against the running max only (`Math.max(maxWidth, textWidth)`);
`textWidth` already includes cell padding.

* fix: clear empty-directory placeholder when a file is added incrementally

When a directory is empty, renderDirectory appends an absolutely-positioned
"No files in this directory." placeholder. If a file then appears via an
incremental path (socket item.added/item.moved -> UIDashboardFileItem), the
new row was inserted but the placeholder was never removed, so the "No files"
text overlapped the real item. insertAtSortedPosition can't clear it because
the placeholder has no `.item.row` class.

Remove the placeholder in UIDashboardFileItem before inserting the row,
matching what the in-app New Folder / New File flows already do.

* fix: stop leaking a document keydown handler per uninstall-modal dismissal

showUninstallModal bound `keydown.uninstall-modal` on document but only
detached it on the Escape and Confirm paths. Dismissing via Cancel or a
backdrop click called `close()`, which just removed the overlay and left the
handler attached (closing over a now-detached overlay). Each open→cancel
cycle stacked another document-level listener for the page lifetime.

Move the `.off('keydown.uninstall-modal')` into `close()` so every dismissal
path cleans up, and drop the now-redundant explicit detaches.

* fix: resolve 2FA setup promise on every close so Dashboard toggle can't freeze

UIWindow2FASetup only resolved its promise from the Done button and, in
on_before_exit, when setup had NOT succeeded. Clicking "Enable" sets
setup_succeeded=true and advances to the recovery-codes screen without
resolving; if the user then closes that screen via the backdrop or Escape
instead of Done, the promise never settles.

The Dashboard Security tab is the only caller: it disables the 2FA toggle and
`await`s this promise, so the hang leaves the toggle permanently disabled and
stuck in the pending state for the rest of the session. Resolve with
setup_succeeded on any exit; resolve_promise is idempotent so a prior
Done-resolve is unaffected.

* fix: refresh Dashboard tabs when navigating via browser back/forward

The sidebar-click and initial-route handlers both call the target tab's
onActivate (its refresh hook), but handleRouteChange — wired to hashchange
and popstate — only swapped the active classes. Since tab switches use
history.pushState (which doesn't fire popstate/hashchange), the only way to
reach handleRouteChange is browser back/forward, and doing so left tabs like
Home/Apps showing stale data.

Look up the tab by id and call onActivate after switching, matching the
sidebar-click path.

* fix: refresh Home "Recently used" apps instead of freezing them for the session

loadRecentApps only hit /get-launch-apps when window.launch_apps.recent was
empty. Once populated (typically at page load), every subsequent init/onActivate
/focus just re-rendered the same cached array, so apps launched during the
session never showed up under "Recently used" until a hard reload.

Drop the empty-list guard so the list refetches on the same triggers that
already refresh the usage cards. The fetch can only make the list fresher.

* fix: sync window.user.otp when enabling 2FA from the Dashboard

The disable path clears window.user.otp (in UIWindowDisable2FA), but the
enable path updated only the card's DOM, never the in-memory flag. TabSecurity
html() reads user.otp to decide the toggle/badge state, so after enabling 2FA
a re-render of the Security tab from the cached window.user showed it as
disabled for an account that actually had 2FA on.

Set window.user.otp = enabling on success so both directions stay consistent.
2026-07-16 20:03:38 -07:00
jelveh 75d3b07e0f fix: phantom shortcut badge on files created via dashboard New menu
renderItem() showed the shortcut badge whenever file.is_shortcut !== 0.
Directory listings return is_shortcut as 0, but the puter.fs.upload()
result used to incrementally render newly-created items omits the field,
so undefined !== 0 was true and every new file got a phantom shortcut
badge. Normalize is_shortcut to 0/1 before using it.
2026-07-16 19:05:28 -07:00
Nariman JelvehandClaude Opus 4.8 9da32c73cb Restore Files tab in dashboard (#3397)
* Restore Files tab in dashboard

Reinstates the Files tab (TabFiles) that was removed in 973d046d, bringing
back the dashboard file browser: directory navigation, list/grid views,
sorting, drag-and-drop, context menus, upload, and trash support.

- Restore src/gui/src/UI/Dashboard/TabFiles.js (identical to pre-removal)
- Re-wire UIDashboard.js: tab registration, initial file path, item socket
  handlers, and route-based file navigation
- Re-add file-tab CSS in dashboard.css and the selection-area rule in style.css
- Re-add dashboard-mode item creation hooks in helpers.js move_items
- Re-add #files/<path> deep-link parsing in initgui.js parseDashboardRoute,
  while preserving Apps as the default dashboard tab

* Only remove stale rows when moving items

* Fix Dashboard Files tab bugs

- Escape filenames in renderItem to prevent stored XSS via HTML in file
  names (matches UIItem); raw name is still preserved for display/rename.
- Read clone data-id from the nested .row in drag drop handlers so
  multi-select drag moves all selected items, not just the grabbed one.
- Resolve sidebar drop target from the element's data-path so dragging
  onto Public/Home works (they aren't keyed in user.directories); fix the
  matching UID-vs-path comparison in the drag-out handler.
- Set data-is_worker/data-worker_url from is_worker itself instead of an
  always-true "!== undefined" check, so worker context-menu behavior only
  applies to actual workers.
- Give delete-confirmation buttons explicit values so permanent delete
  works in non-English locales.
- Refresh the view after a context-menu move-paste, mirroring Ctrl+V.
- Guard the item.updated socket handler against the client's own echo,
  matching the other handlers and UIDesktop.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* Prevent Files tab lockup when readdir fails

renderDirectory sets renderingDirectory=true then awaits puter.fs.readdir
with no error handling. readdir rejects on any backend error (permission,
deleted directory, network), leaving renderingDirectory stuck true so the
re-entry guard blocks all future navigation and the spinner never clears.
Catch the rejection and reset state.

* Isolate render failures so one bad entry can't lock the Files tab

renderItem parsed file.metadata with an unguarded JSON.parse, but metadata
is client-writable and stored verbatim, so '', undefined, or malformed
values throw (item_icon.js guards the identical call). Combined with
Promise.all, one bad entry rejected the whole render and left the tab
frozen. Guard the parse and use Promise.allSettled for per-item isolation.

* Clear selection when clicking empty area in Files tab

The empty-area click handler is an arrow function, so `this` was the
TabFiles object rather than the container element, making `e.target === this`
always false. Only clicks landing exactly on `.files` deselected. Use
e.currentTarget so clicks on the container background deselect as intended.

* Avoid stacking image-preview document click handlers

showImagePreview bound a click.imagepreview handler without removing a
prior one. It is re-invoked during image arrow-navigation while a preview
is open, so identical document handlers accumulated until close. Remove
any existing handler before binding.

* Sort live-inserted items by display name

insertAtSortedPosition compared the new item's raw file.name against
existing rows' data-name (the display name). For trashed items, name is
the UID while the real name is in metadata.original_name, so newly trashed
items landed in the wrong position in Trash until a full re-render. Compare
on the new row's data-name to match sortFiles and the existing rows.

* Use display name in in-place item update

UIDashboardFileItem's in-place update path set data-name and the shown
name from raw file.name, unlike renderItem which uses
metadata.original_name || file.name. An update to an already-visible
trashed item replaced its name with the UID and corrupted data-name
(which sort and type-to-search read). Compute the display name to match.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-16 18:57:19 -07:00
Daniel Salazar 102cef523f fix: app id whitelisting (#3398) 2026-07-16 18:03:34 -07:00
Daniel SalazarandClaude Fable 5 0e1be72f92 test: tests for puter.js (#3396)
* test: tests for puter.js

* fix: ship lockfile for coverage devDeps; tolerate missing base coverage

npm ci failed on CI because package.json gained the babel/istanbul
devDependencies without the matching package-lock.json update. Also make
the coverage workflow's base leg best-effort so a base ref that predates
the coverage script reports without the comparison column instead of
failing the run.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-16 15:52:37 -07:00
Neal Shah ef1afb9cc1 add just app telementry (#3387) 2026-07-16 14:04:54 -04:00
Neal Shah 0462ddd6f5 add support for step-up sessions (#3395)
* add support for step-up sessions

* update step up session
2026-07-16 13:58:49 -04:00
Neal Shah 8d7e70cde4 add kimi k3 (#3394) 2026-07-16 12:47:44 -04:00
Daniel Salazar 52e481128f wip: puter js tests structure (#3393) 2026-07-15 19:09:35 -07:00
Daniel Salazar c237e2433f fix: clean apps on subdomain deletion (#3392) 2026-07-15 13:38:53 -07:00
tamilselvanp57 a4bb57bb79 Fix static hosting domain fallback for self-hosted custom domains (#3382) 2026-07-15 11:45:10 -07:00
Daniel Salazar 861cf5efb8 feat: email api wip (#3391) 2026-07-15 11:30:16 -07:00
Daniel Salazar cf34f9fea9 fix: sanitize get user response (#3390)
* fix: sanitize get user response

* fix: app creation in dev center
2026-07-15 02:02:06 -07:00
jelveh 92a6d30e26 Add calendar to recommended apps 2026-07-14 23:04:24 -07:00
Neal ShahandDaniel Salazar 23e8705b03 Implement workers without cloudflare for local dev testing (#3389)
* typeify subdomains wip

* initial (untested) logic for LocalWorkerService

* Make it work, add lifecycle expiry since workers are process heavy in current implementation

* fix type errors

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-15 01:01:42 -04:00
Daniel Salazar 3240a4670a fix: admin gates (#3386) 2026-07-14 12:58:04 -07:00
Daniel Salazar 3bf83d40e9 fix: ui signup order (#3385) 2026-07-13 18:47:11 -07:00
Neal Shah c3368f553e Add GUI button for custom OIDC (#3384) 2026-07-13 20:32:37 -04:00
velzie 27def94d8b feat: support cross-origin-isolated login (#3338) 2026-07-13 17:36:56 -04:00
Neal Shah 2c139929ac Check if user is suspended on WebDAV basic auth and others (#3369) 2026-07-13 02:46:28 -04:00
Neal Shah b957f797da add support for ndjson logging (#3381)
* add support for ndjson logging

* move install location
2026-07-13 02:43:56 -04:00
jelveh 393500d565 Add calculator to recommended apps
Include 'calculator' in RECOMMENDED_APP_NAMES within RecommendedAppsService.ts so the calculator app is included in the recommended apps list.
2026-07-11 23:04:39 -07:00
jelveh a108837e82 Revert "Add delayed centered spinner for apps tab"
This reverts commit 859e88753f.
2026-07-11 15:21:59 -07:00
jelveh 859e88753f Add delayed centered spinner for apps tab
Show a subtle centered loading spinner for the Apps dashboard when fetching apps if the request is slow. Implements a 1.5s delay before showing the spinner and guarantees a minimum 1.5s visible time to avoid blinking; the spinner is only shown when the container is empty (so background refreshes keep existing tiles). Adds a finish helper to clear the spinner timer and await the minimum visible time before rendering results or errors. Also adds CSS for the spinner element and its animations.
2026-07-11 15:16:04 -07:00
Daniel Salazar 5160b44c4b fix: missing OIDC error messages (#3377)
* fix: missing OIDC error messages

* fix: card verification user prefill out for subs
2026-07-11 08:18:13 -07:00
Daniel Salazar 17a9485fb2 Add spans to drivers and high frequency paths (#3376) 2026-07-10 23:15:09 -07:00
Daniel Salazar d3c44f3d6c fix: cache healthcheck in mem for each node (#3375) 2026-07-10 17:41:09 -07:00
Neal Shah d49ba5281d add support for ignoring specific issues in healthcheck (#3374)
* add support for ignoring specific issues in healthcheck

* add mark-degraded to healthcheck
2026-07-10 17:08:03 -04:00
jelveh c107c80bd8 Open dashboard links at #home anchor 2026-07-10 13:22:58 -07:00
Daniel Salazar cf90e775bf fingerprint: use device info for rate limit too (#3372) 2026-07-09 20:22:56 -07:00
Daniel Salazar 0a46096c41 fix: ai rate limits (#3371) 2026-07-09 16:35:03 -07:00
Daniel Salazar 289a44ae79 fix: OIDC state (#3370) 2026-07-09 14:47:34 -07:00
Neal Shah 827e9bc343 fix possible XSS entrypoints (#3368) 2026-07-09 17:19:55 -04:00
Daniel Salazar f2ffaeb823 fix: add phone errors into kv for debugging (#3367) 2026-07-09 12:25:17 -07:00
Neal Shah 6f594f4b60 add 5.6 models (#3366) 2026-07-09 14:46:57 -04:00
jelveh 326be042f1 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-08 22:33:20 -07:00
jelveh 5b824fb745 Extract resolveTileDisplay and broaden app search
Move logic that determines a tile's displayed title and target link into resolveTileDisplay (use hostname/index_url for external apps) and have buildTileHtml consume it. Expose data-target-link on tiles. Update app search to match the displayed title, raw title, name, and uid so queries find website-shortcut tiles and external apps consistently (matching Home tab behavior).
2026-07-08 22:33:13 -07:00
Reynaldi Chernando cf6cb126b5 fix grok 4.5 release date (#3365) 2026-07-09 12:30:39 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 302df1edd2 update user pays model docs (#3364)
* update user pays model docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-09 08:54:04 +07:00
jelveh e212509ed3 Avoid clipping scaled icons in horizontal scroller
Add a negative top margin and matching top padding to the horizontal scroller to prevent first-row icons from being clipped when they scale on hover. Overflow-x:auto previously caused vertical clipping of scaled icons; margin-top:-8px and padding-top:8px bleed the clip edge upward so scaled icons remain visible without changing scroll behavior.
2026-07-08 18:35:57 -07:00
Daniel Salazar d3e3755d6d chore: cleanup uneeded ai messages (#3363) 2026-07-08 17:55:49 -07:00
jelveh 948ce6e529 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-08 17:51:38 -07:00
jelveh d117dc231e Implement paged apps launcher UI 2026-07-08 17:50:34 -07:00
Daniel Salazar eefa53e5bd fix: misc auth bug fixes (#3362) 2026-07-08 17:50:22 -07:00
Neal Shah db89fbfc52 add grok 4.5 (#3360) 2026-07-08 14:44:05 -04:00
Daniel Salazar 97ec5d115c fix: double counting purchased credit usage (#3359) 2026-07-07 21:43:09 -07:00
Daniel Salazar 722a6cf50f fix: phone retry limit messagin (#3358) 2026-07-07 21:05:41 -07:00
Neal Shah d039dcfee5 fix appTelemetry (#3357) 2026-07-07 20:10:13 -04:00
Daniel Salazar d09aa11f2e feat: add whatsapp support? (#3356) 2026-07-07 16:13:18 -07:00
jelveh 8a2264546d Always use hash for dashboard tabs
Previously the dashboard used a clean root URL for the 'apps' tab and hashes for other tabs. This change makes tab links and history entries consistently use a hash (e.g. #apps, #home) by always generating href as `#<id>` and pushing `#<section>` to history. Simplifies URL handling and ensures the Apps tab also reflects its hash in the address bar.
2026-07-07 15:16:59 -07:00
0801e1dc44 feat: add signup disable config and fix telemetry startup (#3319)
* feat: add signup disable config and fix telemetry startup

* fix: nits for config spread and 403 checks

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-07 13:52:29 -07:00
Daniel Salazar a04d5fa5e0 fix: package-lock (#3355) 2026-07-07 13:43:15 -07:00
e1280e8cc7 Add custom icon support for web links (#3325)
* feat: add custom web link icon support and context menu option

Introduce the ability to change and display custom icons for .weblink files. Added a "Change Icon" context menu item that invokes the changeWeblinkIcon helper, and updated the icon rendering logic to use getWeblinkIcon instead of the default link icon. This allows users to personalize web link appearance in the file manager.

* Fix security and robustness issues in web link custom icons

 # UIWindowSearch and require the icon data URL body to be pure base64 in
  #isValidWeblinkIcon. A shared/downloaded .weblink is untrusted input and
 # the previous prefix-only check let a crafted icon break out of the
 # <img src> attribute.
 # byte-signature MIME sniffing and capping stored icon size (was storing
 # full-resolution images as base64).
 # without picking a file.
 # weblinks no longer fetches every file's contents on each render.
 # path instead of the target weblink).
 # label, add the AGPL license header, and drop redundant stored icon
 # copies.

* Keep SVG weblink icons as vectors instead of rasterizing

Rasterizing a picked SVG to a 256px PNG threw away its scalability for no
real benefit: an SVG rendered in <img> runs in script-free static mode, and
the base64-body validation already prevents attribute injection. Store SVGs
as data:image/svg+xml;base64 (content-sniffed, size-capped), and keep
rasterizing raster formats and oversized SVGs to bound stored size.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-07 13:41:37 -07:00
Aman Rai 70884c2e30 fix: rename start-gui script to start:gui (#3346) 2026-07-07 12:09:49 -07:00
Reynaldi Chernando 917e95e3d7 Add nano banana 2 lite (#3336) 2026-07-07 12:08:36 -07:00
jelvehandClaude Opus 4.8 27222bbb0d Let ctrl/cmd+f use browser find while dashboard is open
The global ctrl/cmd+f shortcut opened UIWindowSearch and preventDefault'd
the keystroke, blocking the browser's native find. Skip that interception
while a dashboard window is open so ctrl/cmd+f falls through to the browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-07 09:43:34 -07:00
jelveh 3720034464 Make Apps the default dashboard tab
Treat the Apps tab as the default (root URL) instead of Home. UIDashboard now reads an initial route from window.dashboard_initial_route?.tab and falls back to 'apps' for unknown/absent routes. Sidebar and section active class logic and tab href/hash generation were updated so 'apps' uses the clean root URL while all other tabs (including home) use #tab. The history.pushState logic was adjusted accordingly. parseDashboardRoute in initgui.js was updated to default to 'apps' and its comment clarified the new behavior.
2026-07-07 09:31:43 -07:00
jelveh de6cab1f9a Reduce myapps tile icon size to 56px
Update .myapps-tile-icon in src/gui/src/css/dashboard.css to change width and height from 60px to 56px, adjusting tile icon sizing for improved layout and spacing.
2026-07-06 21:04:32 -07:00
jelveh 2d1fbc2611 Auto-focus My Apps search on activation
Call focusSearch when the Apps dashboard is activated and add a focusSearch helper. The helper imports isTouchPrimaryDevice and skips focusing on touch-primary devices to avoid showing the on-screen keyboard. It retries up to 12 times (30ms interval) until the search input is visible to handle cases where the dashboard is briefly hidden while entering full-page mode.
2026-07-06 20:10:44 -07:00
jelveh 0afc70218f Use input[type=search] and normalize styling 2026-07-06 20:03:48 -07:00
jelveh 8d3abbc5f0 Increase myapps tile icon size to 60px
Update .myapps-tile-icon width and height from 52px to 60px to improve icon visibility and better match the updated UI design. Other styles (overflow, background, transitions) are left unchanged.
2026-07-06 19:47:17 -07:00
jelveh 7ec14bc7db Add audio-editor and ai-image-project to recommended apps 2026-07-06 19:42:20 -07:00
jelveh 2843010a97 Increase myapps tile size to 100px 2026-07-06 19:37:46 -07:00
jelveh ad4e21ee61 Open external apps via index_url from Apps tab
When an app is external and provides an index_url, use its hostname as the tile title and store the index_url in a new data-target-link attribute. Update the click handler to open the target link in a new tab for external apps, and fall back to opening the internal /app/{name} page for others. This aligns Apps tab behavior with the Home tab and ensures external apps navigate to their website.
2026-07-06 19:21:34 -07:00
Nariman Jelveh 4de39ab6f2 Mark external apps and use the flag for dashboard titles (#3354)
* Mark external apps and use the flag for dashboard titles

An app with no owner_user_id (null/empty) isn't owned by a Puter user —
it's an external, origin-bootstrapped app whose uuid/name/title are all the
opaque app-… id. The dashboard used to detect these client-side by comparing
uuid/name/title and a name.startsWith('app-') check.

Expose an authoritative `external` flag from the API instead:

- /installedApps and /get-launch-apps now return `external`, derived from
  owner_user_id, and no longer leak the raw owner id.
- The dashboard (Home + Apps tabs) shows the index_url hostname for external
  apps based on `external` rather than the uuid/name/title heuristic.

Add/extend extension tests to cover the `external` flag and non-leak.

* Use hostname for opaque external app titles

Detect opaque external app IDs (when name === title === uid/uuid) and replace the displayed title with the hostname from index_url. Adds a uid/uuid fallback and tightens the external-app check in both the Apps and Home dashboard tabs so they behave consistently; preserves target_link for Home entries.

* Add external flag for apps without owner

Expose an `external` property in AppSummary (toAppSummary in RecommendedAppsService.ts) to mark apps that are not owned by a Puter user. The flag is set when `owner_user_id` is null or an empty string, allowing clients to identify origin-bootstrapped/external apps.
2026-07-06 18:00:56 -07:00
Nariman Jelveh fbb646b8eb Return index_url from /installedApps for dashboard app titles (#3352)
The dashboard derives a hostname title for anonymous (app-…) apps whose
uuid/name/title are all identical, falling back to the host of index_url.
That worked for launch/recent apps but not for apps that come only from
/installedApps, because the endpoint didn't return index_url.

Add apps.index_url to the SELECT (alongside apps.icon, relying on the
functional dependency on the apps.id primary key) and assert it in the
extension test.
2026-07-06 16:46:31 -07:00
Neal Shah 46f75ecd6f add type for puter.app.authenticated (#3353) 2026-07-06 18:38:30 -04:00
Daniel Salazar f68caa3bbd feat: allow optional card fallback for failed phone verifications (#3351) 2026-07-06 14:59:55 -07:00
jelveh 0109bf5b72 Use index_url hostname for anonymous app title
Detect anonymous apps whose name/title/uid are identical opaque ids (starting with 'app-') and, when index_url is available, replace the displayed title with the index_url hostname (matching the Home tab behavior). Also persist index_url into the mapped app object and make title a mutable variable to support the change.
2026-07-06 14:43:39 -07:00
Nariman Jelveh c69fb52d5c Make the dashboard the default interface at root (#3345)
* feat: make the dashboard the default interface at root

The dashboard now loads at / (with /dashboard kept as an alias), and the
desktop moves to a new /desktop route. An "Open Desktop" button pinned at
the bottom of the dashboard sidebar switches to the desktop.

Root URLs that carry a desktop-only flow keep booting the desktop:
auth popups (?embedded_in_popup=), app deep links (?app=), direct
downloads (?download=), fullpage mode (?puter.fullpage=), and iframe
embeds.

Desktop-context redirects are retargeted so desktop users stay on the
desktop: logout, the QR "continue on phone" link, the auth cover-page
logo click, and window close/hide URL resets now use /desktop. OIDC
login/signup gains a strictly whitelisted return_to param (/desktop or
/dashboard) so social login returns to the interface it started from.

Also: the dashboard strips ?ref= like the desktop does, ?auth_token=
cleanup preserves the path and tab hash, the logged-out session list
gets the wallpaper back, ?download= also works at /desktop, and the GUI
dev server serves /desktop and /dashboard.

* Add Open Desktop bento card and CSS

Replace the previous desktop-switch UI with a full-width "Open Desktop" bento card that links to /desktop. Removes the pinned sidebar Open Desktop button and the old desktop-switch button/handler, and adds new CSS variables and styles for the desktop card (icon background, shadow, surface/header colors), hover shadow and arrow transition, and layout/padding adjustments.

Changed files: TabHome.js (removed old switch, added anchor card), UIDashboard.js (removed sidebar button), dashboard.css (new desktop color vars and styles; removed legacy desktop-switch/button styles).

* Remove box-shadow from myapps search input

Delete the box-shadow declarations from input.myapps-search::placeholder in dashboard.css. This simplifies the placeholder styling (removes subtle and light shadow layers), likely to reduce visual noise or fix rendering inconsistency.

* Show index_url hostname for opaque app IDs

Only replace an app's title with the hostname from index_url when the app appears to use an opaque id. Add a check for names starting with 'app-' so the hostname is used only if title, name and uuid are identical and the name looks like an opaque app id. This avoids overwriting meaningful user-facing names that may coincidentally match the uuid and clarifies the intent in the comment.

* Update dashboard.css
2026-07-06 13:22:27 -07:00
Neal Shah 3a26f1f1c2 add admin moderation events table (#3348) 2026-07-06 11:49:09 -04:00
Daniel Salazar f1b94a5b69 fix: otel upgrade bump (#3328) 2026-07-06 08:42:25 -07:00
Neal Shah e3fce9191a emit puter.app.authenticated event (#3337) 2026-07-06 11:23:27 -04:00
Nariman Jelveh 2c0a1b9bbb Revert "feat: add sign-in flow for crossOriginIsolated contexts (#3253)" (#3335)
This reverts commit 6fff1fd75f.
2026-06-30 23:32:27 -07:00
jelveh 0d7417f858 Add 'contacts' to recommended apps
Include 'contacts' in the RECOMMENDED_APP_NAMES array so the Contacts app appears in the recommended apps list. Updated src/backend/services/apps/RecommendedAppsService.ts.
2026-06-30 19:08:11 -07:00
Neal Shah 2eee098c66 Add suspension reasons (#3331) 2026-06-30 18:09:00 -04:00
Neal Shah 890d4b3c16 remove temperature from sonnet-5 (#3330) 2026-06-30 17:18:29 -04:00
Neal Shah af9a7c6345 add claude sonnet 5 (#3329) 2026-06-30 17:08:42 -04:00
velzie 6fff1fd75f feat: add sign-in flow for crossOriginIsolated contexts (#3253)
* poc

* route login key through broadcastservice

* cleanup

* security validation and prevent additional webhook sends

* block double-broadcast from redis
2026-06-30 20:17:53 +01:00
velzie e6b7b08bda Add forceRelay option to Peer API and document raw peerconnection property (#3313)
* add forceRelay options

* document peerconnection
2026-06-30 12:28:49 -04:00
Daniel Salazar f4c3cbd53d feat: add logs for failed sms (#3318) 2026-06-30 06:50:47 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI c0c2f35266 Update docs model drift (#3326)
* Update docs model drift

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* standardize

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-30 16:55:42 +07:00
Nikhil Kumar SinghandDevblaze14 ccea980753 feat: add Ideogram 4.0 Together AI image model (#3309)
Co-authored-by: Devblaze14 <[email protected]>
2026-06-29 15:33:01 -04:00
Neal Shah 2bddebd576 add ability to suspend apps (#3322) 2026-06-29 13:06:08 -04:00
Neal Shah b9866ddbeb add suspended_at col (#3321) 2026-06-29 09:51:00 -04:00
Mochammad Fadhlan Al-Ghiffari 6fc79a7184 fix(docs): highlight code blocks after client-side navigation (#3287) (#3288)
* fix(docs): highlight all code blocks after client-side navigation

The docs site highlights code on full page load via an inline
hljs.highlightAll(), which targets the 'pre code' selector and so
highlights every fenced block, including those without a language tag.

The client-side navigation path (pathchange handler) instead selected
only code[class^="language"], so fenced blocks written without a
language hint were left unhighlighted after in-app navigation while
appearing correctly on a hard reload.

Align the pathchange handler with highlightAll() by selecting 'pre code'
and skipping blocks hljs has already processed (data-highlighted=yes),
keeping the pass idempotent across repeated pathchange events.

Fixes #3287

* fix(docs): hoist hljs.configure and scope highlight to docs content

Address review nits on the pathchange highlight pass:
- Move hljs.configure({ ignoreUnescapedHTML: true }) out of the per-element
  loop since it is global config and only needs to run once.
- Scope the selector to '.docs-content pre code' so the re-highlight pass
  only touches the swapped-in docs body and leaves code blocks elsewhere
  on the page untouched.
2026-06-28 23:27:07 +07:00
Nariman Jelveh 50039571b6 Reorder recommended apps and add vault
Adjust RECOMMENDED_APP_NAMES ordering: place 'editor', 'camera', and 'recorder' immediately after 'builder' and move 'app-center' accordingly. Also add 'vault' to the recommended apps list. This changes the resolved ordering of recommended apps at call time.
2026-06-27 17:55:52 -07:00
Daniel Salazar 1216f20567 fix: package-lock issue (#3316) 2026-06-27 14:33:11 -07:00
Daniel Salazar 0752ea761c feat: remove stych, add prelude client (#3312) 2026-06-27 14:07:52 -07:00
Daniel Salazar 98461088f8 fix: decrease puts for unused stamps (#3310) 2026-06-26 12:33:04 -07:00
Neal Shah d4a3224aa3 force locale en-US (#3311) 2026-06-26 15:31:58 -04:00
Daniel Salazar 197ce6faff fix: LegacyFSController (#3308) 2026-06-25 14:47:29 -07:00
MRB Labs c98e293879 fix: unlink OIDC providers when user updates email address (#3307) 2026-06-25 14:46:41 -07:00
Miika Kuisma ebbe382596 tweak context menu's text labels (#3306) 2026-06-25 09:46:05 -07:00
Neal Shah 01d28eb2c5 Await phone verification event (#3305) 2026-06-24 20:08:30 -04:00
Daniel Salazar 7d1c2e7f0a fix: fsService copy (#3304) 2026-06-24 16:32:34 -07:00
Daniel Salazar 22b499af20 fix: cleanup fs responses (#3303) 2026-06-24 14:35:58 -07:00
Neal Shah 7208f86723 bind btoa and atob in emulated worker globalspace (#3302)
* bind btoa and atob in emulated worker globalspace

* Create working directory for workers
2026-06-24 15:56:26 -04:00
Neal Shah 2974a8b01c store app owner ID even when worker is deployed from root ctx (#3301) 2026-06-24 15:21:48 -04:00
Daniel Salazar 7f05a707bd feat: improve kv store incr/decr costs (#3300) 2026-06-24 11:57:36 -07:00
Miika Kuismaandjelveh 1d056253ef contextmenu design tweak (#3298)
* contextmenu design tweak

* icon gap fix (mobile context menu web component)

* fix submenu item padding

* add dark theme support for puter.ui.contextMenu and puter.ui.setMenubar (works only with puter.env === 'web')

* Add theme test & refactor context menu items

Extract shared mediaMenuItems to remove duplication and simplify context menu calls. Add a light-theme context menu handler and a "Run theme test" button that executes runContextMenuThemeTest(), which programmatically renders menus to verify the theme attribute and .puter-theme-dark class for dark, light, and default cases (mirrors existing Playwright assertions). Update testContextMenu and testContextMenuDark to use the new item factory and clean up rendered menus after checks. UI text is updated to show pass/fail results.

---------

Co-authored-by: jelveh <[email protected]>
2026-06-24 11:30:16 -07:00
Neal Shah a22dc5facb fix whoami user token check (#3299) 2026-06-24 14:28:02 -04:00
Neal Shah aeafd4ac79 txt2img megaupdate (#3294)
* Update OpenAIImageProvider with txt2img with input source

* Grok image provider mega update

* make input_images universal
2026-06-24 10:24:09 -04:00
Daniel Salazar 14861df6b2 fix: bad boot error (#3295) 2026-06-24 00:33:38 -07:00
Daniel Salazar 34a7595a6d fix: bad empty file error handling (#3293) 2026-06-23 16:13:41 -07:00
Daniel Salazar b2108c0216 fix: update email for supprot (#3292) 2026-06-23 15:43:12 -07:00
Philip Ifeanyi 5a5a449210 feat: enhance playground UI with page title and style adjustments (#3289) 2026-06-23 22:45:18 +07:00
Daniel Salazar 0405de30bd update taskbar apps (#3286) 2026-06-22 14:26:35 -07:00
Nariman Jelveh 40c2799b06 Add 'builder' to recommended apps list
Include 'builder' in the RECOMMENDED_APP_NAMES array in src/backend/services/apps/RecommendedAppsService.ts so the RecommendedAppsService treats the builder app as a recommended application and surfaces it accordingly.
2026-06-22 14:22:27 -07:00
Daniel Salazar 4bcdeefe68 chore: bump up deps (#3285) 2026-06-22 13:58:45 -07:00
Daniel Salazar b6ba6dd90e fix: [PUT-1199] more prelude signals (#3282) 2026-06-22 08:16:59 -07:00
Daniel Salazar 954f8b7b8d fix: puter self host guides + default bug (#3281)
* fix: oss dev center html_encode hardening

* fix: puter self host guides + default bug
2026-06-21 22:32:43 -07:00
Neal Shah 24d700673c Compaction support for OpenAI and Anthropic (#3279)
* Alpha: compaction support for OpenAI and Anthropic

* update lock

* fix billing for anthropic compactions

* Fix max_tokens bug in together provider

* Fix responses compaction
2026-06-21 21:57:56 -04:00
c6d64029e0 feat: meter Gemini thinking tokens and grounding requests (#3178)
* feat: meter Gemini thinking tokens and grounding requests

- Thinking tokens: Extracted from standard completion tokens to ensure they are billed accurately at the correct model-specific rate.
- Grounding requests: Added flat-fee metering for Google Search by tracking grounding_metadata across both streaming and non-streaming responses.
- Pricing updates: Corrected stale rates for Gemini 2.5 Flash output, cached tokens, thinking tokens, and grounding requests.

* fix: correct Gemini 2.x metering rates and harden grounding capture

Pricing corrections (verified against ai.google.dev/gemini-api/docs/pricing):
- gemini-2.5-flash output is $2.50/M, not $1.00/M: restore
  completion_tokens to 250 and bill thinking_tokens at the same output
  rate (250). The previous 100 under-billed output ~60%, and this is the
  provider's default model.
- gemini-2.5-flash cache read is $0.03/M: restore cached_tokens to 3
  (the 7.5 value over-billed).
- Grounding with Google Search is $35 / 1,000 requests for Gemini 2.x
  models and $14 / 1,000 for 3.x. Set grounding_requests to 3_500_000
  for gemini-2.0-flash, gemini-2.5-flash, gemini-2.5-flash-lite and
  gemini-2.5-pro; 3.x models keep 1_400_000.

Streaming robustness:
- In create_chat_stream_handler, don't let a later extra_content chunk
  without grounding_metadata overwrite an earlier one that carried it,
  so grounding requests are still metered.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-20 15:08:38 -07:00
jelveh 40ff866598 Add 'butler' to recommended apps
Include 'butler' in the RECOMMENDED_APP_NAMES array in RecommendedAppsService so it appears in the recommended apps list. Updates src/backend/services/apps/RecommendedAppsService.ts.
2026-06-19 15:50:57 -07:00
Nariman Jelveh 36888e35d3 Add phone verification UI improvements and helpers (#3274)
Revamps the phone verification dialog and adds reusable phone helper modules.

- Heavily refactors UIWindowPhoneVerificationRequired.js to add a searchable inline country picker, national-number live formatting, country autodetect for "+" input, WebOTP auto-fill support, resend cooldown timer, accessibility improvements, proper teardown/cleanup, and last-used country persistence.
- Introduces helpers/src/helpers/country_codes.js (get_country_list, flag emoji) which builds a localized list of dialable countries from libphonenumber-js and Intl.DisplayNames.
- Introduces helpers/src/helpers/phone.js with thin wrappers around libphonenumber-js: format_phone_as_you_type, inspect_phone, phone_example_for, detect_country_from_input.
- Adds English i18n strings for the new UI in src/gui/src/i18n/translations/en.js.

These changes keep submission canonicalization using libphonenumber-js (E.164), improve UX for selecting/formatting numbers, and make the country list and phone utilities reusable across the app.
2026-06-19 10:28:30 -07:00
Reynaldi Chernando 30432b931a Add PUT and DELETE to preflight allowed methods (#3278) 2026-06-19 12:35:29 -04:00
Reynaldi Chernando e2174a14cf Update router cors preflight docs (#3277) 2026-06-19 20:50:41 +07:00
Daniel Salazar d3cb89d8c9 fix: import in extensions (#3276) 2026-06-18 21:30:00 -07:00
Reynaldi Chernando c8625b3396 Release puterjs npm (#3275)
* Release puterjs npm

* add package lock
2026-06-18 18:26:39 -07:00
Reynaldi Chernando b576f01d12 Sync Puter.js types and docs (#3194)
* AI chat types

* ai voice types

* update tts docs and jsdoc

* stt types and docs

* txt2img docs and types

* txt2vid and img2txt

* apps

* signin

* perms

* fs

* kv

* hosting

* workers

* net

* ui

* jsdoc
2026-06-18 09:34:13 -07:00
Vyasdev a87cafee3c Revise bug report and feature request instructions (#3220)
Updated the wording for reporting bugs and feature requests in Japanese.
2026-06-18 09:27:55 -07:00
Reynaldi Chernando 1c51fde163 Add minimax m3 direct integration + cleanup model list (#3263) 2026-06-18 10:12:44 -04:00
Daniel Salazar b05699db77 fix: AGENTS.md cleanup (#3273) 2026-06-17 17:07:20 -07:00
Daniel Salazar 163e48ab27 feat: viz over user verification types (#3271) 2026-06-17 14:21:55 -07:00
Reynaldi Chernando 0b41d992a4 Add glm 5.2 direct integration (#3269) 2026-06-17 09:40:10 -04:00
Miika Kuisma c2ddfd6867 Merge pull request #3270 from HeyPuter/fix-file-dialog
puter.js - fix issue where content of file dialog wasn't visible
2026-06-17 13:08:43 +03:00
Miika Kuisma 81b9dc1f06 fix issue where content of file dialog wasn't visible 2026-06-17 13:07:13 +03:00
Daniel Salazar 5506ddb2c2 feat: fp cleanup and better devex (#3268) 2026-06-16 22:48:54 -07:00
Neal Shah c74469b01a prelude RCS (#3267) 2026-06-15 19:08:30 -04:00
Daniel Salazar b706693f82 wip: hardening (#3266) 2026-06-15 16:07:44 -07:00
Daniel SalazarandNeal Shah cd800148d5 feat: optional additional verification gates (#3262)
* phone number verificiation initial

* finish phone verification

* feat: add card gate for signups

* chore: npm

* fix: modal order and wording

* fix:wording

---------

Co-authored-by: Neal Shah <[email protected]>
2026-06-15 09:16:00 -07:00
Ricardo Sawir 900b0d7aa6 fix: correct puter container healthcheck URL (#3255)
The healthcheck was using 'puter.localhost:4100/test' which fails for
two reasons:
- 'puter.localhost' does not resolve inside the container; the service
  listens on all interfaces at port 4100, so 'localhost' works
- '/test' endpoint does not exist; use '/' which serves the shell

This caused the container to be perpetually marked as unhealthy.

Fixes #3191
2026-06-13 23:17:44 -07:00
Daniel Salazar 7e21c1f888 fix: fs metadata sanitation (#3257) 2026-06-12 22:56:06 -07:00
Daniel Salazar 00d6f9c230 fix: disallow unverified outlook emails (#3256) 2026-06-12 18:52:24 -07:00
jelveh 4c03080efa Update mcp.md 2026-06-12 16:55:37 +02:00
jelveh 432a11d289 Add Cursor MCP docs and icons 2026-06-12 16:37:53 +02:00
jelveh e9e233404b Add icons for Claude, Codex and OpenCode 2026-06-12 13:20:15 +02:00
jelveh 8101d42604 Capitalize OpenCode in MCP docs 2026-06-12 13:16:41 +02:00
jelveh 88cedd43bc Add opencode MCP example to docs 2026-06-12 13:11:24 +02:00
Daniel Salazar d3a2934d9b feat: fingerprinting for signup checks (#3254) 2026-06-12 00:59:21 -07:00
Daniel Salazar 13a36b6199 fix: secure http calls (#3252) 2026-06-11 13:43:58 -07:00
Daniel Salazar a155464d46 fix: more data for puter site serve event (#3251) 2026-06-10 23:27:13 -07:00
Daniel Salazar 40d1c998bb feat: pass args to all events (#3248)
* feat: pass args to all events

* fix: alias app joining

* fix: actor in event
2026-06-10 14:13:04 -07:00
Daniel Salazar b4837d49b6 perf: some perf gains from fable (#3249) 2026-06-10 13:26:35 -07:00
ProgrammerIn-wonderland e23042b42e add preperation for reputation score persistence (#3250) 2026-06-10 16:09:42 -04:00
Daniel Salazar 240a733285 sec: misc fable hardening (#3244)
* sec: misc fable hardening

* more fixes

* more fixes

* fix: cors issue
2026-06-10 11:19:41 -07:00
Reynaldi Chernando bb202be12c Minor clarification local webserver in deployment (#3247) 2026-06-10 23:04:42 +07:00
Reynaldi Chernando 8c4cb601c9 Add cli docs (#3246) 2026-06-10 20:53:53 +07:00
Reynaldi Chernando 64a7c7b7c4 Add mcp docs (#3245) 2026-06-10 20:31:31 +07:00
wwb1942 2dd2887ff7 ci: add docs build workflow (#3235)
* ci: add docs build workflow

Add CI workflow that runs npm ci and npm run build in src/docs

whenever files under src/docs change in a pull request.

Closes #3225

* ci: also trigger docs build when workflow file changes

* ci: address docs build workflow review
2026-06-10 10:53:40 +07:00
ProgrammerIn-wonderland 4bbe8a2789 add protections for PUT-1094 (#3243) 2026-06-09 22:29:57 -04:00
Dilan Melvin T 3e9ceb673b fix: coerce Mistral image_url parts from object to string (#3177)
* fix: coerce Mistral image_url parts from object to string

Mistral's API expects image_url content parts to be a plain string URL,
not the OpenAI-style { url: string } object that process_input_messages
produces. Without this coercion, sending an image to any Mistral model
that supports vision (mistral-small, mistral-medium, mistral-large,
ministral-*) results in a request error from the Mistral SDK.

Add #coerceImageUrls() as a private method on MistralAIProvider that
maps { type: 'image_url', image_url: { url } } -> { type: 'image_url',
image_url: url } for every content part in every message. Messages with
plain string content are left untouched, as are parts whose image_url is
already a string.

Add four unit tests covering: object-to-string coercion, already-flat
strings, plain string message content, and mixed text+image content.

* style: use plain ASCII dashes in comment section divider
2026-06-09 20:12:44 -04:00
Daniel Salazar 9c4d1ef535 fix: misc bugs + new middleware for before/after routes (#3242) 2026-06-09 17:05:56 -07:00
ProgrammerIn-wonderland bf57518262 MCP: assert worker_exec ends with puter.work (#3241) 2026-06-09 17:43:34 -04:00
Reynaldi Chernando d48fb11de6 Add claude fable 5 (#3238) 2026-06-09 14:05:31 -04:00
ProgrammerIn-wonderland 7ed87f760b make max tokens less lenient (#3240) 2026-06-09 14:03:13 -04:00
Reynaldi Chernando ee37be16f0 Add cli deployment docs (#3239) 2026-06-10 00:40:19 +07:00
Reynaldi Chernando a8cfaeefbd Add subdomain and workers github actions deployment (#3226)
* Add subdomain and workers github actions deployment

* minor adjust
2026-06-09 23:35:34 +07:00
Badri467 58cb8ee048 skip github actions backend for docs only changes (#3236) 2026-06-09 23:28:48 +07:00
Daniel Salazar a72fbd1d47 fix: inform acl of full access flag (#3237) 2026-06-09 08:04:36 -07:00
Daniel Salazar f6408db74e fix: access tokens with full accesses allowed through with new middleware gate (#3234) 2026-06-08 18:47:07 -07:00
ProgrammerIn-wonderland df20993d8b set azure openai as the default (#3233) 2026-06-08 20:33:41 -04:00
ProgrammerIn-wonderland 708fe1bc4e azure ai provider (#3232) 2026-06-08 18:31:49 -04:00
Daniel Salazar e782fa832e fix: kv costs (#3231) 2026-06-08 13:32:36 -07:00
ProgrammerIn-wonderland c947c2c989 fix mcp server cloudfront cache bug (#3230) 2026-06-08 15:42:49 -04:00
Daniel Salazar 3de4613b7d Revert "feat (PUT-1091): have gui tokens labeled and be different than reques…" (#3229)
This reverts commit 1335b46404.
2026-06-08 11:15:24 -07:00
Daniel Salazar def7290c48 feat: emit user.deleted event on user deletion (#3228) 2026-06-08 10:30:24 -07:00
Daniel Salazar 1335b46404 feat (PUT-1091): have gui tokens labeled and be different than requestor ones (#3222)
* feat (PUT-1091): have gui tokens labeled and be different than creator ones

* fix: errors with auth token
2026-06-08 09:03:02 -07:00
Reynaldi Chernando 0e15ac98a9 Update cli readme and set version (#3227) 2026-06-08 22:23:32 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI f127b2509d Add docs fixes for deployment and workers pages (#3223)
* Add docs fixes for deployment and workers pages

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* handle router edge case

* explicit info about wildcard requiring name

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-08 17:43:00 +07:00
Sourabh Sharma 15134c99bf Fix mkdir dedupeName for existing directories (#3215) 2026-06-07 12:49:00 -07:00
Daniel Salazar 8262e3193e perf: don't batch get app stats (#3219) 2026-06-06 18:28:59 -07:00
Reynaldi Chernando 98a21c4ac0 More workers docs improvement (#3218)
* More workers docs improvement

* Add mention of workers alongside mention of user scope
2026-06-06 01:27:24 +07:00
ProgrammerIn-wonderland c52eb1b903 Ns/bug fixes 060426 (#3214)
* Harden backend auth

* Puter.js auth hardening
2026-06-05 13:55:23 -04:00
mohit_parmar_007 d6a71e50bb fix: remove unnecessary cache console.log statements (#3216) 2026-06-05 10:01:59 -07:00
Reynaldi Chernando a4b6ee0dd2 [PUT-1090] Improve worker docs (#3217)
* Improve worker docs

* refine
2026-06-05 21:54:56 +07:00
Daniel Salazar bb61103080 feat: add apps api into mcp (#3213) 2026-06-04 17:19:53 -07:00
Daniel Salazar 9bbc76b2dd drop metering alerts to low severity (#3212) 2026-06-04 14:01:44 -07:00
Daniel Salazar 918aba9c80 fix: reduce alarms further (#3211)
* feat; update bug-bounty

* fix: reduce alarms further
2026-06-04 10:43:50 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 3ceba9e3b3 Minor fix deployment docs (#3210)
* Minor fix deployment docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-04 23:48:25 +07:00
Daniel Salazar a480679a2d vis: make metering service alarm only go off on actual abuse (#3208) 2026-06-03 22:45:18 -07:00
Daniel Salazar e99ae40e93 chore: bump puter-js (#3207) 2026-06-03 18:59:07 -07:00
Daniel Salazar 3cd50fbaf3 move the token migration to root (#3206) 2026-06-03 18:54:22 -07:00
ProgrammerIn-wonderland 60dd1453d4 Fix issues (#3205) 2026-06-03 21:19:33 -04:00
ProgrammerIn-wonderland 07781cd681 allow overriding origin from workers.dev on mcp server (#3204) 2026-06-03 17:48:37 -04:00
ProgrammerIn-wonderland 2387d8dcc9 MCP whoami (#3203) 2026-06-03 17:13:13 -04:00
Reynaldi Chernando 723608f4db Add new puter cli (#3199)
* Add new cli

* update package lock

* fix readme and license
2026-06-04 01:51:31 +07:00
Reynaldi Chernando bd1dcde7b0 Add info for puterjs deployments (#3202)
* Add info for puterjs deployments

* update external links

* hide github actions method
2026-06-04 01:25:57 +07:00
ProgrammerIn-wonderland 5b8e41e66d MCP Server (#3197)
* beginnings of mcp

* oauth support

* add workers and better prompt engineering
2026-06-03 12:23:06 -04:00
Nariman Jelveh acc4796b99 Allow Cache-Control and Pragma headers (#3198) 2026-06-03 14:30:33 +02:00
jelveh 106e85d6ca Force revalidation on FileSystem reads 2026-06-03 14:15:22 +02:00
ProgrammerIn-wonderland 0ffc32f19e add whitelist for signed batch uploads (#3196) 2026-06-02 21:30:34 -04:00
Daniel Salazar d897cc5dd0 refactor: move subscriptions to webhook system (#3193)
* fix: dont sign write urls if they don't have write perm

* wip: new subscriptions

* refactor: move subscriptions to
2026-06-02 17:58:56 -07:00
Daniel Salazar 6d2f277ce2 fix: some error handling (#3190) 2026-05-31 18:58:27 -07:00
ProgrammerIn-wonderland 86f80b29c0 tighten RAO policy (#3188) 2026-05-29 20:52:21 -04:00
Daniel SalazarandProgrammerIn-wonderland 39243735e8 feat (put-1049): worker types (#3185)
* feat (put-1049): worket types

* Implement changes to worker types

---------

Co-authored-by: ProgrammerIn-wonderland <[email protected]>
2026-05-29 20:51:57 -04:00
ProgrammerIn-wonderland b3ff9d74b7 add browserjs to taskbar and RecommendedAppService (#3189) 2026-05-29 20:32:53 -04:00
ProgrammerIn-wonderland 7b7604440f Add serverless only filtering for togetherai chat models (#3187) 2026-05-29 14:33:55 -04:00
ProgrammerIn-wonderland 382d2b3c14 add minimax provider (#3171)
* add minimax provider
2026-05-29 13:36:10 -04:00
Anurag Pappula eda5379c17 fix(docs/search): handle search index loading and failure states (#3184)
* fix(docs/search): show loading state and replay query when index isn't ready

Previously performSearch rendered "No results found" while the
MiniSearch index was still loading, which is misleading on slow
networks. Track the last query as pendingQuery and replay it from
fetchSearchIndex once the index is ready; render a distinct
"Failed to load search index" message on fetch error so the
loading message doesn't persist indefinitely.

Fixes #3180.

* fix(docs/search): close stale-replay race and consolidate status messages

Drop pendingQuery synchronously in the input handler when the query
falls below the minimum length, so a fetchSearchIndex that resolves
within the 300ms debounce window after the user clears the field can
no longer replay the cleared query into an empty input.

Move the idle, loading, and failed copy into a single
renderSearchStatus(kind) helper so the four .search-no-results
renderings can only drift in one place.
2026-05-29 16:00:59 +07:00
Ron HernausandDaniel Salazar d2fee51844 Support PostgreSQL database backend (#3167)
* feat(database): add postgres database client

Adds the PostgreSQL database client, native bootstrap migration, SQL preparation helpers, and database config/factory wiring.\n\nRefs #3165.

* feat(database): make backend queries postgres-aware

Updates runtime SQL call sites for database-specific booleans, identifiers, insert-ignore, upserts, JSON extraction, intervals, and Postgres insert ids.\n\nRefs #3165.

* test(database): cover postgres client behavior

Adds unit coverage for SQL preparation, factory selection, write-result mapping, and transaction rollback/commit ordering, plus an env-gated PostgreSQL integration flow.\n\nRefs #3165.

* docs(self-hosting): document postgres database setup

Adds PostgreSQL configuration examples and migration path guidance for self-hosted deployments.\n\nRefs #3165.

* fix: harden postgres oidc tests

* fix(postgres): normalize query results and SQL prep

* fix(user): preserve normalized cache booleans

* test(postgres): run integration coverage with pgmock

* tests: add way to run all tests with postgres though slow

Also adding note that postgres is not in active use so might not work out the box

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-05-28 14:12:17 -07:00
Daniel Salazar 8ac4bad196 fix: readUrl perm check (#3183) 2026-05-28 11:44:16 -07:00
Reynaldi Chernando f140d7221c Add opus 4.8 (#3182)
* Add opus 4.8

* update alias

* handling opus 48 omit temperature
2026-05-28 13:27:46 -04:00
Daniel Salazar 631f0bda1b chore:deploy latest puter.js (#3181) 2026-05-28 09:58:33 -07:00
Reynaldi Chernando 1121314582 Docs search improvement (#3179)
* Init search improvement

* Add fuzzy implementation

* update package lock

* package lock matched with runner 24.16.0
2026-05-28 23:11:34 +07:00
Daniel Salazar dda6b82f0f fix: missing english i18n (#3176) 2026-05-27 16:31:40 -07:00
ProgrammerIn-wonderland 6c757c3fc8 Don't use passed in authorization in workers (#3175) 2026-05-27 18:09:59 -04:00
Daniel Salazar eba194936e tests: more tests for session and auth (#3174) 2026-05-27 12:32:15 -07:00
Daniel Salazar a7bdac16da tests: Add unit tests for XAISpeechToTextDriver (#3173)
closes #3002
2026-05-27 11:49:16 -07:00
Daniel Salazar 716a8d06e2 fix: session store kind column + duplicate error handling (#3172) 2026-05-27 11:44:08 -07:00
Daniel Salazar e95cf44fec fix: small fixes for perf and username checks (#3169) 2026-05-27 03:10:56 -07:00
Daniel Salazar 8f8efcb349 feat (PUT-1025 PUT-1026 and PUT-1017): better handling old token auth required in gui (#3166)
* feat (PUT-1025 PUT-1026 and PUT-1017): better handling old token forcing in gui

* fix: misc issues with token inval project
2026-05-27 02:26:35 -07:00
Daniel Salazar b188942436 feat (PUT-1016 & PUT-1020) (#3164)
* feat (PUT-1016 & PUT-1020)
temp account preservation on forced relogin
hosted asset cookies to v2 token too

* fix: remove llm dashes and ugly comments

* update agents
2026-05-26 23:35:16 -07:00
Daniel Salazar 9b7e9a98c8 fix: session revocation (#3163) 2026-05-26 22:13:12 -07:00
Daniel Salazar d6f7fb6155 chore: package-lock update (#3161) 2026-05-26 20:33:11 -07:00
Daniel Salazar bd91f5e192 feat: worker sessions get their own kind + per-(user, app, worker_name) row (#3160)
* feat: worker sessions get their own kind + per-(user, app, worker_name) row

Schema
------
- mysql_mig_11.sql + sqlite 0054: add idx_sessions_user_worker_active,
  a partial unique index over (user_id, app_uid, meta.worker_name) for
  kind='worker' rows. Active worker sessions are deduped by that triple
  so each named worker gets its own session row and they don't fight
  the existing idx_sessions_user_app_active (which still constrains
  kind='app' only). app_uid is allowed NULL for user-scoped workers
  with no app binding.

SessionStore
------------
- getOrCreateWorker(userId, { appUid, workerName, ... }): mirrors the
  getOrCreateApp pattern — cache lookup, partial-unique re-SELECT on
  insert-ignore, all keyed on the worker triple. expires_at lands at
  WORKER_WINDOW_SECONDS (~99y) so the worker doesn't have to re-mint
  on any cadence.
- #cacheKeyWorker + #allCacheKeysForRow worker branch so revoke /
  update invalidates the worker cache view alongside the by-uuid one.

AuthService
-----------
- createWorkerSessionToken(user, workerName, meta?) now takes the
  workerName explicitly and routes through getOrCreateWorker. Emits
  the same { session, token, gui_token } shape but both JWTs carry
  { worker: true, worker_name }.
- createWorkerAppToken(actor, appUid, workerName) likewise — JWT
  carries the worker_name claim so a verifier can tell two workers
  under the same app apart without a DB round-trip.
- Both methods 400 on empty workerName.

WorkerDriver
------------
- Five auth-mint call sites swapped over: app-bound deploy (3x:
  appId branch, actor.app fallback, hot-reload redeploy), user-bound
  fallback (2x: cold deploy, hot-reload). All pass `workerName` so
  the worker's session row is naturally idempotent across redeploys.

GUI manage-sessions
-------------------
- sessionTitle adds a kind='worker' branch ("name (app)" for
  app-scoped workers, just "name" for user-scoped), pulling worker_name
  from the meta-spread that listSessions already surfaces.
- en.js adds ui_session_kind_worker.

* fix(workers): MySQL JSON_EXTRACT quoting + revoke cache invalidation +
SQLite NULL-distinct in worker index

Three real bugs in the worker session plumbing from the prior commit,
plus a misleading comment. Schema design kept (worker_name lives in
`meta` rather than a dedicated column) per offline review:

1. `#selectWorkerRow` compared `JSON_EXTRACT(meta, '$.worker_name')`
   directly to a bind parameter. MySQL's `JSON_EXTRACT` returns a
   JSON-typed value with embedded quotes (`"name"`, not `name`), so
   the comparison never matched. After the first INSERT, every
   follow-up getOrCreateWorker call missed the existing row in the
   SELECT, hit INSERT-IGNORE, then missed again in the re-SELECT —
   the caller would receive whatever the INSERT-IGNORE returned (a
   no-op row in conflict cases). Wrap with `JSON_UNQUOTE` on MySQL
   via `db.case`; SQLite's `json_extract` already returns the
   unwrapped scalar so it keeps the literal form.

2. mig_11's generated `worker_unique_key` had the same JSON-quoting
   bug. Mirror the fix: `IFNULL(JSON_UNQUOTE(JSON_EXTRACT(...)), '')`
   so the concatenated unique key is a plain string that lines up
   with what `#selectWorkerRow` now binds against.

3. SQLite UNIQUE indexes treat NULL columns as distinct (per the SQL
   standard), so two user-scoped workers (app_uid NULL) with the same
   worker_name would both insert. Wrap the index expression with
   `IFNULL(app_uid, '')` so they correctly conflict — matches the
   MySQL side's `IFNULL` in the generated column.

4. `removeByUuid` / `revokeCascade` SELECTed only the identity
   columns (no `meta`), so `#allCacheKeysForRow`'s worker branch
   couldn't read `meta.worker_name` and the composite
   `sessions:v2:worker:<user>:<app>:<name>` cache key survived
   revocation. Up to CACHE_TTL_SECONDS (15min) afterwards,
   getOrCreateWorker would short-circuit to the cached (revoked) row.
   Add `meta` to both SELECTs; the existing meta-parsing logic in
   `#allCacheKeysForRow` handles the rest.
2026-05-26 20:05:31 -07:00
Daniel SalazarandClaude Opus 4.7 971a1b5071 chore: drop linear/issue ticket references from source comments (#3159)
54 references across 18 files (PUT-1010, PUT-1014, PUT-1019, PUT-1021,
PUT-1022, PUT-1023, PUT-1024 + sub-tags AUTH-2/4/5, SDK-1, PJS-1/2,
GUI-1/2, ROLLOUT-1) removed from inline comments, doc comments,
test describe blocks, and SQL migration headers. The substantive
explanations stay; only the ticket pointers go.

No behavior change. Full backend test suite: 2172 passed / 16 skipped.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:50:43 -07:00
Daniel SalazarandClaude Opus 4.7 ac5eecb7f3 feat (put-1019 put-1021): v2 auth revoke endpoints + silent v1->v2 to… (#3158)
* feat (put-1019 put-1021): v2 auth revoke endpoints + silent v1->v2 token migration

PUT-1019 (AUTH-5): full revoke-endpoint coverage
- /logout: soft-revoke web session + its asset cookies via revokeCascade
  (app sessions and access tokens survive)
- POST /auth/revoke-session: cascade per row kind (web/app/access_token/asset)
- POST /auth/revoke-all-sessions: revoke all web rows for user; optional
  include_apps=true nuclear option; gated by userProtected (cookie-only)
- revokeAccessToken: soft-revoke matching access_token row in addition to
  removing access_token_permissions
- All revokes are UPDATE revoked_at = now(); no DELETE statements remain

PUT-1021 (SDK-1): backend POST /auth/migrate-token
- v1 access_token/app -> mint matching-kind v2 token, idempotent on
  (auth_id, kind, token_uid)
- v1 web/session -> 409 { code: "reauth_required" } (interactive relogin only)
- Same-origin / signed-referer hardening; rate-limited per IP and auth_id
- Gated by auth.allow_v1_tokens; emits puter_token_v2 cookie for app-in-browser

DB migrations: mysql_mig_10, sqlite 0053 (sessions.access_token_uid column)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1019): reject self-revoke; enrich list-sessions response

- handleRevokeSession refuses uuid === req.actor.session.uid (use /logout
  instead). The cookie that authenticated the call should never be the
  target of a self-revoke — the response can't write fresh auth state
  and the client ends up with an ambiguous identity. revoke-all-sessions
  still has the explicit include_current opt-in for the nuclear case.

- AuthService.listSessions now joins the apps table for kind='app' rows
  (returning { uid, name, title, icon } so the manage-sessions UI can
  render the authorizing app without a second round trip), surfaces
  kind / expires_at / label / last_ip / created_via, and filters out
  asset rows (per-cookie children of web rows, revoked transitively via
  cascade — surfacing them as standalone entries would be confusing).

- Sort order: current session first, then most-recently-active. UI
  relies on this to anchor "you are here" at the top.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1019 put-1021): review nits — origin normalization, cookie fallback, types, migration order

B1: createAccessToken.options.expiresIn widened to string | number.
    The impl (#hardExpiryFromExpiresIn) and existing callers/tests use
    jsonwebtoken-style strings ('1h', '30d'); narrowing to number forced
    unsafe casts at every call site. Cast at the single sign() boundary
    where jsonwebtoken's typed template-literal SignOptions clashes
    with the wider runtime contract.

B2: Inline comment on the DELETE in access_token_permissions. The
    AUTH-5 "no DELETE on revoke" rule scoped to the `sessions` table
    (where the cascade graph + audit trail matter). Permissions rows
    are the grant manifest for an active token — once its session is
    soft-revoked they're dead-weight cache entries. A future audit
    requirement would land as a `revoked_at` column on this table,
    not a behavior change in this PR.

B3: handleMigrateToken now sets the puter_token_v2 cookie (with the
    shared sessionCookieFlags + httpOnly) when the migration result
    is kind='app'. The endpoint is already gated on Origin so the
    caller is by definition in a browser; access tokens deliberately
    skip the cookie since they're programmatic.

B4: #isMigrateTokenOriginAllowed normalizes both incoming origin and
    config.origin / allowlist entries (trim + strip trailing slash +
    lowercase) before equality. A misconfigured `config.origin =
    "https://puter.com/"` would otherwise reject every same-origin
    browser call.

B5: Replaced 4x `this.config.cookie_name!` non-null assertions in
    AuthController with `(this.config.cookie_name ?? 'puter_token')`.
    IConfig is `Partial<IConfigOptional>` so cookie_name is undefined
    at runtime in some deployments / test setups; the fallback matches
    the pattern in userProtected / OIDCController / puterSite.

B6: MySQLDatabaseClient sorts migrations numerically by trailing
    integer instead of lexically. Existing files use unpadded names
    (`mysql_mig_<N>.sql`), so plain `.sort()` ran mysql_mig_10 before
    mysql_mig_2 — a future migration that depended on _2..9 running
    first would break. Non-numeric filenames fall through to
    localeCompare for determinism.

B7: Restored the docstring for SessionStore.getOrCreateApp's
    `opts.auth_id` ("Stable per-user identity (survives re-login);
    carried on every v2 JWT so manage-sessions can group by identity")
    — the previous edit truncated it to a fragment.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test + feat: backend test coverage for PUT-1019/1021 review fixes + worker session methods

Tests
-----
- compareMigrationFilenames (new) covering B6 numeric-sort: confirms
  mysql_mig_10.sql lands after mysql_mig_9.sql; non-numeric files sort
  after numbered ones; stable for already-ordered input.
- listSessions (AuthService.test.ts): excludes kind="asset" rows;
  enriches with kind/expires_at/last_ip/created_via; joins kind="app"
  rows with the apps table; sorts current first then by last_activity
  desc.
- handleRevokeSession (AuthController.test.ts): refuses self-revoke
  with 400; still allows revoking a sibling session.
- handleMigrateToken (AuthController.test.ts): rejects missing/
  disallowed Origin; tolerates trailing slash and uppercase Origin
  (B4 normalization); returns 409 reauth_required for v1 web tokens;
  does NOT set the cookie for access-token migration; DOES set the
  puter_token_v2 cookie (httpOnly + sessionCookieFlags) for
  app-under-user migration.
- SessionStore tests updated to import APP_WINDOW_SECONDS /
  WEB_WINDOW_SECONDS rather than hardcoded 30/90 day values — the
  windows just got bumped to 1y and the assertions need to follow
  the constant.

Refactor
--------
- MySQLDatabaseClient exports compareMigrationFilenames so the sort
  logic is unit-testable in isolation.

Worker tokens
-------------
- AuthService.createWorkerSessionToken(user, meta?): mints a new
  kind="web" row tagged meta.worker=true, expires_at =
  WORKER_WINDOW_SECONDS, returns { session, token, gui_token }
  with worker: true on each JWT.
- AuthService.createWorkerAppToken(actor, appUid): mints a new
  kind="app" row tagged meta.worker=true, expires_at =
  WORKER_WINDOW_SECONDS, returns an app-under-user JWT with
  worker: true. Note the existing idx_sessions_user_app_active
  unique index will collide with an existing non-worker app
  session for the same (user, app) — future schema work can
  carve workers out of that uniqueness.

SessionStore.js: WEB/APP_WINDOW_SECONDS now 1y;
WORKER_WINDOW_SECONDS = 99y added for the worker path.

Full backend suite: 2172 passed / 16 skipped / 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:49:36 -07:00
Daniel SalazarandClaude Opus 4.7 3ae076b73e feat (put-1022 put-1023 put-1024): puter.js reauth handling + GUI v2 modal + silent v1 migration (#3157)
* feat (put-1022 put-1023 put-1024): puter.js reauth handling + GUI v2 modal + silent v1 migration

PUT-1022 (PJS-1): puter.js handles 401 reauth_required
- Single apiCall wrapper intercepts 401 { code: "reauth_required" }
- Clears local token, emits puter.auth.reauth_required { reason, auth_id }
- Queues in-flight requests, re-issues after re-auth completes
- web/app: opens puter.com login popup forwarding auth_id
- gui: no-op (handled by GUI-1 modal)
- workers: surfaces as structured exception
- All token writes go through setAuthToken() (future-proofed for PJS-2)

PUT-1023 (GUI-1): web GUI reauth modal + v2 token storage
- Detects 401 reauth_required across http + websocket connect paths
- Soft modal preserves URL/window state; auth_id forwarded into login form
- Storage key migrated: auth_token -> auth_token_v2
- Cookie cleared on logout: puter_token (legacy) cleared, backend writes puter_token_v2
- Cross-tab propagation via localStorage storage events
- WebSocket revocation surfaces as same modal (no silent failure)

PUT-1024 (PJS-2): puter.js storage versioning + silent v1->v2 migration
- New storage key puter.auth.token.v2
- On SDK init: prefer v2 key; else legacy v1 -> silent POST /auth/migrate-token (SDK-1)
- On success: store v2, clear v1. On failure: fall back to PJS-1 reauth flow
- URL-param tokens (?puter.auth.token=, ?auth_token=) also run through silent migration
- Logout clears both keys; setAuthToken writes v2 only

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1023): manage-sessions polish — hide self-revoke, fix modal stack, richer rows

Three issues surfaced during local testing of the manage-sessions UI:

1. The session row representing the caller's own cookie had a Revoke
   button that, if used, left the client in an ambiguous identity state
   (backend now rejects it too — see put-1019 backend fix). The button
   is now omitted entirely when session.current=true; /logout remains
   the right path for ending the active session.

2. The confirm-revoke prompt rendered behind the manage-sessions
   window because both share the dominant z-index pool. UIAlert calls
   now pass parent_uuid (the manage-sessions window's data-element_uuid)
   plus stay_on_top, so the prompt stacks above its parent.

3. Each row only showed the bare uuid. Now renders: title (app
   title for app sessions, label / "Browser session" / "Access token"
   otherwise), app icon when applicable, kind / current badges,
   created / last-active / expires (timeago, with absolute on hover),
   and last_ip. App metadata is joined server-side per the backend
   listSessions change.

Adds en.js strings for the new labels (ui_session_*).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1022): tighten reauth replay + iframe postMessage hygiene

driverCall_ replay (G1): on 401 reauth_required, re-enter driverCall_
instead of calling replayXhrAfterReauth. The generic helper wires the
retried XHR through setupXhrEventHandlers which resolves with the
parsed response and silently skips driverCall_'s streaming detection,
usage-limit / email-confirmation handling, settings.transform, and
resp.result unwrapping — i.e. it would change the driver call API
contract on retry. One-shot via settings._reauthReplayed so a
fresh-token rejection bubbles up instead of looping.

responseType drift (G2): driverCall_ sets xhr.responseType from
settings AFTER initXhr (which captured the stale value into
xhr._puterReq). Mirror the mutation onto _puterReq so any replay path
builds the retry with the live config rather than the snapshot.

targetOrigin lockdown (G3): triggerReauth's parent.postMessage now
targets this.defaultGUIOrigin instead of '*'. The payload carries
reauth metadata + auth_id and is only meaningful to the GUI; '*'
would leak the signal to whatever frame happened to be embedding us.

event.source pinning (G4): the reauth wait listener also matches
event.source against globalThis.parent, not just event.origin —
origin alone admits any same-origin frame on the GUI domain.

event name alignment (G5): the SDK now emits and listens for
'puter.auth.reauth_required' (matches the documented name in the
PR / commit description). The old 'auth.reauth_required' key isn't
yet consumed externally so this is a safe rename.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:49:17 -07:00
Daniel Salazar 5394ccc45c feat: add verification for v2 auth (#3155) 2026-05-26 15:34:54 -07:00
Daniel Salazar 3649109f40 fix: add back AGENTS.md (#3154) 2026-05-25 16:01:35 -07:00
Daniel Salazar 0ff0ea7743 fix: fs event emissions (#3153) 2026-05-25 14:25:27 -07:00
Daniel Salazar 8be115096f feat (put-1012 put-1014): new tokens version signature and logic (#3152) 2026-05-25 14:19:23 -07:00
Daniel Salazar e66fd2373f feat: start adding support for token expiry and invalidation (#3151)
* feat: new expirable + revokable session

* fix: msql migration
2026-05-25 12:30:20 -07:00
jelveh 0e2cce152d Update PuterAlert.js 2026-05-24 17:03:34 -07:00
Daniel Salazar 101e0bc930 fix: search leaking metadata (#3150) 2026-05-24 16:21:33 -07:00
jelveh ea078250bf Strip non-essential media from docs' markdown 2026-05-24 16:15:24 -07:00
jelveh db79f8e484 Add note: serve Puter.js via HTTP server 2026-05-24 15:48:21 -07:00
Daniel Salazar 6422f2d513 fix: associated app ids as input (#3149) 2026-05-23 13:01:03 -07:00
ProgrammerIn-wonderland c60626f772 Add preamble versioning (#3147)
* Add preamble versioning

* update migrations index
2026-05-23 01:15:58 -04:00
ProgrammerIn-wonderland cafdaf7b50 fix bug with signed url directory listings (#3146) 2026-05-22 18:48:41 -04:00
Daniel Salazar bb0f56d758 fix: bring back puter site config logic (#3145) 2026-05-22 11:15:45 -07:00
NighttandReynaldi Chernando 7438db7a90 Add OpenRouter model release dates (#3144)
* Add OpenRouter model release dates

* simplify implementation

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-05-22 12:58:59 -04:00
Daniel Salazar 948f771838 feat: ai limits tightning and add severity alert (#3142) 2026-05-21 20:46:19 -07:00
Nariman Jelveh 82e34f014d Warn on file:// loads using PuterDialog 2026-05-21 19:15:21 -07:00
Nariman Jelveh 9e4c3f8ba3 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-05-21 14:39:44 -07:00
Nariman JelvehandClaude Opus 4.7 e6acd7386a Bump @heyputer/puter.js to 2.4.2
Record the 2.4.2 release in the repo. The version was published to npm
(via prepublishOnly's `npm version patch`) but the bump was never
committed, leaving package.json stuck at 2.4.1 on GitHub.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-21 14:39:18 -07:00
Daniel Salazar b04e8b046a fix: fixing misc issues around alerting and bugs (#3141)
- decrease ai related alerts and make errors not 500
- decreased bad fs controller checks to be 404s instead of 500s
- force bucket regions for now
- validate webdav perms for locking/unlocking
2026-05-21 14:26:47 -07:00
Nariman Jelveh 3262b984b7 Unify Email/Usage dialogs with Puter dialog style 2026-05-21 14:26:26 -07:00
Nariman Jelveh 71af88c73c Bump @heyputer/puter.js to 2.4.1 2026-05-21 14:06:48 -07:00
Nariman Jelveh 2275f65169 Centralize auth popup handling and consent dialog 2026-05-21 12:06:47 -07:00
ProgrammerIn-wonderland 71b7d6367f fix together provider issues (#3140) 2026-05-21 08:58:48 -04:00
Nariman Jelveh 08369d840b Add cssstyle to docs dependencies (#3139) 2026-05-20 17:41:02 -07:00
Daniel Salazar 94236133b5 fix: event types (#3138)
* fix: agents.md

* fix weird names

* fix: weird names again

* fix: event types

* cleanup types
2026-05-20 16:26:16 -07:00
Daniel Salazar 84b48541bd fix: event types + emit subdomain serve event (#3137)
* fix: agents.md

* fix weird names

* fix: weird names again
2026-05-20 12:47:56 -07:00
Daniel Salazar a33d02f522 fix: agents.md name (#3136) 2026-05-20 12:18:35 -07:00
Reynaldi Chernando f94d4aa7d9 Update gemini 3.5 flash release date (#3135) 2026-05-20 13:25:46 -04:00
Daniel Salazar 89cfb0c634 fix: subdomain htmls (#3134) 2026-05-19 22:04:57 -07:00
ProgrammerIn-wonderland c9096e9eed more validation for signed URLs (#3133) 2026-05-19 19:04:42 -04:00
Daniel Salazar 859ca207c5 feat: new gemini 3.5 flash model (#3131) 2026-05-19 15:16:07 -07:00
Daniel Salazar 58fc122f0d fix: app creation dupped names limit (#3130)
* fix: app creation dupped names limit

* fix: app dupped names
2026-05-19 12:18:59 -07:00
ProgrammerIn-wonderland 0e990da961 import ESM by URL instead of path (#3129) 2026-05-18 19:31:03 -04:00
db8236cc97 update Indonesian section (Indonesian language) (#3117)
* Update id.js

* Update id.js

* Update id.js

* Update id.js

* Update README.id.md

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* refine indonesia i18n

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Co-authored-by: Reynaldi Chernando <[email protected]>
2026-05-18 17:15:01 +07:00
jelveh d731ccf873 Bump puter.js version to 2.3.4 2026-05-17 23:20:41 -07:00
Daniel Salazar 6493d66901 fix: misc write issues with fs + openrouter refresh (#3125)
* feat: refresh openrouter models every 15 min

* fix: if no extensions, skip build step

* fix: misc write issues with fs
2026-05-17 12:34:08 -07:00
Daniel Salazar 9e7f38f473 fix: validate private apps token too (#3124) 2026-05-17 12:28:32 -07:00
Daniel Salazar a568ca0f99 fix: expose upstream ai errors as not 500s (#3123) 2026-05-17 12:01:22 -07:00
Nariman Jelveh 41e7228a9f Update PuterDialog.js 2026-05-16 17:01:40 -07:00
Nariman Jelveh 4568639718 Adjust PuterDialog footer and button spacing 2026-05-16 16:57:35 -07:00
Nariman Jelveh 73ecff0bd7 Tweak PuterDialog spacing and button widths 2026-05-16 16:50:31 -07:00
Nariman Jelveh f2f023880f Remove mobile column-reverse for dialog buttons 2026-05-16 16:42:27 -07:00
Nariman Jelveh f2093e9204 PuterDialog: UI/style tweaks and button reorder 2026-05-16 16:36:53 -07:00
Nariman Jelveh 23497dd371 Add dark mode for puter.js login prompt 2026-05-16 16:04:29 -07:00
Nariman Jelveh 37619fb2a3 Remove embedded background image from PuterDialog 2026-05-16 15:43:15 -07:00
Nariman Jelveh f976686eba Update index.md 2026-05-16 13:12:31 -07:00
ProgrammerIn-wonderland 7af3721543 Make confirming an email unconfirm any other accounts using that email (#3121) 2026-05-16 15:07:38 -04:00
ProgrammerIn-wonderland 93edec0d01 assert normalized (#3119) 2026-05-15 17:01:32 -04:00
Daniel Salazar 54b61f2e24 fix: bad cookie parse test (#3120)
* fix: deepseek alias

* fix: bad cookie parse test
2026-05-15 13:40:43 -07:00
Nariman Jelveh 184ed77493 Use fixed colors for OIDC login divider 2026-05-14 20:26:46 -07:00
Nariman Jelveh 8234b72fc3 Improve login UI: logo, forgot link, OIDC divider 2026-05-14 20:20:21 -07:00
Nariman Jelveh cc9921945e Make auth logo clickable for cover pages 2026-05-14 20:13:33 -07:00
Daniel Salazar ac6f23a41f Update README.md (#3116) 2026-05-14 13:01:34 -07:00
Nariman Jelveh 973d046d33 Remove legacy TabFiles component
Delete the large TabFiles.js file (file browser tab) and update dashboard, styles, helpers, and init logic accordingly. Related changes in UIDashboard.js, dashboard.css, style.css, helpers.js and initgui.js remove or adapt references to the removed component and adjust UI/initialization and styling to account for the file-tab extraction. This reduces duplicate/legacy code and centralizes file browser functionality; review UIDashboard and helper updates to ensure file management features (navigation, upload, context menus, previews) are migrated or re-implemented as needed.
2026-05-14 12:13:48 -07:00
Miika Kuisma dcb50cde8f Merge pull request #3115 from HeyPuter/puter-js-update260514-2
PuterJS - Fix submenu behavior on mobile devices in Menubar web component
2026-05-14 11:45:47 +03:00
Miika Kuisma 08beb55853 PuterJS - Fix submenu behavior on mobile devices in Menubar web component 2026-05-14 11:43:47 +03:00
Miika Kuisma bb0220e163 Merge pull request #3114 from HeyPuter/puter-js-update-260514
Puter JS MenuBar Web Component improvements
2026-05-14 09:14:19 +03:00
Miika Kuisma f3e4c8c6d0 menu item focus tweak 2026-05-14 09:09:40 +03:00
Miika Kuisma 44254671bb - When navigating with keyboard the main level of menubar (file, edit etc) is should also open the menus, now it just highlights the opener
- When hovering main menu items, it must clear hover effects
- Two submenus under each other and diagonal moving
- separator hovering removes highlight but disabled item should do the same
- better way to remove focus
- when mouse leaves context menu highlight should be removed
2026-05-14 08:26:36 +03:00
Daniel Salazar bcc6cc8f00 fix: express proxy chain (#3112) 2026-05-13 18:13:39 -07:00
jelveh 84549a1c31 Add Authorization header to deletion request 2026-05-13 17:21:40 -07:00
Daniel Salazar 0276420213 fix: treat access tokens as non-auth unless opted in (#3111)
* fix: treat access tokens as non-auth unless opted in

* fix: tests
2026-05-13 14:20:15 -07:00
ProgrammerIn-wonderland 6353a08fd5 accept web input in speech2txt (#3110) 2026-05-13 15:53:16 -04:00
Daniel Salazar c265fa308a fix: double rate limits across the board (#3108)
* fix: sqlite auto create path if missing

* fix: double rate limits across the board
2026-05-13 09:30:08 -07:00
Miika Kuisma d3f72bc33a Merge pull request #3105 from HeyPuter/puter-js-menubar-fixes
Puter JS UI Component updates
2026-05-13 11:51:42 +03:00
ProgrammerIn-wonderland 700fc4d27e alibaba provider (#3103)
* Add alibaba provider

* add tests
2026-05-13 03:21:15 -04:00
Miika Kuisma 9c7f30a150 Add theme attribute for each web component, which can be used for forcing dark or light theme (in case the application doesn't have dark mode support it might be odd if just menubar turns dark) 2026-05-13 10:08:59 +03:00
Miika Kuisma 85e45b30f5 fixes https://github.com/HeyPuter/puter/issues/3102 2026-05-13 09:52:21 +03:00
Miika Kuisma 5483e32eab fixes https://github.com/HeyPuter/puter/issues/3101 2026-05-13 09:37:36 +03:00
Miika Kuisma f63c7d8142 - Fixes https://github.com/HeyPuter/puter/issues/3100
- Also fixes similar issues with navigating top level menubar items and submenus
2026-05-13 09:27:44 +03:00
Miika Kuisma 5a2071f852 fix https://github.com/HeyPuter/puter/issues/3099 2026-05-13 09:01:57 +03:00
Miika Kuisma 238ed12e28 - add dark mode support for UI components
- fix https://github.com/HeyPuter/puter/issues/3098
2026-05-13 08:54:23 +03:00
Daniel Salazar d1afe6c3cf fix: puter site redirect for non paid apps (#3104)
* fix: puter site redirect for non paid apps

* add tests
2026-05-12 21:03:33 -07:00
Nariman Jelveh e58dd48e80 Fix link formatting in BUG-BOUNTY.md (#3097) 2026-05-12 17:19:10 -07:00
Daniel Salazar 69f87d4dcd fix: return code (#3096)
* fix: return code

* fix: http error
2026-05-12 16:54:00 -07:00
Daniel Salazar 7b0dd11f7f fix: dev accounts not showing (#3095) 2026-05-12 16:39:42 -07:00
Daniel Salazar 620934fbf4 fix: add user_agent to email signup (#3094) 2026-05-12 15:51:30 -07:00
Daniel Salazar 182fc723f1 expose hasDevAccountAccess at top level (#3093) 2026-05-12 13:56:00 -07:00
Daniel Salazar 5d3c5b6873 fix: whoami (#3092) 2026-05-12 13:44:02 -07:00
slashdevcorpseandCodex f20445bab7 Restrict cross-origin session-cookie auth (#3088)
* Restrict cross-origin cookie auth

Block the auth probe from using ambient session cookies when a browser Origin does not match the request Host. Explicit bearer, body, and x-api-key tokens continue to work for cross-origin SDK calls.

Co-authored-by: Codex <[email protected]>

* Normalize origin checks for cookie auth

Compare normalized origins for browser cookie authentication so default ports and protocol mismatches are handled consistently. Add coverage for default-port and protocol-mismatch cases.

Co-authored-by: Codex <[email protected]>

---------

Co-authored-by: Codex <[email protected]>
2026-05-12 13:40:25 -07:00
Nariman Jelveh 8ad1a58f98 Reorder OIDC Apple and Microsoft buttons 2026-05-12 13:35:18 -07:00
Daniel Salazar 17e475236a fix: mistral ai tests from renames (#3091) 2026-05-12 12:28:32 -07:00
Daniel Salazar b948eaaaa6 chore: update mistralAi providers (#3090) 2026-05-12 12:12:24 -07:00
Nariman Jelveh 9b0c034247 fix: correct xAI TTS pricing to $15.00/1M characters (#3089)
xAI's Voice pricing table lists Text to Speech at $15.00 / 1M characters,
but we were charging $4.20 / 1M characters (420 microcents/char), undercharging
by ~72%. Update the cost constant, listEngines pricing, and test expectation
to 1500 microcents/char.
2026-05-12 11:57:11 -07:00
Miika Kuisma b0dc9c7f1b Merge pull request #3086 from HeyPuter/puter-js-update-260512
PuterJS menubar web component improvements
2026-05-12 12:01:08 +03:00
Miika Kuisma e79abd43e3 PuterJS menubar web component improvements 2026-05-12 12:00:15 +03:00
Miika Kuisma 36c9c478ec Merge pull request #3073 from HeyPuter/puter-js-tests
Puter JS Web Component update
2026-05-12 09:56:58 +03:00
Daniel Salazar 90d01b87ea fix: potential fix for some linux distros (#3085) 2026-05-11 23:28:59 -07:00
Daniel Salazar b0823449bc fix: don't allow dav access control credentials (#3084) 2026-05-11 19:11:02 -07:00
ProgrammerIn-wonderland 760d5d346d add missing GPT 5.4 models (#3083) 2026-05-11 21:34:23 -04:00
Daniel Salazar 4e11d1dd0f fix: updaet deepseek models (#3082) 2026-05-11 18:01:35 -07:00
ProgrammerIn-wonderland 5e8c66142d ai image video fs output (#3079)
* Add puter_output_path

* Fix error reporting in puterjs. Add precheck for puterOutputPath

* add tests

* add tests

* frontend puterjs tests
2026-05-11 20:55:45 -04:00
Daniel Salazar f80016e4e6 feat: rate limit all ai requests (#3081)
* chore: cleanup types for metering

* fix: error responses

* wip: rate limit refactor

* rate-limit for some drivers

* fix: ai driver limits

* tests: rate limiting
2026-05-11 16:18:21 -07:00
Daniel Salazar c8fdbdf8c9 fix: self host mysql (#3080) 2026-05-11 15:36:16 -07:00
ProgrammerIn-wonderland a04267a4e3 Add login with microsoft (#3077) 2026-05-11 15:40:44 -04:00
Daniel Salazar 80a081d62a fix: sql migrations (#3078) 2026-05-11 12:29:56 -07:00
Daniel Salazar bb131c6690 fix: oss icons for new apps (#3076) 2026-05-11 10:52:52 -07:00
Daniel Salazar 4a4a131226 fix: add validation for bad username password input (#3075) 2026-05-11 10:35:12 -07:00
Daniel Salazar 26cff23af4 chore: cleanup types for metering (#3074)
* chore: cleanup types for metering

* fix: error responses
2026-05-11 08:58:13 -07:00
Miika Kuisma 189dae25ef remove hover effect from menubar shortcut labels 2026-05-11 17:58:23 +03:00
Miika Kuisma b2f4a86224 Hide keyboard shortcuts from mobile phone menubar 2026-05-11 11:47:10 +03:00
Miika Kuisma 122811eb7a Puter JS menubar web component now accepts shortcut attribute, improven submenu ux 2026-05-11 11:41:30 +03:00
Miika Kuisma bc98dacf65 Merge branch 'main' into puter-js-tests 2026-05-11 09:55:35 +03:00
Daniel Salazar f00445254d oss: add more apps into oss (#3072) 2026-05-10 22:38:47 -07:00
Daniel Salazar 6686e3a0d7 tests: add tests for SpeechToTextDriver (#3071)
closes #3001
2026-05-10 22:27:35 -07:00
ProgrammerIn-wonderland 5bcb425926 Add tests for Peer, WebDAV, Workers, and WISP. (#3070) 2026-05-10 21:44:30 -04:00
Nariman Jelveh b4e9fa7688 Add dark mode colors to dev console CTA 2026-05-10 17:43:06 -07:00
Daniel Salazar 594cbe03ef tests: big test push with lots of missing areas (#3067)
- Strengthen Broadcast tests
- strengthen oidc tests
- add middleware tests
- add extensions test
- add puter hosting tests
- add Chat Completion Driver tests
2026-05-10 15:06:48 -07:00
Daniel SalazarandClaude Opus 4.7 ec04afe0ad tests: add unit tests for XAITTSProvider (#3060)
Adds offline XAITTSProvider.test.ts covering voice/format selection,
request shape, error paths, and cost reporting. Spies on global fetch
(the provider's egress point) against a real PuterServer + live
MeteringService.

Closes #2998

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:28:04 -07:00
Daniel SalazarandClaude Opus 4.7 539c928920 tests: add unit tests for ElevenLabsTTSProvider (#3066)
Adds offline ElevenLabsTTSProvider.test.ts covering voice/format
selection, request shape, error paths, and cost reporting. Spies on
global fetch (the provider's egress point) against a real PuterServer
+ live MeteringService. The companion integration test stays untouched.

Closes #2999

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:55 -07:00
Daniel SalazarandClaude Opus 4.7 a470857d4e tests: add unit tests for OpenAITTSProvider (#3057)
Adds offline OpenAITTSProvider.test.ts covering voice/format selection,
request shape, error paths, and cost reporting. Mocks the OpenAI SDK at
the module boundary against a real PuterServer + live MeteringService.
The companion integration test stays untouched.

Closes #3000

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:45 -07:00
Daniel SalazarandClaude Opus 4.7 dea40ca8be tests: add unit tests for TogetherVideoProvider (#3058)
Adds offline TogetherVideoProvider.test.ts covering parameter mapping
(togetherai: prefix stripping, seconds default vs no_extra_params,
width/height/fps/steps, reference_images / frame_images filtering),
polling for queued → in_progress → completed jobs (under fake timers),
failure / cancellation / missing-url error paths, and per-video
metering. Mocks together-ai at the module boundary against a real
PuterServer + live MeteringService.

Closes #2994

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:37 -07:00
Daniel SalazarandClaude Opus 4.7 3d593f7cd7 tests: add unit tests for OpenAIVideoProvider (#3059)
Adds offline OpenAIVideoProvider.test.ts covering parameter mapping
(size and seconds snapping to allowed values, input_reference
forwarding), polling for queued/in_progress jobs (under fake timers),
sora-2-pro size tiers (xl/xxl per-second pricing), per-second metering
on default tier, failure handling, and error paths. Mocks the OpenAI
SDK at the module boundary against a real PuterServer + live
MeteringService.

Closes #2993

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:30 -07:00
Daniel SalazarandClaude Opus 4.7 1d36811dd6 tests: add unit tests for GeminiTTSProvider (#3061)
Adds offline GeminiTTSProvider.test.ts covering voice/format selection,
request shape (transcript framing + speechConfig), error paths, and
cost reporting (token-priced input + output:audio batching, including
the PCM-to-WAV wrapping path). Mocks @google/genai at the module
boundary against a real PuterServer + live MeteringService.

Closes #2997

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:21 -07:00
Daniel SalazarandClaude Opus 4.7 4d53faf5b6 tests: add unit tests for GeminiVideoProvider (#3065)
Adds offline GeminiVideoProvider.test.ts covering parameter mapping
(size → aspectRatio/resolution, isHighRes forcing 8s, reference_images
clamping to 3 with image/lastFrame suppression, data-URL parsing,
negative_prompt), polling the long-running operation (under fake
timers), tier-aware metering (default / :1080p / :4k suffixes),
content-filter handling, and error paths. Mocks @google/genai at the
module boundary against a real PuterServer + live MeteringService.

Closes #2992

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:14 -07:00
Daniel SalazarandClaude Opus 4.7 c101fe74ab tests: add unit tests for TTSDriver (#3064)
Adds offline TTSDriver.test.ts covering provider selection/dispatch
(args.provider, legacy driverAlias via Context.driverName, registered
fallback), voice/format param validation, error mapping, and metering
propagation (provider failures must not bill). Mocks each provider's
SDK / fetch boundary against a real PuterServer wired with credentials
for every TTS provider.

Closes #2995

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:08 -07:00
Daniel SalazarandClaude Opus 4.7 b4fad80a8c tests: add unit tests for AWSPollyTTSProvider (#3063)
Adds offline AWSPollyTTSProvider.test.ts covering voice/format
selection (caller-supplied, language-derived, default-per-engine
fallback), SSML handling (TextType=ssml routing), request shape to the
AWS SDK SynthesizeSpeechCommand, engine validation, error paths, and
cost reporting. Mocks @aws-sdk/client-polly at the module boundary
against a real PuterServer + live MeteringService.

Closes #2996

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:27:02 -07:00
Daniel SalazarandClaude Opus 4.7 41e0049045 tests: add unit tests for VideoGenerationDriver (#3062)
Adds offline VideoGenerationDriver.test.ts covering provider
selection/dispatch (args.provider, model-id resolution across the
unified catalog, Context.driverName fallback), parameter validation
(seconds + dimension snapping to model.durationSeconds /
model.dimensions, string coercion), error mapping (provider
HttpErrors pass through, SDK errors don't bill), and metering
propagation (driver-level dispatch reaches the provider's
incrementUsage). Mocks each provider's SDK boundary against a real
PuterServer wired with credentials for every video provider.

Closes #2991

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-10 14:26:56 -07:00
Daniel Salazar 4aaa09bb04 tests: more tests for drivers and perm service (#3056) 2026-05-10 13:46:52 -07:00
Daniel Salazar 3f2c45be26 cleanup: authcontroller with testable methods and tests (#3054)
* cleanup: authcontroller with testable methods and tests

* fix: types

Tested this on zenpacket, seems to all be working
2026-05-10 13:24:17 -07:00
jelveh f7202955ef Move TOS blurb and update copy 2026-05-09 19:08:17 -07:00
jelveh 07432b0967 Update UIWindowSignup.js 2026-05-09 19:00:42 -07:00
jelveh 065eeda72a Add 'Sign up with email' option for OIDC 2026-05-09 18:55:18 -07:00
jelveh 65ce9a67ef Include user-agent in OIDC payload 2026-05-09 15:50:40 -07:00
Nariman Jelveh 4996d7f37f Include user_agent in auth request data (#3053) 2026-05-09 09:25:57 -07:00
Daniel Salazar 86adf4ebc0 fix message at end of selfhost script (#3052) 2026-05-08 23:46:54 -07:00
Daniel Salazar 9a4a0ac9f1 tests: add unit tests for XAIImageProvider (#3045)
Mocks the OpenAI SDK at the module boundary, boots a real PuterServer
for the wired MeteringService, and exercises construction (xAI base
URL, missing-key throw), model catalog (default + verbatim list),
test_mode bypass, prompt validation, the credit gate (402 before SDK),
model resolution (alias and unknown-id fallback), and the success
path covering URL/b64_json branches plus per-image metering with the
expected microcent override.

closes #2988
2026-05-08 23:34:39 -07:00
Daniel Salazar 2543179ae7 tests: add unit tests for OpenAiImageProvider with mocked client (#3050)
Mocks the OpenAI SDK at the module boundary, boots a real PuterServer
for the wired MeteringService, and covers the three model families
the provider treats differently. Exercises construction, model catalog
(dall-e-2 default + verbatim list), test_mode bypass, prompt
validation, the credit gate (402 before SDK), DALL-E request shape
(allowedRatio snap, hd-only-on-DALL-E-3, per-size metering, url and
b64_json output), gpt-image-* token-priced metering (input + output
batched line items at the model rates), and the gpt-image-2 size
normalizer (pixel-budget snap to multiples of 16, 3:1 ratio clamp,
3840 edge cap).

closes #2990
2026-05-08 23:33:52 -07:00
Daniel Salazar 07a049c8ac tests: add unit tests for TogetherImageProvider (#3049)
Mocks the together-ai SDK at the module boundary, boots a real
PuterServer for the wired MeteringService, and exercises construction
(missing-key throw), model catalog (default + verbatim list), test_mode
bypass, prompt validation, the credit gate (402 before SDK), all three
pricing branches (per-MP × ratio, per-image, per-tier with resolution
map rewrite), request shape (togetherai: prefix stripping, dimension
snap to multiples of 8 with min 64, optional knob clamping/forwarding,
input_image → image_base64 aliasing), and output handling for both
url/b64_json branches plus the wrapped error message.

closes #2987
2026-05-08 23:33:46 -07:00
Daniel Salazar 0990fc2667 tests: add unit tests for ReplicateImageGenerationProvider (#3048)
Mocks the replicate SDK and util/secureHttp at the module boundary
(real network egress points), boots a real PuterServer for the wired
MeteringService. Covers construction (missing-key throw), model
catalog (flux-schnell default + verbatim list), test_mode bypass,
prompt validation, the credit gate (402 before SDK), per-image
billing (replicateId routing, aspect_ratio gcd formatting,
single-output metering, string vs array output extraction, missing
URL → 400), megapixel billing (run + output_mp batched components),
the param-aliasing/transform/filter machinery (drops disallowed
keys, renames steps→num_inference_steps + response_format→output_format,
injects go_fast=true default for flux-2-dev, appends ' MP' suffix to
flux-2-pro resolution), and the costs_go_fast pricing branch.

closes #2986
2026-05-08 23:33:39 -07:00
Daniel Salazar 98bb203c84 tests: add unit tests for GeminiImageProvider (#3047)
Mocks @google/genai at the module boundary, boots a real PuterServer
for the wired MeteringService, and covers both the generateContent
(Flash) and generateImages (Imagen) code paths. Exercises
construction (missing-key throw), model catalog (default + verbatim
list), test_mode bypass, prompt validation, the credit gate (402
before SDK), Flash request shape (aspect ratio, allowedRatio fallback,
inlineData → base64 data URL extraction), Flash 3-line metering
(input + output:text + output:image with the IMAGE-modality token
count), Imagen routing (generateImages, per-image metering,
empty-response and rai-filtered failure modes).

closes #2989
2026-05-08 23:33:34 -07:00
Daniel Salazar 0e82a0aed6 tests: add unit tests for ImageGenerationDriver (#3046)
Boots a real PuterServer with API keys for every image provider
configured (so the driver indexes them all), drives
server.drivers.aiImage directly, and mocks each provider SDK at the
module boundary so the driver's routing is exercised without real
network egress. Covers 401 without actor, 400 on unknown model,
catalog dedup + sort, list() and getReportedCosts shape, provider
routing (id-based, lowercased lookups, args.provider override,
Context.driverName fallback), ratio normalization (width/height and
aspect_ratio "w:h" forms), and the ai.log.image audit event firing
both before the upstream call and even when it fails.

closes #2984
2026-05-08 23:33:28 -07:00
Daniel Salazar 3dc9d521b4 tests: add unit tests for CloudflareImageProvider (#3051)
Boots a real PuterServer for the wired MeteringService, stubs global
fetch (the provider talks to the Cloudflare REST API directly with no
SDK). Covers construction, model catalog, test_mode bypass, prompt
validation, the credit gate (402 before fetch), request shape (JSON
body, account-scoped /ai/run/<model> endpoint, apiBaseUrl override,
multipart routing for requiresMultipart models, steps clamping
[1,50]), output extraction (binary image/* → base64 data URL, JSON
envelope image with output_format → MIME, http(s) URL passthrough,
success:false 400, non-2xx 400, missing image 400), and per-scheme
cost components (tile-plus-step and flux2-klein-9b-mp split).

closes #2985
2026-05-08 23:33:21 -07:00
Daniel Salazar 4dbbc7c7c9 tests: add unit tests for GroqAIProvider with mocked client (#3042)
Mocks the groq-sdk at the module boundary, boots a real PuterServer
for the wired MeteringService, and exercises construction, model
catalog/resolution, request shape (max_completion_tokens rename, tools
passthrough, content blanking when paired with tool_calls), non-stream
+ streaming output (including Groq's x_groq.usage envelope deviation),
error mapping, and moderation.

closes #2977
2026-05-08 23:06:23 -07:00
Daniel Salazar 31f22bc339 tests: add unit tests for ClaudeProvider with mocked Anthropic SDK (#3038)
Mocks @anthropic-ai/sdk at the module boundary, boots a real
PuterServer for the wired MeteringService/stores/FSService, and
exercises construction (long timeout for Opus 4.7 thinking), model
catalog/resolution, request shape (tool_choice locking, system message
extraction, OpenAI-shape tool_calls → Claude tool_use blocks, tool
role → user tool_result, opus-4.7 temperature drop, adaptive thinking
config, enabled budget thinking + temperature=1 for older Sonnet),
non-stream output (input/output/cache token metering), streaming via
the message_start/content_block_*/message_delta/message_stop event
graph for both text_delta and input_json_delta tool_use, and the
moderation throw.

closes #2958
2026-05-08 23:06:05 -07:00
Daniel Salazar 87f0294f73 tests: add unit tests for OpenAiChatCompletionsProvider (#3036)
Mocks the OpenAI SDK at the module boundary, boots a real PuterServer
for the wired MeteringService/stores/FSService, and exercises
construction, model catalog (responses_api_only filtering), argument
validation (messages-array gate, web_search routing — error when no
sibling, delegation when one is set), request shape (max_completion_
tokens rename, gpt-5 vs non-gpt-5 reasoning_effort/verbosity gating,
stream_options gating), non-stream + streaming output (cached-token
splitting), the moderations.create flagging branch (above and at the
0.8 threshold), and error mapping.

closes #2959
2026-05-08 23:04:55 -07:00
Daniel Salazar df9c55daf9 tests: add unit tests for OpenAiChatResponsesProvider (#3037)
Mocks the OpenAI SDK at the module boundary, boots a real PuterServer
for the wired MeteringService/stores/FSService, and exercises
construction, the responses_api_only-filtered model catalog (with
no_restrictions toggle), argument validation (messages-array gate),
request shape (input messages, max_output_tokens rename, function-tool
unraveling, Responses-only knobs, gpt-5 vs non-gpt-5 reasoning controls),
model resolution, non-stream output (output_text → message.content,
function_call items → tool_calls, empty-response 400 gate), streaming
(response.output_text.delta text events, response.completed usage,
response.output_item.done function_call → tool_use), moderation, and
error mapping.

closes #2960
2026-05-08 23:01:45 -07:00
Daniel Salazar 9246aff4e4 tests: add tests for VoiceChangerDriver (speech2speech) (#3039)
Boots a real PuterServer with an ElevenLabs apiKey configured, drives
server.drivers.aiSpeech2Speech, and stubs global fetch (the real
network egress for ElevenLabs). Covers getReportedCosts mirroring
costs.ts, argument validation (test_mode bypass, missing actor 401,
missing audio 400), the credit gate (402 before fetch), the success
path (POST URL with voice id, default + override voice/model, optional
knobs forwarded via FormData and search params, audio stream return
shape), per-second metering math, and error mapping when ElevenLabs
returns a non-2xx body.

closes #2963
2026-05-08 23:01:36 -07:00
Daniel Salazar 21c37e6d35 tests: add unit tests for DeepSeekProvider with mocked client (#3040)
Mocks the OpenAI SDK at the module boundary, boots a real PuterServer
for the wired MeteringService, and exercises construction, model
catalog/resolution, request shape (max_tokens, tools, stream_options,
DeepSeek's content-blanking and tool-result system-message workarounds),
non-stream + streaming output paths, error mapping, and moderation.

closes #2976
2026-05-08 23:01:28 -07:00
Daniel Salazar cbdad4a65e tests: add unit tests for TogetherAIProvider with mocked client (#3041)
Mocks the together-ai SDK at the module boundary, boots a real
PuterServer for the wired MeteringService, and clears the kv-cached
model list between tests. Covers construction, model catalog (kv
caching, embedding-type filtering, alias flattening, synthetic
model-fallback-test-1), request shape (togetherai: prefix stripping,
max_tokens omission, tools/stream_options gating), model resolution,
non-stream + streaming output (with input/output cost-key remap),
error mapping, and moderation.

closes #2981
2026-05-08 23:01:20 -07:00
Daniel Salazar 91ecf54746 tests: add unit tests for OpenRouterProvider with mocked client (#3043)
Mocks both the OpenAI SDK and axios at the module boundary, boots a
real PuterServer for the wired MeteringService, and clears the
kv-cached model list between tests. Covers construction (default and
custom apiBaseUrl), model catalog (axios fetch + kv caching, openrouter:
prefix on ids, openrouter/auto filtering), the openrouter/auto disallow
gate, request shape (prefix stripping, usage:{include:true}, stream
options, retry-without-max_tokens on context-length errors, rethrow on
unrelated errors), both cost-calculator branches (cost-bearing vs
per-token fallback) for non-stream and streaming, and moderation.

closes #2980
2026-05-08 23:01:12 -07:00
Daniel Salazar 4bcfe6c7f4 tests: add unit tests for GeminiChatProvider with mocked client (#3044)
Mocks the OpenAI SDK at the module boundary (Gemini speaks the
OpenAI-compat dialect at the v1beta/openai endpoint), boots a real
PuterServer for the wired MeteringService, and exercises construction,
model catalog/resolution, request shape (max_completion_tokens rename,
cache_control stripping, stream_options gating), non-stream + streaming
output (cached-token splitting in the metered usage shape), error
mapping, and moderation.

closes #2961
2026-05-08 23:01:08 -07:00
Daniel SalazarandClaude Opus 4.7 3ada6f1569 tests: XAIProvider uses setupTestServer + real metering:[#2982] (#3035)
Drops the local makeMeteringStub in favour of a live PuterServer-wired
MeteringService. The OpenAI SDK is still mocked at the module boundary
(the real network egress point); the mock now also exposes a default
export so the test server can boot every chat provider (some import
the default and read .OpenAI off it). Aligns with AGENTS.md: "Prefer
test server over mocking deps."

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-08 19:21:52 -07:00
Daniel SalazarandClaude Opus 4.7 64402b01db tests: add tests for OCRDriver: [#2964] (#3028)
* tests: add tests for OCRDriver: [#2964]

Adds offline OCRDriver.test.ts covering both providers:

  • test_mode short-circuit; argument validation (missing actor, missing
    source, unknown provider, AWS/Mistral not configured)
  • aws-textract: raw-bytes vs S3Object source selection (regional
    client when fsEntry has a bucket), block normalisation (PAGE/WORD/
    TABLE filtered, LINE/LAYOUT_TITLE → text/textract:* blocks),
    402 on insufficient credits, per-page metering
  • mistral: image vs PDF chunk packaging (image_url with base64 data
    URL vs document_url with documentName), pass-through of pages /
    annotation / image-limit options, markdown → LINE-block
    normalisation with page indices, per-page metering, additional
    annotations metering when bbox/document annotation formats are set
  • default-provider selection (AWS preferred → Mistral fallback →
    500 when neither is configured)
  • getReportedCosts mirrors costs.ts

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* tests: OCRDriver uses setupTestServer + real fs/metering

Drops the manual config/clients/stores/services stub apparatus and
the loadFileInput mock in favour of the live wired driver from
server.drivers.aiOcr. The Textract and Mistral SDKs are still mocked
at the module boundary (the real network egress points); inputs go
through the real loadFileInput against real fs/store wiring (data
URLs for most cases; FSService.write produces a real fsEntry for the
PDF documentName test). Aligns with AGENTS.md: "Prefer test server
over mocking deps."

The S3Object-source/regional-client assertion was dropped because it
isn't deterministic against the in-memory S3 store and the driver's
per-region TextractClient cache leaks across tests. That branch is
better exercised by a real-cloud integration test.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-08 19:15:57 -07:00
Daniel SalazarandClaude Opus 4.7 6226db73d4 tests: add unit tests for MoonshotProvider with mocked client: [#2979] (#3030)
* tests: add unit tests for MoonshotProvider with mocked client: [#2979]

Adds offline MoonshotProvider.test.ts covering construction, model
resolution and aliasing, request shape (hardcoded max_tokens=1000,
stream_options gating, tool_use → tool_calls hoisting), vision-model
image-inlining hook, non-stream and streaming output, error logging
and rethrow, and metering. Existing integration test is left
untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* tests: MoonshotProvider uses setupTestServer + real metering

Drops the local makeMeteringStub in favour of a live PuterServer-wired
MeteringService. The OpenAI SDK is still mocked at the module boundary
(the real network egress point); inlineHttpImageUrls is mocked because
verifying the provider invokes it for vision-capable models is the
unit's concern. Aligns with AGENTS.md: "Prefer test server over
mocking deps."

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-08 19:15:45 -07:00
Daniel SalazarandClaude Opus 4.7 6ab2a2a6cc tests: add unit tests for MistralAiProvider with mocked client: [#2978] (#3032)
* tests: add unit tests for MistralAiProvider with mocked client: [#2978]

Adds offline MistralAiProvider.test.ts covering construction, model
resolution and aliasing, request shape (camelCase maxTokens, tool_calls
→ toolCalls and tool_call_id → toolCallId rewrite, complete-vs-stream
routing), non-stream output with the camelCase usage coercion,
streaming with the chunk.data unwrap and toolCalls deviation, error
mapping, and metering. Existing integration test is left untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* tests: MistralAiProvider uses setupTestServer + real metering

Drops the local makeMeteringStub in favour of a live PuterServer-wired
MeteringService. The Mistral SDK is still mocked at the module
boundary (the real network egress point); everything else runs through
the real wired graph. Aligns with AGENTS.md: "Prefer test server over
mocking deps."

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-08 19:15:34 -07:00
Daniel SalazarandClaude Opus 4.7 ba03fd7965 tests: add unit tests for PuterAIController: [#2965] (#3031)
* tests: add unit tests for PuterAIController: [#2965]

Adds offline PuterAIController.test.ts with stubbed drivers covering:
route registration (paths and per-route auth options), app-actor
gating across all four chat-proxy routes, body validation (messages
must be an array; openaiCompletions prompt must be a string;
openaiResponses provider must be openai-responses), driver delegation
and request shaping (model + messages + provider defaults), response
shaping for OpenAI chat-completion / text-completion / Responses
envelopes and Anthropic message envelope (text + tool_use blocks),
SSE streaming with [DONE] terminator, model-listing endpoints with
hidden-id filtering and 501 when list() is unavailable, and the
HMAC-gated video proxy's input/expiry/secret guards.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* tests: PuterAIController uses setupTestServer + spy on aiChat.complete

Drops the manual driver/controller stub apparatus (makeStubDriver,
makeController, custom dep builders) in favour of the live wired
controller from server.controllers.puterAi. Each test that needs a
canned chat completion result spies on server.drivers.aiChat.complete,
which keeps the unit-of-test focused on the controller (validation,
response shape, SSE streaming) without dragging in chat-driver model
resolution and credit checks. Aligns with AGENTS.md: "Prefer test
server over mocking deps."

The "500 when URL signature secret missing" assertion was dropped —
the default config wires a secret, so the branch is unreachable in
the live test server. Replaced with the equivalent 403-on-invalid-
signature assertion which actually exercises the HMAC gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-08 19:15:23 -07:00
Daniel SalazarandClaude Opus 4.7 1c188d6325 tests: add unit tests for ZAIProvider with mocked client: [#2983] (#3029)
* tests: add unit tests for ZAIProvider with mocked client: [#2983]

Adds offline ZAIProvider.test.ts covering construction, model
resolution and aliasing, GLM-specific request shape (custom params,
cache_control stripping, user_id derivation, stream_options gating),
non-stream output with reasoning_content normalisation, streaming
deltas (text and tool_use), error mapping, and metering. Existing
integration test is left untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* tests: ZAIProvider uses setupTestServer + real metering

Drops the local makeMeteringStub in favour of a live PuterServer-wired
MeteringService. The OpenAI SDK is still mocked at the module boundary
(the real network egress point); everything else — config, Context,
metering recording — runs through the real wired graph. Aligns with
AGENTS.md: "Prefer test server over mocking deps."

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-08 19:14:15 -07:00
ProgrammerIn-wonderland 8b1226aa1b OIDC referrer (#3034)
* oidc referrer

* optionally infer referrer from window.openerOrigin

* infer referrer from openerOrigin if there

* add log in signupCb

* remove excess logs

* remove FSEntry.js
2026-05-08 22:05:09 -04:00
Daniel Salazar 2c1d320bba ci: push to dockerhub too (#3033) 2026-05-08 17:36:21 -07:00
Daniel Salazar b6414442dd fix: contributing (#3027) 2026-05-08 16:58:05 -07:00
Daniel Salazar f5470f123f tests: add tests for xaiProvider: [#2982] (#3026) 2026-05-08 16:09:25 -07:00
Nariman Jelveh ea9a44c9ad Implement inline 2FA UI, remove old components (#3025)
* Implement inline 2FA UI, remove old components

* Re-enable login button on window close
2026-05-08 15:59:40 -07:00
jelveh 881ee90e76 Remove stray fullscreen hint from translations 2026-05-08 15:16:47 -07:00
jelveh fbb570fb36 Reapply "Remove toolbar clock feature (#3024)"
This reverts commit e781b60bcb.
2026-05-08 15:15:03 -07:00
jelveh e781b60bcb Revert "Remove toolbar clock feature (#3024)"
This reverts commit 78a071ae54.
2026-05-08 15:11:55 -07:00
Nariman Jelveh 78a071ae54 Remove toolbar clock feature (#3024) 2026-05-08 15:08:15 -07:00
Daniel Salazar 9bcd77c85b chore: add legacy codes back to all errors (#3022) 2026-05-08 13:51:55 -07:00
Daniel Salazar 960bbb6965 doc: code architecture doc + contributing guide for backend (#2953)
* architecture doc

* agents and contributor md guides

* fix: userStore
2026-05-08 13:51:41 -07:00
Nariman Jelveh 46a56e474b README edit (#3012)
* Add 'For Users' and 'For Developers' sections

* Update README.md

* Update README.md

* Update README.md
2026-05-08 13:30:33 -07:00
Reynaldi Chernando 666b60bd29 Use GA version of gemini 3.1 flash lite (#3013)
* Use GA version of gemini 3.1 flash lite

* add model name alias
2026-05-08 13:51:56 -04:00
Daniel Salazar 63f128457e remove driver usage endpoint (#3017) 2026-05-08 10:18:26 -07:00
Daniel Salazar 5e8e9a852d test: add unit tests for ai-chat utils (#3016)
Covers `Messages` (normalize_single_message / normalize_messages /
extract_and_remove_system_messages / extract_text), `FunctionCalling`
(normalize_json_schema / normalize_tools_object /
make_openai_tools / make_claude_tools), and `Streaming`
(AIChatStream / AIChatMessageStream / AIChatTextStream /
AIChatToolUseStream). All three are pure data transforms — Streaming
is exercised against a real `AIChatStream` wired to a buffering
Writable, so no method-level spies are needed.

Closes #2956
2026-05-08 10:17:46 -07:00
Daniel Salazar 3166ad933c test: add unit tests for OpenAIUtil shape normalization (#3015)
Covers the Puter↔OpenAI shape conversion (`process_input_messages`,
`process_input_messages_responses_api`), usage extraction
(`extractMeteredUsage`, `create_usage_calculator`), the streaming
handlers (`create_chat_stream_handler` and the responses-API variant),
and the non-stream `handle_completion_output[_responses_api]` paths
including moderation, missing-content, and per-provider deviations.
Streaming is exercised via a real `AIChatStream` wired to a buffering
Writable so no method-level spies are needed.

Closes #2955
2026-05-08 10:17:42 -07:00
Reynaldi Chernando 7e5f64e04b Fix minor typo for tls cert in self hosting docs (#3014) 2026-05-08 09:23:06 -07:00
jelveh 908c02fd99 fix ASCII art for PUTER.JS 2026-05-08 08:04:21 -07:00
jelveh 38fcc6d6bd Detect fullpage apps early and skip desktop load 2026-05-07 22:17:27 -07:00
Daniel Salazar 16652347c6 test: add tests for HomepageController (#3010)
Covers the early-return guard when the homepage service is unwired,
shell rendering at /, /settings, /dashboard, /action, /@:username,
canonical_url + meta inheritance from gui_params, /app/:name (404 +
shell, embedded app metadata, JSON / object metadata, malformed JSON),
and /show/* launching the explorer with the trimmed path.

Closes #2973
2026-05-07 21:41:16 -07:00
Daniel Salazar 5f28b988ec test: add tests for SystemController (#3009)
Covers /healthcheck (live status, 503 on failure, fallback when service
unavailable), /version (semver parsing, env defaults), /contactUs
(validation, persistence, email best-effort, default support address),
/whoarewe, /lsmod (interface grouping + driverName fallback), and the
prepareShutdown drain-flag flip.

Closes #2971
2026-05-07 21:40:27 -07:00
Daniel Salazar 160631b158 test: add tests for OIDCController (#3008)
Covers the provider listing route, the auth-start redirect (404 / 400 /
500 / 302), all three callback flows (login / signup / revalidate)
including state validation, suspended accounts, unconfirmed-email
linking refusal, redirect_uri origin clamping, and the
revalidate-done landing page. Routes are collected via PuterRouter and
invoked with stubbed services that return prefab data.

Closes #2966
2026-05-07 21:38:47 -07:00
Daniel Salazar 3ab9923f0c fix: ai provider integration tests (#3011)
* fix: ai provider integration tests

* change name for ai test
2026-05-07 21:38:40 -07:00
Daniel Salazar cc799e069f test: add unit tests for fileInput driver utility (#3007)
Covers `loadFileInput` (validation, data-URL decoding, FS-path resolution,
maxBytes enforcement, ACL gating) and `inferFilenameFromUrlOrPath`. Stubs
stores/fsService with prefab data so tests don't require a real server.

Closes #2957
2026-05-07 21:38:27 -07:00
Daniel Salazar 9a7d3bac5f test: add tests for DesktopController (#3006)
Covers /set-desktop-bg (partial updates, null clearing, type validation),
/update-taskbar-items (JSON serialization), /set_layout (layout
allowlist, ownership gate, path/uuid resolution), and /set_sort_by
(sort_by/sort_order allowlists, default ordering). Routes are
collected via PuterRouter and driven with stub stores returning
prefab fsentry rows.

Closes #2970
2026-05-07 21:37:34 -07:00
Daniel Salazar 0a9a4f629d test: add tests for StaticPagesController (#3005)
Covers /robots.txt (bot disallow + sitemap), /sitemap.xml (docs +
approved-listing apps), /unsubscribe (missing/invalid uuid, idempotent
re-runs), and /confirm-email-by-token (token mismatch, already-confirmed
short-circuit, duplicate-email rejection, success path with verified-
group promotion + best-effort side channels). Stub stores hold prefab
user rows; the DB stub returns canned EXISTS / SELECT results.

Closes #2975
2026-05-07 21:36:50 -07:00
Daniel Salazar 32c2c0c1a5 test: add tests for AppController (#3004)
Covers /apps, /apps/nameAvailable, /rao, /apps/:name (single + batch),
/query/app (max-entries cap, marketplace shape, hidden-app gating), and
/app-icon (default fallback, data-URL decoding, MIME allowlist, CDN
redirect). Routes are collected via PuterRouter; AppDriver is injected
through the shared `driversContainers` registry, and stub stores return
prefab app rows.

Closes #2969
2026-05-07 21:35:49 -07:00
Daniel Salazar 508d652194 test: add tests for StaticAssetsController (#3003)
Covers each config-gated branch (client_libs_root, puterjs_root,
gui_assets_root, builtin_apps) using real temp directories so the
existsSync checks see real files instead of mocked module behavior.
Asserts both registration outcomes (which routes appear, on which
subdomains) and the file resolution rules for the puter.js / puter.dev.js
fallback.

Closes #2974
2026-05-07 21:35:17 -07:00
jelveh 471ba94ba2 Update index.js 2026-05-07 20:04:43 -07:00
jelveh 66e24a07d4 Update console banner and messaging 2026-05-07 19:49:01 -07:00
jelveh 5ffac54c86 experimental cute CTA 2026-05-07 19:39:06 -07:00
ProgrammerIn-wonderland c5b12b61d9 Unconditionally send Access-Control-Allow-Private-Network, fix LNA fix in GUI (#2954) 2026-05-07 16:38:04 -07:00
Daniel Salazar 8d5e9ef9cf fix: types for open ai driver (#2950)
* fix: types for open ai driver

* fix: add extra time to tests

* fix: test

* fix: together ai tests

* fix: tests
2026-05-07 14:48:47 -07:00
Daniel Salazar 25fe9a3cdf fix: add coverage reports for changes (#2949)
* tests: ai integraton tests

* fix: add coverage reports for this too
2026-05-07 13:55:09 -07:00
Daniel Salazar 05425d20a7 chore: cleanup types for tests (#2948) 2026-05-07 13:40:58 -07:00
Nariman Jelveh ff10e47d67 Show fullscreen overlay when signup is blocked (#2947) 2026-05-07 11:37:23 -07:00
Daniel Salazar 6c19dfbafd fix: linux build (#2946) 2026-05-07 11:23:10 -07:00
Nariman Jelveh 360cbf48af feat: support code in puter.signup.validate error responses (#2945)
Extensions listening on the `puter.signup.validate` event can now set a
`code` field (in addition to `message`) when blocking signups. The code
is forwarded through HttpError and appears in the JSON error response,
giving clients a stable, machine-readable signal to act on.

Covers both the AuthController (password signup) and OIDCService paths.
Adds tests for the new field.
2026-05-07 10:11:08 -07:00
Daniel Salazar bbe93defe9 tests: add some more driver tests (#2944) 2026-05-07 10:04:39 -07:00
Reynaldi Chernando 5e01c13b31 Fix duplicate textract result (#2943) 2026-05-07 09:15:28 -07:00
Daniel Salazar 4b37f35803 test: tests for fs endpoints (#2942) 2026-05-06 22:55:53 -07:00
Daniel Salazar 775c9c49e2 test: add harness and some examples (#2941)
* test: add harness and some examples

* feat: tests ran on pr
2026-05-06 22:43:22 -07:00
Nariman Jelveh e35260924e Scope .myapps-search to input and tweak UI styles 2026-05-06 17:59:54 -07:00
Nariman Jelveh 90aad8918c Support light/dark sidebar logo 2026-05-06 17:54:37 -07:00
Nariman Jelveh 46974beba1 Replace gradients with flat colors in system buttons 2026-05-06 17:47:45 -07:00
Shruc 5e9dc33be1 expose costs in ai chat response (#2921)
* expose costs

* microcents
2026-05-06 15:59:16 -07:00
Daniel Salazar 1e271f188e fix: bring back old app names functionality and fix build (#2937) 2026-05-06 15:48:25 -07:00
jelveh f333597754 Update IPC.js 2026-05-06 15:40:38 -07:00
jelveh 3b33d16356 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-05-06 15:33:21 -07:00
jelveh 2289d61d3c Update UIPrompt.js 2026-05-06 15:33:13 -07:00
Daniel Salazar 8e02f38bad fix: templates (#2936) 2026-05-06 15:31:41 -07:00
angelinazzz1809 5b65a6bc78 Update UIPrompt.js (#2930)
In the app environment, the prompt input was initialized using `options.value`, so `defaultValue` was not reflected.

This change ensures that `defaultValue` is used when provided, while keeping backward compatibility with `options.value`.
2026-05-06 15:27:28 -07:00
ProgrammerIn-wonderland 872d2c01d4 Add support for log in with Apple (#2935)
* Add experimental login with Apple support

* Fix revalidate URL

* Fix OIDC provider revalidate in whoami as well
2026-05-06 16:47:24 -04:00
jelveh b7f199ddbc Normalize string options to object 2026-05-06 11:59:28 -07:00
jelveh 906c00af97 Support options.defaultColor in color picker 2026-05-06 11:48:47 -07:00
Shruc f00d52f879 per model allowed params on replicate image, normalize ratio (#2898)
* replicate allowed params, normalize ratio

* update docs
2026-05-06 14:16:30 -04:00
Devendra Reddy Pennabadianddevareddy05 7a1a885e89 fix: inline http(s) image URLs for Moonshot vision models (#2929) (#2931)
Co-authored-by: devareddy05 <[email protected]>
2026-05-06 13:53:52 -04:00
Daniel Salazar ff8c275317 chore: package-lock.json (#2934) 2026-05-06 10:52:22 -07:00
Devendra Reddy Pennabadianddevareddy05 b3bf979cac fix: support defaultValue in showColorPicker app env (#2895) (#2932)
Co-authored-by: devareddy05 <[email protected]>
2026-05-06 10:46:32 -07:00
Reynaldi Chernando eb9e316ed5 Update Z.AI vision model id to z-ai/z-ai for consistency (#2928) 2026-05-06 02:41:04 -04:00
Daniel Salazar 224ea5f7f8 fix: extensions (#2926) 2026-05-05 20:55:53 -07:00
Daniel Salazar c1dacab4c2 extension events after boot fixes (#2925) 2026-05-05 17:42:55 -07:00
Daniel Salazar 8d0b8d4905 fix: app errors (#2924) 2026-05-05 17:20:31 -07:00
jelveh e7190f442e Update README.md 2026-05-05 16:49:01 -07:00
Miika Kuisma b593eb5630 Tests update 2026-04-29 16:45:09 +03:00
2161 changed files with 537115 additions and 55411 deletions

No files matched your search

+7 -2
View File
@@ -2,9 +2,9 @@
# docker-compose.full.yml up -d`. None of the defaults below are safe for
# anything beyond a local laptop test.
# ── Public-facing ports (nginx) ---------------------------------------
# ── Public-facing ports (Caddy) ---------------------------------------
HTTP_PORT=80
# HTTPS_PORT=443 # uncomment after you enable TLS in nginx/nginx.conf
# HTTPS_PORT=443 # uncomment after you enable TLS in caddy/Caddyfile
# ── MariaDB ------------------------------------------------------------
MARIADB_ROOT_PASSWORD=replace-with-strong-password
@@ -16,3 +16,8 @@ MARIADB_PASSWORD=replace-with-strong-password
S3_ACCESS_KEY=puter
S3_SECRET_KEY=replace-with-strong-secret
S3_BUCKET=puter-local
# ── Inbound SMTP (compose profile `smtp`) ------------------------------
# Host port mapped to the receiver's 2525. 25 is what an MX record means;
# many hosting providers block it.
SMTP_PORT=25
+72
View File
@@ -0,0 +1,72 @@
# Default owner, including backend and SDK areas without a specialist below.
# The last matching rule takes precedence.
- @Salazareo
# GUI
/src/gui/ @jelveh @Salazareo
# Documentation, developer site, and CLI
/doc/ @reynaldichernando @Salazareo
/src/docs/ @reynaldichernando @Salazareo
/src/dev-center/ @reynaldichernando @Salazareo
/src/cli/ @reynaldichernando @Salazareo
# Backend AI
/src/backend/controllers/puterai/ @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/ai-\*/ @404oops @ProgrammerIn-wonderland @Salazareo
# Backend: AI, hosting, workers, networking, and WebDAV
/src/backend/controllers/hosting/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/webdav/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/wisp/ @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/util/ @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/workers/ @ProgrammerIn-wonderland @Salazareo
/src/backend/services/localworker/ @ProgrammerIn-wonderland @Salazareo
/src/backend/stores/subdomain/ @ProgrammerIn-wonderland @Salazareo
/extensions/workerSandbox\* @ProgrammerIn-wonderland @Salazareo
/src/worker/ @ProgrammerIn-wonderland @Salazareo
/src/worker-types/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/peer/ @ProgrammerIn-wonderland @Salazareo
# Backend: sharing, permissions, and teams
/src/backend/controllers/share/ @jfcastro92 @Salazareo
/src/backend/controllers/team/ @jfcastro92 @Salazareo
/src/backend/services/share/ @jfcastro92 @Salazareo
/src/backend/services/permission/ @jfcastro92 @Salazareo
/src/backend/services/acl/ @jfcastro92 @Salazareo
/src/backend/services/team/ @jfcastro92 @Salazareo
/src/backend/services/apps/AppPermissionService\* @jfcastro92 @Salazareo
/src/backend/stores/share/ @jfcastro92 @Salazareo
/src/backend/stores/permission/ @jfcastro92 @Salazareo
/src/backend/stores/group/ @jfcastro92 @Salazareo
/src/backend/stores/team/ @jfcastro92 @Salazareo
# SDK: AI, hosting, workers, and networking
/src/puter-js/src/modules/ai/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/hosting/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Workers* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/networking/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/lib/networkUtils* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/ai.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/hosting.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/workers.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/net.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Peer\* @ProgrammerIn-wonderland @Salazareo
# SDK: sharing and permissions
/src/puter-js/src/modules/perms/ @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/share* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/unshare* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/getShares* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/listShared* @jfcastro92 @Salazareo
/src/puter-js/tests/api/suites/perms.suite.ts @jfcastro92 @Salazareo
/src/puter-js/tests/api/suites/sharing.suite.ts @jfcastro92 @Salazareo
+20
View File
@@ -0,0 +1,20 @@
version: 2
updates:
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
@@ -0,0 +1,49 @@
name: AI Provider Integration Tests
# Hits real provider APIs with cheap models, so this workflow is
# manually triggered only. Each provider's test reads its credential
# from PUTER_TEST_AI_<PROVIDER>_API_KEY and skips itself silently if
# the secret is missing.
on:
workflow_dispatch:
permissions:
contents: read
jobs:
ai-test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run AI provider integration tests
# Vitest treats positional args as filename substring filters.
# The pattern matches every `*.integration.test.ts` file and
# nothing else — the regular backend test suite stays out.
run: npm run test:backend -- integration.test
env:
CI: 'true'
PUTER_TEST_AI_CLAUDE_API_KEY: ${{ secrets.PUTER_TEST_AI_CLAUDE_API_KEY }}
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
PUTER_TEST_AI_GEMINI_API_KEY: ${{ secrets.PUTER_TEST_AI_GEMINI_API_KEY }}
PUTER_TEST_AI_GROQ_API_KEY: ${{ secrets.PUTER_TEST_AI_GROQ_API_KEY }}
PUTER_TEST_AI_MISTRAL_API_KEY: ${{ secrets.PUTER_TEST_AI_MISTRAL_API_KEY }}
PUTER_TEST_AI_DEEPSEEK_API_KEY: ${{ secrets.PUTER_TEST_AI_DEEPSEEK_API_KEY }}
PUTER_TEST_AI_XAI_API_KEY: ${{ secrets.PUTER_TEST_AI_XAI_API_KEY }}
PUTER_TEST_AI_OPENROUTER_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENROUTER_API_KEY }}
PUTER_TEST_AI_TOGETHER_API_KEY: ${{ secrets.PUTER_TEST_AI_TOGETHER_API_KEY }}
PUTER_TEST_AI_MOONSHOT_API_KEY: ${{ secrets.PUTER_TEST_AI_MOONSHOT_API_KEY }}
PUTER_TEST_AI_ZAI_API_KEY: ${{ secrets.PUTER_TEST_AI_ZAI_API_KEY }}
PUTER_TEST_AI_ELEVENLABS_API_KEY: ${{ secrets.PUTER_TEST_AI_ELEVENLABS_API_KEY }}
+211
View File
@@ -0,0 +1,211 @@
name: Backend Tests
# Always triggers on PRs so it can be a required check; the `changes` job below
# decides whether backend/extension paths actually changed (test-only changes
# match the same globs, since tests are co-located with the code), and `test`
# still reports success when they didn't. Pushes to main produce the coverage
# PRs compare against. puter.js SDK changes are covered by puterjs-tests.yaml.
on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches: [main]
permissions:
contents: read
pull-requests: write
# Reads main's coverage artifact for the PR comparison.
actions: read
jobs:
changes:
runs-on: ubuntu-latest
outputs:
backend: ${{ steps.filter.outputs.backend }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
id: filter
with:
filters: |
backend:
- 'src/backend/**'
- 'extensions/**'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'tsconfig.build.json'
- '.github/workflows/backend-tests.yaml'
# `tsconfig.build.json` builds with `noCheck: true`, so nothing else in CI
# runs the type checker and a missing export compiles to `undefined`,
# surfacing only when the call is finally reached at runtime. This diffs
# against tools/typecheck-baseline.json and fails only on *new* errors.
#
# The consuming repo runs an equivalent gate, but only once someone bumps the
# submodule pointer — too late to keep the error off this repo's default
# branch. Hence a gate here, on this repo's own pull requests.
typecheck:
needs: changes
if: github.event_name == 'pull_request' && needs.changes.outputs.backend == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Type check (new errors only)
run: npm run typecheck
# The suite is CPU-bound, so it's split across runners. Each shard writes a
# blob report (results plus coverage) that the `coverage` job merges.
test-shard:
needs: changes
if: needs.changes.outputs.backend == 'true'
name: test shard ${{ matrix.shard }}/${{ strategy.job-total }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- name: Checkout
uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run backend tests with coverage
run: >-
npm run test:backend --
--coverage
--shard=${{ matrix.shard }}/${{ strategy.job-total }}
--reporter=default
--reporter=blob
env:
CI: 'true'
- name: Upload blob report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: backend-blob-${{ matrix.shard }}
path: .vitest-reports/
include-hidden-files: true
retention-days: 1
# Required check, so its PR name is pinned. A skipped job would satisfy it,
# so this always runs and fails on any shard that didn't pass.
test:
needs: [changes, test-shard]
if: ${{ !cancelled() }}
name: test (${{ github.event_name == 'push' && 'main' || 'pr' }})
runs-on: ubuntu-latest
steps:
- name: Check shard results
env:
CHANGES_RESULT: ${{ needs.changes.result }}
BACKEND_CHANGED: ${{ needs.changes.outputs.backend }}
SHARDS_RESULT: ${{ needs.test-shard.result }}
run: |
if [ "$CHANGES_RESULT" != "success" ]; then
echo "changes job: $CHANGES_RESULT"
exit 1
fi
if [ "$BACKEND_CHANGED" != "true" ]; then
echo "No backend changes."
exit 0
fi
if [ "$SHARDS_RESULT" != "success" ]; then
echo "test shards: $SHARDS_RESULT"
exit 1
fi
coverage:
needs: [changes, test-shard]
if: ${{ !cancelled() && needs.test-shard.result == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Download blob reports
uses: actions/download-artifact@v8
with:
pattern: backend-blob-*
path: .vitest-reports
merge-multiple: true
- name: Merge shard coverage
run: npx vitest run --config src/backend/vitest.config.ts --merge-reports --coverage
env:
CI: 'true'
- name: Upload coverage artifact
uses: actions/upload-artifact@v7
with:
name: backend-coverage-${{ github.event_name == 'push' && 'main' || 'pr' }}
path: src/backend/coverage/
retention-days: ${{ github.event_name == 'push' && 30 || 14 }}
# Latest main coverage from this repo's own runs; a fork can't publish
# under this repository id.
- name: Download main coverage
id: base
if: github.event_name == 'pull_request'
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
run: |
id=$(gh api "repos/$REPO/actions/artifacts?name=backend-coverage-main&per_page=20" --jq '
[.artifacts[]
| select(.expired | not)
| select(.workflow_run.head_branch == "main")
| select(.workflow_run.head_repository_id == .workflow_run.repository_id)
][0].id // empty')
if [ -z "$id" ]; then
echo "No main coverage artifact; reporting without a comparison."
exit 0
fi
gh api "repos/$REPO/actions/artifacts/$id/zip" > coverage-base.zip
unzip -q coverage-base.zip -d coverage-base
echo "found=true" >> "$GITHUB_OUTPUT"
- name: Report coverage on PR
if: github.event_name == 'pull_request'
uses: davelosert/vitest-coverage-report-action@v2
with:
json-summary-path: src/backend/coverage/coverage-summary.json
json-final-path: src/backend/coverage/coverage-final.json
json-summary-compare-path: ${{ steps.base.outputs.found == 'true' && 'coverage-base/coverage-summary.json' || '' }}
@@ -0,0 +1,25 @@
name: Dependabot auto-merge
on: pull_request
permissions:
contents: write
pull-requests: write
jobs:
automerge:
runs-on: ubuntu-latest
if: github.actor == 'dependabot[bot]'
steps:
- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v3
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
- name: Enable auto-merge for patch updates
if: steps.metadata.outputs.update-type == 'version-update:semver-patch'
run: gh pr merge --auto --squash "$PR_URL"
env:
PR_URL: ${{ github.event.pull_request.html_url }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+83 -65
View File
@@ -6,80 +6,98 @@ name: Docker Image CI
# picks in the Actions UI. Used to fast-forward :latest/:main when the
# branch has moved ahead of the most recent release tag.
on:
push:
tags:
- '[0-9][0-9].[0-9][0-9]'
- '[0-9][0-9].[0-9][0-9].[0-9]*'
workflow_dispatch:
inputs:
push_latest:
description: 'Push :latest tag'
type: boolean
default: true
push_main:
description: 'Push :main tag'
type: boolean
default: true
push:
tags:
- '[0-9][0-9].[0-9][0-9]'
- '[0-9][0-9].[0-9][0-9].[0-9]*'
workflow_dispatch:
inputs:
push_latest:
description: 'Push :latest tag'
type: boolean
default: true
push_main:
description: 'Push :main tag'
type: boolean
default: true
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-image:
runs-on: ubuntu-latest
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
permissions:
contents: read
packages: write
steps:
- name: Validate calver tag
if: github.event_name == 'push'
env:
REF_NAME: ${{ github.ref_name }}
run: |
if ! [[ "$REF_NAME" =~ ^[0-9]{2}\.[0-9]{2}(\.[0-9]+)?$ ]]; then
echo "Tag '$REF_NAME' does not match YY.MM or YY.MM.p calver format."
exit 1
fi
steps:
- name: Validate calver tag
if: github.event_name == 'push'
env:
REF_NAME: ${{ github.ref_name }}
run: |
if ! [[ "$REF_NAME" =~ ^[0-9]{2}\.[0-9]{2}(\.[0-9]+)?$ ]]; then
echo "Tag '$REF_NAME' does not match YY.MM or YY.MM.p calver format."
exit 1
fi
- name: Checkout repository
uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v7
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Log in to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}"
# type=ref,event=tag only emits on tag pushes, so manual runs skip
# the calver tag and just publish whichever of latest / main the
# dispatch inputs asked for. On a tag push, both default to true.
tags: |
type=ref,event=tag
type=raw,value=latest,enable=${{ github.event_name == 'push' || inputs.push_latest }}
type=raw,value=main,enable=${{ github.event_name == 'push' || inputs.push_main }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
platforms: linux/amd64,linux/arm64
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
heyputer/puter
# type=ref,event=tag only emits on tag pushes, so manual runs skip
# the calver tag and just publish whichever of latest / main the
# dispatch inputs asked for. On a tag push, both default to true.
tags: |
type=ref,event=tag
type=raw,value=latest,enable=${{ github.event_name == 'push' || inputs.push_latest }}
type=raw,value=main,enable=${{ github.event_name == 'push' || inputs.push_main }}
- name: Build and push Docker image
uses: docker/build-push-action@v7
with:
platforms: linux/amd64,linux/arm64
context: .
push: true
provenance: mode=max
sbom: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Sync README to Docker Hub
uses: peter-evans/dockerhub-description@v5
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
repository: heyputer/puter
readme-filepath: ./README.md
+27
View File
@@ -0,0 +1,27 @@
name: Docs Build
on:
pull_request:
paths:
- 'src/docs/**'
jobs:
build-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build docs
run: npm run build
working-directory: src/docs
+3 -3
View File
@@ -9,7 +9,7 @@ jobs:
update-release-pr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v7
- name: Setup Git
run: |
git config user.name github-actions
@@ -20,7 +20,7 @@ jobs:
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
branch: release
- uses: actions/checkout@v4
- uses: actions/checkout@v7
if: steps.find-pr.outputs.number == ''
with:
ref: release
@@ -31,7 +31,7 @@ jobs:
git reset --hard main
- name: Create/Update PR
if: steps.find-pr.outputs.number == ''
uses: peter-evans/create-pull-request@v6
uses: peter-evans/create-pull-request@v8
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: Update release branch
+206
View File
@@ -0,0 +1,206 @@
name: Puter.js API Tests
# Always triggers so it can be a required PR check; the `changes` job below
# decides whether puter.js-relevant paths actually changed (the suites live
# in src/puter-js/tests, so test-only changes are covered by the same glob,
# plus the pieces the runners depend on: the worker preamble, the in-memory
# test env in testUtil, and the backend vitest config the API-tests config
# extends), and every other job is skipped — not absent — when they didn't,
# which still satisfies a required check. Backend and extension changes run
# backend-tests.yaml instead.
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
changes:
runs-on: ubuntu-latest
outputs:
puterjs: ${{ steps.filter.outputs.puterjs }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
id: filter
with:
filters: |
puterjs:
- 'src/puter-js/**'
- 'src/worker/**'
- 'src/backend/testUtil.ts'
- 'src/backend/vitest.config.ts'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/puterjs-tests.yaml'
# The JSDoc in src/puter-js/src is the source of truth for the SDK's public
# types. The declarations shipped to npm are generated from it at build time
# and never committed, so what needs guarding is the JSDoc itself: this
# generates the declarations and type-checks the published surface without
# skipLibCheck, which is how broken re-exports used to go unnoticed.
#
# Its own job rather than a step in `test`: that one builds bundles and
# installs a browser, and this needs neither.
types:
needs: changes
if: needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Check the puter.js JSDoc produces declarations that type-check
run: npm run check:puterjs:types
test:
needs: changes
if: needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
# The runners exercise the built artifacts: the SDK bundle
# (src/puter-js/dist) for node + browser, and the worker preamble
# (src/worker/dist) for workerd.
- name: Build puter.js SDK and worker preamble
run: npm run build:workerLib
- name: Install Playwright chromium
run: npx playwright install --with-deps chromium
- name: Run puter.js API tests (node, browser, workerd)
run: npm run test:puterjs
env:
CI: 'true'
# SDK coverage on both refs, reported as a PR comment — same shape as
# backend-tests.yaml. This run uses the istanbul-instrumented bundle
# (the `test` job above keeps exercising the production build).
# `needs.changes` gates the steps below, not this job, on purpose: a
# job-level `if` on a matrix job skips the whole job as one unexpanded
# check ("coverage (${{ matrix.artifact }})", literally, since there's no
# matrix context to fill it in) instead of two real per-artifact checks —
# confusing to read, and unusable if this ever gets pinned as a required
# check. Letting the matrix always expand keeps the check names stable.
coverage:
needs: changes
# Overrides the default matrix naming, which would otherwise bake the
# base/head branch name (matrix.ref) into the check name — making it a
# different string on every PR and impossible to pin as a required check.
name: coverage (${{ matrix.artifact }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- ref: ${{ github.base_ref }}
artifact: base
- ref: ${{ github.head_ref }}
artifact: pr
steps:
- name: Checkout ${{ matrix.ref }}
if: needs.changes.outputs.puterjs == 'true'
uses: actions/checkout@v7
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Set up Node.js
if: needs.changes.outputs.puterjs == 'true'
uses: actions/setup-node@v7
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
if: needs.changes.outputs.puterjs == 'true'
run: npm ci
- name: Install Playwright chromium
if: needs.changes.outputs.puterjs == 'true'
run: npx playwright install --with-deps chromium
# The base leg is best-effort — it only enriches the PR comment with
# a comparison, and the base ref may predate the coverage script.
- name: Run puter.js API tests with coverage
if: needs.changes.outputs.puterjs == 'true'
continue-on-error: ${{ matrix.artifact == 'base' }}
run: npm run test:puterjs:coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: needs.changes.outputs.puterjs == 'true' && !cancelled()
uses: actions/upload-artifact@v7
with:
name: puterjs-coverage-${{ matrix.artifact }}
path: src/puter-js/coverage/coverage-*.json
retention-days: 14
if-no-files-found: ignore
report-coverage:
needs: [changes, coverage]
if: always() && needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Download PR coverage
uses: actions/download-artifact@v8
with:
name: puterjs-coverage-pr
path: src/puter-js/coverage
# May not exist (base ref without the coverage script, or a failed
# base run) — then report without the comparison column.
- name: Download base coverage
id: base-coverage
continue-on-error: true
uses: actions/download-artifact@v8
with:
name: puterjs-coverage-base
path: coverage-base
- name: Report coverage on PR
if: steps.base-coverage.outcome == 'success'
uses: davelosert/vitest-coverage-report-action@v2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
json-final-path: src/puter-js/coverage/coverage-final.json
json-summary-compare-path: coverage-base/coverage-summary.json
- name: Report coverage on PR (no base comparison)
if: steps.base-coverage.outcome != 'success'
uses: davelosert/vitest-coverage-report-action@v2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
json-final-path: src/puter-js/coverage/coverage-final.json
+2 -2
View File
@@ -45,14 +45,14 @@ jobs:
fi
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Create alias tag
env:
+6
View File
@@ -66,7 +66,13 @@ AGENTS.md
coverage/
.vitest-reports/
*.log
undefined
servers.json
config.*.json
volatile/
# Local build override
docker-compose.override.yml
+2 -1
View File
@@ -1,4 +1,5 @@
{
"tabWidth": 4,
"singleQuote": true
"singleQuote": true,
"plugins": ["prettier-plugin-jsdoc"]
}
+204
View File
@@ -0,0 +1,204 @@
# AGENTS.md
Guidance for AI coding agents working in this repository, and the source of truth for how we write code, tests, and docs here. Read it before making changes. FOLLOW GUIDANCE AS CLOSELY AS POSSIBLE. If you think a rule is wrong, raise an issue or flag a maintainer instead of doing what you think is right.
## Documentation Index
Use these as the source of truth before exploring further:
- [README.md](README.md) — project overview and quickstart.
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), actors, and extensions.
- [doc/contributing-apis.md](doc/contributing-apis.md) — adding and maintaining public APIs end to end (backend surface → puter.js → types → docs → tests). Follow it for any API work.
- [doc/pagination.md](doc/pagination.md) — the one pagination convention for list APIs (limit/cursor/offset/includeTotal, envelope shape, cursor semantics).
- [doc/alarms.md](doc/alarms.md) — raising alarms and picking a severity (what pages, what only gets recorded), plus the config that routes them.
- [doc/self-hosting.md](doc/self-hosting.md) — running Puter outside hosted infra.
- [CONTRIBUTING.md](CONTRIBUTING.md) — contributor entry point and PR conventions.
- [SECURITY.md](SECURITY.md) — how to report vulnerabilities (do not file them publicly).
- [BUG-BOUNTY.md](BUG-BOUNTY.md) — bounty program scope.
- [TRADEMARK.md](TRADEMARK.md) — trademark usage.
---
## Repo-wide conventions
These apply everywhere — backend, puter.js, and GUI.
### Language & files
- Write ES modules, not CommonJS — we transpile and build as needed.
- TypeScript preferred for new files in the backend and extensions; convert existing JS there opportunistically when you're already touching a file. The GUI and puter.js are plain JavaScript — don't introduce TypeScript files in them.
- In plain-JS files, typing with JSDoc (`@type`/`@param`/`@returns`, `@typedef` for shared shapes, using the TypeScript type system) is encouraged. API types must not be `unknown` or untyped `...args`: spell out the real parameter and return shapes, unless the value passes through transparently to an upstream layer that owns its type. puter.js has stricter rules ([below](#types)).
- Make new types findable: descriptive `PascalCase` names, exported from the obvious entry point. A type used from many places goes in the owning module's `types.ts` (e.g. [src/backend/controllers/types.ts](src/backend/controllers/types.ts)); a type with a single consumer can stay next to it.
- Naming: `camelCase` for variables and functions, `PascalCase` for classes and files containing a class (`AuthService.ts`). **Prefer `camelCase` for all new code**, including parameters and internal object properties. `snake_case` is reserved for external contracts that already use it: wire/JSON keys, stable API error codes, public option names that already ship as `snake_case`, and established `puter_*` namespaced fields. Don't rename those (it's a breaking change), and don't add new `snake_case` beside them; a camelCase local can carry a value into a wire key (`{ operation_id: operationId }`).
### Comments
Prefer self-documenting code. Comment only when the _why_ is non-obvious or a usage detail would trip the next reader. **A comment is a sentence or two**: one or two lines of `//`, a few of `/** ... */` JSDoc. State the constraint and stop. Don't restate the code, narrate history ("used to swallow the failure, which meant…"), argue the design, or explain parameters the signature already shows. Reasoning that needs paragraphs belongs in a design doc.
Don't reference the current task, PR, or version; those rot. **No ticket references** (`PUT-1234`, `// fix for FOO-99`) in code, comments, or test names: describe the why in domain terms. Section dividers use plain ASCII (`// -- Section --`), never box-drawing characters.
### Security & privacy
Before opening a PR, scan the diff for:
- Logs, error messages, or responses leaking internal paths, secrets, tokens, env vars, or other users' data.
- Debug routes, test credentials, commented-out auth checks.
- Endpoints returning more than the caller actually needs.
When in doubt, return less. Auth-, permission-, or data-export-related changes deserve an explicit callout in the PR description.
Commit messages and PR descriptions never credit anyone or mention a bug report. For security fixes, state what the change does mechanically ("scoped tokens do xyz now"), not the vulnerability or how it could be abused.
### Reuse and abstraction
- **Search before you build.** Before writing a type, helper, cache/TTL map, retry, lock, pagination, validator, serializer, error mapper, host/email normalizer, or provider adapter, grep `src/backend/util/`, the owning layer, and `extensions/` for one that exists, and extend it instead of adding a parallel copy. If you find two copies, consolidate them or flag it. Before opening the PR, check the diff for new helpers that duplicate existing ones.
- **One path per operation.** Single-item variants call the batch version. Legacy and v2 endpoints share one service method. A second entry point (OIDC, WebDAV, admin, cron) calls the same service as the first instead of re-implementing it.
- **No abstraction without a second caller.** No base class, interface, factory, registry, decorator, or option bag with one implementation. No wrappers that only forward, no interface methods nobody calls, no methods only tests call.
- **Variants are data.** Providers or models that differ only in constants (base URL, prices, key names) are entries in a table read by one implementation, not one file each.
### Hot paths
- No per-row queries or awaits in loops: use the store's batch method (add one if missing) or [src/backend/util/concurrency.ts](src/backend/util/concurrency.ts).
- Bound everything: queries get `LIMIT`/pagination, in-memory maps get eviction, outbound calls get a timeout and a size cap.
- Build clients, catalogs, and config-derived lookups once, not per request.
- Listeners on fleet-wide events (`fs.write.file`, route lifecycle) return before any I/O for requests they don't concern.
### Working rules of thumb
- **Run it, don't just compile it.** "It type-checks" is not "it works." Exercise the code path end-to-end at least once.
- **Read the neighbors before writing.** Match the shape of similar things already in the tree. If you think the existing pattern is wrong, raise it — don't quietly diverge.
- **Test new behavior.** Every new function, endpoint, driver method, or logic branch gets a test; every bug fix gets a regression test that fails before the fix. If something is genuinely hard to test, skip it but say so in the PR.
- **Boy Scout Rule, proportional to the change.** In files you're already touching, fix the typo, the dead import, the missing test, the bit you had to read twice. Don't ride a refactor along with a bug fix.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't have written, debugged, or defended in review is not.
- **A limit change is not done until the docs change with it.** Every rate limit, concurrency cap, quota, and allowance in the code is published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) — an undisclosed limit is one developers discover as a service failure. If a PR moves any of these numbers, the same PR updates that page. The numbers live in `src/backend/controllers/fs/limits.ts` (filesystem), `src/backend/drivers/util/aiLimits.ts` (all AI drivers), per-driver `rateLimit`/`concurrent` configs, and `src/backend/data/subPolicies/` (free-tier allowances).
---
## Local development
To test a local Docker build, use the gitignored `docker-compose.override.yml` described in [Self-Hosting → Building from source](doc/self-hosting.md#building-from-source-instead-of-pulling). Keep local build settings there, not in `docker-compose.yml`.
When budget checks block unrelated local API tests, you may set `"unlimitedMetering": true` without further approval. Merge it into the gitignored `config.json` (`npm start`) or `puter/config/config.json` (Docker) and restart Puter. It covers every account on that instance, guests included, and usage is still recorded. Keep normal settings when testing budget or subscription enforcement, and leave shared defaults unchanged.
---
## Backend
Controllers → drivers (optional) → services → stores → clients, all configured from `IConfig`. Each layer depends only on the layers beneath it and receives them through its constructor; `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires them. [doc/architecture.md](doc/architecture.md) is the full reference.
A public API can be a controller or a driver. Prefer a controller when you need fine-grained control over routes and gates; [doc/contributing-apis.md](doc/contributing-apis.md#controller-or-driver) has the decision guide.
Layer rules:
- **Don't reach across layers.** Controllers don't poke clients; services don't register routes; lower layers never import from controllers. If you want to, the abstraction is wrong; fix the abstraction.
- **Gates run at the edge.** Controllers and drivers validate input and apply auth gates; services assume the caller is already authenticated and authorized.
- **Don't call sideways within a layer for code reuse.** Two services needing the same logic means a util/helper, not a service-to-service dependency.
- **A table's store owns every query against it.** Services, controllers, and extensions don't run raw SQL/redis against a table that has a store; add the store method so cache invalidation stays with the write.
- **Drivers and controllers stay thin.** Domain rules a second entry point needs (signup, app CRUD, worker deploy, AI routing/billing) live in a service, not in the first driver or controller that needed them.
- **Prefer explicit arguments over [`Context`](src/backend/core/context.ts) (ALS).** Use it only for request-scoped values that would otherwise thread through many layers; today that's mostly `actor` and `req`.
### Extensions
[extensions/](extensions/) is for **non-crucial parts of the system**: Puter must still work with the extension removed. If core needs to call it, it belongs in core ([whoami](extensions/whoami.ts) is the cautionary example). Inside an extension, follow the same layered structure, unless it only needs a few route handlers, in which case the `extension.get/post/...` helpers are enough on their own. The `extension` API is in [doc/architecture.md](doc/architecture.md#extension-api).
### Backend tests
- Vitest; test files sit next to the code they test (`*.test.ts` / `*.test.js`). Run with `npm run test:backend`.
- **Mock data, not methods.** Stub inputs (fixtures, fake rows, payloads), not the function under test or the layer beneath it — over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service).
- **Prefer the test server over mocking deps.** `setupPuterTestEnv()` in [src/backend/testUtil.ts](src/backend/testUtil.ts) boots a fully in-memory backend; hit a real database/client shape where reasonable — integration shapes catch what mocked unit tests miss.
- **Test code, not docs.** Never write a test that reads a file under `src/docs/` and asserts on its wording or numbers. Docs are kept in step by the PR (see the limits rule above) and checked in review; a test that greps a markdown page fails on every rewording and verifies nothing about behavior.
---
## puter.js (the SDK)
[src/puter-js/](src/puter-js/) is the public SDK. It ships live from `https://js.puter.com/v2/` with no version pinning, so every change reaches every existing app at once; see [Rule zero](doc/contributing-apis.md#rule-zero-dont-break-callers).
Layout: SDK modules in [src/puter-js/src/modules/](src/puter-js/src/modules/) (one file or directory per area — `FileSystem/`, `kv/`, `ai/`, …), shared plumbing in [src/puter-js/src/lib/](src/puter-js/src/lib/), generated (gitignored) type declarations in `src/puter-js/types/`, API tests in [src/puter-js/tests/api/](src/puter-js/tests/api/), UI e2e tests in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/), developer docs in [src/docs/](src/docs/).
Every SDK change carries all five of these, or the PR is incomplete. [doc/contributing-apis.md](doc/contributing-apis.md) has the steps for each.
1. **Backward compatibility**, unless a maintainer explicitly signs off on a break ([rules](doc/contributing-apis.md#maintaining-an-existing-api)).
2. **Tests** in the cross-platform API suite, or the Playwright harness for desktop-rendered `puter.ui.*` ([steps](doc/contributing-apis.md#6-tests)).
3. **Docs** in [src/docs/src/](src/docs/src/) that match the implementation exactly ([steps](doc/contributing-apis.md#5-docs), [writing rules](#developer-docs)).
4. **Types**, per the rules below.
5. **Error handling**: `{ message, code }` with stable `snake_case` codes, as in [modules/kv/](src/puter-js/src/modules/kv/) ([steps](doc/contributing-apis.md#3-puterjs)).
### Types
puter.js source is plain JavaScript typed via JSDoc, and **the JSDoc is the source of truth for the SDK's types**. `src/puter-js/types/` is `tsc --emitDeclarationOnly` output: the SDK build (`npm run build` in `src/puter-js`) generates it, the npm tarball ships it, and git ignores it. Never commit or hand-edit it. `npm run check:puterjs:types` generates it and type-checks the result without `skipLibCheck`; run it after any type change. The one hand-written declaration file is [src/puter-js/index.d.ts](src/puter-js/index.d.ts): it decides what is public and re-exports the generated names, so name a new type there if consumers should be able to import it.
- Public (exposed) methods carry JSDoc types for parameters and return value, with one `@overload` block per accepted call form; those overloads _are_ the published signature. Typing private helpers is at your discretion; annotate where it helps the next reader.
- Tag members `@internal`, not `@private`, to strip them from the generated declarations.
- Declarations must match runtime behavior exactly. A wrong type is worse than a missing one.
Declare a shape where it belongs and reference it elsewhere with an `import(...)` type. A shape more than one file needs lives in the module's `types.js`; one shared across modules lives in [src/puter-js/src/lib/types.js](src/puter-js/src/lib/types.js); one with a single consumer can stay next to it:
```js
/** @typedef {import('./types.js').KVOptConfig} KVOptConfig */
```
Use `@typedef {Object}` + `@property` for any shape whose fields need documenting; it's the only JSDoc form that carries a per-field doc comment into the generated declarations. Use the inline form for small internal shapes with nothing to say per field, and prefer `unknown` over `*`:
```js
/** @typedef {{ key: string; value: unknown }} KVEntry */
```
---
## Developer docs
[src/docs/src/](src/docs/src/) is the published puter.js documentation: one reference page per method (`<Area>/<method>.md`, [steps](doc/contributing-apis.md#5-docs)) and recipes in [src/docs/src/recipes/](src/docs/src/recipes/).
### Page structure
The reference has two kinds of page:
- **API overview pages** (`<Area>.md`, such as [KV.md](src/docs/src/KV.md)) give a quick top-level picture of what the API is for and what it can do, with short examples and a list of its functions that links every function page in the area. Leave out the details. The first sentence of the intro says what the API is about. This is a judgment call too: when the API depends on a Puter concept the reader may not know, such as Teams, the first sentence can name it and the sentences after it explain it.
- **Function pages** (`<Area>/<method>.md`) hold the details: parameters, defaults, return values, errors, limits, and edge cases.
An overview page answers "what is this API and is it what I need?", and a function page answers "how exactly does this call behave?". A change to one function usually only touches that function's page, so a new `kv.list()` option goes in `KV/list.md` without a matching update to `KV.md`. Don't copy a function's options, limits, or edge cases onto the overview page for completeness. Where a detail goes is a judgment call, not a hard rule: update the overview when a change affects how someone would describe the API as a whole.
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
### Writing docs
These apply to reference pages and recipes alike.
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
- **State the rule, then its consequence, in short sentences.** No metaphor in place of a rule. Name a concrete action ("you can create any files and folders inside it") over an abstract permission ("yours to organize freely"). Cut a sentence that only repeats the next one.
- **Describe behavior directly.** No quoted inner-voice phrases. Write "`isNew` is `true` when the person did not have access before", not "this tells 'I just gave this to someone' from 'they already had it'".
- **Keep references easy to follow.** Name what a list or code block belongs to in the sentence directly above it. Name a thing once and keep that name. Introduce an actor as "a user" before giving them a role: "A user can create a team. The user who creates the team becomes its owner."
- **Sentence mechanics.** Never start a sentence with inline code: write "To store data, use the [`set()`](/KV/set/) method." Link every inline mention of an API to its page, every time. No em dashes, and no colons or semicolons mid-sentence; a colon that introduces a code block is fine.
- **Show the correct way.** No "don't" and no wrong-way-first examples, in recipes and playground examples alike. Mention a wrong approach only when developers already hit it in practice, and state how it fails as a fact: "`add()` reads a plain object as a path map, so the call rejects with …".
### Recipes
Recipes are tested, copy-pasteable patterns for a task, one layer above the per-method reference, often combining several APIs. Agents find them through the `## Recipes` section of `llms.txt`.
- **Files.** One `.md` per recipe in [src/docs/src/recipes/](src/docs/src/recipes/), with `title`, `description`, `tags`, and optional `order` front matter; they're picked up automatically. Tags must exist in [src/docs/src/recipe-tags.js](src/docs/src/recipe-tags.js) or the build fails. Quote a `description` that contains a colon followed by a space. Every top-level `.md` in that directory is built as a recipe, so keep other markdown out of it.
- **Title.** Short, Title Case, starts with a verb, and leaves out "to Your App" ("Add Team Support", "Deploy Workers Programmatically"). The slug is the title in kebab case, with no API-name prefix. In `llms.txt` an agent sees only the title and description before deciding to open a recipe, so word them to match the task the reader is trying to do.
- **Description.** Written for someone who has never heard of Puter. Start with "Learn how to", put the action first, and say what the reader's app gains: "Learn how to add file sharing to your app with Puter.js, so users can share files in their Puter account with other people." No API names, no mechanism, no list of example cases, and no "not just …" contrast.
- **Intro.** Explain the situation and why you'd do this, tied to something the reader already knows ("`puter.kv` is your database in Puter.js"). No "Use this when" bullet list.
- **Structure.** Get to the first working call as early as possible, then add validation, rate limits, and sign-in after it. Explain a Puter concept the first time it appears, such as what `me.puter` is or that `puter.workers.exec()` opens the Puter sign-in window. Split recipes by what the reader needs, not by technique, and give a family of recipes a shared tag.
- **Link to the reference for full option lists.** Mention only the options the recipe uses, then link to the function page for the rest ("All the sort options are listed on [`puter.fs.readdir()`](/FS/readdir/)."). A copied list of options, rules, or scopes goes stale when the reference changes.
- **Code.** Platform-neutral: just the code, with top-level `await`. No `<script>` or `<html>` wrapper, no IIFE.
- **Tradeoffs.** Justify a choice with one scenario the reader can picture right away, in about two short paragraphs, or fold each point into the section where it matters. The description states the benefit only.
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit goes in the section it applies to; other facts belong on the linked reference page.
---
## GUI
[src/gui/](src/gui/) is the Puter desktop: deliberately plain JavaScript + jQuery with HTML-string templates. Don't introduce a UI framework or a new rendering pattern.
**Conformity over novelty**: match the existing design and code structure even where you'd choose differently. A visually or structurally divergent addition is a defect even when it works.
- **Reuse existing UI primitives before writing new ones.** Windows and dialogs are `UIWindow*` functions in [src/gui/src/UI/](src/gui/src/UI/); generic pieces already exist (`UIAlert`, `UIPrompt`, `UIContextMenu`, `UINotification`, `UIPopover`, widgets in [UI/Components/](src/gui/src/UI/Components/)). A new window should read like its neighbors: an async function taking an options object, composing an HTML string, wiring behavior with jQuery, delegating to `UIWindow(...)`.
- **Match the visual language.** Use existing CSS classes (`button`, `button-primary`, window chrome, form styles) and copy the layout patterns of neighboring windows; new styles go in [src/gui/src/css/](src/gui/src/css/) following existing conventions. Verify anything positional (menus, overlays, z-index) on both desktop and mobile viewports.
- **i18n every user-facing string.** No hardcoded UI text — use `i18n('key')` and add the key to [src/gui/src/i18n/translations/en.js](src/gui/src/i18n/translations/en.js). Run `npm run check-translations` before opening the PR.
- **Shared logic goes in helpers/services.** Reusable non-UI logic belongs in [src/gui/src/helpers/](src/gui/src/helpers/) or [src/gui/src/services/](src/gui/src/services/), not copy-pasted between windows.
- **Tests.** Vitest is wired for the GUI (`src/**/*.test.js`; see [appOrder.test.js](src/gui/src/UI/Dashboard/appOrder.test.js) for the shape). Extract pure logic into functions and test those. Desktop behavior driven through puter.js (`puter.ui.*`) is covered by the Playwright harness in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/) — add a spec there when you change how the desktop renders SDK-driven UI.
+60 -16
View File
@@ -6,38 +6,82 @@ We at **Puter** are committed to maintaining a secure experience for our users a
The following are in scope for this program:
* **The Puter open-source project** (available at [github.com/HeyPuter](https://github.com/HeyPuter/puter))
* **`puter.com`**
* **`api.puter.com`**
- **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
- **`puter.com`**
- **`api.puter.com`**
Out-of-scope:
* Third-party services, applications, or libraries not maintained by Puter.
* Social engineering attacks (e.g., phishing against staff).
* Denial of Service (DoS), spam, or volumetric attacks.
* Physical security issues.
- Third-party services, applications, or libraries not maintained by Puter.
- Social engineering attacks (e.g., phishing against staff).
- Denial of Service (DoS), spam, or volumetric attacks.
- Physical security issues.
## Known Non-Issues (Please Check Before Submitting)
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
- **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is out of scope. This explicitly includes **missing `Content-Security-Policy: sandbox` on any endpoint that serves a file inline** — `/token-read`, `/file`, `/fs/read`, WebDAV, and any MIME type the `applyInlineContentSecurity` allow-list does not cover (`text/xsl`, `image/svg+xml`, `application/xhtml+xml`, …). Script execution on the API origin has nothing to reach: there is no session cookie, no `localStorage`, and no ambient credential there. WebDAV additionally has to return an accurate MIME type for real WebDAV clients. The one exception we _do_ evaluate on its merits is anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host — if your report does not name a specific credential that is actually reachable from the API origin, it is this non-issue.
- **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
- **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
- **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
- **App metadata or app user-count "leaks".** This information is currently public by design. This covers **private apps** too: private apps are an unreleased, admin-only beta for paid listings in the Puter App Center. Their `index_url` and other metadata are not secret, the flag is not user-settable, and no production app relies on it. Reports that a private app's metadata is reachable through a sibling route (taskbar items, `/get-launch-apps`, `index_url` drift on the hosting gate, etc.) are normalization work, not vulnerabilities.
- **Worker hot-reload driven by a share recipient.** Sharing a worker's source file with write access is how you let a collaborator change that worker's code and have it deploy live. The redeploy running as the file owner, with the owner's worker credentials, is the intended behavior of that grant — the owner chose to hand out write access to executable code. If you do not want that, do not share the source file with write access.
- **Share-mode downgrades that do not cascade.** Demoting a delegate does not retroactively revoke the grants that delegate already issued downstream. Shares are independent grants once issued, not a live-evaluated chain. Revoking the downstream grant is a separate, supported action.
- **Concurrency / TOCTOU overspend of credits or storage quota.** Firing concurrent requests to exceed an AI credit hold or a storage quota by some margin is known and accepted; the per-account concurrency and rate-limit gates are the control we rely on, and the residual overage is bounded and billed. Report these only if you can demonstrate _unbounded_ overspend.
- **Session and credential lifecycle around 2FA enrollment.** Enrolling in 2FA is not a credential-revocation event: existing authenticated sessions stay valid, and outstanding password-reset links stay valid. Likewise, account-management actions behind the `/user-protected/*` gate are password-gated by design, not TOTP-gated. A recovery code that stops being accepted fails _closed_ — it denies access rather than granting it, so it is a bug report, not a vulnerability.
- **Group permissions** (`/auth/revoke-user-group` and siblings). The group-permission surface is unfinished and not exposed anywhere in the product. Findings there are not reachable by any user.
- **Correctness bugs with no security boundary crossed.** For example, an API that reports success while silently leaving data behind. We want these reported — please open a GitHub issue — but they are not bounty-eligible.
- **Missing Subresource Integrity (SRI) on external scripts/styles.** Accepted as hardening, not as a vulnerability, absent a demonstrated compromise of the CDN in question.
- **General "token in a URL" / token-lifetime / token-storage designs** — signed directory URLs exposing children, a write signature implying read, app tokens outliving a web session, or the session token being held in `localStorage` on `puter.com`. These are current intended behaviors. Shorter-lived tokens with a refresh flow are on the roadmap; their absence is not a vulnerability on its own. Show a reproducible attack that does not already presuppose script execution on `puter.com`.
- **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
- **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
- **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
- **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
- **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
- **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
## Rules of Engagement
To participate, you must:
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code or screenshots where applicable.
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see _Proof of Reproduction_ below).
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
5. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
6. **Check the Known Non-Issues list**: Reports matching an item in the "Known Non-Issues" section above are not eligible and will be closed.
Reports that do not meet these guidelines may not be eligible for a reward.
## Proof of Reproduction
Reports **MUST** demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
- Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
- The **observed** result versus the **expected** result.
- For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
- For server-side bugs: a runnable proof-of-concept and screenshot or video showing the attack process.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and not eligible for bounty**.
If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Please submit **one issue per report**.
Bundled "audit packs" of many speculative findings will not be elligible for bounty; send each confirmed issue separately.
Additionally please **DO NOT** submit too many issues at once; these will be treated as a bundled report, and will not be elligible for bounty.
## Reporting Process
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
Include:
* A description of the vulnerability
* Steps to reproduce
* Potential impact
* Suggested remediation (if available)
- A description of the vulnerability
- Steps to reproduce
- Potential impact
- Suggested remediation (if available)
- Paypal id where we can send the bounty (if eligible)
We aim to acknowledge receipt within **72 hours** and provide a resolution timeline.
@@ -45,10 +89,10 @@ We aim to acknowledge receipt within **72 hours** and provide a resolution timel
We offer monetary rewards based on the severity of the vulnerability, as determined by our internal assessment (using CVSS as a guide).
* **Critical: \$1,000 – \$2,000**
* **High: \$500 – \$1,000**
* **Medium: \$200 – \$500**
* **Low: \$50 – \$100**
- **Critical: \$1,000 – \$2,000**
- **High: \$500 – \$1,000**
- **Medium: \$200 – \$500**
- **Low: \$50 – \$100**
Non-security issues, suggestions, and best practices feedback are always welcome, but may not qualify for a reward.
If multiple researchers report the same issue, the bounty will be awarded to the first eligible report we receive.
+1
View File
@@ -0,0 +1 @@
FOLLOW ./AGENTS.md
+29
View File
@@ -0,0 +1,29 @@
# Contributing to Puter
Thanks for contributing. These rules aren't strictly enforced, but following them makes every PR easier. If anything's unclear, ping a core maintainer or open the PR and ask.
- New to the backend? Start with [doc/architecture.md](doc/architecture.md).
- Code, test, and docs conventions for the whole repo live in [AGENTS.md](AGENTS.md). It's written for AI agents; the conventions apply to human contributors too.
- Adding or changing a public API (an endpoint, driver method, or puter.js method)? Follow [doc/contributing-apis.md](doc/contributing-apis.md): backward compatibility, [developer docs](src/docs/), types, and tests all move in the same PR.
- Found a vulnerability? Report it privately per [SECURITY.md](SECURITY.md).
---
## Before you open a PR
Check your diff against these sections of AGENTS.md:
- [Working rules of thumb](AGENTS.md#working-rules-of-thumb): run it end to end, test new behavior and bug fixes, understand every line you commit (AI assistance is fine), leave touched files a bit better, and publish limit changes.
- [Reuse and abstraction](AGENTS.md#reuse-and-abstraction) and [hot paths](AGENTS.md#hot-paths).
- [Security & privacy](AGENTS.md#security--privacy): expose nothing the caller doesn't need.
## Opening a PR
- One thing per PR where possible.
- Describe **what** and **why**; the diff shows how.
- Mention how you tested user-visible changes.
- Drafts welcome.
---
Questions? Message a core maintainer. Welcome aboard.
+2
View File
@@ -79,6 +79,8 @@ ENV PUTER_CONFIG_PATH=/etc/puter/config.json
ENV NODE_OPTIONS=--enable-source-maps
EXPOSE 4100
# The inbound SMTP receiver, when run from this image with the `smtp` profile.
EXPOSE 2525
USER node
+17 -12
View File
@@ -10,12 +10,10 @@
·
<a href="https://apps.puter.com/">App Store</a>
·
<a href="https://builder.puter.com/">AI Builder</a>
·
<a href="https://developer.puter.com" target="_blank">Developers</a>
·
<a href="https://discord.com/invite/PQcx7Teh8u">Discord</a>
·
<a href="https://reddit.com/r/puter">Reddit</a>
·
<a href="https://twitter.com/HeyPuter">X</a>
</p>
@@ -25,7 +23,15 @@
## Puter
Puter is an advanced, open-source, self-hostable internet computer designed to be feature-rich, fast, and highly extensible. Puter's goal is to provide you with every app and feature you need to work, create, and play under one roof.
Puter is an advanced, open-source, self-hostable internet computer designed to be feature-rich, fast, and highly extensible.
### For Users
Puter's goal is to provide you with every app and feature you need to work, create, and play under one roof. From a simple [Notepad](https://online-notepad.com) and [Voice Recorder](https://voice-recorder.com) to [Spreadsheet](https://apps.puter.com/app/spreadsheet) and [Camera](https://online-camera.com), Puter wants to be the all-in-one solution for your digital life.
### For Developers
Puter provides everything you need to build and publish web apps and games. From [AI](https://developer.puter.com/ai/) to [Cloud Storage](https://developer.puter.com/object-storage/) and [Database](https://developer.puter.com/key-value-database/) to [Serverless Workers](https://developer.puter.com/serverless-workers/), Puter has you covered. Puter also helps you get users! Once you build your app, you can publish it on our [App Store](https://apps.puter.com/) to reach and monetize users.
<br/>
@@ -40,7 +46,9 @@ cd puter
npm install
npm start
```
**→** This should launch Puter at http://puter.localhost:4100 (or the next available port).
**→** This should launch Puter at http://puter.localhost:4100
To run this checkout with Docker, follow [Building from source](./doc/self-hosting.md#building-from-source-instead-of-pulling). Create a local `docker-compose.override.yml` to select the local build; keeping these settings out of `docker-compose.yml` avoids conflicts when pulling updates and keeps local configuration out of pull requests.
<br/>
@@ -59,8 +67,6 @@ curl -fsSL https://puter.com/selfhost | sh
irm https://puter.com/selfhost?os=windows | iex
```
<br/>
**→** For more details, see [Self-Hosting Puter](./doc/self-hosting.md).
<br/>
@@ -76,11 +82,8 @@ Puter is available as a hosted service at [**puter.com**](https://puter.com).
Connect with the maintainers and community through these channels:
- Bug report or feature request? Please [open an issue](https://github.com/HeyPuter/puter/issues/new/choose).
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
- Mastodon: [mastodon.social/@puter](https://mastodon.social/@puter)
- Security issues? [[email protected]](mailto:[email protected])
- Security issues or abuse reports? [[email protected]](mailto:security@puter.com)
- Email maintainers at [[email protected]](mailto:[email protected])
We are always happy to help you with any questions you may have. Don't hesitate to ask!
@@ -114,7 +117,9 @@ This repository, including all its contents, sub-projects, modules, and componen
- [Korean / 한국어](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ko.md)
- [Malay / Bahasa Malaysia](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.my.md)
- [Malayalam / മലയാളം](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ml.md)
- [Dutch / Nederlands](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.nl.md)
- [Polish / Polski](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pl.md)
- [Odia / ଓଡ଼ିଆ](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.od.md)
- [Portuguese / Português](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pt.md)
- [Punjabi / ਪੰਜਾਬੀ](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pa.md)
- [Romanian / Română](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ro.md)
+73
View File
@@ -0,0 +1,73 @@
# Reverse proxy in front of Puter — mirrors what the prod ALB does:
# accepts every Host header, forwards to the Puter container, and lets
# the Puter app handle subdomain-based routing internally (api.*,
# site.*, app.*, dev.*, plus the per-user subdomains under those).
#
# To enable TLS:
# 1. Drop a wildcard fullchain.pem + privkey.pem into ./puter/tls/
# (see "Step 3 — TLS" in doc/self-hosting.md).
# 2. Uncomment the `:443` block at the bottom of this file, and swap the
# `:80` block for the redirect shown alongside it.
# 3. Uncomment the `443:443` port mapping under the `caddy` service in
# docker-compose.yml, and HTTPS_PORT in .env.
# 4. Set `"protocol": "https"` and `"pub_port": 443` in config.json.
{
# Certs are supplied by the operator, not issued by Caddy. Puter serves
# per-user sites and apps on dynamic subdomains (<name>.site.<domain>,
# <name>.app.<domain>) — only a DNS-01 wildcard cert covers those, and
# DNS-01 needs a provider plugin that isn't in the stock caddy image.
# `off` stops Caddy attempting ACME on boot (it would fail and leave the
# site unreachable) and stops it inventing its own http→https redirects.
auto_https off
}
# Shared handling, imported by the HTTP and HTTPS site blocks below so the
# two can't drift apart.
(puter_routes) {
# Rough size cap that mirrors prod ALB defaults; tune for your uploads.
# Puter chunks large uploads, so 1 GiB per request is plenty.
request_body {
max_size 1024MiB
}
# RustFS — see the `s3` service in docker-compose.yml. Browsers PUT/GET
# here for presigned-URL uploads / downloads. Matched on the `s3.`
# subdomain of whatever the install domain is, so signature verification
# works (Caddy preserves the original Host end-to-end) and HTTPS stays
# clean — no mixed content from a port-9000 host publish.
@s3 header_regexp Host ^s3\.
handle @s3 {
reverse_proxy s3:9000
}
# Everything else, on every Host, goes to Puter — which routes on that
# Host internally. Caddy forwards it unchanged and adds X-Forwarded-For
# / -Proto / -Host, which is what `trust_proxy` in config.json counts.
handle {
reverse_proxy puter:4100 {
# Stream responses through unbuffered — Puter uses SSE and
# socket.io. WebSocket upgrades need no config of their own;
# Caddy proxies them by default.
flush_interval -1
}
}
}
# ── HTTP (port 80) ─────────────────────────────────────────────────────
# A site address with no hostname is the catch-all: it answers for every
# Host, which is what Puter's subdomain routing requires.
:80 {
import puter_routes
}
# ── HTTPS (port 443) — uncomment after dropping certs in ./puter/tls/ ──
# Replace the `:80` block above with a redirect to force HTTPS everywhere:
#
# :80 {
# redir https://{host}{uri} permanent
# }
#
# :443 {
# tls /etc/caddy/tls/fullchain.pem /etc/caddy/tls/privkey.pem
# import puter_routes
# }
+6 -1
View File
@@ -5,7 +5,7 @@
"protocol": "http",
"domain": "puter.localhost",
"cookie_name": "puter_auth_token",
"jwt_secret": "dev-jwt-secret-change-me",
"jwt_secret_v2": "dev-jwt-secret-v2-change-me",
"url_signature_secret": "dev-url-signature-secret-change-me",
"allow_all_host_values": true,
"allow_no_host_header": true,
@@ -21,6 +21,11 @@
"default_user_group": "78b1b1dd-c959-44d2-b02c-8735671f9997",
"default_temp_group": "b7220104-7905-4985-b996-649fdcdb3c8f",
"storage_capacity": 104857600,
"disable_user_signup": false,
"teams_enabled": false,
"max_teams_per_user": 1,
"max_seats_per_team_free": 3,
"max_seats_per_team_paid": 40,
"strict_email_verification_required": false,
"gui_assets_root": "./src/gui",
"puterjs_root": "./src/puter-js/dist",
+240 -8
View File
@@ -16,6 +16,12 @@
// `dev` opens a browser on boot, skips blocked-email checks, and runs the
// dev-time webpack watcher; `prod` serves pre-built bundles.
"env": "dev",
// Console output format. `json` replaces the global console so every call
// emits one structured JSON line (level, timestamp, msg, and the active
// request's trace id) — one event per call, so a line-oriented log
// collector can't split stack traces across events and level filtering
// works. Unset (the default) leaves console output human-readable.
"log_format": "text",
"version": "0.0.0",
// Stable identity for this server node — used by pager alerts and
// graceful-shutdown coordination.
@@ -26,6 +32,10 @@
"port": 4100,
// Externally-visible port (set this when behind a reverse proxy on 80/443).
"pub_port": 4100,
// Idle keep-alive timeout in ms. Must stay above the idle timeout of any
// proxy in front, or the proxy reuses connections this server has closed
// and its clients see 502s.
"keep_alive_timeout": 620000,
"protocol": "http",
"domain": "puter.localhost",
// Fully-qualified externally-visible URL. Computed from protocol/domain/
@@ -67,10 +77,13 @@
// ── Auth / session ──────────────────────────────────────────────────
// ALWAYS replace these for any public install — `openssl rand -hex 64`.
"jwt_secret": "change-me",
"jwt_secret_v2": "change-me",
"url_signature_secret": "change-me",
"cookie_name": "puter_auth_token",
"min_pass_length": 6,
// When true, anonymous users must log in instead of creating temp or
// permanent accounts.
"disable_user_signup": false,
"allow_system_login": false,
"strict_email_verification_required": false,
"captcha": {
@@ -155,17 +168,30 @@
// ── Database ────────────────────────────────────────────────────────
"database": {
// `sqlite` for single-node / dev; `mysql` for self-host with MariaDB.
// `sqlite` for single-node/dev, `mysql` for MariaDB/MySQL,
// or `postgres` for PostgreSQL.
//
// NOTE: `postgres` support is a community contribution and is not
// exercised by Puter.com production. It boots, passes its own
// integration tests against pgmock, and runs the common user/app/
// fsentry/session/permission/OIDC flows — but less-traveled code
// paths may surface MySQL/SQLite-isms that haven't been ported yet.
// Expect rough edges and please file issues if you hit one. For
// production self-hosting today, `mysql` (MariaDB) and `sqlite` are
// the supported defaults.
"engine": "sqlite",
// sqlite — file path on disk
"path": "volatile/runtime/puter-database.sqlite",
"targetVersion": 0,
// mysql — connection details
// mysql/postgres — connection details. PostgreSQL defaults to port
// 5432 when `engine` is `postgres`; MySQL/MariaDB normally use 3306.
"host": "",
"port": 3306,
"user": "",
"password": "",
"database": "",
// postgres may also use a URL instead of host/user/password fields:
// "connectionString": "postgres://puter:secret@localhost:5432/puter",
// Optional read-replica pool. Reads route here when populated.
"replica": {
"host": "",
@@ -174,9 +200,10 @@
"password": "",
"database": ""
}
// mysql self-host bootstrap: set `migrationPaths` to apply the bundled
// schema on first boot. Idempotent — safe to leave on.
// mysql/postgres self-host bootstrap: set `migrationPaths` to apply the
// bundled schema on first boot. Idempotent — safe to leave on.
// "migrationPaths": ["./src/backend/clients/database/migrations/mysql"]
// "migrationPaths": ["./src/backend/clients/database/migrations/postgres"]
},
// ── DynamoDB (KV store) ─────────────────────────────────────────────
@@ -228,6 +255,154 @@
}
},
// ── Inbound mail (separate process) ─────────────────────────────────
// Mailboxes addressed at one of your own domains. `secret` gates the
// ingress endpoint that files a message into the recipient's ~/.mail.
//
// `localServer` additionally runs an SMTP listener that accepts mail from
// other mail servers and hands each message to that endpoint. It runs as
// its own process - `npm run start:smtp`, or the `smtp` compose profile -
// and stores nothing itself. Without it, nothing delivers to the endpoint
// unless you point something else at it.
//
// The listener is unauthenticated by nature: anything on the internet may
// hand it a message for an address in `localDomains`. It performs no spam,
// SPF, DKIM or DMARC checks. Read the self-hosting guide before exposing
// it.
// "userEmail": {
// "secret": "a long random string",
// "localServer": true,
// "localDomains": ["example.com"],
// // Container-internal by default; publish 25 onto it from outside.
// "localPort": 2525,
// // Defaults to <api_base_url>/email/ingress. The endpoint is served
// // on the `api` subdomain, so when this points somewhere internal,
// // `localIngressHost` has to name that virtual host.
// "localIngressUrl": "http://puter:4100/email/ingress",
// "localIngressHost": "api.example.com",
// "localHostname": "mx.example.com",
// "localMaxRecipients": 50,
// // Concurrent connections. Each message is held in memory while it
// // is delivered, so this also bounds memory use.
// "localMaxClients": 20
// },
// ── Sharing ─────────────────────────────────────────────────────────
// Email a recipient who already has an account about a new share. On
// unless set to false; they opt out via the unsubscribe link, or by
// blocking a sender. An invite to an address with no account always goes.
// "share_email_notifications": false,
//
// New shares one account may create per UTC day. Default 200.
// "share_daily_limit": 200,
//
// How often a share may interrupt its recipient — the notification pushed
// to their screen and the email with it. Over budget the share still
// succeeds and their notification is still brought up to date; only the
// interruption is dropped. The recipient bounds are what stop many senders
// burying one person between them. A non-positive value removes a bound.
// "share_notify_limits": {
// "pairWindowSeconds": 900,
// "pairDaily": 20,
// "recipientHourly": 10,
// "recipientDaily": 50,
// "emailBatchSeconds": 90
// },
// ── Teams / teams ──────────────────────────────────────────────
// Off by default, and off is the state every existing install stays in.
// The tables ship either way and sit inert; this gates whether `/teams`
// is registered at all, so with it off the paths do not exist rather than
// existing and refusing. It is also the backout: turning it off removes
// the feature without touching data.
"teams_enabled": false,
//
// Live teams one user may own. Soft-deleted ones do not count, so
// deleting yours frees the slot. Default 1.
"max_teams_per_user": 1,
//
// Seats one team may provision. Each seat is a real Puter account taking a
// name from the global username pool, so this is what bounds a team's blast
// radius. The owner's plan decides which of the two applies; defaults are 3
// and 40. A plan registered with `teamSeatCap` uses that instead. Lowering
// a cap below a team's current seat count blocks new provisioning and
// disables nobody. Deployments upgrading from the old flat default of 50:
// without a billing extension every owner reads as free, so set
// `max_seats_per_team` to keep the old behavior.
"max_seats_per_team_free": 3,
"max_seats_per_team_paid": 40,
//
// One flat cap for every team, whatever the owner pays. Setting it
// overrides both of the above, so a deployment that does not sell seats can
// ignore plans entirely. Unset by default — leave it that way to keep the
// free cap meaningful.
// "max_seats_per_team": 50,
//
// Staged rollout: only these email domains may create a team, and the tab
// renders for them even with `gui_params.teams_ui` off. Members of an
// existing team always pass. Unset means everyone, gated by `teams_ui`.
// "teams_allowed_email_domains": ["puter.com"],
// ── Notifications ───────────────────────────────────────────────────
// How long a notification is kept, in days from creation. Acknowledged or
// not, a row past this is swept. Off by default; any value above 0 arms
// the sweep.
// "notificationRetentionDays": 14,
// ── Alarms / alerting ───────────────────────────────────────────────
// Where system alarms go. Severity is the routing decision — each
// transport takes everything at or above its own `minSeverity`:
//
// critical — an unhandled server error; pages on-call.
// error — pages as well; prefer critical or warning.
// warning — look at it today; no page.
// info — a record in the chat channel only.
//
// Both transports are off unless enabled, so a self-hosted node just
// logs its alarms to the console.
"pager": {
// Severity for call sites that don't pick one. Default "critical".
"defaultSeverity": "critical",
// Retier or silence an alarm without a deploy. Keys are alarm ids,
// or a prefix ending in `*`; the exact id wins over a pattern, and
// the longest matching pattern wins among patterns. Values are a
// severity or "mute". This is applied last, so it overrides both the
// call site and any known-error rule.
"severityOverrides": {
// "cronMonitor:*": "info",
// "http_500:GET:/some/flapping/route:*": "mute"
},
"pagerduty": {
"enabled": false,
"routingKey": "",
// Lowest severity that reaches PagerDuty. Default "warning",
// which keeps `info` out of the paging system entirely.
"minSeverity": "warning"
},
// Slack incoming webhook — the low-noise destination for everything
// that shouldn't page.
"slack": {
"enabled": false,
"webhookUrl": "",
// Optional; defaults to the channel the webhook was created for.
"channel": "#alerts",
"username": "puter-alarms",
// Severity window posted to Slack. The ceiling defaults to
// "info" when PagerDuty is configured — what pages belongs in
// the pager, not in chat — and to "critical" when Slack is the
// only transport.
"minSeverity": "info",
"maxSeverity": "info",
// Don't repost the same alarm id within this window. The first
// occurrence always posts; the next post that gets through
// reports how many occurrences piled up. 0 disables throttling.
"repeatThrottleMs": 900000
}
},
// ── Rate limiting ───────────────────────────────────────────────────
// `memory` for single-node, `redis` for multi-node (default), `kv` for
// dynamo-backed counters.
@@ -242,16 +417,46 @@
// ─ Chat / completion ─
"claude": { "apiKey": "" },
"openai-completion": { "apiKey": "" },
"azure-openai": {
"apiKey": "",
"apiURL": ""
},
"gemini": { "apiKey": "" },
// Meta Model API (Muse Spark). `muse-spark-1.2-contributor`
// trades a ~12x discount for Meta training on its prompts and
// completions, so it is only served to callers that name it.
"meta": { "apiKey": "" },
"groq": { "apiKey": "" },
"deepseek": { "apiKey": "" },
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
},
"openrouter": {
"apiKey": "",
"apiBaseUrl": "https://openrouter.ai/api/v1"
},
"infron": {
"apiKey": "",
"apiBaseUrl": "https://llm.onerouter.pro/v1"
},
// BytePlus ModelArk. One key powers chat (Seed/GLM/DeepSeek), image
// generation (Seedream) and video generation (Seedance); `apiBaseUrl`
// selects the region; see
// https://docs.byteplus.com/en/docs/ModelArk/1330310 for options.
"byteplus": {
"apiKey": "",
"apiBaseUrl": "https://ark.ap-southeast.bytepluses.com/api/v3"
},
"zai": { "apiKey": "" },
"alibaba": { "apiKey": "" },
"hoonify": {
"apiKey": "",
"apiBaseUrl": "https://api.hoonify.ai/v1"
},
"together-ai": { "apiKey": "" },
// Local Ollama. `enabled: false` skips the auto-probe at startup
// (otherwise Puter logs ECONNREFUSED on every boot when no Ollama
@@ -273,7 +478,6 @@
"xai-image-generation": { "apiKey": "" },
// ─ Video generation ─
"openai-video-generation": { "apiKey": "" },
"together-video-generation": { "apiKey": "" },
"gemini-video-generation": { "apiKey": "" },
@@ -290,6 +494,7 @@
"secret_key": "",
"region": "us-west-2"
},
"speechify": { "apiKey": "" },
"aws-textract": {
"access_key": "",
"secret_key": "",
@@ -347,6 +552,33 @@
],
// ── Metering ────────────────────────────────────────────────────────
// When true, all metering checks pass — no per-actor limits enforced.
"unlimitedMetering": false
// When true, every account resolves to an unlimited policy: usage is still
// recorded, but nothing is ever refused for lack of budget. This is the
// setting for a deployment with no way to buy more — without it, accounts
// are held to the free monthly allowance and start getting 402s from the AI
// surfaces, file transfers and KV once they pass it.
"unlimitedMetering": false,
// Whether an account that has spent its whole allowance is refused the
// operations that spend it — file transfers, KV calls. Recording is
// unaffected either way. `workers` extends the same refusal to
// worker-driven calls, which are exempt by default because a deployed
// worker has nowhere to surface a payment prompt. `subscriptions` covers
// the separate question of surfaces reserved for paid plans (the
// vendor-compatible AI endpoints): turn it off on a deployment with no
// paid plans, or every account is refused them.
// "meteringEnforcement": { "enabled": true, "workers": false, "subscriptions": true },
// Fleet-wide spend rate, in micro-cents per minute, past which metering
// raises the `metering:excessiveGlobalUsageRate` alarm. Omit it (the
// default) to leave the check off: the only useful value is a multiple of
// what this deployment's normal traffic costs, so it has to be measured
// rather than guessed, and a stale number here alarms on healthy growth.
// "maxGlobalUsagePerMinute": 200000000
// How many full months past its own a monthly usage record is kept before
// it expires. Defaults to 3; set to 0 to keep every month's records
// forever. Changing it, including to 0, applies to records first written
// after the change; records already stamped keep their expiry.
// "meteringRetentionMonths": 3
}
+145
View File
@@ -0,0 +1,145 @@
# Alarms
`clients.alarm` is the one way code reports that something is wrong. It
de-dupes by alarm id, counts occurrences, and routes each alarm to alert
transports by **severity**.
```ts
this.clients.alarm.create(
`metering_write_failed:${userUuid}`, // de-dupe key
'Metering write failed', // what a human reads
{ userUuid, appId, error }, // context fields
'info', // severity
);
```
## Severity is the routing decision
| Severity | Meaning | Goes to |
| ---------- | ------------------------------------------------------ | ----------- |
| `critical` | An unhandled server error. Someone gets woken up. | Pager |
| `error` | Same urgency as critical; prefer one of the other two. | Pager |
| `warning` | Worth a look today. Nobody is paged. | Pager (low) |
| `info` | A record of something expected-but-notable. | Chat |
Each transport declares the severity window it accepts, so the value a call
site passes is what decides where the alarm lands. The two windows don't
overlap by default: anything that pages lives in the paging system, and chat
is the record of what didn't. The bar for `critical` is
deliberately high: an unhandled 5xx out of the HTTP error handler is the main
thing that still pages. Anything a human can look at tomorrow is `warning`,
and anything that's just worth recording is `info`.
Omitting the severity takes `pager.defaultSeverity` (itself `critical`), so
pass one explicitly unless you really mean "page someone".
### Choosing one
- Did the server fail to do its job in a way nobody expected? → `critical`
- Is a background job, rate, or dependency degraded, or has an account spent
far past its credits? → `warning`
- Is this a user doing something notable (tripping an abuse heuristic)?
→ `info`
- Did one of *our own* limits reject a caller — a rate limit, a concurrency
cap, a quota? → don't alarm at all. The limit doing its job is not an
event; the 429 is the whole signal, and alarming on it only produces noise
proportional to traffic. An *upstream provider* rate-limiting us is the
opposite case and still alarms (`upstream_rate_limited`, `info`) — that one
is not something we chose. The exception is a free model: nothing is billed,
nothing is actionable, and the throttling is the price of the model, so the
AI chat driver marks those `noAlarm` and only paid models still record.
`noAlarm` is the code-level mute: an `HttpError` carrying it skips the
terminal gate entirely, no matter its status or code. It is for failures the
call site *already knows* are expected and traffic-proportional — reach for it
there, not as a way to quiet an alarm you haven't diagnosed
(`severityOverrides` below is the knob for that).
An extension whose signals are all one tier can default its own local
`raiseAlarm` helper to that tier instead of repeating it at every call site —
see [extensions/cronMonitor](../../../extensions/cronMonitor/index.js).
## One incident per occurrence, unless you say otherwise
Alarms always de-dupe *in process* — repeats of an id bump its occurrence
count rather than creating a second alarm. What that means for the pager is a
separate decision, and by default every occurrence opens its own PagerDuty
incident: two failed scans an hour apart are two things that happened, and
closing one shouldn't hide the other.
Pass `{ dedup: true }` as a fifth argument when repeats of the id really are
one recurring fault, and they collapse onto a single incident carrying the
occurrence count:
```ts
this.clients.alarm.create(alarmId, message, fields, 'critical', {
dedup: true,
});
```
The HTTP error handler uses it: its id is route + error signature, so a hot
loop of the same crash is one incident with N occurrences instead of N pages.
Dedup and Slack's repeat throttle mean a responder sees only the latest
occurrence, so the handler attaches the `HttpError`'s `fields` to the alarm as
`details` — put what a responder needs (which upstreams failed, and how)
there. Everything in `fields` is also returned to the client in the response
body, so it has to be safe to show the caller.
Reach for it anywhere else only when the id is that specific — otherwise a
per-request alarm can flood the pager.
## Configuration
Everything lives under `pager` in config (see
[config.template.jsonc](../config.template.jsonc) for the annotated version).
Both transports are off unless enabled, so a self-hosted node just logs
alarms to the console.
```jsonc
"pager": {
"defaultSeverity": "critical",
"severityOverrides": { "cronMonitor:*": "info" },
"pagerduty": { "enabled": true, "routingKey": "…", "minSeverity": "warning" },
"slack": {
"enabled": true,
"webhookUrl": "…",
"channel": "#alerts",
"minSeverity": "info",
"maxSeverity": "info",
"repeatThrottleMs": 900000,
},
}
```
Slack's `maxSeverity` defaults to `info` whenever PagerDuty is configured, and
to `critical` when it isn't — a node with only a webhook still sees
everything. Raise it to have chat mirror the paging tiers as well.
### Retiering without a deploy
`severityOverrides` is the escape hatch for an alarm that turns out to be
noisier or more serious than its call site assumed. Keys are alarm ids or a
prefix ending in `*`; the exact id beats a pattern, and the longest matching
prefix wins among patterns. Values are a severity, or `mute` to drop the
alarm before any transport sees it.
It is applied *after* the call site's severity and any known-error rule, so
config always has the last word.
### Repeat throttling
Two levels, and they stack.
The client itself backs off before a repeat is reported at all: every
occurrence is reported up to a small burst, then one per interval. A fault
recurring hundreds of times a second still counts every occurrence, but it
does not write a log line or build an alert payload for each one.
The count lasts only as long as the client remembers the alarm. One that
hasn't recurred for 24 hours is forgotten, and the client holds at most 1000
at once, dropping the least recently raised first. Either way the next
occurrence is a new alarm, with its count and burst starting over.
Below that, the chat transport won't repost the same alarm id within
`repeatThrottleMs` (default 15 minutes). The first occurrence always posts,
and the next one that gets through reports how many piled up in between —
so a hot loop reads as one message with a count, not a wall of them.
+115
View File
@@ -0,0 +1,115 @@
# Backend Architecture
Loosely inspired by the Controller–Service–Repository pattern with dependency injection. The backend is a stack of layers; each depends only on the layers beneath it and receives them through its constructor from `PuterServer` ([src/backend/server.ts](../src/backend/server.ts)), which instantiates the layers in order. The rules for working within the layers are in [AGENTS.md](../AGENTS.md#backend).
## Layers
```mermaid
block-beta
columns 1
REQ["HTTP request"]
CTRL["Controllers — route handlers, gates, I/O shaping"]
DRV["Drivers (optional) — RPC handlers on /drivers/*"]
SVC["Services — business logic, no auth"]
STR["Stores — persistence / domain shapes"]
CLI["Clients — sql, redis, s3, dynamo, email, …"]
CFG["Config — IConfig"]
REQ --> CTRL
CTRL --> DRV
DRV --> SVC
SVC --> STR
STR --> CLI
CLI --> CFG
style REQ fill:#0ea5e9,stroke:#0369a1,color:#fff
style CTRL fill:#1d4ed8,stroke:#1e3a8a,color:#fff
style DRV fill:#2563eb,stroke:#1e40af,color:#fff
style SVC fill:#4f46e5,stroke:#3730a3,color:#fff
style STR fill:#7c3aed,stroke:#5b21b6,color:#fff
style CLI fill:#9333ea,stroke:#6b21a8,color:#fff
style CFG fill:#334155,stroke:#1e293b,color:#fff
```
| Layer | Lives in | Responsibility |
| --- | --- | --- |
| **Controllers** | [src/backend/controllers/](../src/backend/controllers/) | Route handlers. Parse + validate input, apply per-route gates (auth, subdomain, rate limit, body parsers — see `RouteOptions`), call into services, format responses. |
| **Drivers** | [src/backend/drivers/](../src/backend/drivers/) | Optional. RPC-style handlers exposed over the `/drivers/*` surface (`puter-kvstore`, `puter-chat-completion`, …). A driver is a thin shell that validates RPC inputs and calls into services/stores; controllers can hold a typed reference to drivers when they need the same logic over HTTP. |
| **Services** | [src/backend/services/](../src/backend/services/) | Business logic. Assume the caller is already authenticated/authorized — services do not run auth gates themselves. |
| **Stores** | [src/backend/stores/](../src/backend/stores/) | Persistence and storage logic. Wraps clients with the domain shape services consume (rows, entities, KV namespaces). |
| **Clients** | [src/backend/clients/](../src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, dynamodb, email, event bus, …). Knows protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | The flat, typed config object every layer receives at construction. |
Extensions sit alongside this stack and can register into any layer; see [Extensions](#extensions).
## Entry point: `PuterServer`
`PuterServer` is the bootstrap. It:
1. Loads any configured extension directories (`config.extensions`) so extensions can register before instantiation begins.
2. Instantiates each layer in order — clients → stores → services → drivers → controllers — merging in anything extensions have registered for that layer.
3. Wires global middleware, mounts controller routes through `PuterRouter` (which translates `RouteOptions` into the gate/parser middleware chain), and mounts extension routes through the same materializer.
4. Fires `onServerStart` hooks across every layer once HTTP is listening, and `onServerPrepareShutdown` / `onServerShutdown` on the way down.
## Context (ALS)
[`Context`](../src/backend/core/context.ts), backed by `AsyncLocalStorage`, carries per-request state without threading it through every function signature. The request-context middleware opens a scope per request after the auth probe runs, so anything inside a request handler can call `Context.get('actor')` / `Context.get('req')`. It's used sparingly, mostly for `actor` and `req`; see the [Context rule](../AGENTS.md#backend).
## Actors
An [`Actor`](../src/backend/core/actor.ts) is who a request is acting as. Its two app fields are not interchangeable: `app` is the app the actor carries *directly* (an app-under-user token), while `effectiveApp` is the app it ultimately acts as — its own, or the one that issued its access token. An access-token actor has no `app` of its own, so a gate that reads `app` answers "no app" for it and falls open.
`makeActor` resolves `effectiveApp` once at construction, and `assertResolvedActor` at the request edge rejects any actor that skipped it — so reading `effectiveApp` needs no fallback. Reach for `app` only where the direct app genuinely is the question, and say why in a comment.
- `actor.effectiveApp` — which app is acting. This is the default; nearly every scoping rule, namespace key and attribution wants it.
- `isAppActor(actor)` — a direct app actor specifically. Use it where a token must *not* be treated as its issuer, e.g. the permission scanners that read an app's own grant rows.
- `isPlainUserActor(actor)` — the account acting through nothing at all; `isAccountContext(actor)` for the wider "the account's own reach", which also admits a full-access token.
## Extensions
Extensions live in [extensions/](../extensions/), parallel the layered stack, and hold the parts of the system Puter still works without. [AGENTS.md](../AGENTS.md#extensions) has the rule for what belongs in one.
- **Good extensions**: [thumbnails](../extensions/thumbnails.ts), [serverInfo](../extensions/serverInfo.ts), [devWatcher](../extensions/devWatcher.ts) — opt-in features cleanly bolted on.
- **Should probably be core**: [metering](../extensions/metering.ts), [appTelemetry](../extensions/appTelemetry.ts) — clients now expect these to be present, so the "extension" framing is misleading.
- **Shouldn't have been an extension**: [whoami](../extensions/whoami.ts) — it's load-bearing for every authenticated client.
### Extension API
The `extension` global ([src/backend/extensions.ts](../src/backend/extensions.ts)) exposes:
- **Layer registration** for first-class additions:
- `extension.registerClient(name, ClientClass)`
- `extension.registerStore(name, StoreClass)`
- `extension.registerService(name, ServiceClass)`
- `extension.registerDriver(name, DriverClass)`
- `extension.registerController(name, ControllerClass)`
- **Lightweight wrappers** for the common case where a full class isn't worth it:
- `extension.on(event, handler)` — subscribe to event-bus events.
- `extension.get(path, opts?, handler)` / `.post` / `.put` / `.delete` / `.patch` / `.head` / `.options` / `.all` / `.use` — register routes. The `opts` shape is the same `RouteOptions` controllers use, so `subdomain`, `requireAuth`, `adminOnly`, body parsers, etc. all work identically.
- **Cross-layer access**: `extension.import('client' | 'store' | 'service' | 'controller' | 'driver')` returns a lazy proxy to instantiated objects, and `extension.config` exposes the live config.
```ts
import { extension } from '@heyputer/backend/src/extensions';
const services = extension.import('service');
extension.get('/healthcheck/deep', { subdomain: 'api', adminOnly: true }, async (_req, res) => {
res.json(await services.health.getStatus());
});
extension.on('puter.signup.success', (_key, data) => {
console.log('new user', data.username);
});
```
The `app.recommended` event lets extensions customize desktop recommendations.
Its `appNames` array starts as a fresh copy of the default ordered names on each
call. Listeners may mutate or replace it, including setting it to an empty array.
Async listeners are awaited before names are resolved to app summaries; names
that do not exist are skipped.
```ts
extension.on('app.recommended', (_key, data) => {
data.appNames = ['editor', 'camera'];
});
```
+100
View File
@@ -0,0 +1,100 @@
# Contributing APIs
How to add a new public API to Puter, and how to maintain one that already exists. "Public API" means anything applications can call: HTTP endpoints, `/drivers/*` methods, and the puter.js methods that wrap them. This guide is written for human contributors and AI agents alike — [AGENTS.md](../AGENTS.md) defers to it for API work.
Companion docs: [architecture.md](architecture.md) (backend layering), [pagination.md](pagination.md) (list APIs), [src/puter-js/tests/api/README.md](../src/puter-js/tests/api/README.md) (SDK test environment).
## Rule zero: don't break callers
puter.js is served live from `https://js.puter.com/v2/` with no version pinning — every app in existence picks up your change the moment it deploys, and the backend endpoints underneath have the same property. Assume every observable behavior (parameter handling, response fields, error codes, ordering) has someone depending on it.
Every change is backward compatible unless a maintainer has explicitly agreed to a break beforehand.
## Core or extension?
The first decision is where the API lives. If **nothing in core will call it, prefer an extension** over wiring it into core. [AGENTS.md](../AGENTS.md#extensions) has the rule and its test; [architecture.md](architecture.md#extension-api) documents the `extension` API.
## Adding a new API
Work through all seven steps; the PR is complete when every one is.
### 1. Design the surface first
- Sketch the signature, options, return shape, and error cases before writing code. Find the two or three most similar existing APIs and match their conventions.
- New parameter and field names follow the [naming rule](../AGENTS.md#language--files) (`camelCase`).
- Anything returning a list follows the [pagination convention](pagination.md): `limit`/`cursor` in, `{ items, cursor, total }` envelope out.
- Prefer an options object over a growing list of positional parameters, but keep the common case callable with a single argument where siblings do.
### 2. Backend
Follow the layered stack ([architecture.md](architecture.md), [layer rules](../AGENTS.md#backend)): a controller or driver at the edge, business logic in a service, persistence in a store. Return exactly what the caller needs and no more — every response field you ship is permanent — and use stable `snake_case` error codes.
#### Controller or driver?
Both are supported ways to define an API. **Prefer a controller when you need fine-grained control** — URL shape, HTTP verbs, per-route gates, response and streaming formats. A **driver** fits when the API is a set of RPC methods implementing one of the named interfaces on `/drivers/*` (`puter-kvstore`, `puter-chat-completion`, …) and the generic driver plumbing — one call envelope, interchangeable implementations, per-method policies — covers what you need.
- **Controller:** extend `PuterController` ([src/backend/controllers/types.ts](../src/backend/controllers/types.ts)) and declare routes with the `@Controller(prefix)` class decorator plus `@Get`/`@Post`/… method decorators ([src/backend/core/http/decorators.ts](../src/backend/core/http/decorators.ts)), each taking `(path, routeOptions)` — or override `registerRoutes(router)` imperatively. Core controllers register in [src/backend/controllers/index.ts](../src/backend/controllers/index.ts); extensions use `extension.registerController(...)` or the plain route helpers.
- **Driver:** extend `PuterDriver` ([src/backend/drivers/types.ts](../src/backend/drivers/types.ts)) and mark it with the `@Driver(interfaceName, opts)` decorator ([src/backend/drivers/decorators.ts](../src/backend/drivers/decorators.ts)), which also declares its policies. Core drivers register in [src/backend/drivers/index.ts](../src/backend/drivers/index.ts); extensions use `extension.registerDriver(...)`.
#### Middleware and gates
- **Controller routes** (and extension routes — same options) take [`RouteOptions`](../src/backend/core/http/types.ts): auth gates (`requireAuth`, `requireUserActor`, `noUserSession`, `adminOnly`, `allowedAppIds`, and the access-token controls), `subdomain` routing, per-route `rateLimit`, body parsers, and arbitrary extra `middleware`. The auth flavors are subtle and default-deny — read the JSDoc on each field before picking.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), `requireSubscription`, `requireReputation`, and `noUserSession`. The `/drivers/call` surface enforces them.
- **Paying accounts only:** `requireSubscription`, as a route option on a controller endpoint or in the per-method `@Driver({ requireSubscription })` block on a driver (`/drivers/call` is one shared route, so a driver's requirement can't live in route options). `true` accepts any plan that isn't free, so a plan an extension registers counts without core naming it; an array of policy ids (`['business', 'pro']`) accepts only those; `false` or leaving it out means no requirement. An empty array is a boot error, since it would read as subscribers-only while admitting everyone. The check is a map lookup on the metering service's cached per-actor subscription. It asks which plan the account is on; `requireCredits` asks whether it has budget left. Deployments with no paid plans (self-hosted installs) turn it off with `meteringEnforcement.subscriptions: false`.
- **Trusted-enough accounts only:** `requireReputation`, as a route option or in the per-method `@Driver({ requireReputation })` block. Opt-in; implies `requireAuth`. The value names a tier, and `reputationGate.tiers` in config sets the score that tier takes, so a deployment can retune it without editing the surface. A tier the running config doesn't define is inert and everyone passes, so an install that doesn't score its accounts never turns traffic away on a score it never computed. `reputationGate.enabled: false` stops every declared gate at once. Denials are a bare 403 `reputation_required` that names neither the score nor the tier. Nothing in the tree declares one yet.
- **A specific verified factor:** `requireVerified` (email, and only under `strict_email_verification_required`), `requirePhoneVerified`, `requireCardVerified`. Opt-in, and the factor must actually have been verified, unlike the default-on gate that turns away accounts still carrying a pending verification the abuse harness asked for. `requireAnyVerified: ['phone', 'card']` is the OR of the last two: any listed factor verified at any point passes, as does a paid plan when `card` is listed (a paying account already has a card on file). Otherwise only the factors the deployment can verify are asked for (an SMS provider configured, a card gate an extension reports on); with none verifiable the gate is inert rather than locking the route on a self-hosted install. A denial carries the first verifiable factor's code plus `factors` (the verifiable ones, in the route's order), so a client can lead with one flow and offer the other. `verifiedFactorGate.enabled: false` in config stops every declared gate at once, e.g. during an SMS-provider outage.
- **Metered spend:** an endpoint that spends metered resources on the caller's behalf (moving file content, object-store requests, anything else the account is billed for) declares `requireCredits: true`, which turns an account with no budget left away with a 402 before the handler runs. Endpoints that only describe or delete things deliberately don't: an account that has run out still has to see what it has, clear it, and reach its billing pages. Drivers have no route options, so they call `assertActorHasCredits` themselves ([src/backend/services/metering/enforcement.ts](../src/backend/services/metering/enforcement.ts)) — see `KVStoreDriver`, which does it once for every method.
### 3. puter.js
- Add the method to the matching module in [src/puter-js/src/modules/](../src/puter-js/src/modules/), matching the calling conventions of its siblings (promise-returning; positional shortcut plus options form where that's the local pattern).
- Validate cheap preconditions client-side and throw `{ message, code }` objects; pass backend errors through unchanged rather than swallowing or re-wrapping them.
### 4. Types
Type the method in JSDoc on the implementation, following the [puter.js type rules](../AGENTS.md#types) (overloads, where shapes live, `index.d.ts`), then run `npm run check:puterjs:types`. Declarations are generated from the JSDoc, so there is nothing to keep in sync by hand.
### 5. Docs
- Add the method page at [src/docs/src/](../src/docs/src/)`<Area>/<method>.md` — frontmatter (`title`, `description`, `platforms`), syntax, parameters, return value, and at least one runnable example — and update the area overview (`<Area>.md`). Copy the structure of an existing page.
- The docs are the contract users code against: signatures, defaults, and return shapes match the implementation exactly.
### 6. Tests
- **Backend:** see [backend tests](../AGENTS.md#backend-tests).
- **SDK:** add cases to [src/puter-js/tests/api/suites/](../src/puter-js/tests/api/suites/)`<area>.suite.ts` (register new suites in `suites/index.ts`; there is no globbing). One suite runs on node, browser, and workerd via `npm run test:puterjs`; never write per-platform tests. The runners execute the built bundle, so run `npm run build:workerLib` first or the suite silently tests stale code.
- **Desktop-rendered UI** (`puter.ui.*`): add a Playwright spec per [src/puter-js/TESTING.md](../src/puter-js/TESTING.md).
### 7. Security pass
Run the [security & privacy](../AGENTS.md#security--privacy) check on the diff, and confirm the auth gates are present.
## Maintaining an existing API
Changes are **additive by default**:
- Existing call signatures keep working, including both positional and options-object forms where a method supports them. Say in the PR how existing callers are unaffected.
- New parameters are optional, with defaults that reproduce the old behavior exactly.
- Never rename, repurpose, or remove existing parameters, response fields, or error codes. Don't change types, ordering guarantees, or which fields are present when.
- New behavior that could surprise existing callers goes behind an opt-in flag.
- Docs, types, and tests move in the same PR as the behavior; a signature change with stale docs is a bug.
- Bug fixes come with a regression test that fails before the fix. Be suspicious of fixes that change observable behavior: someone may depend on the bug. When in doubt, ask a maintainer.
### Breaking changes
Rare and deliberate. In order: explicit maintainer sign-off, a documented migration path, and a rollout plan — typically new surface added first, old surface deprecated, old surface removed much later, if ever. Never break as a side effect of a refactor.
### Deprecating
The old surface keeps working. Mark it `@deprecated` in the type declarations, note the replacement on its docs page, and stop using it in examples. Removal is a separate, maintainer-approved decision.
## Definition of done
- [ ] Backward compatible (or the break was explicitly approved)
- [ ] Right home: extension if core never calls it; layered structure either way
- [ ] puter.js method matches sibling conventions; errors are `{ message, code }` with stable codes
- [ ] Types updated and matching runtime behavior
- [ ] Docs page + area overview updated, with a runnable example
- [ ] Tests: backend + three-platform SDK suite (+ e2e for desktop-rendered UI)
- [ ] Security pass on the diff
- [ ] You ran it end to end
+1 -1
View File
@@ -113,7 +113,7 @@ Terhubung dengan maintainer dan komunitas melalui saluran-saluran berikut:
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
- Mastodon: [mastodon.social/@puter](https://mastodon.social/@puter)
- Isu keamanan? [[email protected]](mailto:[email protected])
- Email maintainers di [[email protected]](mailto:[email protected])
- Email pengelola di [[email protected]](mailto:[email protected])
Kami selalu senang membantu Anda dengan pertanyaan apa pun yang Anda miliki. Jangan ragu untuk bertanya!
+1 -1
View File
@@ -104,7 +104,7 @@ Puterは[**puter.com**](https://puter.com)でホストサービスとして利
メンテナーやコミュニティと以下のチャンネルを通じてつながりましょう:
- バグ報告や機能リクエストがありますか? [issueを開く](https://github.com/HeyPuter/puter/issues/new/choose) してください。
- バグ報告や機能リクエストは [新規Issue](https://github.com/HeyPuter/puter/issues/new/choose) まで
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
+101
View File
@@ -0,0 +1,101 @@
# Pagination convention for list APIs
Every list endpoint follows one wire contract. This document is the source of
truth for adding pagination to a new or existing list surface.
## Wire contract
Requests accept:
| Param | Type | Meaning |
| --- | --- | --- |
| `limit` | number | Maximum items per page. Each endpoint documents its cap and default. |
| `cursor` | string \| null | Opaque continuation token. `null` (or any presence of the key) requests the first page. |
| `offset` | number | Skip N items. Legacy/discouraged — see below. Cannot be combined with `cursor`. |
| `includeTotal` | boolean | Adds `total` to the response. |
Paginated responses are an envelope:
```json
{ "items": [...], "cursor": "…", "total": 123 }
```
- `cursor` is present only while more pages exist. Clients iterate until it
is absent.
- `total` is present only when the request set `includeTotal`.
- **Pages may be short.** Post-query filtering (TTL expiry, permission
checks) can shrink a page below `limit` — or even to zero — while a
`cursor` is still returned. Never use `items.length < limit` as an
end-of-list signal.
## Backward compatibility
Requests without pagination params keep returning the full result in the
legacy shape (bare array) forever — old clients never break.
The envelope trigger depends on the endpoint's history:
- Endpoints where `limit` pre-dates the convention (readdir, subdomain
`select`) return the envelope only when the request contains `cursor`
(including `null`) or `includeTotal`; `limit`/`offset`-only requests keep
the bare array.
- Endpoints where every pagination param is new (kv `list`, workers
`getFilePaths`) return the envelope when any pagination param is present.
## Cursors
Cursors are opaque base64-encoded JSON, produced and consumed only by the
backend (`src/backend/util/pagination.ts`). A cursor over a global sequence
(an auto-increment `id`) is sealed with `sealCursor`/`openCursor` instead, so
two pages can't be decoded into how fast the table grows; `openCursor` still
reads a plain cursor. What a cursor wraps is an implementation detail per
store:
- DynamoDB-backed lists wrap `LastEvaluatedKey`.
- SQL-backed lists wrap a keyset position — `(sortValue, id)` of the last
row — and the query seeks past it (`WHERE (col, id) > (?, ?) ORDER BY col,
id`). Any SQL list gaining a cursor must have a deterministic `ORDER BY`
ending in a unique tiebreaker (`id`).
Cursors that carry a sort also pin it: a request that passes a cursor plus a
conflicting sort is rejected with 400.
## Offset
SQL-backed endpoints support `offset` natively. DynamoDB-backed endpoints
(kv) emulate it by advancing past skipped items with `Select: COUNT` queries
— no item data is transferred, but read capacity is still consumed for
everything skipped, and the caller is metered for it. **Offset is supported
for parity, not recommended** — cost grows linearly with the offset, so it is
capped (kv: 5000). Use cursors.
## Totals (`includeTotal`)
- SQL: `SELECT COUNT(*)` with the same WHERE clause as the listing.
- DynamoDB: a `Select: COUNT` loop over the query (TTL-filtered), metered to
the caller. Cost is proportional to the total item count — request the
total on the first page only, not every page.
- Where visibility is decided per-actor after the query (protected apps in
the catalog listing), `total` approximates the visible set: it counts
non-protected plus caller-owned rows, and misses rows visible only through
explicit permission grants.
## fetchUntilFull (kv only)
DynamoDB applies `Limit` before its filter expression, so TTL-filtered pages
are structurally short. `fetchUntilFull: true` makes the backend keep
fetching (bounded number of continuation queries) until the page holds
`limit` items or the keyset is exhausted. Requires `limit`. If the bound is
hit, the response simply carries a cursor — still convention-legal.
## Adding pagination to a new endpoint
1. Use `encodeCursor`/`decodeCursor`/`normalizeLimit`/`normalizeOffset` from
`src/backend/util/pagination.ts` — `sealCursor`/`openCursor` when the
cursor wraps a global id.
2. Push equality filters into the query so pages and counts operate on the
true result set; only genuinely per-actor filtering may remain post-query
(short-pages rule covers it).
3. Fetch `limit + 1` rows to detect whether a next page exists (SQL), or use
`LastEvaluatedKey` (DynamoDB).
4. Keep the no-params request returning the legacy full result.
+311 -31
View File
@@ -1,6 +1,6 @@
# Self-Hosting Puter
`docker-compose.yml` brings up Puter **plus every external service it needs** — MariaDB, Valkey, DynamoDB-local, RustFS S3, nginx — wired together. Closest thing to a production deployment you can self-manage on a single host.
`docker-compose.yml` brings up Puter **plus every external service it needs** — MariaDB, Valkey, DynamoDB-local, RustFS S3, Caddy — wired together. Closest thing to a production deployment you can self-manage on a single host.
## One-line installer (recommended)
@@ -8,19 +8,19 @@
curl -fsSL https://raw.githubusercontent.com/HeyPuter/puter/main/install.sh | sh
```
Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker-compose.yml` from the OSS repo, and runs `docker compose up -d`. Re-running is safe — it won't overwrite existing config (set `PUTER_FORCE=1` to rotate). Use the manual steps below if you want to inspect or tweak each step yourself.
Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker-compose.yml` + `caddy/Caddyfile` from the OSS repo, and runs `docker compose up -d`. Re-running is safe — it won't overwrite existing config (set `PUTER_FORCE=1` to rotate). Use the manual steps below if you want to inspect or tweak each step yourself.
## Requirements
- **Docker** with the `compose` plugin.
- A **domain** with DNS access — you need a wildcard record (`*.your-domain.com` → server IP). Puter routes by subdomain (`api.<domain>`, `site.<domain>`, `app.<domain>`).
- Optional: **TLS certs** (or `certbot` to grab them — see Step 4).
- Optional: **TLS certs** (or `certbot` to grab them — see Step 3).
## What's running
| Container | Image | Role |
| --------------- | ------------------------ | ---------------------------------------------------------- |
| `puter-nginx` | `nginx:1.27-alpine` | Reverse proxy on 80 (and 443 if TLS); forwards to Puter |
| `puter-caddy` | `caddy:2.11-alpine` | Reverse proxy on 80 (and 443 if TLS); forwards to Puter |
| `puter` | `ghcr.io/heyputer/puter` | The app |
| `puter-mariadb` | `mariadb:11` | SQL database — schema applied automatically on first boot |
| `puter-valkey` | `valkey/valkey:8-alpine` | Redis-compatible cache + rate-limiter |
@@ -35,6 +35,12 @@ Optional services (compose profile `ai`, opt-in):
| `puter-ollama` | `ollama/ollama` | Local LLM provider (CPU; GPU passthrough opt-in) |
| `puter-ollama-init` | `ollama/ollama` | One-shot — pulls the default model (`tinyllama`) on first boot |
Optional services (compose profile `smtp`, opt-in):
| Container | Image | Role |
| ------------ | ------------------------ | -------------------------------------------------------- |
| `puter-smtp` | `ghcr.io/heyputer/puter` | Receives mail over SMTP and files it into user mailboxes |
State lives under `./puter/data/<service>/`.
---
@@ -47,7 +53,7 @@ State lives under `./puter/data/<service>/`.
MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
cat > .env <<EOF
@@ -64,7 +70,7 @@ S3_SECRET_KEY=$S3_SECRET_KEY
S3_BUCKET=puter-local
EOF
mkdir -p puter/config puter/data puter/tls
mkdir -p puter/config puter/data puter/data/caddy puter/tls
cat > puter/config/config.json <<EOF
{
"domain": "puter.localhost",
@@ -77,7 +83,7 @@ cat > puter/config/config.json <<EOF
"private_app_hosting_domain": "app.puter.localhost",
"private_app_hosting_domain_alt": "dev.puter.localhost",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
@@ -122,6 +128,10 @@ cat > puter/config/config.json <<EOF
"ollama": { "enabled": false }
},
"meteringEnforcement": {
"subscriptions": false
},
"trust_proxy": 1
}
EOF
@@ -131,20 +141,32 @@ Replace `puter.localhost`, `site.puter.localhost`, `host.puter.localhost`, `dev.
Why these knobs:
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start=gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL schema on boot. `mysql_mig_1.sql` (tables) and `mysql_mig_2.sql` (default apps: editor, viewer, pdf, camera, player, recorder, git, dev-center, puter-linux). Idempotent — safe to re-run.
- `jwt_secret_v2` — the HMAC secret Puter signs and verifies auth tokens with (`kid: 'v2'` JWT header). The pre-v2 token format is retired: a token signed with the old `jwt_secret` no longer verifies, and holders are asked to sign in again. If you are upgrading from a release that had `jwt_secret`, drop it from your config — it is ignored.
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start:gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
- `meteringEnforcement.subscriptions: false` — a few surfaces (the OpenAI/Anthropic-compatible AI endpoints) are declared paid-plan-only. A self-hosted install has no paid plans, so every account sits on the free policy and those routes would answer `402 subscription_required` for everyone. Turning the plan gates off opens them; nothing else about metering changes.
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
- `s3.s3Config.forcePathStyle: true` — RustFS / MinIO / fauxqs need path-style URLs (`<endpoint>/<bucket>`). Real AWS S3 wants virtual-hosted (`<bucket>.<endpoint>`) — drop this flag (or set `false`) when you swap to real S3.
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. nginx routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
- `trust_proxy: 1` — nginx terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the nginx container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (nginx). Bump to `2` if you put Cloudflare in front of nginx; never set `true` (it trusts every hop and makes XFF forgeable).
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. Caddy routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
- **Bucket CORS** — the browser uploads file bytes straight to RustFS with a cross-origin `PUT` to the presigned URL (Dev Center deploy, `puter.fs.upload`). That only works if the bucket answers the `OPTIONS` preflight with `Access-Control-Allow-*` headers. The `s3-init` container applies those rules on every boot (`put-bucket-cors`, origins `*` — the signed URL is the credential, and the SDK sends no cookies). If deploys fail with `No 'Access-Control-Allow-Origin' header is present`, re-run it: `docker compose run --rm s3-init`. Verify with:
```bash
curl -sk -X OPTIONS "https://s3.<domain>/<bucket>/test" \
-H "Origin: https://<domain>" \
-H "Access-Control-Request-Method: PUT" \
-H "Access-Control-Request-Headers: content-type" -D - -o /dev/null | grep -i access-control
```
You should see `access-control-allow-origin`, `allow-methods: GET, HEAD, PUT, POST, DELETE`, `allow-headers: *`, and `max-age: 3600`.
- `trust_proxy: 1` — Caddy terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the Caddy container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (Caddy). Bump to `2` if you put Cloudflare in front of Caddy; never set `true` (it trusts every hop and makes XFF forgeable).
> If you ever change `MARIADB_PASSWORD` after first boot, `.env` alone won't update MariaDB — its credentials are baked into `./puter/data/mariadb/` on first init. Either rotate the password inside MariaDB by hand or `docker compose down && rm -rf ./puter/data/mariadb` to start fresh.
## Step 2 — Point DNS at the server \[Optional\]
In your DNS provider, add records for the main domain plus the subdomains Puter and nginx route on (`api.*`, `site.*`, `app.*`, `s3.*`):
In your DNS provider, add records for the main domain plus the subdomains Puter and Caddy route on (`api.*`, `site.*`, `app.*`, `s3.*`):
```
A puter.localhost → <your server's public IP>
@@ -159,7 +181,7 @@ A dev.puter.localhost → <your server's public IP>
A *.dev.puter.localhost → <your server's public IP>
```
The wildcards are required — Puter routes via subdomains (`api.*`, `app.*`, etc.) and nginx routes browser S3 traffic via `s3.*` to RustFS.
The wildcards are required — Puter routes via subdomains (`api.*`, `app.*`, etc.) and Caddy routes browser S3 traffic via `s3.*` to RustFS.
## Step 3 — TLS (recommended for public installs) \[Optional\]
@@ -178,13 +200,15 @@ sudo certbot certonly --manual --preferred-challenges dns \
The cert needs to cover `*.puter.localhost` so that `s3.puter.localhost` (browser S3 endpoint), plus Puter's own `api.*` / `app.*` subdomains, all validate.
> **Why not Caddy's automatic HTTPS?** Caddy issues certs over ACME by itself, but only for hostnames it knows up front, and only over HTTP-01 with the stock image. Puter serves every user's site and app on a subdomain it invents at runtime (`<name>.site.<domain>`, `<name>.app.<domain>`), which needs a **wildcard** cert — and wildcards require DNS-01, which requires a DNS-provider plugin that isn't in `caddy:2.11-alpine`. So `caddy/Caddyfile` sets `auto_https off` and reads the cert you supply below. If you'd rather have Caddy manage certs, build an image with the plugin for your DNS provider (`xcaddy build --with github.com/caddy-dns/<provider>`) and swap the `tls` line for a `tls { dns <provider> … }` block.
Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
**Wire nginx to use them:**
**Wire Caddy to use them:**
1. Open [nginx/nginx.conf](../nginx/nginx.conf), uncomment **both** `# server { listen 443 ssl … }` blocks (one for `s3.*`, one for the catch-all).
2. (Optional) Replace the body of the port-80 blocks with `return 301 https://$host$request_uri;` to force HTTPS everywhere.
3. In [docker-compose.yml](../docker-compose.yml), uncomment the `443:443` port mapping under the `nginx` service.
1. Open [caddy/Caddyfile](../caddy/Caddyfile) and uncomment the `# :443 { … }` block at the bottom.
2. (Optional but recommended) Replace the plain `:80 { import puter_routes }` block with the `redir` version shown alongside it, to force HTTPS everywhere.
3. In [docker-compose.yml](../docker-compose.yml), uncomment the `443:443` port mapping under the `caddy` service.
4. In `.env`, uncomment `HTTPS_PORT=443`.
5. In `config.json`, switch:
```json
@@ -192,11 +216,39 @@ Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
```
…and update the S3 public endpoint:
```json
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.local", ... } }
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.localhost", ... } }
```
## Running behind your own reverse proxy
The bundled `puter-caddy` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Traefik, nginx, HAProxy, a cloud load balancer, another Caddy). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
You can keep the bundled Caddy as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
**The rules that actually matter** (getting any of these wrong is what causes the redirect loops and "Invalid Host header" failures people hit):
1. **Don't rewrite the `Host` header.** Puter routes entirely on Host — `api.<domain>`, `site.<domain>`, `app.<domain>` are all distinguished by the incoming host. Forward the original host through unchanged. Do **not** rewrite external hostnames to an internal name like `puter.local`; that forces you to remap every subdomain by hand and still breaks signed-URL and CORS checks. Most proxies preserve Host by default — just don't override it.
2. **The external domain must equal `domain` in your `config.json`.** If users reach the box at `puter.example.com`, then `domain` must be `puter.example.com` and the hosting domains must be its real subdomains (`site.puter.example.com`, `app.puter.example.com`, …) — exactly as the installer/`config.json` lays them out. Puter rejects hosts it doesn't recognize with `Invalid Host header`.
3. **Set `protocol` to match the public scheme.** When your proxy terminates TLS, set `"protocol": "https"` (installer: `PUTER_PROTOCOL=https`). Puter builds its origins, redirects, signed S3 URLs, and OIDC callback URLs from this — leave it `http` behind an HTTPS proxy and you get redirect loops, broken logins, and mixed-content errors.
4. **Forward the standard proxy headers.** Puter needs:
- `Host` — the original request host (rule 1).
- `X-Forwarded-Proto` — the **external** scheme (`https`), so Puter knows TLS was terminated upstream.
- `X-Forwarded-For` — the real client IP (rate limiting + audit logs).
- `Upgrade` / `Connection` — passed through for WebSocket / socket.io upgrades, or the realtime connection silently fails.
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled Caddy if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
6. **Route the wildcard to Puter.** Your proxy must forward `*.<domain>` (covering `api.*`, `app.*`, and `s3.<domain>`) and `*.site.<domain>` (and any other hosting domains) to Puter — same wildcard DNS as Step 2. Puter does the per-subdomain routing internally; the proxy just needs to hand it the traffic with the Host intact.
With those in place the rest of the stack is unchanged — `docker compose up -d` as below.
## Step 4 — Bring it up
Both [docker-compose.yml](../docker-compose.yml) and [caddy/Caddyfile](../caddy/Caddyfile) need to sit next to your `.env` — clone the repo, or copy those two paths out of it. The Caddyfile is bind-mounted read-only; if it's missing, Docker creates a *directory* at that path and `puter-caddy` dies with "not a directory".
```bash
docker compose up -d
```
@@ -211,15 +263,15 @@ Healthy startup:
```
[config] override from /etc/puter/config.json
[mysql] running migrations from /opt/puter/dist/src/backend/clients/database/migrations/mysql: 2 file(s)
[mysql] running migrations from /opt/puter/dist/src/backend/clients/database/migrations/mysql: <n> file(s)
[mysql] applied mysql_mig_1.sql (...)
[mysql] applied mysql_mig_2.sql (9 statements)
...
```
Then open **<https://puter.local>** (or `http://` if you skipped TLS). Login is `admin` — the temp password is printed once in the puter container logs on first boot:
Then open **<https://puter.localhost>** (or `http://` if you skipped TLS). Login is `admin`; its temporary password is printed in the puter container logs on every boot until you change it:
```bash
docker compose logs puter | grep tmp_password
docker compose logs puter | grep "password for admin"
```
Change it in Settings after first login.
@@ -228,6 +280,26 @@ Change it in Settings after first login.
All optional. Drop any of the blocks below into `puter/config/config.json` and `docker compose restart puter`. See [config.template.jsonc](../config.template.jsonc) for the full list. Per-key documentation lives in [src/backend/types.ts](../src/backend/types.ts).
### PostgreSQL database
> **Community contribution — expect rough edges.** PostgreSQL support was contributed by the community and is not run by Puter.com production. It boots, applies the bundled schema, and exercises the common user/app/fsentry/session/permission/OIDC flows in the integration tests, but less-traveled SQL call sites may still need porting. If you hit a query that doesn't work on Postgres, please open an issue. For production self-hosting today, MariaDB/MySQL and SQLite are the supported defaults.
The bundled Docker Compose stack still defaults to MariaDB. To use PostgreSQL instead, run a PostgreSQL service yourself, point Puter at it, and use the PostgreSQL migration path:
```json
"database": {
"engine": "postgres",
"host": "postgres",
"port": 5432,
"user": "puter",
"password": "...",
"database": "puter",
"migrationPaths": ["/opt/puter/dist/src/backend/clients/database/migrations/postgres"]
}
```
You may use `"connectionString": "postgres://puter:...@postgres:5432/puter"` instead of the host/user/password fields. Puter only needs normal PostgreSQL connection details and the migration path; CloudNativePG is one possible Kubernetes operator, but no operator-specific manifests are required by Puter.
### Email (SMTP)
Used for password resets, email confirmation, and notifications. Without it those flows silently fail.
@@ -244,6 +316,18 @@ Used for password resets, email confirmation, and notifications. Without it thos
To require email confirmation before login, also set `"strict_email_verification_required": true`.
To read the mail yourself instead of sending it, point the transport at a local SMTP sink — [MailHog](https://github.com/mailhog/MailHog) catches everything and serves it at `http://localhost:8025`. Publish both ports; a container that only exposes them is unreachable from the host:
```sh
docker run -d -p 1025:1025 -p 8025:8025 mailhog/mailhog
```
```json
"email": { "from": "\"Puter\" <[email protected]>", "host": "127.0.0.1", "port": 1025, "secure": false, "ignoreTLS": true }
```
Share email is on once a transport exists: a recipient who already has an account is emailed as well as notified in-app, and they opt out with the unsubscribe link the mail carries. Set `"share_email_notifications": false` to keep those in the app only — an address with no account is still emailed, since there is nothing else to reach them with.
### Sign in with Google (or another OIDC provider)
```json
@@ -260,6 +344,39 @@ To require email confirmation before login, also set `"strict_email_verification
Add `https://puter.<your-domain>/auth/oidc/callback/login` to the OAuth client's authorized redirect URIs in the Google Cloud Console. For non-Google providers, replace `google` with a custom id and supply `authorization_endpoint` / `token_endpoint` / `userinfo_endpoint` explicitly.
### Sign in with Apple
```json
"oidc": {
"providers": {
"apple": {
"client_id": "com.example.your-service-id",
"team_id": "YOUR_TEAM_ID",
"key_id": "YOUR_KEY_ID",
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----"
}
}
}
```
The `client_id` is your Apple Services ID. `team_id`, `key_id`, and `private_key` come from the Apple Developer Portal (Keys section — create a key with "Sign in with Apple" enabled). The `private_key` is the contents of the `.p8` file Apple provides. Add `https://puter.<your-domain>/auth/oidc/callback/login` and `https://puter.<your-domain>/auth/oidc/callback/signup` as return URLs in the Apple Services ID configuration.
### Sign in with Microsoft
```json
"oidc": {
"providers": {
"microsoft": {
"client_id": "YOUR_APPLICATION_CLIENT_ID",
"client_secret": "YOUR_CLIENT_SECRET_VALUE",
"tenant_id": "YOUR_TENANT_ID"
}
}
}
```
Register an app in the Azure Portal (Microsoft Entra ID → App registrations). The `client_id` is the Application (client) ID, `client_secret` is a client secret value, and `tenant_id` is the Directory (tenant) ID. Use `common` as `tenant_id` to allow any Microsoft account (personal and organizational); omit it to default to `common`. Add `https://puter.<your-domain>/auth/oidc/callback/login` and `https://puter.<your-domain>/auth/oidc/callback/signup` as redirect URIs under Authentication in the app registration.
### AI providers
Any provider with a key set is auto-enabled. Same shape as `ollama` above:
@@ -277,14 +394,56 @@ Full provider list (chat, image, video, TTS, OCR) is in the template.
### Per-user storage quota
Default is 100 MB per user.
Storage is unlimited by default (bounded by host disk). To cap each user:
```json
"storage_capacity": 5368709120, // 5 GB
"is_storage_limited": true
```
Set `is_storage_limited: false` for unlimited (bounded by host disk).
`storage_capacity` defaults to 100 MB.
### Usage metering and budgets
Puter meters what an account costs to serve — bytes sent back, object-store
requests, KV capacity, AI tokens — against a monthly budget, and refuses the
operations that spend it once that budget is gone. The refusal is a `402` with
code `insufficient_funds`; reads that only describe things, and every kind of
deletion, stay available so an account can always see what it has and clear it.
On a self-hosted install this is almost certainly not what you want. There is
nowhere to buy more, so accounts are held to the free monthly allowance
(US$0.50 of measured cost for a registered account, roughly 2.5 GiB of
downloads) and start being refused after that. Turn it off:
```json
"unlimitedMetering": true
```
Every account then resolves to an unlimited policy. Usage is still recorded, so
the dashboard still shows what is being consumed; nothing is ever refused for
lack of budget.
To keep the budgets but stop them blocking anything — recording only:
```json
"meteringEnforcement": { "enabled": false }
```
Calls driven by a deployed worker are exempt from enforcement by default,
because a worker has no prompt to show and nobody watching it fail. Set
`"meteringEnforcement": { "workers": true }` to include them.
Monthly usage records expire on their own once they're old enough to be of no
further use — `meteringRetentionMonths` full months past the month they
belong to, 3 by default. Set it to `0` to keep every month's records forever:
```json
"meteringRetentionMonths": 6
```
Changing it, including to `0`, applies to records first written after the
change; records already stamped keep their expiry.
### Captcha on signup / login
@@ -296,9 +455,19 @@ Built-in proof-of-work captcha — no external service needed.
`difficulty` is one of `easy` / `medium` / `hard`.
### Block disposable email TLDs
### Disable new signups
Only enforced when `env: "prod"`.
Force visitors to log in with an existing account instead of creating a
temporary or permanent one.
```json
"disable_user_signup": true
```
### Block disposable email domains
Addresses on these domains (suffix match, so subdomains too) can't sign up or be
set as an account's email.
```json
"blockedEmailDomains": ["mailinator.com", "tempmail.com", "guerrillamail.com"]
@@ -345,14 +514,125 @@ Without `--profile ai`, the `ollama` containers stay down and Puter (with `enabl
For GPU acceleration (NVIDIA), uncomment the `deploy:` block under the `ollama` service in [docker-compose.yml](../docker-compose.yml). Requires `nvidia-container-toolkit` on the host.
## Optional: receiving mail over SMTP
Puter can give each account a mailbox at a domain you control. Two pieces are
involved, and they are configured together under `userEmail`:
- an **ingress endpoint** (`POST /email/ingress`) that files a message into the
recipient's `~/.mail`. It exists whenever `userEmail.secret` is set.
- the **`puter-smtp` receiver**, which accepts mail from other mail servers and
hands each message to that endpoint. It runs as its own process, behind the
`smtp` compose profile.
> **Read this before you expose it.** An inbound mail server is unauthenticated
> by nature: anyone on the internet may hand it a message for any address in
> `localDomains`. Puter performs **no** spam filtering and **no** SPF, DKIM or
> DMARC checks — whatever is delivered is stored in the recipient's mailbox and
> counts against their storage. If that is not acceptable, put a filtering relay
> in front of it and point `localDomains` at that instead.
1. Configure it in `puter/config/config.json`:
```json
"userEmail": {
"secret": "a long random string",
"localServer": true,
"localDomains": ["example.com"]
}
```
`secret` is shared between the two halves; generate it with
`openssl rand -hex 32`. `localDomains` lists the domains this server accepts
mail for — anything else is refused at `RCPT TO`, which is what stops it
being used as an open relay. It has no default, and the receiver refuses to
start without it.
The receiver posts to `<api_base_url>/email/ingress` unless you set
`localIngressUrl`. The endpoint is served on the `api` subdomain, so if you
point it at an internal address to skip the reverse proxy, name that virtual
host too or the request will 404:
```json
"localIngressUrl": "http://puter:4100/email/ingress",
"localIngressHost": "api.example.com"
```
`localIngressHost` defaults to the host of `api_base_url`, which is usually
what you want.
2. Bring it up with the `smtp` profile:
```bash
docker compose --profile smtp up -d
docker compose logs -f puter-smtp
```
3. Point DNS at it:
```dns
mx.example.com. A 203.0.113.10
example.com. MX 10 mx.example.com.
```
Also set a `PTR` record for the address that resolves back to
`mx.example.com` — many senders refuse mail from a server without one.
Inbound TCP port 25 must reach the host. **Most cloud providers block port
25 by default and residential ISPs block it permanently**; if you cannot get
it unblocked, this will not work no matter how it is configured. The
container itself binds 2525 (it runs unprivileged) and compose publishes
`${SMTP_PORT:-25}` onto it, so you can move the host port if you need to.
A message is held in memory while it is being delivered, so peak memory is
bounded by `localMaxClients` (default 20) times the 25 MiB message ceiling.
Lower `localMaxClients` on a small host. There is no queue and no retry
schedule — if delivery fails, the sending server is told to try again later and
retrying is its job.
Mail is stored as `message/rfc822` objects under `~/.mail/objects/<date>/` and
read back through `puter.email` in the SDK.
Every setting is documented in
[config.template.jsonc](../config.template.jsonc). Two deliberate limits, both
in the name of keeping the self-hosted path simple: it does not offer STARTTLS,
so mail arrives in the clear, and it applies no per-sender rate limiting. Put a
filtering relay in front if either matters to you.
Note this is a *receiving* server only: it never sends, and it must not be used
for authenticated submission. Outbound transactional mail is the separate
`email` block above.
## Building from source instead of pulling
If you want to test local Dockerfile changes against the full stack, uncomment the `build:` block in [docker-compose.yml](../docker-compose.yml) under the `puter` service, change `pull_policy: always` → `pull_policy: never`, then:
To run a local build against the full stack, use a source checkout and complete
the configuration steps above. Create `docker-compose.override.yml` in the
repository root, next to [docker-compose.yml](../docker-compose.yml):
```yaml
services:
puter:
pull_policy: never
build:
context: .
```
If that file already exists, merge these settings into its `puter` service.
Compose merges the override automatically: `build.context: .` builds this
checkout, and `pull_policy: never` stops it pulling the published image. Git
ignores the override, so local build settings stay out of pull requests and
don't conflict with `docker-compose.yml` when you pull updates.
From the repository root, build and start the stack:
```bash
docker compose up -d --build
```
Run the same command after changing or pulling source code to rebuild the image.
To return to the published image, remove the local build settings (or the override
file if those are its only settings), then run `docker compose up -d`.
---
## Managing running backend
@@ -377,14 +657,14 @@ Migrations re-apply idempotently across pulls. Volumes are preserved.
## Troubleshooting
**Site loads but I get "Bad Gateway" / nginx errors.**
**Site loads but I get a 502 / "Bad Gateway" from Caddy.**
The puter container failed to come up. `docker compose logs puter` will tell you which dependency rejected it (most often DB password mismatch between `.env` and `config.json`).
**Login screen says "admin password not set".**
First-boot temp password is logged once. Find it: `docker compose logs puter | grep "tmp_password"`. After login, change it in Settings.
**Can't find the admin password.**
It's logged on every boot until you change it: `docker compose logs puter | grep "password for admin"`. After login, change it in Settings.
**Healthcheck reports unhealthy but the site works.**
The healthcheck hits `puter.localhost:4100/test` from inside the container. If you changed `domain` or `port`, the check still uses defaults. The site itself is fine.
The compose healthcheck requests `http://localhost:4100/` from inside the container. If you changed `port` or set `allow_all_host_values: false`, the check fails while the site itself is fine.
**Nothing resolves at `puter.example.com` after DNS changes.**
DNS propagates slowly. `dig puter.example.com` and `dig api.puter.example.com` should both return your server IP. If not, give it 5–60 minutes.
+85 -31
View File
@@ -2,7 +2,7 @@
# Self-hosted Puter — full stack.
#
# Brings up Puter + every external service it needs:
# - nginx : reverse proxy (mirrors prod ALB; handles TLS + Host fan-out)
# - caddy : reverse proxy (mirrors prod ALB; handles TLS + Host fan-out)
# - valkey : redis-compatible cache / rate-limiter backend
# - mariadb : SQL database (Puter applies its schema on first boot)
# - dynamo : DynamoDB-local (KV store; Puter creates the table itself)
@@ -23,7 +23,8 @@
#
# Production:
# - Always replace the default passwords / S3 keys / Puter secrets.
# - Front Puter with TLS-terminating reverse proxy (Caddy / nginx).
# - Enable TLS on the bundled Caddy (see caddy/Caddyfile) or front the
# stack with your own TLS-terminating proxy.
# - Move state-bearing volumes to a backed-up location.
services:
@@ -58,7 +59,10 @@ services:
fi
wait $$SERVER_PID
volumes:
- ./puter/data/valkey:/data
# `:z` is an SELinux relabel hint for Fedora/RHEL hosts (no-op
# everywhere else) — without it those distros deny container
# access to the bind mount and the service loops on EACCES.
- ./puter/data/valkey:/data:z
healthcheck:
test:
["CMD-SHELL", "valkey-cli -p 6379 cluster info | grep -q cluster_state:ok"]
@@ -77,7 +81,7 @@ services:
MARIADB_USER: ${MARIADB_USER:-puter}
MARIADB_PASSWORD: ${MARIADB_PASSWORD:-puter-change-me}
volumes:
- ./puter/data/mariadb:/var/lib/mysql
- ./puter/data/mariadb:/var/lib/mysql:z
healthcheck:
# `healthcheck.sh` ships with the mariadb image; --connect verifies
# the server is accepting auth, not just listening on the socket.
@@ -102,7 +106,7 @@ services:
- "-dbPath"
- "/home/dynamodblocal/data"
volumes:
- ./puter/data/dynamo:/home/dynamodblocal/data
- ./puter/data/dynamo:/home/dynamodblocal/data:z
s3:
# RustFS — S3-compatible object storage. Drop-in alternative:
@@ -114,8 +118,8 @@ services:
RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY:-puter}
RUSTFS_SECRET_KEY: ${S3_SECRET_KEY:-puter-secret-change-me}
volumes:
- ./puter/data/s3:/data
# Internal-only — browsers reach RustFS via nginx (`s3.<domain>`),
- ./puter/data/s3:/data:z
# Internal-only — browsers reach RustFS via Caddy (`s3.<domain>`),
# which preserves the Host header for S3 signature validation and
# rides the same TLS termination as Puter. Uncomment to also expose
# 9000 directly on the host for `aws-cli` / debugging.
@@ -135,8 +139,10 @@ services:
start_period: 5s
s3-init:
# One-shot container that creates the `puter-local` bucket on first
# boot. Exits 0 once the bucket exists; stays exited 0 thereafter.
# One-shot container that creates the `puter-local` bucket and
# applies its CORS rules on every boot (put-bucket-cors is
# idempotent). Without CORS, browser uploads to presigned URLs
# fail the preflight check.
image: amazon/aws-cli:latest
container_name: puter-s3-init
depends_on:
@@ -159,6 +165,26 @@ services:
echo "creating bucket $$bucket"
aws --endpoint-url "$$endpoint" s3 mb "s3://$$bucket"
fi
# Browser uploads (Dev Center deploy, puter.fs.upload) PUT to
# presigned URLs cross-origin, so the bucket must answer the
# OPTIONS preflight. Origins are open because the presigned URL
# itself is the credential and the SDK sends no cookies.
cat > /tmp/cors.json <<'EOF'
{
"CORSRules": [
{
"AllowedOrigins": ["*"],
"AllowedMethods": ["GET", "HEAD", "PUT", "POST", "DELETE"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag", "x-amz-request-id"],
"MaxAgeSeconds": 3600
}
]
}
EOF
aws --endpoint-url "$$endpoint" s3api put-bucket-cors \
--bucket "$$bucket" --cors-configuration file:///tmp/cors.json
echo "bucket CORS configured"
restart: "no"
# ── Optional: local LLM ───────────────────────────────────────────
@@ -180,7 +206,7 @@ services:
container_name: puter-ollama
restart: unless-stopped
volumes:
- ./puter/data/ollama:/root/.ollama
- ./puter/data/ollama:/root/.ollama:z
# Uncomment to expose Ollama directly on the host (`localhost:11434`)
# for `ollama` CLI / OpenAI-API compatible tools. Internal-only by default.
# ports:
@@ -226,15 +252,8 @@ services:
puter:
image: ghcr.io/heyputer/puter:main
pull_policy: always
# Uncomment to build from this directory instead of pulling the published
# image. Also flip pull_policy to `never` so compose doesn't overwrite
# your local build by re-pulling :latest.
# build:
# context: .
# # buildx-only: cross-compile to both archs in a single push
# # platforms:
# # - linux/amd64
# # - linux/arm64
# For local builds, create a docker-compose.override.yml;
# see doc/self-hosting.md#building-from-source-instead-of-pulling.
container_name: puter
restart: unless-stopped
depends_on:
@@ -246,7 +265,7 @@ services:
condition: service_started
s3-init:
condition: service_completed_successfully
# Internal-only: nginx reaches it on the compose network. Uncomment
# Internal-only: Caddy reaches it on the compose network. Uncomment
# to also expose port 4100 directly on the host (useful for debugging).
# ports:
# - "4100:4100"
@@ -257,33 +276,68 @@ services:
PGID: 1000
volumes:
# Drop your config.json here — see selfhosted/full-stack.md.
- ./puter/config:/etc/puter
- ./puter/config:/etc/puter:z
# Persistent runtime data (anything your config points at /var/puter).
- ./puter/data/puter:/var/puter
- ./puter/data/puter:/var/puter:z
healthcheck:
test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
test: wget --no-verbose --tries=1 --spider http://localhost:4100/ || exit 1
interval: 30s
timeout: 3s
retries: 3
start_period: 30s
nginx:
image: nginx:1.27-alpine
container_name: puter-nginx
# Inbound mail. Behind the `smtp` profile, so it only starts when asked for:
# docker compose --profile smtp up -d
# Accepts SMTP from other mail servers and hands each message to the ingress
# endpoint, which files it into the recipient's mailbox. Needs `userEmail`
# configured — see doc/self-hosting.md#optional-receiving-mail-over-smtp.
puter-smtp:
profiles: ["smtp"]
image: ghcr.io/heyputer/puter:main
pull_policy: always
container_name: puter-smtp
restart: unless-stopped
command: ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/smtp/index.js"]
depends_on:
puter:
condition: service_healthy
ports:
# The container runs unprivileged and binds 2525; 25 is what an MX record
# means, and many providers block it.
- "${SMTP_PORT:-25}:2525"
environment:
PUID: 1000
PGID: 1000
volumes:
# The same config.json the backend reads; only `userEmail` applies here.
- ./puter/config:/etc/puter:z
caddy:
image: caddy:2.11-alpine
container_name: puter-caddy
restart: unless-stopped
depends_on:
puter:
condition: service_started
ports:
- "${HTTP_PORT:-80}:80"
# Uncomment when you enable TLS in nginx/nginx.conf:
# Uncomment when you enable TLS in caddy/Caddyfile:
# - "${HTTPS_PORT:-443}:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
# TLS certs (fullchain.pem + privkey.pem). Read-only inside.
- ./puter/tls:/etc/nginx/tls:ro
- ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro,z
# TLS certs (wildcard fullchain.pem + privkey.pem). Read-only inside.
- ./puter/tls:/etc/caddy/tls:ro,z
# Caddy's own state — the cert store it would use if you swap this
# config over to ACME. Kept under ./puter/data with everything else
# so one directory is the whole backup.
- ./puter/data/caddy:/data:z
healthcheck:
test: ["CMD-SHELL", "wget -qO- --tries=1 --timeout=2 http://localhost/ || exit 1"]
# Hits Caddy's local-only admin API rather than proxying through to
# Puter: this reports whether the proxy itself is up and configured,
# and keeps working once the `:80` block becomes an HTTPS redirect.
# 127.0.0.1, not localhost — the admin endpoint is IPv4-only and
# busybox wget tries ::1 first.
test: ["CMD-SHELL", "wget -qO- --tries=1 --timeout=2 http://127.0.0.1:2019/config/ >/dev/null || exit 1"]
interval: 10s
timeout: 3s
retries: 5
+225
View File
@@ -0,0 +1,225 @@
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
// Importing the module registers the `appTelemetry` driver into the shared
// extensionStore, so `setupTestServer` instantiates it on `server.drivers`.
import { AppTelemetryDriver } from './appTelemetry.ts';
let server: PuterServer;
let driver: AppTelemetryDriver;
// `get_users` reads `Context.get('actor')` for the ownership gate. Wrap
// actor-dependent calls in `runWithContext` so the ALS lookup resolves.
const callWithActor = <T>(
actor: { user: { uuid: string; id?: number } } | undefined,
fn: () => Promise<T>,
) => runWithContext({ actor }, fn);
beforeAll(async () => {
server = await setupTestServer();
driver = (server.drivers as unknown as Record<string, AppTelemetryDriver>)
.appTelemetry;
// Guard: the driver must have been wired onto the server. If this is
// undefined the extension didn't register, and every test below would
// fail with a confusing "cannot read property of undefined".
expect(driver).toBeInstanceOf(AppTelemetryDriver);
});
afterAll(async () => {
await server?.shutdown();
});
const seedOwnedApp = async (prefix: string) => {
const owner = await server.stores.user.create({
username: `${prefix}_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `${prefix}_${slug}`,
title: `${prefix} ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
return { owner, app: app! };
};
const seedUser = async (prefix: string, email: string | null = null) =>
server.stores.user.create({
username: `${prefix}_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email,
});
const grantAuthenticated = async (appId: number, userId: number) => {
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, 'flag:app-is-authenticated', null],
);
};
const grantEmailRead = async (
appId: number,
userId: number,
userUuid: string,
) => {
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, `user:${userUuid}:email:read`, null],
);
};
describe('appTelemetry driver — get_users', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
await expect(driver.get_users({})).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('app_uuid'),
});
});
it('throws HttpError(400) for a non-numeric limit', async () => {
await expect(
driver.get_users({ app_uuid: 'app-anything', limit: 'banana' }),
).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('limit'),
});
});
it('throws HttpError(400) when offset exceeds the allowed maximum', async () => {
await expect(
driver.get_users({ app_uuid: 'app-anything', offset: 1_000_001 }),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws HttpError(404) when the app cannot be found', async () => {
await expect(
driver.get_users({ app_uuid: 'app-does-not-exist' }),
).rejects.toMatchObject({
statusCode: 404,
message: 'App not found',
});
});
it('throws HttpError(403) when the caller does not own the app', async () => {
// Seed an owner + their app, then call as a *different* user. The
// real permission service should reject: the caller has no
// `apps-of-user:<owner>:write`.
const { app } = await seedOwnedApp('owner');
const intruder = await server.stores.user.create({
username: `intruder_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
await callWithActor(
{ user: { uuid: intruder.uuid, id: intruder.id as number } },
async () => {
await expect(
driver.get_users({ app_uuid: app.uid }),
).rejects.toMatchObject({
statusCode: 403,
message: 'Permission denied',
});
},
);
});
it('returns an empty list for an owned app with no authenticated users', async () => {
const { owner, app } = await seedOwnedApp('owner2');
const result = await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
);
expect(result).toEqual([]);
});
it('omits user_email for a user who did not grant email:read', async () => {
const { owner, app } = await seedOwnedApp('noemail');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(row.user).toBe(member.username);
expect(row.user_uuid).toBe(member.uuid);
// No grant → the field must be absent (not just null), so the email
// never leaks to the app owner.
expect(Object.prototype.hasOwnProperty.call(row, 'user_email')).toBe(
false,
);
});
it('returns user_email when the user granted email:read to the app', async () => {
const { owner, app } = await seedOwnedApp('withemail');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
await grantEmailRead(
app.id as number,
member.id as number,
member.uuid,
);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(row.user_uuid).toBe(member.uuid);
expect(row.user_email).toBe('[email protected]');
});
it('does not leak email granted to a *different* app', async () => {
const { owner, app } = await seedOwnedApp('appA');
const { app: otherApp } = await seedOwnedApp('appB');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
// Grant email:read against the OTHER app only.
await grantEmailRead(
otherApp.id as number,
member.id as number,
member.uuid,
);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(Object.prototype.hasOwnProperty.call(row, 'user_email')).toBe(
false,
);
});
});
describe('appTelemetry driver — user_count', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
await expect(driver.user_count({})).rejects.toMatchObject({
statusCode: 400,
});
});
it('throws HttpError(404) for an unknown app_uuid', async () => {
await expect(
driver.user_count({ app_uuid: 'app-not-here' }),
).rejects.toMatchObject({ statusCode: 404 });
});
it('returns 0 for an app with no authenticated users', async () => {
const { app } = await seedOwnedApp('appcount');
await expect(driver.user_count({ app_uuid: app.uid })).resolves.toBe(0);
});
});
+107 -52
View File
@@ -1,10 +1,23 @@
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import { PuterDriver } from '@heyputer/backend/src/drivers/types';
import type {
DriverConcurrentConfig,
DriverRateLimitConfig,
} from '@heyputer/backend/src/drivers/meta';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '@heyputer/backend/src/services/metering/consts';
import { extension } from '@heyputer/backend/src/extensions';
const clients = extension.import('client');
const stores = extension.import('store');
const services = extension.import('service');
// App-telemetry lets an app owner enumerate the users who have
// authenticated into their app. v1 shipped this as a driver on the
// `app-telemetry` interface (methods `get_users` / `user_count`) and
// puter-js's `puter.apps(...).getUsers()` still calls it that way
// (`puter.drivers.call('app-telemetry', 'app-telemetry', 'get_users', …)`).
// This is the v2 port of that driver — same interface/method/return shapes
// so existing puter-js callers work unchanged.
const DEFAULT_LIMIT = 100;
const MAX_LIMIT = 1000;
@@ -40,81 +53,123 @@ const parseIntParam = (
return parsed;
};
extension.get(
'/app-telemetry/users',
{ subdomain: 'api', requireAuth: true },
async (req, res) => {
const { app_uuid } = req.query as Record<string, string>;
/**
* Driver exposing the `app-telemetry` interface.
*
* The `/drivers/call` permission gate checks
* `service:app-telemetry:ii:app-telemetry`, which every actor already holds via
* the blanket `service` grant (hardcoded-permissions.js +
* `default_implicit_user_app_permissions`). The real authorization — "is the
* caller the app owner?" — is enforced inside `get_users` below, exactly as v1
* did.
*/
export class AppTelemetryDriver extends PuterDriver {
readonly driverInterface = 'app-telemetry';
readonly driverName = 'app-telemetry';
readonly isDefault = true;
// Declaring nothing here would leave both methods on the generic
// 600/minute driver default, which does not fit a paginated scan that
// can ask for MAX_LIMIT rows at MAX_OFFSET. This is a dashboard read —
// nobody calls it in a loop.
readonly rateLimit: DriverRateLimitConfig = {
default: {
limit: 60,
window: 60_000,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 30,
[DEFAULT_TEMP_SUBSCRIPTION]: 10,
},
},
};
readonly concurrent: DriverConcurrentConfig = {
default: {
limit: 5,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 2,
[DEFAULT_TEMP_SUBSCRIPTION]: 2,
},
},
};
/** Users who have authenticated into the given app (owner-only). */
async get_users({
app_uuid,
limit,
offset,
}: {
app_uuid?: string;
limit?: unknown;
offset?: unknown;
} = {}): Promise<
Array<{ user: string; user_uuid: string; user_email?: string | null }>
> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
const safeLimit = parseIntParam(req.query.limit, {
const safeLimit = parseIntParam(limit, {
key: 'limit',
min: 1,
max: MAX_LIMIT,
fallback: DEFAULT_LIMIT,
});
const safeOffset = parseIntParam(req.query.offset, {
const safeOffset = parseIntParam(offset, {
key: 'offset',
min: 0,
max: MAX_OFFSET,
fallback: 0,
});
const app = await stores.app.getByUid(app_uuid);
const app = await this.stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
// `apps-of-user:<uuid>:write` — the implicator keys on the owner's
// UUID, not the numeric id. Look up the owner explicitly. v1 got
// this for free because its entity-storage layer eager-joined the
// owner row; v2's AppStore.getByUid returns the raw row with only
// `owner_user_id` populated.
// The `apps-of-user:<uuid>:write` implicator keys on the owner's
// UUID, not the numeric id. Look up the owner explicitly — the raw
// app row only carries `owner_user_id`. (v1 got the owner for free
// because its entity-storage layer eager-joined the owner row.)
const ownerId = (app as { owner_user_id?: number }).owner_user_id;
if (!ownerId) throw new HttpError(404, 'App owner not found');
const owner = (await stores.user.getById(ownerId)) as {
uuid?: string;
} | null;
const owner = await this.stores.user.getById(ownerId);
if (!owner?.uuid) throw new HttpError(404, 'App owner not found');
const actor = Context.get('actor');
const ownsApp = await services.permission
.check(actor!, `apps-of-user:${owner.uuid}:write`)
.catch(() => false);
// `/drivers/call` requires auth, so there is always an actor.
const actor = Context.get('actor')!;
let ownsApp = false;
try {
ownsApp = await this.services.permission.check(
actor,
`apps-of-user:${owner.uuid}:write`,
);
} catch {
// A failed check is a denial.
}
if (!ownsApp) throw new HttpError(403, 'Permission denied');
const users = await clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
INNER JOIN user u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
[(app as Record<string, unknown>).id, safeLimit, safeOffset],
// An email surfaces only for a user who granted *this app*
// `user:<their-uuid>:email:read` — the grant `puter.perms.requestEmail()`
// obtains and `whoami` honours. Authenticating alone doesn't share it.
const users = await this.stores.permission.listAppAuthenticatedUsers(
(app as { id: number }).id,
{ limit: safeLimit, offset: safeOffset },
);
res.json(
(users as Array<{ username: string; uuid: string }>).map((e) => ({
user: e.username,
user_uuid: e.uuid,
})),
return users.map((u) =>
u.emailShared
? { user: u.username, user_uuid: u.uuid, user_email: u.email }
: { user: u.username, user_uuid: u.uuid },
);
},
);
}
extension.get(
'/app-telemetry/user-count',
{ subdomain: 'api', requireAuth: true },
async (req, res) => {
const { app_uuid } = req.query as Record<string, string>;
/** Count of users who have authenticated into the given app. */
async user_count({
app_uuid,
}: { app_uuid?: string } = {}): Promise<number> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
const app = await stores.app.getByUid(app_uuid);
const app = await this.stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
const [row] = (await clients.db.read(
`SELECT COUNT(*) AS n FROM user_to_app_permissions
WHERE permission = 'flag:app-is-authenticated' AND app_id = ?`,
[(app as Record<string, unknown>).id],
)) as Array<{ n: number }>;
return this.stores.permission.countAppUsers((app as { id: number }).id);
}
}
res.json({ count: row?.n ?? 0 });
},
);
extension.registerDriver('appTelemetry', AppTelemetryDriver);
+555
View File
@@ -0,0 +1,555 @@
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import { extensionStore } from '../src/backend/extensions.ts';
import type { IConfig } from '../src/backend/types';
import './devWatcher.ts';
type Lifecycle = {
onServerStart: () => Promise<void>;
onServerShutdown: () => Promise<void>;
};
// The extension registers itself on import; that registry entry is the
// only handle on the service class.
const DevWatcherService = extensionStore.services.devWatcher as unknown as new (
config: IConfig,
clients: unknown,
stores: unknown,
services: unknown,
) => Lifecycle;
const makeService = (config: Record<string, unknown>): Lifecycle =>
new DevWatcherService(config as unknown as IConfig, {}, {}, {});
let workdir: string;
const write = (relative: string, contents: string): string => {
const abs = path.join(workdir, relative);
writeFileSync(abs, contents);
return abs;
};
/** Poll until `predicate` holds or the budget runs out. */
const waitFor = async (
predicate: () => boolean,
label: string,
timeoutMs = 15_000,
): Promise<void> => {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (predicate()) return;
await new Promise((r) => setTimeout(r, 20));
}
throw new Error(`timed out waiting for ${label}`);
};
const logLines = (spy: { mock: { calls: unknown[][] } }): string[] =>
spy.mock.calls.map((call) => call.map(String).join(' '));
describe('devWatcher extension', () => {
let log: ReturnType<typeof vi.spyOn>;
let warn: ReturnType<typeof vi.spyOn>;
let error: ReturnType<typeof vi.spyOn>;
beforeAll(() => {
workdir = mkdtempSync(path.join(tmpdir(), 'devwatch-test-'));
});
afterAll(() => {
rmSync(workdir, { recursive: true, force: true });
delete (extensionStore.services as Record<string, unknown>).devWatcher;
});
beforeEach(() => {
log = vi.spyOn(console, 'log').mockImplementation(() => {});
warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
error = vi.spyOn(console, 'error').mockImplementation(() => {});
});
afterEach(() => {
log.mockRestore();
warn.mockRestore();
error.mockRestore();
});
const started = () =>
logLines(log).some((line) =>
line.startsWith('[devwatch] starting watchers from'),
);
// -- start gating -------------------------------------------------
it('does nothing when the server is not running in dev', async () => {
await makeService({ env: 'production' }).onServerStart();
expect(started()).toBe(false);
});
it('does nothing when devwatch is explicitly disabled', async () => {
await makeService({
env: 'dev',
devwatch: { enabled: false },
}).onServerStart();
expect(started()).toBe(false);
});
it('does nothing when the server opts out with no_devwatch', async () => {
await makeService({
env: 'dev',
no_devwatch: true,
devwatch: { enabled: true },
}).onServerStart();
expect(started()).toBe(false);
});
it('starts in dev even with no explicit devwatch block', async () => {
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
expect(started()).toBe(true);
await service.onServerShutdown();
});
it('starts outside dev when devwatch is explicitly enabled', async () => {
const service = makeService({
env: 'production',
devwatch: {
enabled: true,
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
expect(started()).toBe(true);
await service.onServerShutdown();
});
it('ignores a devwatch config that is not an object', async () => {
// Falls back to `{}`, so the dev-env rule decides — and the
// default webpack entries would be used, so pin it off instead.
await makeService({
env: 'production',
devwatch: 'yes',
}).onServerStart();
expect(started()).toBe(false);
});
it('only starts once even if the lifecycle hook fires again', async () => {
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await service.onServerStart();
expect(
logLines(log).filter((line) =>
line.startsWith('[devwatch] starting watchers from'),
),
).toHaveLength(1);
await service.onServerShutdown();
});
it('waits out the configured ready delay before resolving', async () => {
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 60,
},
});
const start = Date.now();
await service.onServerStart();
expect(Date.now() - start).toBeGreaterThanOrEqual(50);
await service.onServerShutdown();
});
// -- child commands -----------------------------------------------
it('spawns a command, line-buffers its output and flushes the tail', async () => {
write(
'chatty.js',
[
"process.stdout.write('one\\ntwo\\n');",
"process.stderr.write('bad thing\\n');",
// No trailing newline: only the stream `end` flushes it.
"process.stdout.write('tail-no-newline');",
].join('\n'),
);
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [
{
name: 'chatty',
directory: '.',
command: 'node',
args: ['chatty.js'],
},
],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch:chatty:1] tail-no-newline'),
),
'child stdout',
);
const lines = logLines(log);
expect(lines).toContain('[devwatch:chatty:1] one');
expect(lines).toContain('[devwatch:chatty:1] two');
expect(logLines(warn)).toContain('[devwatch:chatty:2] bad thing');
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch:chatty:exit] process exited'),
),
'child exit',
);
await service.onServerShutdown();
});
it('passes literal and computed env values to the child', async () => {
write(
'env.js',
'process.stdout.write(`STATIC=${process.env.STATIC_VALUE} ORIGIN=${process.env.FROM_CONFIG} MISSING=${process.env.BLOWS_UP}\\n`);',
);
const service = makeService({
env: 'dev',
origin: 'http://puter.localhost:4100',
devwatch: {
root: workdir,
commands: [
{
name: 'envtest',
directory: '.',
command: 'node',
args: ['env.js'],
env: {
STATIC_VALUE: 'literal',
FROM_CONFIG: ({
global_config,
}: {
global_config: Record<string, unknown> | null;
}) => String(global_config?.origin ?? ''),
// Reading through a null is the "config not
// loaded yet" shape the extension deliberately
// stays quiet about.
BLOWS_UP: () => {
const nothing = null as unknown as {
x: string;
};
return nothing.x;
},
NOISY: () => {
throw new Error('unexpected failure');
},
},
},
],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await waitFor(
() => logLines(log).some((l) => l.includes('STATIC=literal')),
'env output',
);
const line = logLines(log).find((l) => l.includes('STATIC=literal'))!;
expect(line).toContain('ORIGIN=http://puter.localhost:4100');
expect(line).toContain('MISSING=undefined');
// A null-property read is expected noise and stays silent; any
// other failure is reported.
const warnings = logLines(warn);
expect(
warnings.some((w) => w.includes('could not evaluate env function')),
).toBe(true);
expect(warnings.some((w) => w.includes('for BLOWS_UP'))).toBe(false);
expect(warnings.some((w) => w.includes('for NOISY'))).toBe(true);
await service.onServerShutdown();
});
it('kills a still-running child on shutdown', async () => {
write('forever.js', 'setInterval(() => {}, 1000);');
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [
{
name: 'forever',
directory: '.',
command: 'node',
args: ['forever.js'],
},
],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await service.onServerShutdown();
expect(logLines(log)).toContain('[devwatch:forever] stopping process');
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch:forever:exit] process exited'),
),
'child killed',
);
});
// -- webpack watchers ---------------------------------------------
const makeWebpackProject = (
name: string,
configFile: string,
contents: string,
packageJson?: string,
): string => {
const dir = path.join(workdir, name);
rmSync(dir, { recursive: true, force: true });
writeFileSync(
path.join(
(() => {
const { mkdirSync } =
require('node:fs') as typeof import('node:fs');
mkdirSync(path.join(dir, 'src'), { recursive: true });
return dir;
})(),
'src/entry.js',
),
"console.log('hello');\n",
);
writeFileSync(path.join(dir, configFile), contents);
if (packageJson) {
writeFileSync(path.join(dir, 'package.json'), packageJson);
}
return name;
};
it('compiles a CommonJS webpack config and reports later rebuilds', async () => {
const dir = makeWebpackProject(
'cjs-project',
'webpack.config.cjs',
`module.exports = {
mode: 'development',
entry: './src/entry.js',
output: { path: __dirname + '/out', filename: 'bundle.js' },
};`,
);
let onConfigSaw: Record<string, unknown> | undefined;
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [
{
name: 'cjs',
directory: dir,
onConfig: (cfg: Record<string, unknown>) => {
onConfigSaw = cfg;
},
},
],
ready_delay_ms: 0,
},
});
await service.onServerStart();
expect(onConfigSaw).toBeDefined();
// The watcher context is anchored at <root>/<directory> so relative
// entries resolve regardless of the server's cwd.
expect(onConfigSaw!.context).toBe(path.join(workdir, dir));
// First build is silent by design; force a rebuild and assert the
// update line shows up.
await new Promise((r) => setTimeout(r, 500));
writeFileSync(
path.join(workdir, dir, 'src/entry.js'),
"console.log('hello again');\n",
);
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch] updated cjs using Webpack'),
),
'webpack rebuild',
);
await service.onServerShutdown();
});
it('resolves an ESM webpack config declared through package.json type', async () => {
const dir = makeWebpackProject(
'esm-project',
'webpack.config.js',
`export default {
mode: 'development',
entry: './src/entry.js',
output: { filename: 'bundle.js' },
};`,
JSON.stringify({ name: 'esm-project', type: 'module' }),
);
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [{ directory: dir }],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await service.onServerShutdown();
// Falling back to the directory as the display name is the
// documented behaviour when `name` is omitted.
expect(error).not.toHaveBeenCalled();
});
it('calls a config exported as a function and honours an explicit context', async () => {
const dir = makeWebpackProject(
'fn-project',
'webpack.config.cjs',
`module.exports = () => ({
mode: 'development',
context: 'src',
entry: './entry.js',
name: process.env.WEBPACK_MARKER,
output: { path: __dirname + '/out', filename: 'bundle.js' },
});`,
);
let seen: Record<string, unknown> | undefined;
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [
{
name: 'fn',
directory: dir,
env: { WEBPACK_MARKER: 'set-during-load' },
onConfig: (cfg: Record<string, unknown>) => {
seen = cfg;
},
},
],
ready_delay_ms: 0,
},
});
await service.onServerStart();
// Relative `context` resolves against <root>/<directory>.
expect(seen!.context).toBe(path.join(workdir, dir, 'src'));
// The entry's env map is applied while the config module runs, and
// restored afterwards.
expect(seen!.name).toBe('set-during-load');
expect(process.env.WEBPACK_MARKER).toBeUndefined();
await service.onServerShutdown();
});
it('reports a failing compilation instead of crashing the server', async () => {
const dir = makeWebpackProject(
'broken-project',
'webpack.config.cjs',
`module.exports = {
mode: 'development',
entry: './src/does-not-exist.js',
output: { path: __dirname + '/out', filename: 'bundle.js' },
};`,
);
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [{ name: 'broken', directory: dir }],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await waitFor(
() =>
logLines(error).some((l) =>
l.includes('[devwatch] failed to update broken'),
),
'webpack failure',
);
expect(
logLines(error).some((l) =>
l.includes('[devwatch] error information: broken'),
),
).toBe(true);
await service.onServerShutdown();
});
it('fails loudly when a directory has no webpack config at all', async () => {
const dir = path.join('no-config');
const { mkdirSync } = await import('node:fs');
mkdirSync(path.join(workdir, dir), { recursive: true });
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [{ name: 'missing', directory: dir }],
ready_delay_ms: 0,
},
});
await expect(service.onServerStart()).rejects.toThrow(
'could not find webpack config for: no-config',
);
await service.onServerShutdown();
});
});
+5 -7
View File
@@ -1,11 +1,11 @@
import { extension } from '@heyputer/backend/src/extensions';
import { PuterService } from '@heyputer/backend/src/services/types.js';
import { nativeImport } from '@heyputer/backend/src/util/nativeImport.js';
import { spawn, type ChildProcess } from 'node:child_process';
import { existsSync, readFileSync } from 'node:fs';
import { createRequire } from 'node:module';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { extension } from '@heyputer/backend/src/extensions';
import { PuterService } from '@heyputer/backend/src/services/types.js';
import { nativeImport } from '@heyputer/backend/src/util/nativeImport.js';
const requireFromHere = createRequire(__filename);
const webpack = requireFromHere('webpack') as typeof import('webpack');
@@ -121,9 +121,7 @@ const defaultWebpackEntries: WebpackEntry[] = [
class DevWatcherService extends PuterService {
#processes: Array<{ name: string; proc: ChildProcess }> = [];
#watchers: Array<{
close: (handler: (err?: Error | null) => void) => void;
}> = [];
#watchers: ReturnType<ReturnType<typeof webpack>['watch']>[] = [];
#started = false;
#packageRoot = findPackageRoot();
@@ -159,7 +157,7 @@ class DevWatcherService extends PuterService {
this.#watchers.map(
(watcher) =>
new Promise<void>((resolve) => {
watcher.close((err) => {
watcher?.close((err) => {
if (err) {
console.warn(
'[devwatch] failed to close webpack watcher:',
+218
View File
@@ -0,0 +1,218 @@
import type { Request, Response } from 'express';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { extension } from '../src/backend/extensions.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleInstalledApps } from './installedApps.ts';
interface CapturedResponse {
body: unknown;
}
const makeReq = (query: Record<string, unknown> = {}): Request =>
({ query }) as unknown as Request;
const makeRes = () => {
const captured: CapturedResponse = { body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const seedUser = async () => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.user.create({
username: `iauser_${slug}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
};
const seedApp = async (ownerUserId: number, overrides = {}) => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.app.create(
{
name: `iaapp_${slug}`,
title: `Installed App ${slug}`,
index_url: `https://example.com/${slug}`,
...overrides,
},
{ ownerUserId },
);
};
const grantInstalled = async (appId: number, userId: number) => {
// Mimic the `flag:app-is-authenticated` perm row the handler joins on.
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, 'flag:app-is-authenticated', null],
);
};
describe('installedApps extension — handleInstalledApps', () => {
it('throws HttpError(400) when orderBy is not in the allowlist', async () => {
const user = await seedUser();
const { res } = makeRes();
await expect(
runWithContext(
{
actor: {
user: { uuid: user.uuid, id: user.id as number },
},
},
() =>
handleInstalledApps(
// SQL injection attempt — must be rejected.
makeReq({ orderBy: 'apps.id; DROP TABLE apps;--' }),
res,
),
),
).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('Invalid orderBy'),
});
});
it('returns an empty list for a user with no installed apps', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
expect(captured.body).toEqual([]);
});
it('returns the caller’s installed apps with an iconUrl field', async () => {
const user = await seedUser();
const app = await seedApp(user.id as number);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list).toHaveLength(1);
expect(list[0].uid).toBe(app!.uid);
expect(list[0].name).toBe(app!.name);
expect(list[0].title).toBe(app!.title);
// index_url is required so the dashboard can derive a hostname title
// for anonymous (app-…) apps.
expect(list[0].index_url).toBe(app!.index_url);
expect(Object.prototype.hasOwnProperty.call(list[0], 'iconUrl')).toBe(
true,
);
// An owned app is not external, and the raw owner id must not leak.
expect(list[0].external).toBe(false);
expect(
Object.prototype.hasOwnProperty.call(list[0], 'owner_user_id'),
).toBe(false);
});
it('pairs iconUrl with the direct subdomain URL for a generated icon', async () => {
const user = await seedUser();
// The resize pipeline leaves an http(s) icon column behind, which is
// what says the sized PNG is on the icons subdomain.
const app = await seedApp(user.id as number, {
icon: 'https://api.puter.com/app-icon/x',
});
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
const { protocol, static_hosting_domain } = extension.config;
expect(list[0].iconCdnUrl).toBe(
`${protocol}://puter-app-icons.${static_hosting_domain}/${app!.uid}-256.png`,
);
});
it('reports no subdomain URL for an app with no generated icon', async () => {
const user = await seedUser();
const app = await seedApp(user.id as number);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list[0].iconCdnUrl).toBeNull();
});
it('flags apps with no owner_user_id as external', async () => {
const user = await seedUser();
const slug = Math.random().toString(36).slice(2, 8);
// createFromOrigin bootstraps an app with owner_user_id = null.
const app = await server.stores.app.createFromOrigin(
`app-${slug}`,
`https://external-${slug}.example.com`,
);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list).toHaveLength(1);
expect(list[0].external).toBe(true);
expect(
Object.prototype.hasOwnProperty.call(list[0], 'owner_user_id'),
).toBe(false);
});
it('clamps page/limit to safe ranges (page>=1, 1<=limit<=100)', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
// page=0 and limit=999 should be clamped without throwing.
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() =>
handleInstalledApps(
makeReq({ page: 0, limit: 999, desc: '1' }),
res,
),
);
expect(Array.isArray(captured.body)).toBe(true);
});
});
+63 -63
View File
@@ -1,74 +1,74 @@
import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import { extension } from '@heyputer/backend/src/extensions';
import { getAppIconUrl } from '@heyputer/backend/src/util/appIcon.js';
import {
INSTALLED_APPS_ORDERS,
type InstalledAppsOrder,
} from '@heyputer/backend/src/stores/permission/PermissionStore.js';
import {
getAppIconCdnUrl,
getAppIconUrl,
} from '@heyputer/backend/src/util/appIcon.js';
const clients = extension.import('client');
const stores = extension.import('store');
const ALLOWED_ORDER_BY = [
'id',
'name',
'uid',
'title',
'installed_at',
] as const;
const ORDER_BY_FIELD_MAP: Record<string, string> = {
id: 'apps.id',
name: 'apps.name',
uid: 'apps.uid',
title: 'apps.title',
installed_at: 'installed_at',
export const handleInstalledApps = async (
req: Request,
res: Response,
): Promise<void> => {
// Behind `requireUserActor`, so there is always an actor.
const actor = Context.get('actor')!;
const orderBy = String(
req.query.orderBy ?? 'installed_at',
) as InstalledAppsOrder;
if (!INSTALLED_APPS_ORDERS.includes(orderBy)) {
throw new HttpError(
400,
`Invalid orderBy. Allowed: ${INSTALLED_APPS_ORDERS.join(', ')}`,
);
}
const page = Math.max(Number(req.query.page) || 1, 1);
const limit = Math.min(Math.max(Number(req.query.limit) || 100, 1), 100);
const offset = (page - 1) * limit;
const installedApps = await stores.permission.listAppsGrantedByUser(
actor.user.id!,
{ orderBy, descending: Boolean(req.query.desc), limit, offset },
);
const apiBaseUrl = extension.config.api_base_url as string | undefined;
res.json(
installedApps.map((app) => {
// An app with no owner_user_id (null/empty) isn't owned by a Puter
// user — it's an "external" app. Derive a flag and don't leak the
// raw owner id to the client.
const { owner_user_id, ...rest } = app;
const external = owner_user_id == null || owner_user_id === '';
return {
...rest,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
// Direct subdomain URL for the client to try before iconUrl.
iconCdnUrl: getAppIconCdnUrl(app, extension.config),
external,
};
}),
);
};
extension.get(
'/installedApps',
{ subdomain: 'api', requireUserActor: true },
async (req, res) => {
const actor = Context.get('actor');
if (!actor?.user?.id)
throw new HttpError(401, 'Authentication required');
const orderBy = String(req.query.orderBy ?? 'installed_at');
if (!(ALLOWED_ORDER_BY as readonly string[]).includes(orderBy)) {
throw new HttpError(
400,
`Invalid orderBy. Allowed: ${ALLOWED_ORDER_BY.join(', ')}`,
);
}
const page = Math.max(Number(req.query.page) || 1, 1);
const limit = Math.min(
Math.max(Number(req.query.limit) || 100, 1),
100,
);
const offset = (page - 1) * limit;
const orderByField = ORDER_BY_FIELD_MAP[orderBy];
const sortDirection = req.query.desc ? 'DESC' : 'ASC';
const installedApps = (await clients.db.read(
`SELECT
apps.name,
apps.uid,
apps.title,
apps.description,
apps.icon,
MIN(perm.dt) AS installed_at
FROM apps
LEFT JOIN user_to_app_permissions AS perm ON apps.id = perm.app_id
WHERE perm.user_id = ?
GROUP BY apps.id, apps.name, apps.uid, apps.title, apps.description
ORDER BY ${orderByField} ${sortDirection}
LIMIT ?
OFFSET ?`,
[actor.user.id, limit, offset],
)) as Array<Record<string, unknown>>;
const apiBaseUrl = extension.config.api_base_url as string | undefined;
res.json(
installedApps.map((app) => ({
...app,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
})),
);
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: {
scope: 'installed-apps',
limit: 120,
window: 60_000,
key: 'user',
},
},
handleInstalledApps,
);
+312
View File
@@ -0,0 +1,312 @@
import type { Request, Response } from 'express';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { makeActor, type Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import {
handleMeteringAllCosts,
handleMeteringGlobalUsage,
handleMeteringUsage,
handleMeteringUsageForApp,
} from './metering.ts';
interface CapturedResponse {
body: unknown;
}
const makeReq = (
init: { params?: Record<string, unknown> } = {},
): Request =>
({
params: init.params ?? {},
query: {},
}) as unknown as Request;
const makeRes = () => {
const captured: CapturedResponse = { body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const seedUser = async () => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.user.create({
username: `muser_${slug}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
};
describe('metering extension — handleMeteringUsage', () => {
it('returns usage details merged with allowanceInfo for an authenticated user', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleMeteringUsage(makeReq(), res),
);
// We don't assert the inner shape (provider-specific) — only that
// the handler returned a JSON object that carries `allowanceInfo`.
expect(typeof captured.body).toBe('object');
expect(captured.body).not.toBeNull();
expect(
(captured.body as Record<string, unknown>).allowanceInfo,
).toBeDefined();
});
describe('usage split across apps', () => {
const seedUsage = async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const metering = server.services.metering;
await metering.incrementUsage(
makeActor({ user: owner, app: { uid: 'app-mine' } }),
'kv:read',
1,
10,
);
await metering.incrementUsage(
makeActor({ user: owner, app: { uid: 'app-other' } }),
'kv:write',
1,
20,
);
await metering.incrementUsage(
makeActor({ user: owner }),
'kv:list',
1,
40,
);
await server.stores.meteringBuffer.flushCycle();
return owner;
};
const usageAs = async (actor: Actor) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleMeteringUsage(makeReq(), res),
);
return captured.body as {
usage: Record<string, unknown> & { total: number };
appTotals: Record<string, unknown>;
allowanceInfo: Record<string, unknown>;
};
};
it('reports only the calling app’s own usage to an app actor', async () => {
const owner = await seedUsage();
const body = await usageAs(
makeActor({ user: owner, app: { uid: 'app-mine' } }),
);
expect(body.usage.total).toBe(10);
expect(Object.keys(body.usage).sort()).toEqual([
'kv:read',
'total',
]);
expect(body.appTotals).toEqual({
'app-mine': { total: 10, count: 1 },
});
expect(body.allowanceInfo).toMatchObject({
remaining: expect.any(Number),
monthUsageAllowance: expect.any(Number),
});
});
it('still reports the whole account to the user', async () => {
const owner = await seedUsage();
const body = await usageAs(makeActor({ user: owner }));
expect(body.usage.total).toBe(70);
expect(body.usage).toHaveProperty('kv:read');
expect(body.usage).toHaveProperty('kv:write');
expect(body.usage).toHaveProperty('kv:list');
expect(body.appTotals).toMatchObject({
'app-mine': { total: 10, count: 1 },
'app-other': { total: 20, count: 1 },
});
});
});
});
describe('metering extension — handleMeteringUsageForApp', () => {
it('throws HttpError(400) when no appId is supplied', async () => {
const user = await seedUser();
const { res } = makeRes();
await expect(
runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() =>
handleMeteringUsageForApp(
makeReq({ params: { appIdOrName: '' } }),
res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws HttpError(404) when looking up an unknown app by name', async () => {
const user = await seedUser();
const { res } = makeRes();
await expect(
runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() =>
handleMeteringUsageForApp(
makeReq({ params: { appIdOrName: 'no-such-app' } }),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
describe('scoped to the calling app', () => {
const appUsageAs = async (actor: Actor, appIdOrName: string) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleMeteringUsageForApp(
makeReq({ params: { appIdOrName } }),
res,
),
);
return captured.body as { total: number };
};
const seedApps = async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const mine = `app-${uuidv4()}`;
const other = `app-${uuidv4()}`;
await server.services.metering.incrementUsage(
makeActor({ user: owner, app: { uid: mine } }),
'kv:read',
1,
10,
);
await server.services.metering.incrementUsage(
makeActor({ user: owner, app: { uid: other } }),
'kv:write',
1,
20,
);
await server.stores.meteringBuffer.flushCycle();
return {
owner,
mine,
other,
asMine: makeActor({ user: owner, app: { uid: mine } }),
};
};
it('lets an app read its own usage', async () => {
const { mine, asMine } = await seedApps();
expect((await appUsageAs(asMine, mine)).total).toBe(10);
});
it('refuses an app another app’s usage', async () => {
const { other, asMine } = await seedApps();
await expect(appUsageAs(asMine, other)).rejects.toMatchObject({
statusCode: 403,
});
});
it('refuses an app another app’s usage by name', async () => {
const { owner, other, asMine } = await seedApps();
const name = `other-${uuidv4()}`;
await server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[other, name, name, `https://${name}.example/`, owner.id],
);
await expect(appUsageAs(asMine, name)).rejects.toMatchObject({
statusCode: 403,
});
});
it('refuses an app the usage outside any app', async () => {
const { asMine } = await seedApps();
await expect(appUsageAs(asMine, 'os-global')).rejects.toMatchObject(
{ statusCode: 403 },
);
});
it('still lets the user read any of their apps', async () => {
const { owner, mine, other } = await seedApps();
const asUser = makeActor({ user: owner });
expect((await appUsageAs(asUser, mine)).total).toBe(10);
expect((await appUsageAs(asUser, other)).total).toBe(20);
});
it('lets the user read their usage outside any app', async () => {
const { owner } = await seedApps();
const asUser = makeActor({ user: owner });
await server.services.metering.incrementUsage(
asUser,
'kv:read',
1,
5,
);
await server.stores.meteringBuffer.flushCycle();
expect((await appUsageAs(asUser, 'os-global')).total).toBe(5);
});
});
});
describe('metering extension — handleMeteringGlobalUsage', () => {
it('returns the global usage payload from MeteringService', async () => {
const { res, captured } = makeRes();
await handleMeteringGlobalUsage(makeReq(), res);
// Just confirm a JSON body was returned. Inner shape comes from
// MeteringService and is covered elsewhere.
expect(captured.body).toBeDefined();
});
});
describe('metering extension — handleMeteringAllCosts', () => {
it('returns a { costs: [...] } payload', async () => {
const { res, captured } = makeRes();
await handleMeteringAllCosts(makeReq(), res);
const body = captured.body as { costs: unknown };
expect(Array.isArray(body.costs)).toBe(true);
});
it('caches the costs catalogue across calls (same array reference)', async () => {
const a = makeRes();
const b = makeRes();
await handleMeteringAllCosts(makeReq(), a.res);
await handleMeteringAllCosts(makeReq(), b.res);
const costsA = (a.captured.body as { costs: unknown[] }).costs;
const costsB = (b.captured.body as { costs: unknown[] }).costs;
// Cache fields the same array instance — this is the property we
// actually want to lock down (no rewalk of every driver/controller
// per request).
expect(costsA).toBe(costsB);
});
});
+198 -53
View File
@@ -3,11 +3,18 @@ import { HttpError } from '@heyputer/backend/src/core/http';
import {
controllersContainers,
driversContainers,
servicesContainers,
} from '@heyputer/backend/src/exports';
import { extension } from '@heyputer/backend/src/extensions';
import { GLOBAL_APP_KEY } from '@heyputer/backend/src/services/metering/consts';
import {
creditMultiplierFrom,
toCredits,
} from '@heyputer/backend/src/services/metering/utils';
import type { Request, Response } from 'express';
const services = extension.import('service');
const clients = extension.import('client');
const stores = extension.import('store');
// Cached on first request — the underlying cost catalogues are baked into
// driver/controller source so they only change on deploy.
@@ -17,7 +24,7 @@ function collectAllCosts(): Record<string, unknown>[] {
const all: Record<string, unknown>[] = [];
const collect = (
source: Record<string, unknown>,
kind: 'driver' | 'controller',
kind: 'driver' | 'controller' | 'service',
) => {
for (const [name, instance] of Object.entries(source)) {
const fn = (
@@ -42,73 +49,211 @@ function collectAllCosts(): Record<string, unknown>[] {
};
collect(driversContainers as Record<string, unknown>, 'driver');
collect(controllersContainers as Record<string, unknown>, 'controller');
// Services report the costs that aren't tied to one endpoint — egress,
// which is metered for every response there is.
collect(servicesContainers as Record<string, unknown>, 'service');
return all;
}
// -- Credits scaling ---------------------------------------------------
//
// When the deployment configures a credit multiplier, every monetary amount
// is scaled to display credits BEFORE it leaves the server — raw metered
// amounts are never reported. Counts, units, and byte figures pass through
// untouched. A response that was scaled says so (`unit: 'credits'`); without
// a configured multiplier amounts pass through raw and clients render them
// as dollars.
/** The per-usage-type records of a UsageByType map, costs scaled. */
const scaleUsageByType = (
usage: Record<string, unknown>,
multiplier: number,
): Record<string, unknown> => {
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(usage)) {
// `allowanceUsed` is the month's allowance-charged spend — monetary,
// scalar, scaled like the total.
if (key === 'total' || key === 'allowanceUsed') {
out[key] = toCredits(Number(value) || 0, multiplier);
} else if (
value &&
typeof value === 'object' &&
typeof (value as { cost?: unknown }).cost === 'number'
) {
out[key] = {
...(value as Record<string, unknown>),
cost: toCredits((value as { cost: number }).cost, multiplier),
};
} else {
out[key] = value;
}
}
return out;
};
export const handleMeteringUsage = async (
_req: Request,
res: Response,
): Promise<void> => {
// Behind `requireAuth`, so there is always an actor.
const actor = Context.get('actor')!;
const [actorUsage, allowanceInfo] = await Promise.all([
services.metering.getActorCurrentMonthUsageDetails(actor),
services.metering.getAllowedUsage(actor),
]);
const multiplier = creditMultiplierFrom(extension.config);
if (!multiplier) {
res.json({ ...actorUsage, allowanceInfo });
return;
}
res.json({
usage: scaleUsageByType(
actorUsage.usage as unknown as Record<string, unknown>,
multiplier,
),
appTotals: Object.fromEntries(
Object.entries(actorUsage.appTotals).map(([appId, totals]) => [
appId,
{ ...totals, total: toCredits(totals.total, multiplier) },
]),
),
allowanceInfo: {
...allowanceInfo,
remaining: toCredits(allowanceInfo.remaining, multiplier),
monthUsageAllowance: toCredits(
allowanceInfo.monthUsageAllowance,
multiplier,
),
// Monetary addon fields scale; absent ones stay absent rather
// than materializing as NaN/null. purchasedStorage is bytes.
addons: Object.fromEntries(
Object.entries(allowanceInfo.addons ?? {}).map(([k, v]) => [
k,
(k === 'purchasedCredits' ||
k === 'consumedPurchaseCredits') &&
typeof v === 'number'
? toCredits(v, multiplier)
: v,
]),
),
unit: 'credits',
},
});
};
export const handleMeteringUsageForApp = async (
req: Request,
res: Response,
): Promise<void> => {
// Behind `requireAuth`, so there is always an actor.
const actor = Context.get('actor')!;
let appId = String(req.params.appIdOrName ?? '');
if (!appId) throw new HttpError(400, 'appId parameter is required');
// If not a UUID-shaped app UID or the global sentinel, look up by name.
// Which apps an actor may read is MeteringService's call, not this route's.
if (!appId.startsWith('app-') && appId !== GLOBAL_APP_KEY) {
const app = await stores.app.getByName(appId);
if (!app) throw new HttpError(404, 'App not found');
appId = app.uid;
}
const appUsage =
await services.metering.getActorCurrentMonthAppUsageDetails(
actor,
appId,
);
const multiplier = creditMultiplierFrom(extension.config);
if (!multiplier) {
res.json(appUsage);
return;
}
res.json({
...scaleUsageByType(
appUsage as unknown as Record<string, unknown>,
multiplier,
),
unit: 'credits',
});
};
export const handleMeteringGlobalUsage = async (
_req: Request,
res: Response,
): Promise<void> => {
const globalUsage = await services.metering.getGlobalUsage();
res.json(globalUsage);
};
// First hit walks the registries; subsequent hits serve the in-memory cache.
export const handleMeteringAllCosts = async (
_req: Request,
res: Response,
): Promise<void> => {
if (!cachedAllCosts) {
const raw = collectAllCosts();
const multiplier = creditMultiplierFrom(extension.config);
// Price the catalogue in the same unit the usage endpoints report:
// per-unit credits when a multiplier is configured, raw otherwise.
cachedAllCosts = multiplier
? raw.map(({ ucentsPerUnit, costs_currency: _cc, ...rest }) =>
typeof ucentsPerUnit === 'number'
? {
...rest,
creditsPerUnit: toCredits(
ucentsPerUnit,
multiplier,
),
}
: rest,
)
: raw;
}
res.json({ costs: cachedAllCosts });
};
/** Dashboard reads over the per-actor KV aggregates. */
const USAGE_READ_LIMIT = {
scope: 'metering-usage',
limit: 120,
window: 60_000,
key: 'user' as const,
};
extension.get(
'/metering/usage',
{ subdomain: 'api', requireAuth: true },
async (req, res) => {
const actor = Context.get('actor');
if (!actor?.user) throw new HttpError(401, 'Authentication required');
const [actorUsage, allowanceInfo] = await Promise.all([
services.metering.getActorCurrentMonthUsageDetails(actor),
services.metering.getAllowedUsage(actor),
]);
res.json({ ...actorUsage, allowanceInfo });
},
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringUsage,
);
extension.get(
'/metering/usage/:appIdOrName',
{ subdomain: 'api', requireAuth: true },
async (req, res) => {
const actor = Context.get('actor');
if (!actor?.user) throw new HttpError(401, 'Authentication required');
let appId = String(req.params.appIdOrName ?? '');
if (!appId) throw new HttpError(400, 'appId parameter is required');
// If not a UUID-shaped app UID, look up by name
if (!appId.startsWith('app-')) {
const appRows = (await clients.db.read(
'SELECT `uid` FROM `apps` WHERE `name` = ? LIMIT 1',
[appId],
)) as Array<{ uid: string }>;
if (appRows.length > 0) {
appId = appRows[0].uid;
} else {
throw new HttpError(404, 'App not found');
}
}
const appUsage =
await services.metering.getActorCurrentMonthAppUsageDetails(
actor,
appId,
);
res.json(appUsage);
},
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringUsageForApp,
);
extension.get(
'/metering/globalUsage',
{ subdomain: 'api', adminOnly: true },
async (_req, res) => {
const globalUsage = await services.metering.getGlobalUsage();
res.json(globalUsage);
{
subdomain: 'api',
adminOnly: true,
// Sums across every shard of the global aggregate. Admin-gated, so
// this is loop protection — but one accidental poll is an
// expensive minute.
rateLimit: {
scope: 'metering-global-usage',
limit: 10,
window: 60_000,
key: 'user',
},
},
handleMeteringGlobalUsage,
);
// First hit walks the registries; subsequent hits serve the in-memory cache.
extension.get(
'/metering/allCosts',
{ subdomain: 'api', requireAuth: true },
async (_req, res) => {
if (!cachedAllCosts) {
cachedAllCosts = collectAllCosts();
}
res.json({ costs: cachedAllCosts });
},
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringAllCosts,
);
+3
View File
@@ -0,0 +1,3 @@
{
"type": "commonjs"
}
+103
View File
@@ -0,0 +1,103 @@
import type { Request, Response } from 'express';
import { describe, expect, it, vi } from 'vitest';
import { handleServerInfo } from './serverInfo.ts';
interface CapturedResponse {
body: Record<string, unknown> | undefined;
}
const makeRes = () => {
const captured: CapturedResponse = { body: undefined };
const res = {
json: vi.fn((value: Record<string, unknown>) => {
captured.body = value;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
describe('serverInfo extension — handleServerInfo', () => {
it('returns the full server info payload', async () => {
const { res, captured } = makeRes();
await handleServerInfo({} as Request, res);
expect(captured.body).toBeDefined();
const body = captured.body!;
// OS section
const os = body.os as Record<string, unknown>;
expect(typeof os.platform).toBe('string');
expect(typeof os.type).toBe('string');
expect(typeof os.release).toBe('string');
expect(os.pretty).toBe(`${os.type} ${os.release}`);
// CPU section
const cpu = body.cpu as Record<string, unknown>;
expect(typeof cpu.model).toBe('string');
expect(typeof cpu.cores).toBe('number');
expect((cpu.cores as number) > 0).toBe(true);
// RAM — totalGB / freeGB are stringified two-decimal values
const ram = body.ram as Record<string, unknown>;
expect(typeof ram.total).toBe('number');
expect(typeof ram.free).toBe('number');
expect(ram.totalGB).toMatch(/^\d+\.\d{2}$/);
expect(ram.freeGB).toMatch(/^\d+\.\d{2}$/);
// Uptime fields are numeric (seconds/days/hours/minutes) plus pretty.
const uptime = body.uptime as Record<string, unknown>;
expect(typeof uptime.seconds).toBe('number');
expect(typeof uptime.days).toBe('number');
expect(typeof uptime.hours).toBe('number');
expect(typeof uptime.minutes).toBe('number');
expect(uptime.pretty).toMatch(/^\d+d \d+h \d+m$/);
// Disk may fall through to N/A when statfs throws (e.g. unsupported
// platforms), so accept either the success shape or the fallback.
const disk = body.disk as Record<string, unknown>;
expect(['N/A']).toContain(disk.total === 'N/A' ? 'N/A' : 'N/A'); // keep shape-only assertion
expect(typeof disk.total).toBe('string');
expect(typeof disk.free).toBe('string');
expect(typeof disk.used).toBe('string');
expect(Array.isArray(body.loadavg)).toBe(true);
expect(typeof body.hostname).toBe('string');
});
it('falls back to N/A disk stats when statfs throws', async () => {
const fs = await import('node:fs/promises');
const statfsSpy = vi
.spyOn(fs.default, 'statfs')
.mockRejectedValue(new Error('statfs unsupported'));
const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
const { res, captured } = makeRes();
await handleServerInfo({} as Request, res);
const disk = (captured.body as Record<string, unknown>).disk as Record<
string,
string
>;
expect(disk).toEqual({ total: 'N/A', free: 'N/A', used: 'N/A' });
expect(errSpy).toHaveBeenCalled();
statfsSpy.mockRestore();
errSpy.mockRestore();
});
it('reports an unknown CPU model when os.cpus() comes back empty', async () => {
const os = await import('node:os');
const cpusSpy = vi.spyOn(os.default, 'cpus').mockReturnValue([]);
const { res, captured } = makeRes();
await handleServerInfo({} as Request, res);
const cpu = captured.body!.cpu as Record<string, unknown>;
expect(cpu.model).toBe('Unknown');
expect(cpu.cores).toBe(0);
cpusSpy.mockRestore();
});
});
+65 -59
View File
@@ -1,67 +1,73 @@
import type { Request, Response } from 'express';
import { extension } from '@heyputer/backend/src/extensions';
import fs from 'fs/promises';
import os from 'os';
export const handleServerInfo = async (
_req: Request,
res: Response,
): Promise<void> => {
const osData = {
platform: os.platform(),
type: os.type(),
release: os.release(),
pretty: `${os.type()} ${os.release()}`,
};
const cpus = os.cpus();
const cpuData = {
model: cpus[0]?.model || 'Unknown',
cores: cpus.length,
};
const ramData = {
total: os.totalmem(),
free: os.freemem(),
totalGB: (os.totalmem() / 1073741824).toFixed(2),
freeGB: (os.freemem() / 1073741824).toFixed(2),
};
const uptimeSeconds = os.uptime();
const uptimeData = {
seconds: uptimeSeconds,
days: Math.floor(uptimeSeconds / 86400),
hours: Math.floor((uptimeSeconds % 86400) / 3600),
minutes: Math.floor((uptimeSeconds % 3600) / 60),
pretty: `${Math.floor(uptimeSeconds / 86400)}d ${Math.floor((uptimeSeconds % 86400) / 3600)}h ${Math.floor((uptimeSeconds % 3600) / 60)}m`,
};
let diskData: Record<string, string> = {
total: 'N/A',
free: 'N/A',
used: 'N/A',
};
try {
const stats = await fs.statfs('/');
const totalGB = (stats.blocks * stats.bsize) / 1073741824;
const freeGB = (stats.bfree * stats.bsize) / 1073741824;
const usedGB = (totalGB - freeGB).toFixed(2);
diskData = {
total: totalGB.toFixed(2),
free: freeGB.toFixed(2),
used: usedGB,
};
} catch (err) {
console.error('Disk stats error:', err);
}
res.json({
os: osData,
cpu: cpuData,
ram: ramData,
uptime: uptimeData,
disk: diskData,
loadavg: os.loadavg(),
hostname: os.hostname(),
});
};
extension.get(
'/serverInfo',
{ subdomain: 'api', adminOnly: true },
async (_req, res) => {
const osData = {
platform: os.platform(),
type: os.type(),
release: os.release(),
pretty: `${os.type()} ${os.release()}`,
};
const cpus = os.cpus();
const cpuData = {
model: cpus[0]?.model || 'Unknown',
cores: cpus.length,
};
const ramData = {
total: os.totalmem(),
free: os.freemem(),
totalGB: (os.totalmem() / 1073741824).toFixed(2),
freeGB: (os.freemem() / 1073741824).toFixed(2),
};
const uptimeSeconds = os.uptime();
const uptimeData = {
seconds: uptimeSeconds,
days: Math.floor(uptimeSeconds / 86400),
hours: Math.floor((uptimeSeconds % 86400) / 3600),
minutes: Math.floor((uptimeSeconds % 3600) / 60),
pretty: `${Math.floor(uptimeSeconds / 86400)}d ${Math.floor((uptimeSeconds % 86400) / 3600)}h ${Math.floor((uptimeSeconds % 3600) / 60)}m`,
};
let diskData: Record<string, string> = {
total: 'N/A',
free: 'N/A',
used: 'N/A',
};
try {
const stats = await fs.statfs('/');
const totalGB = (stats.blocks * stats.bsize) / 1073741824;
const freeGB = (stats.bfree * stats.bsize) / 1073741824;
const usedGB = (totalGB - freeGB).toFixed(2);
diskData = {
total: totalGB.toFixed(2),
free: freeGB.toFixed(2),
used: usedGB,
};
} catch (err) {
console.error('Disk stats error:', err);
}
res.json({
os: osData,
cpu: cpuData,
ram: ramData,
uptime: uptimeData,
disk: diskData,
loadavg: os.loadavg(),
hostname: os.hostname(),
});
},
handleServerInfo,
);
File diff suppressed because it is too large. Load diff
+413 -125
View File
@@ -1,18 +1,95 @@
import {
DeleteObjectCommand,
GetObjectCommand,
HeadObjectCommand,
PutObjectCommand,
S3Client,
} from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { extension } from '@heyputer/backend/src/extensions';
import type { FSEntryStore } from '@heyputer/backend/src/stores/fs/FSEntryStore';
import { isMissingObjectError } from '@heyputer/backend/src/stores/fs/S3ObjectStore';
import crypto from 'node:crypto';
import type { Readable } from 'node:stream';
import sharp from 'sharp';
const clients = extension.import('client');
const stores = extension.import('store');
type ThumbnailEntryStore = Pick<FSEntryStore, 'updateEntryThumbnailByUuid'>;
const MAX_THUMBNAIL_BYTES = 2 * 1024 * 1024;
const MAX_THUMBNAIL_PIXELS = 64e6;
// Namespace every object this extension writes. An fs object's key is its
// bare fsentry uuid and the default config points `thumbnailStore.name` at
// the same bucket as `s3_bucket`, so without a prefix of our own there is no
// way to tell a thumbnail we minted from any other object in the deployment.
const THUMBNAIL_KEY_PREFIX = 'thumbnails/';
const UUID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
const isUuid = (value: unknown): value is string =>
typeof value === 'string' && UUID_PATTERN.test(value);
// `thumbnails/<entry uuid>/<random uuid>`: the entry segment binds the object
// to the entry it was minted for, which is what writes and deletes check.
const mintThumbnailKey = (entryUuid: string): string =>
`${THUMBNAIL_KEY_PREFIX}${entryUuid}/${crypto.randomUUID()}`;
/**
* Extract the object key from a stored thumbnail pointer, or null when the
* pointer isn't one this extension minted.
*
* `fsentries.thumbnail` is writable through the FS API, so neither half of the
* stored string is trusted: the bucket is discarded (callers always pass their
* own) and the key must sit under {@link THUMBNAIL_KEY_PREFIX} as either
* `<uuid>` (minted before keys were bound to entries) or `<entry uuid>/<uuid>`.
* Honouring an arbitrary key would lend this extension's storage credentials to
* whatever object the caller named — in the shared-bucket layout that is every
* user's file, since an fs object's key is its fsentry uuid. Bare-uuid keys
* from before the prefix fail the check and are treated as absent; they are
* indistinguishable from a planted pointer, so there is nothing safer to do
* with them than stop signing them.
*/
const resolveThumbnailKey = (pointer: string): string | null => {
let key: string;
if (pointer.startsWith('s3://')) {
const rest = pointer.slice('s3://'.length);
const slash = rest.indexOf('/');
if (slash === -1) return null;
key = rest.slice(slash + 1);
} else {
let pathname: string;
try {
pathname = new URL(pointer).pathname;
} catch {
return null;
}
const segments = pathname.replace(/^\/+/, '').split('/');
segments.shift(); // bucket
key = segments.join('/');
}
if (!key.startsWith(THUMBNAIL_KEY_PREFIX)) return null;
const segments = key.slice(THUMBNAIL_KEY_PREFIX.length).split('/');
return segments.length <= 2 && segments.every(isUuid) ? key : null;
};
// The entry a resolved key was minted for. Null for an unbound key, whose
// holder can't be told apart from a row that copied or planted the pointer.
const thumbnailKeyOwner = (key: string): string | null => {
const segments = key.slice(THUMBNAIL_KEY_PREFIX.length).split('/');
return segments.length === 2 ? segments[0] : null;
};
// Whether a stored thumbnail names an object in our storage. Anything else is
// an external image URL the client supplied: shown as given, never copied or
// deleted.
const isStoragePointer = (pointer: string, bucketEndpoint: string): boolean =>
pointer.startsWith('s3://') ||
// Legacy format — remove after full migration
(pointer.startsWith('https') &&
pointer.includes(new URL(bucketEndpoint).hostname));
// S3 client + bucket config — lazily resolved after boot from config.
let s3Client: S3Client | null = null;
let s3PresignClient: S3Client | null = null;
@@ -62,13 +139,6 @@ function getPresignClient(): S3Client {
return resolveClients().presign;
}
function base64ParseDataUrl(dataURL: string) {
dataURL = dataURL.slice(5);
const mimeType = dataURL.split(';')[0];
const data = Buffer.from(dataURL.split(',')[1], 'base64');
return { mimeType, data };
}
// Strictly decode a data: URL and validate the decoded image. Encoded-string
// length lies about decoded byte count (whitespace, padding) and says nothing
// about pixel count — a 2MB PNG can decompress to hundreds of MB of raster.
@@ -95,13 +165,73 @@ async function decodeAndValidateThumbnail(
return { mimeType, data };
}
// ── thumbnail.created ───────────────────────────────────────────────
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer.
// A storage pointer a client hands back after a presigned upload is kept only
// if it names a key minted for this entry and an object within the size bound.
async function vetUploadedThumbnail(
pointer: string,
entryUuid: unknown,
deps: { s3: S3Client; bucketName: string },
): Promise<string | null> {
const key = resolveThumbnailKey(pointer);
if (!key || thumbnailKeyOwner(key) !== entryUuid) return null;
extension.on('thumbnail.created', async (event: Record<string, unknown>) => {
let size: number | undefined;
try {
const head = await deps.s3.send(
new HeadObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
size = head.ContentLength;
} catch (err) {
if (!isMissingObjectError(err)) {
console.warn(
'[thumbnails] failed to check uploaded thumbnail',
err,
);
}
return null;
}
if (typeof size !== 'number') return null;
if (size > MAX_THUMBNAIL_BYTES) {
// Only reachable on a store that doesn't enforce the signed length.
// The key is this entry's own, so removing it is safe.
try {
await deps.s3.send(
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
} catch (err) {
console.warn(
'[thumbnails] failed to remove oversized thumbnail',
err,
);
}
return null;
}
return `s3://${deps.bucketName}/${key}`;
}
// -- thumbnail.created -----------------------------------------------
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer bound to the entry.
export async function handleThumbnailCreated(
event: Record<string, unknown>,
deps: { s3: S3Client; bucketName: string; bucketEndpoint: string },
): Promise<void> {
const url = event.url;
if (typeof url !== 'string' || !url.startsWith('data:')) return;
if (typeof url !== 'string') return;
if (!url.startsWith('data:')) {
if (isStoragePointer(url, deps.bucketEndpoint)) {
event.url = await vetUploadedThumbnail(url, event.uuid, deps);
}
return;
}
const entryUuid = event.uuid;
if (!isUuid(entryUuid)) {
event.url = null;
return;
}
const decoded = await decodeAndValidateThumbnail(url);
if (!decoded) {
@@ -109,137 +239,295 @@ extension.on('thumbnail.created', async (event: Record<string, unknown>) => {
return;
}
const key = crypto.randomUUID();
event.url = `s3://${thumbnailBucketName}/${key}`;
const key = mintThumbnailKey(entryUuid);
event.url = `s3://${deps.bucketName}/${key}`;
await getClient().send(
await deps.s3.send(
new PutObjectCommand({
Bucket: thumbnailBucketName,
Bucket: deps.bucketName,
Key: key,
Body: decoded.data,
ContentType: decoded.mimeType,
}),
);
});
}
// ── thumbnail.upload.prepare ────────────────────────────────────────
export const handleThumbnailUploadPrepare = async (
event: Record<string, unknown>,
deps: { s3Presign: S3Client; bucketName: string },
): Promise<void> => {
if (!event || !Array.isArray(event.items)) return;
const presignClient = deps.s3Presign;
for (const item of event.items as Array<Record<string, unknown>>) {
if (!item || typeof item !== 'object') {
throw new Error('thumbnail.upload.prepare item is invalid');
}
const contentType =
typeof item.contentType === 'string' ? item.contentType.trim() : '';
if (!contentType) continue;
const size = item.size;
if (
typeof size !== 'number' ||
!Number.isInteger(size) ||
size < 0 ||
size > MAX_THUMBNAIL_BYTES
)
continue;
if (!isUuid(item.item_uid)) continue;
const key = mintThumbnailKey(item.item_uid);
const command = new PutObjectCommand({
Bucket: deps.bucketName,
Key: key,
ContentType: contentType,
// Signing covers `content-length`, so a body of any other size
// fails the signature instead of landing.
ContentLength: size,
});
item.uploadUrl = await getSignedUrl(presignClient, command, {
expiresIn: 900,
});
item.thumbnailUrl = `s3://${deps.bucketName}/${key}`;
}
};
export const handleThumbnailRead = async (
entry: Record<string, unknown>,
deps: {
s3: S3Client;
s3Presign: S3Client;
bucketName: string;
bucketEndpoint: string;
fsEntry: ThumbnailEntryStore;
},
): Promise<void> => {
const thumb = entry.thumbnail;
if (typeof thumb !== 'string' || !thumb) return;
const presignClient = deps.s3Presign;
if (isStoragePointer(thumb, deps.bucketEndpoint)) {
const key = resolveThumbnailKey(thumb);
if (!key) {
// Not a pointer we minted — refuse to sign it rather than hand
// out a presigned read of whatever object it names.
entry.thumbnail = null;
return;
}
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
{ expiresIn: 604800 },
);
} else if (thumb.startsWith('data')) {
// Move an inline thumbnail into storage. The row only takes the new
// pointer while it still holds this data URL, so of concurrent reads
// one wins and the rest discard their upload.
const uuid = entry.uuid ?? entry.uid;
if (!isUuid(uuid)) return;
// Same bar as an upload; anything else stays inline, unmigrated.
const decoded = await decodeAndValidateThumbnail(thumb);
if (!decoded) return;
const key = mintThumbnailKey(uuid);
await deps.s3.send(
new PutObjectCommand({
Bucket: deps.bucketName,
Key: key,
Body: decoded.data,
ContentType: decoded.mimeType,
}),
);
let migrated: boolean;
try {
migrated = await deps.fsEntry.updateEntryThumbnailByUuid(
uuid,
`s3://${deps.bucketName}/${key}`,
thumb,
);
} catch (err) {
// The row may or may not point at the object now, so keep it.
console.warn('[thumbnails] inline migration failed', err);
return;
}
if (!migrated) {
try {
await deps.s3.send(
new DeleteObjectCommand({
Bucket: deps.bucketName,
Key: key,
}),
);
} catch (err) {
console.warn(
'[thumbnails] failed to remove unused thumbnail',
err,
);
}
return;
}
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
{ expiresIn: 604800 },
);
}
};
export const handleFsCopyNodeThumbnail = async (
payload: { copy?: { thumbnail?: string | null; uuid?: string } | null },
deps: {
s3: S3Client;
bucketName: string;
bucketEndpoint: string;
fsEntry: ThumbnailEntryStore;
},
): Promise<void> => {
const copy = payload.copy;
const thumbnailUrl = copy?.thumbnail;
if (!copy || !isUuid(copy.uuid) || typeof thumbnailUrl !== 'string') return;
if (!isStoragePointer(thumbnailUrl, deps.bucketEndpoint)) return;
// Same trust rule as the read and remove paths: only touch objects this
// extension minted.
const sourceKey = resolveThumbnailKey(thumbnailUrl);
if (!sourceKey) return;
// The copied row points at the SAME object as its source. Give the copy
// an object of its own, bound to it, so removing either entry leaves the
// other's thumbnail alone.
const newKey = mintThumbnailKey(copy.uuid);
try {
// Nothing larger than an upload may produce gets duplicated; such an
// object came in through an unbounded upload URL.
const head = await deps.s3.send(
new HeadObjectCommand({ Bucket: deps.bucketName, Key: sourceKey }),
);
if (
typeof head.ContentLength !== 'number' ||
head.ContentLength > MAX_THUMBNAIL_BYTES
) {
throw new Error('thumbnail exceeds the size bound');
}
// Read and rewrite rather than CopyObject: a copy source skips any path
// in the client's endpoint, so it can miss an object Bucket/Key reach.
const source = await deps.s3.send(
new GetObjectCommand({ Bucket: deps.bucketName, Key: sourceKey }),
);
// The object can be replaced after the HEAD; bound what gets buffered.
if ((source.ContentLength ?? 0) > MAX_THUMBNAIL_BYTES) {
(source.Body as Readable | undefined)?.destroy();
throw new Error('thumbnail exceeds the size bound');
}
const body = await source.Body?.transformToByteArray();
if (!body) throw new Error('thumbnail has no body');
await deps.s3.send(
new PutObjectCommand({
Bucket: deps.bucketName,
Key: newKey,
Body: body,
ContentType: source.ContentType,
}),
);
} catch (err) {
// The shared object is gone or oversized — drop the pointer rather
// than leave the row advertising a thumbnail it doesn't have.
await deps.fsEntry.updateEntryThumbnailByUuid(copy.uuid, null);
console.warn('[thumbnails] failed to duplicate thumbnail on copy', err);
return;
}
await deps.fsEntry.updateEntryThumbnailByUuid(
copy.uuid,
`s3://${deps.bucketName}/${newKey}`,
);
};
export const handleFsRemoveNodeThumbnail = async (
payload: { target: { thumbnail?: string | null; uuid?: string } },
deps: { s3: S3Client; bucketName: string; bucketEndpoint: string },
): Promise<void> => {
const { thumbnail: thumbnailUrl, uuid } = payload.target;
if (!thumbnailUrl || !isStoragePointer(thumbnailUrl, deps.bucketEndpoint))
return;
// The pointer decides which object gets deleted, and a key is visible to
// anyone who can list the entry holding it. Only a key minted for this
// entry is deleted; an unbound one may be shared or planted, so it stays.
const key = resolveThumbnailKey(thumbnailUrl);
if (!key || thumbnailKeyOwner(key) !== uuid) return;
await deps.s3.send(
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
};
extension.on(
'thumbnail.created',
async (_key, event: Record<string, unknown>) => {
await handleThumbnailCreated(event, {
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
});
},
);
// -- thumbnail.upload.prepare ----------------------------------------
// Generate pre-signed upload URLs so the client can PUT directly to S3.
extension.on(
'thumbnail.upload.prepare',
async (event: Record<string, unknown>) => {
if (!event || !Array.isArray(event.items)) return;
const presignClient = getPresignClient();
for (const item of event.items as Array<Record<string, unknown>>) {
if (!item || typeof item !== 'object') {
throw new Error('thumbnail.upload.prepare item is invalid');
}
const contentType =
typeof item.contentType === 'string'
? item.contentType.trim()
: '';
if (!contentType) continue;
if (item.size !== undefined) {
const size = Number(item.size);
if (
!Number.isFinite(size) ||
size < 0 ||
size > MAX_THUMBNAIL_BYTES
)
continue;
}
const key = crypto.randomUUID();
const command = new PutObjectCommand({
Bucket: thumbnailBucketName,
Key: key,
ContentType: contentType,
});
item.uploadUrl = await getSignedUrl(presignClient, command, {
expiresIn: 900,
});
item.thumbnailUrl = `s3://${thumbnailBucketName}/${key}`;
}
async (_key, event: Record<string, unknown>) => {
await handleThumbnailUploadPrepare(event, {
s3Presign: getPresignClient(),
bucketName: thumbnailBucketName,
});
},
);
// ── thumbnail.read ──────────────────────────────────────────────────
// -- thumbnail.read --------------------------------------------------
// Convert s3:// or legacy https:// thumbnails to signed URLs.
extension.on('thumbnail.read', async (entry: Record<string, unknown>) => {
const thumb = entry.thumbnail;
if (typeof thumb !== 'string' || !thumb) return;
const presignClient = getPresignClient();
if (thumb.startsWith('s3://')) {
const [bucket, key] = thumb.slice(5).split('/');
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: bucket, Key: key }),
{ expiresIn: 604800 },
);
} else if (
thumb.startsWith('https') &&
thumb.includes(new URL(extensionBucketEndpoint).hostname)
) {
// Legacy format — remove after full migration
const [bucket, key] = new URL(thumb).pathname.slice(1).split('/');
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: bucket, Key: key }),
{ expiresIn: 604800 },
);
} else if (thumb.startsWith('data')) {
// Inline data-URL migration: upload to S3 and update the DB entry.
const key = crypto.randomUUID();
const { mimeType, data } = base64ParseDataUrl(thumb);
const newUrl = `s3://${thumbnailBucketName}/${key}`;
await getClient().send(
new PutObjectCommand({
Bucket: thumbnailBucketName,
Key: key,
Body: data,
ContentType: mimeType,
}),
);
// Best-effort async DB update
const uuid = entry.uuid ?? entry.uid;
if (uuid) {
clients.db
.write(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[newUrl, uuid],
)
.catch((err: unknown) =>
console.warn('[thumbnails] inline migration failed', err),
);
}
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: thumbnailBucketName, Key: key }),
{ expiresIn: 604800 },
);
}
extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
await handleThumbnailRead(entry, {
s3: getClient(),
s3Presign: getPresignClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
fsEntry: stores.fsEntry,
});
});
// ── fs.remove.node ──────────────────────────────────────────────────
// -- fs.copy.node ----------------------------------------------------
// A copied entry initially shares its source's thumbnail object; duplicate
// it so removing either entry can't break the other's thumbnail.
extension.on('fs.copy.node', async (_key, payload) => {
await handleFsCopyNodeThumbnail(
payload as {
copy?: { thumbnail?: string | null; uuid?: string } | null;
},
{
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
fsEntry: stores.fsEntry,
},
);
});
// -- fs.remove.node --------------------------------------------------
// Delete S3 thumbnail when the file is removed.
extension.on(
'fs.remove.node',
async ({ target }: { target: Record<string, unknown> }) => {
const thumbnailUrl = target.thumbnail as string | undefined;
if (!thumbnailUrl || !thumbnailUrl.startsWith('s3://')) return;
const [bucket, key] = thumbnailUrl.slice(5).split('/');
await getClient().send(
new DeleteObjectCommand({ Bucket: bucket, Key: key }),
);
},
);
extension.on('fs.remove.node', async (_key, payload) => {
await handleFsRemoveNodeThumbnail(payload, {
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
});
});
+355
View File
@@ -0,0 +1,355 @@
import { EventEmitter } from 'node:events';
import { Readable } from 'node:stream';
import type { Request, Response } from 'express';
import { beforeEach, describe, expect, test, vi } from 'vitest';
const SECRET = 'ingress-secret';
// Only `extension` is stubbed; the secret comparison and the routing are the
// real code paths, and the fake service/store layer is the boundary.
const state = vi.hoisted(() => ({
config: { userEmail: { secret: 'ingress-secret' } } as Record<
string,
unknown
>,
routes: [] as Array<{
path: string;
handler: (req: unknown, res: unknown) => unknown;
}>,
users: new Map<string, Record<string, unknown>>(),
writes: [] as Array<{
userId: number;
path: string;
homeRegion?: string;
content?: unknown;
}>,
feedback: [] as Array<{ to: string; from: string | null; raw: Buffer }>,
takeFeedback: true,
}));
vi.mock('@heyputer/backend/src/extensions', () => ({
extension: {
get config() {
return state.config;
},
post(
path: string,
_opts: unknown,
handler: (req: unknown, res: unknown) => unknown,
) {
state.routes.push({ path, handler });
},
import(layer: string) {
if (layer === 'client') {
return {
event: {
async emitAndWait(
key: string,
data: {
to: string;
from: string | null;
raw: Buffer;
handled: boolean;
},
) {
if (key !== 'email.ingress.feedback') return;
state.feedback.push(data);
if (state.takeFeedback) data.handled = true;
},
},
};
}
if (layer === 'store') {
return {
user: {
getByUsername: async (username: string) =>
state.users.get(username) ?? null,
},
};
}
return {
fs: {
async write(
userId: number,
{
fileMetadata,
fileContent,
}: {
fileMetadata: { path: string };
fileContent: unknown;
},
_uploadTracker?: unknown,
_storageAllowanceMax?: number,
homeRegion?: string,
) {
state.writes.push({
userId,
path: fileMetadata.path,
homeRegion,
content: fileContent,
});
return { fsEntry: { path: fileMetadata.path } };
},
},
};
},
},
}));
import './index.js';
const permanent = (id: number, username: string) => ({
id,
username,
email: `${username}@example.com`,
password: '$2b$10$hash',
});
/** No password and no email is what makes an account temporary. */
const temp = (id: number, username: string) => ({
id,
username,
email: null,
password: null,
});
const ingress = async (
to: string,
over: { secret?: string; contentLength?: string | null } = {},
) => {
const route = state.routes.find((r) => r.path === '/email/ingress');
if (!route) throw new Error('route not registered');
const body = Buffer.from('From: [email protected]\r\n\r\nhello\r\n');
const req = Readable.from([body], { objectMode: false }) as unknown as {
headers: Record<string, string>;
query: Record<string, string>;
};
// `contentLength: null` sends no length header at all, which is the one
// framing the HTTP parser does not bound for us.
req.headers =
over.contentLength === null
? {}
: {
'content-length': over.contentLength ?? String(body.length),
};
req.query = {
SECRET: over.secret ?? SECRET,
to,
from: '[email protected]',
subject: 'hi',
};
let status = 200;
let ended = false;
// `end` takes (chunk, encoding) and no callback, the way the compression
// middleware leaves it, and emits `finish` — which is what the refusal
// paths hang their drain/destroy on. A stub that accepts a callback
// instead hides the bug that took the 413 path to production as a 500.
const emitter = new EventEmitter();
const res = {
status(code: number) {
status = code;
return res;
},
end(chunk?: unknown, encoding?: BufferEncoding) {
if (chunk) Buffer.byteLength(chunk as string, encoding);
ended = true;
emitter.emit('finish');
},
once(event: string, listener: () => void) {
emitter.once(event, listener);
return res;
},
} as unknown as Response;
await route.handler(req as unknown as Request, res);
return {
status,
ended,
drained: (req as unknown as Readable).readableFlowing,
destroyed: (req as unknown as Readable).destroyed,
};
};
beforeEach(() => {
state.users.clear();
state.writes.length = 0;
state.feedback.length = 0;
state.takeFeedback = true;
});
describe('temporary accounts cannot receive', () => {
test('mail for a temp account is accepted and dropped', async () => {
state.users.set('tempy', temp(1, 'tempy'));
const { status, ended, drained } = await ingress('[email protected]');
// Accepted, so the relay neither retries nor bounces to the sender.
expect(status).toBe(200);
expect(ended).toBe(true);
// The body is read off the wire rather than left unconsumed.
expect(drained).toBe(true);
expect(state.writes).toEqual([]);
});
test('mail for a permanent account still lands in the mailbox', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status } = await ingress('[email protected]');
expect(status).toBe(200);
expect(state.writes).toHaveLength(1);
expect(state.writes[0].userId).toBe(2);
expect(state.writes[0].path).toMatch(/^\/bob\/\.mail\/objects\//);
});
test('an unknown address is still rejected outright', async () => {
const { status } = await ingress('[email protected]');
expect(status).toBe(404);
expect(state.writes).toEqual([]);
});
});
describe('inbound mail is placed in the recipient home region', () => {
/** A permanent account with explicit placement columns. */
const placed = (
id: number,
username: string,
over: { home?: string | null; signup_server?: string | null },
) => ({
...permanent(id, username),
home: null,
signup_server: null,
...over,
});
test("uses the recipient's own home region when they have one", async () => {
state.users.set('bob', placed(2, 'bob', { home: 'frankfurt' }));
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('frankfurt');
});
test('falls back to the server that served their signup', async () => {
// Every account that predates the `home` column is this case.
state.users.set(
'bob',
placed(2, 'bob', { home: null, signup_server: 'london' }),
);
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('london');
});
test('falls back to the primary region when neither is set', async () => {
state.users.set('bob', placed(2, 'bob', {}));
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('oregon');
});
test('home wins over signup_server when both are set', async () => {
state.users.set(
'bob',
placed(2, 'bob', { home: 'sydney', signup_server: 'london' }),
);
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('sydney');
});
});
describe('refusals dispose of the body nobody read', () => {
test('an unknown recipient drains it and keeps the connection', async () => {
// The secret checked out and the length is under the cap, so the body
// was ours to read: discard it rather than resetting a usable socket.
const { status, drained, destroyed } =
await ingress('[email protected]');
expect(status).toBe(404);
expect(drained).toBe(true);
expect(destroyed).toBe(false);
});
test('a bad secret closes rather than reading the upload', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
secret: 'not-the-secret',
});
expect(status).toBe(403);
// Nothing authenticated this caller; their bytes are not worth reading.
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
test('no content-length is refused with 411', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
contentLength: null,
});
expect(status).toBe(411);
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
test('a declared length over the cap is refused with 413', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
contentLength: String(25 * 1024 * 1024 + 1),
});
expect(status).toBe(413);
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
});
describe('feedback addresses', () => {
test('are handed to a listener and reach no mailbox', async () => {
const { status, ended } = await ingress('[email protected]');
expect(status).toBe(200);
expect(ended).toBe(true);
expect(state.feedback).toHaveLength(1);
expect(state.feedback[0].to).toBe('[email protected]');
expect(state.feedback[0].from).toBe('[email protected]');
expect(state.feedback[0].raw.toString()).toContain('hello');
expect(state.writes).toEqual([]);
});
test('match the local part case-insensitively', async () => {
await ingress('[email protected]');
expect(state.feedback).toHaveLength(1);
});
test('fall through to normal routing when nobody takes them', async () => {
state.takeFeedback = false;
state.users.set('abuse', permanent(3, 'abuse'));
const { status } = await ingress('[email protected]');
expect(status).toBe(200);
expect(state.feedback).toHaveLength(1);
// The body was already read, so the buffered copy is what is stored.
expect(state.writes).toHaveLength(1);
expect(Buffer.isBuffer(state.writes[0].content)).toBe(true);
});
test('an unclaimed one with no such account is still a 404', async () => {
state.takeFeedback = false;
const { status } = await ingress('[email protected]');
expect(status).toBe(404);
});
test('only count on a Puter mail domain', async () => {
state.users.set('abuse', permanent(3, 'abuse'));
await ingress('[email protected]');
expect(state.feedback).toEqual([]);
});
test('are refused over the feedback size cap without being read', async () => {
const { status, destroyed } = await ingress('[email protected]', {
contentLength: String(5 * 1024 * 1024 + 1),
});
expect(status).toBe(413);
expect(destroyed).toBe(true);
expect(state.feedback).toEqual([]);
});
});
+161
View File
@@ -0,0 +1,161 @@
import { extension } from '@heyputer/backend/src/extensions';
import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
import type { Request } from 'express';
import {
MAX_MESSAGE_BYTES,
isPuterEmailAddress,
isTempUser,
puterEmailUsername,
storeInboxMessage,
} from '@heyputer/backend/src/services/email/mailbox.js';
// copied form peer secrets checker
const COMPARE_KEY = randomBytes(32);
const secretsEqual = (a: string, b: string): boolean =>
timingSafeEqual(
createHmac('sha256', COMPARE_KEY).update(a).digest(),
createHmac('sha256', COMPARE_KEY).update(b).digest(),
);
const USER_EMAIL_CONFIG = (extension.config as Record<string, unknown>)
.userEmail as { secret?: string; feedbackAddresses?: unknown } | undefined;
const INGRESS_SECRET = USER_EMAIL_CONFIG?.secret;
/**
* Local parts that take complaint feedback rather than naming a mailbox. The
* same names are reserved at signup, so no account can be handed them.
*/
const DEFAULT_FEEDBACK_ADDRESSES = ['fbl', 'abuse', 'postmaster'];
const FEEDBACK_ADDRESSES = new Set(
(Array.isArray(USER_EMAIL_CONFIG?.feedbackAddresses)
? USER_EMAIL_CONFIG.feedbackAddresses
: DEFAULT_FEEDBACK_ADDRESSES
).map((local) => String(local).toLowerCase()),
);
/**
* Feedback is handed over whole, so it is read into memory; a complaint report
* quotes one message and has no business near the mailbox ceiling.
*/
const MAX_FEEDBACK_BYTES = 5 * 1024 * 1024;
const isFeedbackAddress = (to: string): boolean =>
isPuterEmailAddress(to) &&
FEEDBACK_ADDRESSES.has(puterEmailUsername(to).toLowerCase());
const readBody = async (req: Request): Promise<Buffer> => {
const chunks: Buffer[] = [];
for await (const chunk of req) chunks.push(chunk as Buffer);
return Buffer.concat(chunks);
};
if (!INGRESS_SECRET) {
console.warn(
'[useremailrx] config.userEmail.secret unset - email ingress disabled',
);
} else {
extension.post(
'/email/ingress',
{
subdomain: 'api',
},
async (req, res) => {
// Answer first, then deal with the body nobody read. Touching the
// request before the response has flushed takes the response down
// with the connection and the caller sees a reset with no status,
// so this hangs off `finish` — and it has to be the event rather
// than `res.end(cb)`, because the compression middleware replaces
// `res.end` with an `(chunk, encoding)` signature that has no
// callback parameter, so a callback lands in `chunk` and dies in
// `Buffer.byteLength`.
//
// `drain` discards the rest of a body we were entitled to read and
// leaves the connection reusable. `close` is for bodies we do not
// want to read at all: an unauthenticated caller's, or one whose
// declared length is missing or over the cap.
const refuse = (status: number, how: 'drain' | 'close') => {
res.once('finish', () =>
how === 'drain' ? req.resume() : req.destroy(),
);
res.status(status).end();
};
const presented = req.query.SECRET;
if (
typeof presented !== 'string' ||
!secretsEqual(presented, INGRESS_SECRET)
) {
// Nothing has authenticated this caller, so refuse to spend
// bandwidth reading whatever they were uploading.
refuse(403, 'close');
return;
}
const size = Number(req.headers['content-length']);
if (!Number.isInteger(size) || size <= 0) {
refuse(411, 'close');
return;
}
if (size > MAX_MESSAGE_BYTES) {
refuse(413, 'close');
return;
}
const stores = extension.import('store');
const services = extension.import('service');
const to = String(req.query.to ?? '');
let content: Buffer | Request = req;
if (isFeedbackAddress(to)) {
if (size > MAX_FEEDBACK_BYTES) {
refuse(413, 'close');
return;
}
const raw = await readBody(req);
const event = {
to,
from:
typeof req.query.from === 'string'
? req.query.from
: null,
raw,
handled: false,
};
await extension
.import('client')
.event.emitAndWait('email.ingress.feedback', event, {});
if (event.handled) {
res.end();
return;
}
// Nobody took it: route it like any other message.
content = raw;
}
const user = await stores.user.getByUsername(
puterEmailUsername(to),
);
if (!user) {
refuse(404, 'drain');
return;
}
if (isTempUser(user)) {
if (content === req) req.resume();
res.end();
return;
}
await storeInboxMessage(services.fs, user, {
subject: req.query.subject,
content,
size: Buffer.isBuffer(content) ? content.length : size,
});
const { SECRET: _secret, ..._loggableQuery } = req.query;
// console.log('got email: ', puterUser, loggableQuery, size);
res.end();
},
);
}
+12
View File
@@ -0,0 +1,12 @@
{
"name": "emailrx",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "emailrx",
"version": "1.0.0"
}
}
}
+6
View File
@@ -0,0 +1,6 @@
{
"name": "emailrx",
"version": "1.0.0",
"main": "index.ts",
"description": "Internal service which handles routing puter emails to users"
}
+534
View File
@@ -0,0 +1,534 @@
import type { Request, Response } from 'express';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { makeActor, type Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { configContainer } from '../src/backend/exports.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleWhoami } from './whoami.ts';
interface CapturedResponse {
statusCode: number;
body: unknown;
}
const makeReq = (query: Record<string, unknown> = {}): Request =>
({ query }) as unknown as Request;
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer({
// Feature flag allowlist is enforced in the handler. We seed
// both an allow-listed flag and an internal flag to verify the
// internal one never reaches the response.
feature_flags: {
create_shortcut: true,
payment_bypass: true,
},
teams_enabled: true,
} as never);
});
afterAll(async () => {
await server?.shutdown();
});
const seedUser = async () => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.user.create({
username: `wuser_${slug}`,
uuid: uuidv4(),
password: 'hashedpw',
email: `${slug}@example.com`,
});
};
describe('whoami extension — handleWhoami', () => {
// The sidebar label needs this at boot, which is why it rides whoami
// rather than a call of its own.
describe('the team an account belongs to', () => {
// A seat is created, never adopted, so it must have no password.
const seedSeat = async () => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.user.create({
username: `wseat_${slug}`,
uuid: uuidv4(),
password: null,
email: null,
});
};
const seatOf = async (teamName: string) => {
const owner = await seedUser();
const seat = await seedSeat();
const team = await server.stores.team.create({
ownerUserId: owner.id as number,
name: teamName,
handle: `wt-${Math.random().toString(36).slice(2, 9)}`,
});
await server.stores.team.addMember(team.uid, seat.id as number, {
orgOwned: true,
});
return { seat, team };
};
it('names the team for a seat', async () => {
const { seat, team } = await seatOf('Acme Corp');
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: seat.uuid, id: seat.id as number } } },
() => handleWhoami(makeReq(), res),
);
expect((captured.body as { team?: unknown }).team).toEqual({
uid: team.uid,
name: 'Acme Corp',
});
});
it('says nothing for an account that is not a seat', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
expect(captured.body).not.toHaveProperty('team');
});
it('withholds it from an app actor', async () => {
// Same class as the phone number: a seat's employer is not an
// app's business.
const { seat } = await seatOf('Acme Corp');
const { res, captured } = makeRes();
await runWithContext(
{
actor: makeActor({
user: { uuid: seat.uuid, id: seat.id as number },
app: { uid: 'app-1' },
}),
},
() => handleWhoami(makeReq(), res),
);
expect(captured.body).not.toHaveProperty('team');
});
});
it('returns 401 when no actor is on the context', async () => {
const { res, captured } = makeRes();
await runWithContext({ actor: undefined }, () =>
handleWhoami(makeReq(), res),
);
expect(captured.statusCode).toBe(401);
expect(captured.body).toEqual({ error: 'Authentication required' });
});
it('returns 404 when the actor’s user no longer exists', async () => {
const { res, captured } = makeRes();
await runWithContext(
{
actor: {
user: { uuid: 'ghost-uuid', id: 99_999_999 },
},
},
() => handleWhoami(makeReq(), res),
);
expect(captured.statusCode).toBe(404);
expect(captured.body).toEqual({ error: 'User not found' });
});
it('returns full user details for a user actor', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.username).toBe(user.username);
expect(body.uuid).toBe(user.uuid);
expect(body.email).toBe(user.email);
expect(body.is_temp).toBe(false);
expect(body.oidc_only).toBe(false);
// is_user_token is present (true) for user actors.
expect(body.is_user_token).toBe(true);
// Account creation time, in unix seconds.
expect(typeof body.created_ts).toBe('number');
expect(body.created_ts).toBeGreaterThan(0);
// `directories` is only sent to user actors — confirm it’s present.
expect(body.directories).toBeDefined();
// taskbar_items is only sent to user actors.
expect(body).toHaveProperty('taskbar_items');
});
it('reports the SMS-to-card fallback only once a send has opened it', async () => {
const user = await server.stores.user.create({
username: `wuser_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'hashedpw',
email: `${Math.random().toString(36).slice(2, 8)}@example.com`,
requires_phone_verification: true,
} as never);
const prev = configContainer.phone_verification_card_fallback;
configContainer.phone_verification_card_fallback = {
enabled: true,
} as never;
try {
const before = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), before.res),
);
// Phone-gated, but the user has attempts left — no offer yet.
expect(
before.captured.body as Record<string, unknown>,
).toMatchObject({ card_fallback_available: false });
// Exhausting SMS attempts stamps this flag; whoami is then the only
// thing that can still tell a reloading GUI about the offer, since
// further sends are rejected by the route's own rate limit.
await server.stores.kv.set({
key: `card-fallback-open:${user.id}`,
value: true,
});
const after = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), after.res),
);
expect(
after.captured.body as Record<string, unknown>,
).toMatchObject({ card_fallback_available: true });
} finally {
configContainer.phone_verification_card_fallback = prev;
}
});
it('only forwards allow-listed feature flags', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const flags = (captured.body as Record<string, unknown>)
.feature_flags as Record<string, boolean>;
// Allowed flag is forwarded as a coerced boolean.
expect(flags.create_shortcut).toBe(true);
// Internal flag must never leak.
expect(flags.payment_bypass).toBeUndefined();
});
it('strips desktop_bg_*, created_ts and human_readable_age fields for app actors', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{
actor: makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
},
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.app_name).toBe('app-test-actor');
// is_user_token is stripped for app actors.
expect(body.is_user_token).toBeUndefined();
expect(body.desktop_bg_url).toBeUndefined();
expect(body.desktop_bg_color).toBeUndefined();
expect(body.desktop_bg_fit).toBeUndefined();
expect(body.human_readable_age).toBeUndefined();
// Account age, in either form, is not exposed to apps.
expect(body.created_ts).toBeUndefined();
// Directories are user-only.
expect(body.directories).toBeUndefined();
});
describe('fields added by whoami.details listeners', () => {
const addAccountDetails = (
_key: unknown,
event: { details: Record<string, unknown> },
) => {
Object.assign(event.details, {
subscribed: true,
paid_storage: 1024,
subscription: { tier: 'pro', status: 'trialing' },
accountEligible: true,
});
};
const whoamiWithListener = async (
actor: Actor,
listener: typeof addAccountDetails = addAccountDetails,
) => {
const { res, captured } = makeRes();
server.clients.event.on('whoami.details', listener);
try {
await runWithContext({ actor }, () =>
handleWhoami(makeReq(), res),
);
} finally {
server.clients.event.off('whoami.details', listener);
}
return captured.body as Record<string, unknown>;
};
it('reach an app actor only as the documented subscribed / paid_storage', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
);
expect(body.subscribed).toBe(true);
expect(body.paid_storage).toBe(1024);
expect(body).not.toHaveProperty('subscription');
expect(body).not.toHaveProperty('accountEligible');
// Fields the handler built itself still ship.
expect(body.username).toBe(user.username);
expect(body.app_name).toBe('app-test-actor');
});
it('are withheld from an app reached through an access token it issued', async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const body = await whoamiWithListener(
makeActor({
user: owner,
accessToken: {
uid: 'tok-1',
issuer: makeActor({
user: owner,
app: { uid: 'app-issuer' },
}),
},
}),
);
expect(body.subscribed).toBe(true);
expect(body).not.toHaveProperty('subscription');
expect(body).not.toHaveProperty('accountEligible');
});
it('all reach a user actor', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({ user: { uuid: user.uuid, id: user.id as number } }),
);
expect(body).toMatchObject({
subscribed: true,
paid_storage: 1024,
subscription: { tier: 'pro', status: 'trialing' },
accountEligible: true,
});
});
it('cannot add taskbar_items for an app actor, which core omits', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
(_key, event) => {
event.details.taskbar_items = [{ name: 'injected' }];
},
);
expect(body).not.toHaveProperty('taskbar_items');
});
});
it('redacts tmp_password from metadata for user actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
hasDevAccountAccess: true,
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
const metadata = body.metadata as Record<string, unknown>;
expect(metadata.tmp_password).toBeUndefined();
// Other metadata keys still reach the user's own client.
expect(metadata.hasDevAccountAccess).toBe(true);
expect(body.hasDevAccountAccess).toBe(true);
});
it('never sends user metadata to app actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
});
const { res, captured } = makeRes();
await runWithContext(
{
actor: makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
},
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.metadata).toBeUndefined();
expect(JSON.stringify(body)).not.toContain('bootstrap-secret');
});
it('never exposes phone, card fingerprint or signup identity', async () => {
const user = await seedUser();
await server.stores.user.update(user.id as number, {
phone: '+15551234567',
card_fingerprint: 'fp_ABC123',
requires_phone_verification: false,
requires_card_verification: false,
});
// Guard against a vacuous assertion below: the columns really do hold
// the values we then expect never to see on the wire.
const stored = await server.stores.user.getById(user.id as number, {
cached: false,
force: true,
});
expect(stored?.phone).toBe('+15551234567');
expect(stored?.card_fingerprint).toBe('fp_ABC123');
for (const actor of [
makeActor({ user: { uuid: user.uuid, id: user.id as number } }),
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
]) {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.phone).toBeUndefined();
expect(body.card_fingerprint).toBeUndefined();
expect(body.password).toBeUndefined();
expect(body.otp_secret).toBeUndefined();
expect(body.signup_ip).toBeUndefined();
// Not just absent as a key — the values must not appear anywhere
// in the payload (nested under metadata, taskbar items, …).
const serialized = JSON.stringify(body);
expect(serialized).not.toContain('5551234567');
expect(serialized).not.toContain('fp_ABC123');
// The verification flags, which the GUI acts on, still ship.
expect(body).toHaveProperty('requires_phone_verification');
expect(body).toHaveProperty('requires_card_verification');
}
});
it('scrubs sensitive keys added to metadata, and leaves the cached row intact', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
billing: { card_fingerprint: 'fp_NESTED', tier: 'pro' },
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
const metadata = body.metadata as Record<string, unknown>;
expect(metadata.tmp_password).toBeUndefined();
// Nested sensitive keys are removed too; siblings survive.
expect(metadata.billing).toEqual({ tier: 'pro' });
expect(JSON.stringify(body)).not.toContain('fp_NESTED');
// The scrub works on a copy — server-side state is untouched.
const fresh = await server.stores.user.getById(user.id as number, {
cached: false,
force: true,
});
expect(fresh?.metadata?.tmp_password).toBe('bootstrap-secret');
expect(
(fresh?.metadata?.billing as Record<string, unknown>)
?.card_fingerprint,
).toBe('fp_NESTED');
});
it('marks the user as oidc_only when password is null', async () => {
const slug = Math.random().toString(36).slice(2, 8);
const oidcUser = await server.stores.user.create({
username: `oidc_${slug}`,
uuid: uuidv4(),
password: null,
email: `${slug}@oidc.test`,
});
const { res, captured } = makeRes();
await runWithContext(
{
actor: {
user: {
uuid: oidcUser.uuid,
id: oidcUser.id as number,
},
},
},
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.oidc_only).toBe(true);
// No email yet means temp account.
expect(body.is_temp).toBe(false);
});
});
+126 -158
View File
@@ -1,6 +1,11 @@
import { Context } from '@heyputer/backend/src/core';
import { extension } from '@heyputer/backend/src/extensions';
import { getTaskbarItems } from '@heyputer/backend/src/util/taskbarItems.js';
import { APP_ICON_SIZES } from '@heyputer/backend/src/util/appIcon.js';
import {
buildUserDetails,
scrubSensitive,
} from '@heyputer/backend/src/util/userDetails.js';
import type { Request, Response } from 'express';
import TimeAgo from 'javascript-time-ago';
import localeEn from 'javascript-time-ago/locale/en';
@@ -13,166 +18,129 @@ const timeago = (() => {
return new TimeAgo('en-US');
})();
// Allowlist of `config.feature_flags` keys safe to surface via /whoami.
// Anything not listed here stays server-side, so internal flags
// (payment_bypass, staff_only_*, etc.) cannot leak by accident. Add a
// flag here when, and only when, the client actually needs to read it.
const CLIENT_VISIBLE_FEATURE_FLAGS: ReadonlySet<string> = new Set([
'create_shortcut',
'download_directory',
'prompt_user_when_navigation_away_from_puter',
// What an app actor may receive from `whoami.details` listeners. Anything
// else a listener adds describes the account, and only ships to user actors.
const APP_VISIBLE_LISTENER_KEYS: ReadonlySet<string> = new Set([
'subscribed',
'paid_storage',
]);
export const handleWhoami = async (
req: Request,
res: Response,
): Promise<void> => {
const actor = Context.get('actor');
if (!actor?.user?.id) {
res.status(401).json({ error: 'Authentication required' });
return;
}
const isUser = !actor.effectiveApp;
const user = await stores.user.getById(actor.user.id);
if (!user) {
res.status(404).json({ error: 'User not found' });
return;
}
const rawIconSize =
typeof req.query?.icon_size === 'string'
? Number(req.query.icon_size)
: undefined;
const iconSize =
rawIconSize !== undefined && APP_ICON_SIZES.includes(rawIconSize)
? rawIconSize
: undefined;
const details = await buildUserDetails(
user,
{ config: extension.config, clients, stores, services },
{ isUser, iconSize, noIcons: !iconSize },
);
details.human_readable_age = user.timestamp
? timeago.format(new Date(user.timestamp as string))
: null;
// Strip sensitive fields for app actors
if (!isUser) {
const canReadEmail = await services.permission
.check(actor, `user:${user.uuid}:email:read`)
.catch(() => false);
if (!canReadEmail) {
delete details.email;
delete details.unconfirmed_email;
}
delete details.desktop_bg_url;
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.created_ts;
delete details.is_user_token;
delete details.metadata;
// An app has no business reading the code its user earns credit with.
delete details.referral_code;
}
const app = actor.effectiveApp;
if (app) {
details.app_name = app.uid;
}
// A key core left undefined isn't built, so a listener can't fill it for an app.
const builtKeys = isUser
? null
: new Set(
Object.keys(details).filter((key) => details[key] !== undefined),
);
try {
await clients.event.emitAndWait(
'whoami.details',
{ user, details, isUser },
{},
);
} catch {
/* best-effort */
}
if (builtKeys) {
for (const key of Object.keys(details)) {
if (!builtKeys.has(key) && !APP_VISIBLE_LISTENER_KEYS.has(key)) {
delete details[key];
}
}
}
const subscription = details.subscription as
{ offering?: Record<string, unknown> } | undefined;
if (subscription?.offering) {
delete subscription.offering.group;
delete subscription.offering.benefits;
delete subscription.offering.price_id;
}
// Last word on what ships, after every listener has had its say.
scrubSensitive(details);
res.json(details);
};
extension.get(
'/whoami',
{ subdomain: 'api', requireAuth: true, allowUnconfirmed: true },
async (req, res) => {
const actor = Context.get('actor');
if (!actor?.user?.id) {
res.status(401).json({ error: 'Authentication required' });
return;
}
const isUser = !actor.app;
const user = await stores.user.getById(actor.user.id);
if (!user) {
res.status(404).json({ error: 'User not found' });
return;
}
const oidcOnly = user.password === null;
const ALLOWED_ICON_SIZES = new Set([16, 32, 64, 128, 256, 512]);
const rawIconSize =
typeof req.query?.icon_size === 'string'
? Number(req.query.icon_size)
: undefined;
const iconSize =
rawIconSize !== undefined && ALLOWED_ICON_SIZES.has(rawIconSize)
? rawIconSize
: undefined;
const noIcons = !iconSize;
// Feature flags come from `config.feature_flags`. We only forward keys
// listed in CLIENT_VISIBLE_FEATURE_FLAGS so internal flags can't leak.
// Non-boolean values (e.g. `"true"` as a string) are coerced so the
// client never has to guess.
const rawFlags = extension.config.feature_flags ?? {};
const feature_flags: Record<string, boolean> = {};
for (const [k, v] of Object.entries(rawFlags)) {
if (CLIENT_VISIBLE_FEATURE_FLAGS.has(k)) {
feature_flags[k] = Boolean(v);
}
}
const details: Record<string, unknown> = {
username: user.username,
uuid: user.uuid,
email: user.email,
unconfirmed_email: user.email,
email_confirmed: user.email_confirmed || user.username === 'admin',
requires_email_confirmation: user.requires_email_confirmation,
desktop_bg_url: user.desktop_bg_url,
desktop_bg_color: user.desktop_bg_color,
desktop_bg_fit: user.desktop_bg_fit,
is_temp: user.password === null && user.email === null,
oidc_only: oidcOnly,
taskbar_items: isUser
? await getTaskbarItems(
user,
{
clients,
stores,
services,
apiBaseUrl: String(
extension.config.api_base_url ?? '',
),
},
{ iconSize, noIcons },
)
: undefined,
otp: !!user.otp_enabled,
feature_flags,
human_readable_age: user.timestamp
? timeago.format(new Date(user.timestamp as string))
: null,
};
// OIDC revalidate URL for password-less accounts
if (oidcOnly) {
try {
const providers = await services.oidc.getEnabledProviderIds();
const provider = providers?.[0];
if (provider) {
const callbackUrl =
services.oidc.getCallbackUrl?.('login') ?? '';
const origin = callbackUrl.replace(
/\/auth\/oidc\/callback\/login$/,
'',
);
details.oidc_revalidate_url = `${origin}/auth/oidc/${provider}/start?flow=revalidate&user_uuid=${encodeURIComponent(user.uuid)}`;
}
} catch {
// OIDC not configured
}
}
// Directories — only sent to user actors
if (isUser) {
const directories: Record<string, unknown> = {};
const nameToProp: Record<string, string> = {
desktop_uuid: `/${user.username}/Desktop`,
appdata_uuid: `/${user.username}/AppData`,
documents_uuid: `/${user.username}/Documents`,
pictures_uuid: `/${user.username}/Pictures`,
videos_uuid: `/${user.username}/Videos`,
trash_uuid: `/${user.username}/Trash`,
};
for (const k in nameToProp) {
directories[nameToProp[k]] = user[k];
}
details.directories = directories;
}
// Last activity
if (user.last_activity_ts) {
try {
details.last_activity_ts = Math.round(
new Date(user.last_activity_ts as string).getTime() / 1000,
);
} catch {
/* ignore parse error */
}
}
// Strip sensitive fields for app actors
if (!isUser) {
const canReadEmail = await services.permission
.check(actor, `user:${user.uuid}:email:read`)
.catch(() => false);
if (!canReadEmail) {
delete details.email;
delete details.unconfirmed_email;
}
delete details.desktop_bg_url;
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
}
if (actor.app) {
details.app_name = actor.app.uid;
}
try {
await clients.event.emitAndWait(
'whoami.details',
{ user, details, isUser },
{},
);
} catch {
/* best-effort */
}
res.json(details);
{
subdomain: 'api',
requireAuth: true,
allowUnconfirmed: true,
// The GUI polls this, and each call fans out to every `whoami`
// event listener — so it costs more than the response suggests.
//
// It is also the call everything else leans on to find out who it is
// talking to, so it rides along with unrelated work rather than
// arriving at its own pace: the ceiling has to clear whatever the
// busiest session is doing, not what a person clicks.
rateLimit: {
scope: 'whoami',
limit: 1_800,
window: 60_000,
key: 'user',
},
},
handleWhoami,
);
+41
View File
@@ -0,0 +1,41 @@
import type { Request, Response } from 'express';
import { describe, expect, it, vi } from 'vitest';
import { handleWorkerSandboxPage } from './workerSandbox.ts';
interface CapturedResponse {
contentType: string | undefined;
body: string | undefined;
}
const makeRes = () => {
const captured: CapturedResponse = {
contentType: undefined,
body: undefined,
};
const res = {
type: vi.fn((mime: string) => {
captured.contentType = mime;
return res;
}),
send: vi.fn((value: string) => {
captured.body = value;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
describe('workerSandbox extension — handleWorkerSandboxPage', () => {
it('sends the playground HTML page with text/html content type', () => {
const { res, captured } = makeRes();
handleWorkerSandboxPage({} as Request, res);
expect(captured.contentType).toBe('html');
expect(typeof captured.body).toBe('string');
expect(captured.body).toContain('<!doctype html>');
expect(captured.body).toContain('Puter Worker Sandbox Playground');
// The page is meant to load the public puter SDK
expect(captured.body).toContain('https://js.puter.com/v2/');
});
});
+6 -3
View File
@@ -1,3 +1,4 @@
import type { Request, Response } from 'express';
import { extension } from '@heyputer/backend/src/extensions';
const page = `
@@ -89,10 +90,12 @@ const page = `
</html>
`;
export const handleWorkerSandboxPage = (_req: Request, res: Response): void => {
res.type('html').send(page);
};
extension.get(
'/',
{ requireAuth: false, subdomain: 'worker-sandbox' },
(_req, res) => {
res.type('html').send(page);
},
handleWorkerSandboxPage,
);
+20 -12
View File
@@ -11,7 +11,7 @@
# What this does, in order:
# 1. Checks that docker (with the compose plugin) exists.
# 2. Creates ./puter-selfhosted/ (override with $env:PUTER_DIR).
# 3. Downloads docker-compose.yml + nginx.conf from the OSS repo.
# 3. Downloads docker-compose.yml + caddy/Caddyfile from the OSS repo.
# 4. Generates fresh secrets and writes .env + puter/config/config.json.
# 5. Runs `docker compose up -d` and prints how to find the admin password.
#
@@ -23,7 +23,7 @@
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_PORT HTTP port for Caddy (default: 80)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
[CmdletBinding()]
@@ -77,10 +77,10 @@ if ($LASTEXITCODE -ne 0) {
# ── Step 2: install dir ─────────────────────────────────────────────
$null = New-Item -ItemType Directory -Force -Path $PuterDir
Set-Location $PuterDir
$null = New-Item -ItemType Directory -Force -Path 'puter/config', 'puter/data', 'puter/tls'
$null = New-Item -ItemType Directory -Force -Path 'puter/config', 'puter/data', 'puter/data/caddy', 'puter/tls'
Write-Log "install dir: $((Get-Location).Path)"
# ── Step 3: docker-compose.yml + nginx config ──────────────────────
# ── Step 3: docker-compose.yml + Caddy config ──────────────────────
Write-Log "downloading docker-compose.yml from $PuterUrl"
try {
Invoke-WebRequest -Uri "$PuterUrl/docker-compose.yml" -OutFile 'docker-compose.yml' -UseBasicParsing
@@ -88,17 +88,19 @@ try {
Die "could not fetch $PuterUrl/docker-compose.yml — $_"
}
Write-Log "downloading nginx/nginx.conf from $PuterUrl"
$null = New-Item -ItemType Directory -Force -Path 'nginx'
# The Caddyfile is domain-agnostic — it answers on every Host and leaves
# the subdomain routing to Puter — so there's nothing to template in.
Write-Log "downloading caddy/Caddyfile from $PuterUrl"
$null = New-Item -ItemType Directory -Force -Path 'caddy'
# If the path was previously auto-created as a directory by a failed
# `compose up`, remove it so we can write the file there.
if (Test-Path 'nginx/nginx.conf' -PathType Container) {
Remove-Item 'nginx/nginx.conf' -Recurse -Force
if (Test-Path 'caddy/Caddyfile' -PathType Container) {
Remove-Item 'caddy/Caddyfile' -Recurse -Force
}
try {
Invoke-WebRequest -Uri "$PuterUrl/nginx/nginx.conf" -OutFile 'nginx/nginx.conf' -UseBasicParsing
Invoke-WebRequest -Uri "$PuterUrl/caddy/Caddyfile" -OutFile 'caddy/Caddyfile' -UseBasicParsing
} catch {
Die "could not fetch $PuterUrl/nginx/nginx.conf — $_"
Die "could not fetch $PuterUrl/caddy/Caddyfile — $_"
}
# ── Step 4: secrets, .env, config.json ──────────────────────────────
@@ -113,12 +115,16 @@ if ($writeConfig) {
$mariadbRootPw = New-HexSecret 32
$mariadbPw = New-HexSecret 32
$s3SecretKey = New-HexSecret 32
# Two JWT secrets: $jwtSecret is verify-only for legacy v1 tokens
# already in circulation; $jwtSecretV2 signs every new token.
$jwtSecret = New-HexSecret 64
$jwtSecretV2 = New-HexSecret 64
$urlSigSecret = New-HexSecret 64
$envContent = @"
HTTP_PORT=$PuterPort
# HTTPS_PORT=443 # uncomment after enabling TLS (see doc/selfhosting/full-stack.md)
# HTTPS_PORT=443 # uncomment after enabling TLS in caddy/Caddyfile
# # (see "Step 3 — TLS" in doc/self-hosting.md)
MARIADB_ROOT_PASSWORD=$mariadbRootPw
MARIADB_DATABASE=puter
@@ -142,6 +148,8 @@ S3_BUCKET=puter-local
private_app_hosting_domain = "app.$PuterDomain"
private_app_hosting_domain_alt = "dev.$PuterDomain"
jwt_secret = $jwtSecret
jwt_secret_v2 = $jwtSecretV2
allow_v1_tokens = $true
url_signature_secret = $urlSigSecret
database = [ordered]@{
engine = 'mysql'
@@ -200,4 +208,4 @@ Write-Log " cd $PuterDir; docker compose logs -f puter"
Write-Log ''
Write-Log "open http://${PuterDomain}:${PuterPort} once the puter container is healthy."
Write-Log 'first-boot admin password is logged once — grab it with:'
Write-Log " cd $PuterDir; docker compose logs puter | Select-String tmp_password"
Write-Log " cd $PuterDir; docker compose logs puter | Select-String password"
+77 -20
View File
@@ -7,7 +7,8 @@
# What this does, in order:
# 1. Checks that docker (with the compose plugin), curl, and openssl exist.
# 2. Creates ./puter-selfhosted/ (override with PUTER_DIR=...).
# 3. Downloads docker-compose.yml from the OSS repo (raw.githubusercontent.com).
# 3. Downloads docker-compose.yml + caddy/Caddyfile from the OSS repo
# (raw.githubusercontent.com).
# 4. Generates fresh secrets and writes .env + puter/config/config.json.
# 5. Runs `docker compose up -d` and prints the first-boot admin password.
#
@@ -16,11 +17,22 @@
# the compose file + brings the stack up. Set PUTER_FORCE=1 to overwrite.
#
# Tunable env vars:
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for Caddy (default: 80)
# PUTER_PROTOCOL public scheme: http | https (default: http)
# Set https once TLS is terminating in front of Puter —
# either the bundled Caddy with your certs (see "Step 3 —
# TLS" in doc/self-hosting.md) or your own proxy (Traefik,
# a cloud LB). Puter then builds https:// origins, S3
# URLs, and redirects. Leave it http and the installer
# serves plain HTTP on PUTER_PORT.
# PUTER_TRUST_PROXY number of reverse-proxy hops in front (default: 1)
# 1 = the bundled Caddy (or a single external proxy);
# 2 = two hops (e.g. Cloudflare → Caddy → Puter).
# PUTER_ENV prod | dev (default: prod)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
set -eu
@@ -28,6 +40,9 @@ PUTER_DIR="${PUTER_DIR:-puter-selfhosted}"
PUTER_URL="${PUTER_URL:-https://raw.githubusercontent.com/HeyPuter/puter/main}"
PUTER_DOMAIN="${PUTER_DOMAIN:-puter.localhost}"
PUTER_PORT="${PUTER_PORT:-80}"
PUTER_PROTOCOL="${PUTER_PROTOCOL:-http}"
PUTER_TRUST_PROXY="${PUTER_TRUST_PROXY:-1}"
PUTER_ENV="${PUTER_ENV:-prod}"
PUTER_FORCE="${PUTER_FORCE:-0}"
log() { printf '\033[1;36m[puter-install]\033[0m %s\n' "$*"; }
@@ -50,23 +65,37 @@ docker compose version >/dev/null 2>&1 \
mkdir -p "$PUTER_DIR"
cd "$PUTER_DIR"
mkdir -p puter/config puter/data puter/tls
# Pre-create per-service data dirs and make them writable by any UID.
# Several upstream images run as non-root inside the container (rustfs
# uses UID 10001; dynamo is pinned to 1000 in compose), and rustfs's
# entrypoint runs as that same non-root user so it can't chown an
# already-existing bind-mounted dir. On hosts where the user that ran
# this script has a UID that doesn't match — or where docker is running
# rootless — those containers loop on EACCES at startup. 0777 on the
# bind-mount roots sidesteps the mismatch without guessing each image's
# internal UID. (Docker Desktop on macOS/Windows papers over this with
# its VM layer; native Linux docker on Debian/Alpine doesn't.)
mkdir -p puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter puter/data/caddy
chmod 0777 puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter puter/data/caddy
log "install dir: $(pwd)"
# ── Step 3: docker-compose.yml + nginx config ──────────────────────
# ── Step 3: docker-compose.yml + Caddy config ──────────────────────
log "downloading docker-compose.yml from $PUTER_URL"
curl -fsSL "$PUTER_URL/docker-compose.yml" -o docker-compose.yml \
|| die "could not fetch $PUTER_URL/docker-compose.yml"
# nginx is mounted as `./nginx/nginx.conf:/etc/nginx/nginx.conf:ro` — if
# Caddy is mounted as `./caddy/Caddyfile:/etc/caddy/Caddyfile:ro,z` — if
# the host file is missing, docker silently creates a directory at that
# path and the mount fails with "not a directory" at container start.
log "downloading nginx/nginx.conf from $PUTER_URL"
mkdir -p nginx
# The Caddyfile is domain-agnostic — it answers on every Host and leaves
# the subdomain routing to Puter — so there's nothing to template in.
log "downloading caddy/Caddyfile from $PUTER_URL"
mkdir -p caddy
# If the path was previously auto-created as a dir by a failed `compose up`,
# remove it so curl can write the file.
[ -d nginx/nginx.conf ] && rmdir nginx/nginx.conf 2>/dev/null || true
curl -fsSL "$PUTER_URL/nginx/nginx.conf" -o nginx/nginx.conf \
|| die "could not fetch $PUTER_URL/nginx/nginx.conf"
[ -d caddy/Caddyfile ] && rmdir caddy/Caddyfile 2>/dev/null || true
curl -fsSL "$PUTER_URL/caddy/Caddyfile" -o caddy/Caddyfile \
|| die "could not fetch $PUTER_URL/caddy/Caddyfile"
# ── Step 4: secrets, .env, config.json ──────────────────────────────
write_config=1
@@ -80,12 +109,17 @@ if [ "$write_config" = "1" ]; then
MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
# Two JWT secrets: `jwt_secret` is verify-only for legacy v1 tokens
# already in circulation; `jwt_secret_v2` signs every new token.
# Both are required at boot (`jwt_secret` only when verifying v1).
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
cat > .env <<EOF
HTTP_PORT=$PUTER_PORT
# HTTPS_PORT=443 # uncomment after enabling TLS (see doc/selfhosting/full-stack.md)
# HTTPS_PORT=443 # uncomment after enabling TLS in caddy/Caddyfile
# # (see "Step 3 — TLS" in doc/self-hosting.md)
MARIADB_ROOT_PASSWORD=$MARIADB_ROOT_PASSWORD
MARIADB_DATABASE=puter
@@ -97,13 +131,18 @@ S3_SECRET_KEY=$S3_SECRET_KEY
S3_BUCKET=puter-local
EOF
# `meteringEnforcement.subscriptions` is off here because a self-hosted
# install has no paid plans to be on: every account resolves to the free
# policy, so leaving the plan gates armed would 402 the surfaces that
# declare one (the OpenAI/Anthropic-compatible AI endpoints) for everybody.
# The code default is on — this is a property of the install, not of Puter.
log "writing puter/config/config.json"
cat > puter/config/config.json <<EOF
{
"domain": "$PUTER_DOMAIN",
"protocol": "http",
"protocol": "$PUTER_PROTOCOL",
"pub_port": $PUTER_PORT,
"env": "prod",
"env": "$PUTER_ENV",
"static_hosting_domain": "site.$PUTER_DOMAIN",
"static_hosting_domain_alt": "host.$PUTER_DOMAIN",
@@ -111,6 +150,8 @@ EOF
"private_app_hosting_domain_alt": "dev.$PUTER_DOMAIN",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"allow_v1_tokens": true,
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
@@ -141,7 +182,7 @@ EOF
"s3": {
"s3Config": {
"endpoint": "http://s3:9000",
"publicEndpoint": "http://s3.$PUTER_DOMAIN",
"publicEndpoint": "$PUTER_PROTOCOL://s3.$PUTER_DOMAIN",
"accessKeyId": "puter",
"secretAccessKey": "$S3_SECRET_KEY",
"region": "us-east-1",
@@ -155,11 +196,27 @@ EOF
"ollama": { "enabled": false }
},
"trust_proxy": 1
"meteringEnforcement": {
"subscriptions": false
},
"trust_proxy": $PUTER_TRUST_PROXY
}
EOF
fi
# `https` only works if something in front is actually terminating it. The
# bundled Caddy does that from ./puter/tls once you enable its `:443` block
# (see "Step 3 — TLS" in doc/self-hosting.md); your own edge proxy is fine
# too. Neither in place means Puter hands out https:// URLs for an endpoint
# that only speaks HTTP — broken logins and mixed content.
if [ "$PUTER_PROTOCOL" = "https" ] && [ ! -f puter/tls/fullchain.pem ]; then
warn "PUTER_PROTOCOL=https but ./puter/tls/fullchain.pem is missing."
warn "The bundled Caddy is still serving plain HTTP on port $PUTER_PORT."
warn "Either drop a wildcard cert into ./puter/tls and uncomment the :443"
warn "block in caddy/Caddyfile, or terminate TLS in your own proxy."
fi
# ── Step 5: bring it up ─────────────────────────────────────────────
log "docker compose up -d"
docker compose up -d
@@ -169,6 +226,6 @@ log "stack starting. first boot takes ~30s while MariaDB initialises."
log "follow puter logs:"
log " cd $PUTER_DIR && docker compose logs -f puter"
log ""
log "open http://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
log "open $PUTER_PROTOCOL://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
log "first-boot admin password is logged once — grab it with:"
log " cd $PUTER_DIR && docker compose logs puter | grep tmp_password"
log " cd $PUTER_DIR && docker compose logs puter | grep password"
-135
View File
@@ -1,135 +0,0 @@
# Reverse proxy in front of Puter — mirrors what the prod ALB does:
# accepts every Host header, forwards to the Puter container, and lets
# the Puter app handle subdomain-based routing internally (api.*,
# site.*, app.*, etc).
#
# To enable TLS:
# 1. Drop your fullchain.pem + privkey.pem into ./puter/tls/.
# 2. Uncomment the 443 server{} block below.
# 3. Update server_name to your domain (and wildcard subdomains).
worker_processes auto;
events {
worker_connections 4096;
}
http {
# Required for Puter's WebSocket / socket.io upgrades.
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
# Rough size cap that mirrors prod ALB defaults; tune for your
# uploads. Puter chunks large uploads, so 1 GiB per request is plenty.
client_max_body_size 1024m;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
proxy_buffering off;
server_tokens off;
upstream puter_backend {
server puter:4100;
keepalive 32;
}
upstream s3_backend {
# RustFS — see `s3` service in docker-compose.full.yml. Browsers
# PUT/GET here for presigned-URL uploads / downloads. Routed via
# the `s3.<domain>` subdomain so signature verification works
# (Host header preserved end-to-end) and so HTTPS stays clean
# (no mixed-content from a port-9000 host publish).
server s3:9000;
keepalive 32;
}
# ── HTTP (port 80) ─────────────────────────────────────────────
server {
listen 80;
listen [::]:80;
server_name ~^s3\.;
location / {
proxy_pass http://s3_backend;
proxy_http_version 1.1;
# Critical: preserve the original Host so RustFS validates
# the request against the same host the URL was signed for.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
# Note: when you enable TLS, replace this block with a redirect:
# return 301 https://$host$request_uri;
location / {
proxy_pass http://puter_backend;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
# ── HTTPS (port 443) — uncomment after dropping certs in ./puter/tls/ ─
# server {
# listen 443 ssl;
# listen [::]:443 ssl;
# http2 on;
# server_name ~^s3\.;
#
# ssl_certificate /etc/nginx/tls/fullchain.pem;
# ssl_certificate_key /etc/nginx/tls/privkey.pem;
# ssl_protocols TLSv1.2 TLSv1.3;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_session_cache shared:SSL:10m;
# ssl_session_timeout 10m;
#
# location / {
# proxy_pass http://s3_backend;
# proxy_http_version 1.1;
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# }
# }
#
# server {
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
# http2 on;
# server_name _;
#
# ssl_certificate /etc/nginx/tls/fullchain.pem;
# ssl_certificate_key /etc/nginx/tls/privkey.pem;
# ssl_protocols TLSv1.2 TLSv1.3;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_session_cache shared:SSL:10m;
# ssl_session_timeout 10m;
#
# location / {
# proxy_pass http://puter_backend;
# proxy_http_version 1.1;
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# proxy_set_header X-Forwarded-Host $host;
# proxy_set_header X-Forwarded-Port $server_port;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection $connection_upgrade;
# }
# }
}
+7378 -6860
View File
File diff suppressed because it is too large. Load diff
+46 -31
View File
@@ -1,6 +1,6 @@
{
"name": "puter.com",
"version": "2.5.1",
"version": "26.07",
"author": "Puter Technologies Inc.",
"license": "AGPL-3.0-only",
"description": "Desktop environment in the browser!",
@@ -10,56 +10,60 @@
"lib": "lib"
},
"devDependencies": {
"@babel/core": "^7.29.7",
"@eslint/js": "^9.35.0",
"@playwright/test": "^1.56.1",
"@stylistic/eslint-plugin": "^5.3.1",
"@types/express": "^5.0.0",
"@types/mime-types": "^3.0.1",
"@typescript-eslint/eslint-plugin": "^8.46.1",
"@typescript-eslint/parser": "^8.46.1",
"@vitest/coverage-v8": "^4.0.14",
"@vitest/ui": "^4.0.14",
"babel-loader": "^10.1.1",
"babel-plugin-istanbul": "^8.0.0",
"chalk": "^4.1.0",
"clean-css": "^5.3.2",
"dotenv": "^16.4.5",
"esbuild": "^0.28.0",
"eslint": "^9.35.0",
"eslint-config-prettier": "^10.1.8",
"eslint-plugin-prettier": "^5.5.5",
"eslint-rule-composer": "^0.3.0",
"globals": "^15.15.0",
"html-webpack-plugin": "^5.6.0",
"husky": "^9.1.7",
"license-check-and-add": "^4.0.5",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-reports": "^3.2.0",
"nodemon": "^3.1.0",
"prettier": "^3.8.3",
"simple-git": "^3.32.3",
"prettier-plugin-jsdoc": "^1.8.1",
"typescript": "^5.4.5",
"uglify-js": "^3.17.4",
"vite-plugin-static-copy": "^3.3.0",
"vitest": "^4.1.5",
"webpack": "^5.88.2",
"webpack-cli": "^5.1.1",
"yaml": "^2.8.1"
"webpack-cli": "^5.1.1"
},
"scripts": {
"test": "npx vitest run --config=src/backend/vitest.config.ts && node src/backend/tools/test.mjs",
"test:puterjs-api": "vitest run tests/puterJsApiTests",
"test:backend": "npm run build:ts; vitest run --config=src/backend/vitest.config.ts",
"test:backend-coverage": "npm run build:ts; vitest run --config=src/backend/vitest.config.ts",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run build:ts",
"dev": "npm run build:ts && node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"build": "npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"test:backend": "npm run setupExtensions && npm run build:workerLib && vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "npm run setupExtensions && npm run build:workerLib && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"test:puterjs": "npm run setupExtensions && vitest run --config src/puter-js/tests/api/vitest.config.ts",
"test:puterjs:node": "npm run test:puterjs -- src/puter-js/tests/api/runners/node.test.ts",
"test:puterjs:browser": "npm run test:puterjs -- src/puter-js/tests/api/runners/browser.test.ts",
"test:puterjs:workerd": "npm run test:puterjs -- src/puter-js/tests/api/runners/workerd.test.ts",
"test:puterjs:coverage": "npm run build:workerLib:coverage && npm run setupExtensions && rm -rf src/puter-js/coverage && PUTER_COVERAGE=1 vitest run --config src/puter-js/tests/api/vitest.config.ts && node ./tools/puterjsCoverageReport.mjs",
"build:workerLib:coverage": "cd src/puter-js && npm run build:coverage && cd ../worker && npm run build",
"start:gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node ./tools/start.mjs",
"start:smtp": "node ./tools/start.mjs --smtp",
"dev": "npm start",
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
"check-translations": "node tools/check-translations.js",
"prepare": "husky",
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
"postinstall": "node ./tools/extensionSetup.mjs"
"check:puterjs:types": "node tools/checkPuterjsTypes.mjs",
"typecheck": "node tools/typecheck.mjs",
"typecheck:update": "node tools/typecheck.mjs --update",
"setupExtensions": "node ./tools/extensionSetup.mjs"
},
"workspaces": [
"src/*",
"tools/*",
"experiments/js-parse-and-output"
"src/*"
],
"nodemonConfig": {
"ext": "js, json, mjs, jsx, svg, css",
@@ -69,19 +73,30 @@
]
},
"dependencies": {
"@ai-sdk/openai": "^3.0.25",
"@aws-sdk/client-s3": "^3.1020.0",
"@aws-sdk/s3-request-presigner": "^3.1028.0",
"@heyputer/putility": "^1.0.2",
"ai": "^6.0.73",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.14",
"miniflare": "^4.20260617.1",
"open": "^10.1.0"
},
"optionalDependencies": {
"sharp": "^0.34.4"
},
"engines": {
"node": ">=24.0.0"
},
"allowScripts": {
"@google/genai": false,
"bcrypt": true,
"better-sqlite3": true,
"browser-tabs-lock": true,
"classic-level": true,
"esbuild": true,
"fsevents": true,
"protobufjs": true,
"sharp": true,
"workerd": true
},
"overrides": {
"undici": "^7.30.0"
}
}
-137
View File
@@ -1,137 +0,0 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { PuterClient } from './types';
export class EventClient extends PuterClient {
#eventListeners = {};
onServerStart() {
this.emit('serverStart', {}, {});
}
onServerPrepareShutdown() {
this.emit('serverPrepareShutdown', {}, {});
}
onServerShutdown() {
this.emit('serverShutdown', {}, {});
}
/**
* Dispatch an event to every matching subscriber.
*
* Match semantics: emit walks every dot-separated prefix of `key`,
* looking up `<prefix>.*` listeners for prefixes shorter than the full
* key, and exact-key listeners on the final iteration. So emitting
* `outer.gui.item.removed` fires subscribers on:
*
* - `outer.*`
* - `outer.gui.*`
* - `outer.gui.item.*`
* - `outer.gui.item.removed`
*
* Subscribers are still keyed in a single map — wildcards just live
* under their literal `<prefix>.*` string. No regex, no per-emit
* scan of every listener.
*
* @param {string} key
* @param {unknown} data
* @param {object} meta
*/
emit(key, data, meta) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey =
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`;
const listeners = this.#eventListeners[matchKey];
if (!listeners) continue;
for (const listener of listeners) {
this.#emitEvent(listener, key, data, meta);
}
}
}
/**
* Like `emit`, but awaits every matched listener before resolving.
*
* Use this when the emitter needs to act on mutations the handlers made
* to `data` — e.g. validation hooks where a listener can set
* `data.allow = false` to reject, or pre-commit pipelines where every
* stage must complete before the next step runs. Regular `emit` is
* fire-and-forget and can't observe handler state changes.
*
* Listeners run sequentially in the order they're registered so an
* earlier handler's mutation is visible to later ones. A listener that
* throws is logged (same as `emit`) and the chain continues.
*
* @param {string} key
* @param {unknown} data
* @param {object} meta
* @returns {Promise<void>}
*/
async emitAndWait(key, data, meta) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey =
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`;
const listeners = this.#eventListeners[matchKey];
if (!listeners) continue;
for (const listener of listeners) {
try {
await listener(key, data, meta);
} catch (e) {
console.error('Error in event listener for event', key, e);
}
}
}
}
/**
* Subscribe to an event by exact key OR a wildcard prefix.
*
* Wildcards: a key ending in `.*` matches every event whose name
* starts with the prefix. `outer.*` matches `outer.gui.item.removed`,
* `outer.fs.write-hash`, and any other dot-extended descendant.
* Exact keys still match exactly. See `emit()` for the dispatch order.
*
* Callback receives the full `(key, data, meta)` tuple as passed
* to `emit()` — wildcard subscribers can branch on the triggering
* event name.
*
* @param {string} key
* @param {(key: string, data: unknown, meta: object) => void} callback
*/
on(key, callback) {
if (!this.#eventListeners[key]) {
this.#eventListeners[key] = [];
}
this.#eventListeners[key].push(callback);
}
async #emitEvent(listener, key, data, meta) {
try {
await listener(key, data, meta);
} catch (e) {
console.error('Error in event listener for event', key, e);
}
}
}
@@ -0,0 +1,692 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
ALARM_IDLE_TTL_MS,
AlarmClient,
MAX_TRACKED_ALARMS,
} from './AlarmClient';
import type { IConfig, IPagerConfig } from '../../types';
import type { AlertPayload } from './types';
const pdEvent = vi.hoisted(() => vi.fn(async () => ({})));
vi.mock('@pagerduty/pdjs', () => ({ event: pdEvent }));
const makeClient = (pager: IPagerConfig = {}) =>
new AlarmClient({ serverId: 'test-node', pager } as unknown as IConfig);
/** Register a capturing handler in place of a real transport. */
const capture = (
client: AlarmClient,
minSeverity?: 'critical' | 'error' | 'warning' | 'info',
maxSeverity?: 'critical' | 'error' | 'warning' | 'info',
) => {
const seen: AlertPayload[] = [];
client.addAlertHandler(
async (alert) => {
seen.push(alert);
},
{ name: 'capture', minSeverity, maxSeverity },
);
return seen;
};
describe('AlarmClient severity routing', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('defaults to critical when the call site says nothing', () => {
const client = makeClient();
const seen = capture(client);
client.create('boom', 'everything is on fire');
expect(seen).toHaveLength(1);
expect(seen[0].severity).toBe('critical');
});
it('honours a configured default severity', () => {
const client = makeClient({ defaultSeverity: 'info' });
const seen = capture(client);
client.create('boom', 'everything is on fire');
expect(seen[0].severity).toBe('info');
});
it('skips handlers whose floor the alarm does not reach', () => {
const client = makeClient();
const paging = capture(client, 'warning');
const chat = capture(client, 'info');
client.create('rate-limit', 'a user hit a limit', {}, 'info');
expect(paging).toHaveLength(0);
expect(chat).toHaveLength(1);
});
it('skips handlers whose ceiling the alarm exceeds', () => {
const client = makeClient();
const chat = capture(client, 'info', 'info');
client.create('rate-limit', 'a user hit a limit', {}, 'info');
client.create('outage', 'everything is on fire', {}, 'critical');
expect(chat.map((alert) => alert.id)).toEqual(['rate-limit']);
});
it('retiers an alarm from config', () => {
const client = makeClient({
severityOverrides: { 'noisy:*': 'info' },
});
const paging = capture(client, 'warning');
const chat = capture(client, 'info');
client.create('noisy:thing', 'used to page', {}, 'critical');
expect(paging).toHaveLength(0);
expect(chat[0].severity).toBe('info');
});
it('mutes an alarm from config', () => {
const client = makeClient({
severityOverrides: { 'noisy:thing': 'mute' },
});
const chat = capture(client, 'info');
client.create('noisy:thing', 'not worth reporting');
client.create('noisy:thing', 'still not worth reporting');
expect(chat).toHaveLength(0);
});
it('lets config override a known-error rule', () => {
const client = makeClient({
severityOverrides: { 'known:thing': 'critical' },
});
client.setKnownErrors([
{
match: { id: 'known:thing' },
action: { type: 'severity', value: 'info' },
},
]);
const paging = capture(client, 'warning');
client.create('known:thing', 'known but escalated');
expect(paging[0].severity).toBe('critical');
});
it('still respects a no-alert known-error rule', () => {
const client = makeClient();
client.setKnownErrors([
{ match: { id: 'known:quiet' }, action: { type: 'no-alert' } },
]);
const chat = capture(client, 'info');
client.create('known:quiet', 'suppressed');
expect(chat).toHaveLength(0);
});
});
describe('AlarmClient alert payload', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
});
it('reports occurrence counts across repeats', () => {
const client = makeClient();
const seen = capture(client);
client.create('flap', 'first');
client.create('flap', 'second');
expect(seen[0]).toMatchObject({ repeatCount: 1, isRepeat: false });
expect(seen[1]).toMatchObject({ repeatCount: 2, isRepeat: true });
});
it('renders fields as strings and lifts the stack out of the error', () => {
const client = makeClient();
const seen = capture(client);
const error = new Error('kaboom');
client.create('boom', 'failed', { error, status: 500 }, 'critical');
expect(seen[0].fields.status).toBe('500');
expect(seen[0].trace).toBe(error.stack);
expect(seen[0].shortId).toMatch(/^[a-z]+-[a-z]+-[a-z]+$/);
});
it('keeps one failing handler from starving the others', async () => {
const client = makeClient();
client.addAlertHandler(
async () => {
throw new Error('transport down');
},
{ name: 'broken' },
);
const seen = capture(client);
client.create('boom', 'failed');
await Promise.resolve();
expect(seen).toHaveLength(1);
});
it('suppresses alarms once draining', () => {
const client = makeClient();
const seen = capture(client);
client.onServerPrepareShutdown();
client.create('boom', 'too late');
expect(seen).toHaveLength(0);
});
});
describe('AlarmClient transport registration', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('splits info to Slack and everything above it to PagerDuty', async () => {
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
slack: { enabled: true, webhookUrl: 'https://hooks.example/abc' },
});
await client.onServerStart();
client.create('quiet:thing', 'informational', {}, 'info');
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
expect(pdEvent).not.toHaveBeenCalled();
client.create('loud:thing', 'paging', {}, 'critical');
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(1));
// The pager has it; chat doesn't repeat it.
expect(fetchMock).toHaveBeenCalledTimes(1);
vi.unstubAllGlobals();
});
it('sends every severity to Slack when there is no pager', async () => {
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const client = makeClient({
slack: { enabled: true, webhookUrl: 'https://hooks.example/abc' },
});
await client.onServerStart();
client.create(
'loud:thing',
'nowhere else to send this',
{},
'critical',
);
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
vi.unstubAllGlobals();
});
it('lets config widen the Slack ceiling back out', async () => {
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
slack: {
enabled: true,
webhookUrl: 'https://hooks.example/abc',
maxSeverity: 'critical',
},
});
await client.onServerStart();
client.create('loud:thing', 'paging', {}, 'critical');
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
vi.unstubAllGlobals();
});
it('gives each occurrence its own PagerDuty incident by default', async () => {
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
});
await client.onServerStart();
client.create('scan:failed', 'first', {}, 'warning');
client.create('scan:failed', 'second', {}, 'warning');
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
const keys = pdEvent.mock.calls.map(
([arg]: [{ data: { dedup_key: string } }]) => arg.data.dedup_key,
);
expect(keys[0]).not.toBe(keys[1]);
});
it('collapses repeats of a dedup alarm onto one incident', async () => {
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
});
await client.onServerStart();
const raise = () =>
client.create(
'http_500:POST:/notif/mark-ack:deadlock',
'HTTP 500 on POST /notif/mark-ack: deadlock',
{},
'critical',
{ dedup: true },
);
raise();
raise();
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
const keys = pdEvent.mock.calls.map(
([arg]: [{ data: { dedup_key: string } }]) => arg.data.dedup_key,
);
expect(keys[0]).toBe('http_500:POST:/notif/mark-ack:deadlock');
expect(keys[1]).toBe(keys[0]);
});
it('skips transports that are enabled but not configured', async () => {
const client = makeClient({
pagerduty: { enabled: true },
slack: { enabled: true },
});
await client.onServerStart();
const seen = capture(client);
client.create('boom', 'nowhere to send this');
// Only the capturing handler is registered.
expect(seen).toHaveLength(1);
expect(pdEvent).not.toHaveBeenCalled();
});
});
describe('AlarmClient alarm registry', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
});
it('finds an alarm by its full id or its short id', () => {
const client = makeClient();
client.create('disk-pressure', 'nearly full');
const alarm = client.get('disk-pressure');
expect(alarm?.message).toBe('nearly full');
expect(client.get(alarm!.shortId)).toBe(alarm);
});
it('returns nothing for an id it has never seen', () => {
expect(makeClient().get('never-raised')).toBeUndefined();
});
it('forgets an alarm once cleared, under both ids', () => {
const client = makeClient();
client.create('disk-pressure', 'nearly full');
const shortId = client.get('disk-pressure')!.shortId;
client.clear('disk-pressure');
expect(client.get('disk-pressure')).toBeUndefined();
expect(client.get(shortId)).toBeUndefined();
});
it('keeps the short id of another alarm that shares it when one is cleared', () => {
const client = makeClient();
// These two ids hash to the same short id.
client.create('alarm:274', 'first');
const shortId = client.get('alarm:274')!.shortId;
client.create('alarm:643', 'second');
expect(client.get('alarm:643')!.shortId).toBe(shortId);
client.clear('alarm:274');
expect(client.get(shortId)?.id).toBe('alarm:643');
});
it('ignores a clear for an alarm that is not active', () => {
const client = makeClient();
expect(() => client.clear('never-raised')).not.toThrow();
});
it('abbreviates long ids in the log line but keeps the short id usable', () => {
const log = vi.spyOn(console, 'error').mockImplementation(() => {});
const client = makeClient();
const longId = 'a-very-long-alarm-identifier-that-wraps';
client.create(longId, 'noisy');
const shortId = client.get(longId)!.shortId;
expect(log).toHaveBeenCalledWith(
`[alarm] ACTIVE ${shortId} (${longId.slice(0, 20)}...) :: noisy`,
);
});
it('accumulates fields across repeats of the same alarm', () => {
const client = makeClient();
const seen = capture(client);
client.create('flap', 'first', { a: 1 });
client.create('flap', 'second', { b: 2 });
expect(seen[1].fields).toEqual({ a: '1', b: '2' });
expect(client.get('flap')?.occurrences).toHaveLength(2);
});
it('caps retained occurrences while still counting every repeat', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 500; i++) {
client.create('hot', `occurrence ${i}`, { i });
}
const alarm = client.get('hot')!;
expect(alarm.count).toBe(500);
expect(alarm.occurrences).toHaveLength(20);
expect(alarm.timestamps).toHaveLength(20);
// The window kept is the most recent one, not the oldest.
expect(alarm.occurrences[19].message).toBe('occurrence 499');
// Trimming history must not rewind what the transports are told.
expect(seen.at(-1)).toMatchObject({ repeatCount: 500, isRepeat: true });
});
it('names anonymous handlers by their registration order', () => {
const client = makeClient();
client.addAlertHandler(async () => {
throw new Error('down');
});
expect(() => client.create('boom', 'x')).not.toThrow();
});
it('only logs the drain notice once', () => {
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
log.mockClear();
const client = makeClient();
client.onServerPrepareShutdown();
// A second prepare must not restart the drain.
client.onServerPrepareShutdown();
client.create('a', 'x');
client.create('b', 'y');
const drainLogs = log.mock.calls.filter(
([message]) =>
message === '[alarm] suppressing alarm while draining',
);
expect(drainLogs).toHaveLength(1);
});
it('substitutes placeholders for an empty id and message', () => {
const client = makeClient();
const seen = capture(client);
client.create('', '');
expect(seen[0]).toMatchObject({
id: 'something-bad',
message: 'something bad happened',
});
});
});
describe('AlarmClient alarm eviction', () => {
beforeEach(() => {
vi.useFakeTimers();
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
});
afterEach(() => {
vi.useRealTimers();
});
it('forgets alarms that stay quiet past the idle window, under both ids', () => {
const client = makeClient();
const ids = Array.from(
{ length: 50 },
(_, i) => `metering_usage_dropped:user-${i}`,
);
for (const id of ids) client.create(id, 'usage dropped');
const shortIds = ids.map((id) => client.get(id)!.shortId);
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS + 1);
client.create('unrelated', 'the next alarm sweeps the registry');
expect(ids.filter((id) => client.get(id))).toEqual([]);
expect(shortIds.filter((shortId) => client.get(shortId))).toEqual([]);
expect(client.get('unrelated')).toBeDefined();
});
it('treats a recurrence after the idle window as a new alarm', () => {
const client = makeClient();
const seen = capture(client);
client.create('scan:failed', 'first');
client.create('scan:failed', 'second');
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS + 1);
client.create('scan:failed', 'back again');
expect(seen.at(-1)).toMatchObject({ repeatCount: 1, isRepeat: false });
// A fresh start keeps per-occurrence incident keys from colliding.
expect(seen.at(-1)!.dedupKey).not.toBe(seen[0].dedupKey);
});
it('still treats a repeat inside the window as a repeat after a sweep', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 10; i++) client.create('flap', 'same fault');
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS - 1000);
client.create('other', 'sweeps the registry');
client.create('flap', 'same fault');
// Occurrence 11 is past the burst, so nothing new goes out for it.
expect(seen.map((alert) => alert.id)).toEqual([
...Array(10).fill('flap'),
'other',
]);
expect(client.get('flap')?.count).toBe(11);
});
it('keeps an alarm that keeps recurring for longer than the window', () => {
const client = makeClient();
for (let i = 0; i < 6; i++) {
client.create('hot', 'still broken');
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS / 2);
client.create(`noise:${i}`, 'sweeps the registry');
}
expect(client.get('hot')?.count).toBe(6);
});
it('drops the least recently raised alarm once full', () => {
const client = makeClient();
for (let i = 0; i < MAX_TRACKED_ALARMS; i++) {
client.create(`user:${i}`, 'per-user alarm');
vi.advanceTimersByTime(1);
}
// Raised again, so user:1 is now the one seen longest ago.
client.create('user:0', 'per-user alarm');
client.create('one-more', 'over the cap');
expect(client.get('user:1')).toBeUndefined();
expect(client.get('user:0')?.count).toBe(2);
expect(client.get('user:2')).toBeDefined();
expect(client.get('one-more')).toBeDefined();
});
it('never holds more than the cap', () => {
const client = makeClient();
const ids = Array.from(
{ length: MAX_TRACKED_ALARMS * 3 },
(_, i) => `user:${i}`,
);
for (const id of ids) client.create(id, 'per-user alarm');
const held = ids.filter((id) => client.get(id));
expect(held).toHaveLength(MAX_TRACKED_ALARMS);
expect(held[0]).toBe(`user:${MAX_TRACKED_ALARMS * 2}`);
});
});
describe('AlarmClient known-error rules', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
});
it('suppresses alerts for a matching rule', () => {
const client = makeClient();
const seen = capture(client);
client.setKnownErrors([
{ match: { id: 'noisy' }, action: { type: 'no-alert' } },
]);
client.create('noisy', 'ignore me');
client.create('noisy', 'ignore me again');
expect(seen).toHaveLength(0);
});
it('leaves alarms with a different id alone', () => {
const client = makeClient();
const seen = capture(client);
client.setKnownErrors([
{ match: { id: 'noisy' }, action: { type: 'no-alert' } },
]);
client.create('real', 'page me');
expect(seen).toHaveLength(1);
});
it('only matches when the message matches too', () => {
const rules = [
{
match: { id: 'timeout', message: 'upstream timed out' },
action: { type: 'no-alert' as const },
},
];
// Separate clients: suppression is recorded on the alarm, so a
// second occurrence of the same id would inherit it.
const matching = makeClient();
const matchingSeen = capture(matching);
matching.setKnownErrors(rules);
matching.create('timeout', 'upstream timed out');
expect(matchingSeen).toHaveLength(0);
const differing = makeClient();
const differingSeen = capture(differing);
differing.setKnownErrors(rules);
differing.create('timeout', 'something else entirely');
expect(differingSeen.map((alert) => alert.message)).toEqual([
'something else entirely',
]);
});
it('only matches when every named field matches', () => {
const client = makeClient();
const seen = capture(client);
client.setKnownErrors([
{
match: { id: 'http', fields: { status: 404 } },
action: { type: 'no-alert' },
},
]);
client.create('http', 'not found', { status: 404 });
client.create('http-2', 'server error', { status: 500 });
expect(seen.map((alert) => alert.id)).toEqual(['http-2']);
});
it('retiers a matching alarm to a lower severity', () => {
const client = makeClient();
const paging = capture(client, 'error');
const chat = capture(client, 'info');
client.setKnownErrors([
{
match: { id: 'flaky' },
action: { type: 'severity', value: 'info' },
},
]);
client.create('flaky', 'transient', {}, 'critical');
expect(paging).toHaveLength(0);
expect(chat).toHaveLength(1);
expect(chat[0].severity).toBe('info');
});
});
describe('AlarmClient repeat reporting', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('reports every occurrence up to the burst, then backs off', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 12; i++) client.create('flap', 'same fault');
expect(seen).toHaveLength(10);
expect(seen[9].repeatCount).toBe(10);
});
it('reports again once the interval is reached', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 500; i++) client.create('flap', 'same fault');
expect(seen).toHaveLength(11);
expect(seen[10].repeatCount).toBe(500);
});
it('keeps a fast-repeating fault out of the log after the burst', () => {
const client = makeClient();
capture(client);
const warn = vi.mocked(console.warn);
warn.mockClear();
for (let i = 0; i < 100; i++) client.create('flap', 'same fault');
// Occurrences 2-10 only; the first is an ACTIVE line on console.error.
expect(warn.mock.calls).toHaveLength(9);
});
it('skips the payload entirely when no transport accepts the severity', () => {
const client = makeClient();
const seen = capture(client, 'critical');
client.create('quiet', 'nobody is listening', {}, 'info');
expect(seen).toHaveLength(0);
});
});
+325 -115
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -20,64 +20,84 @@
import { event as pdEvent } from '@pagerduty/pdjs';
import { inspect } from 'node:util';
import { createHash } from 'node:crypto';
import type { IConfig } from '../../types';
import type { IConfig, PagerSeverity, SeverityRule } from '../../types';
import { PuterClient } from '../types';
import {
meetsMinSeverity,
resolveSeverityOverride,
withinMaxSeverity,
} from './severity';
import { createSlackAlertHandler } from './slack';
import type {
Alarm,
AlarmFields,
AlarmOptions,
AlertHandler,
AlertPayload,
KnownErrorRule,
} from './types';
// ── Types ────────────────────────────────────────────────────────────
export type {
Alarm,
AlarmFields,
AlarmOptions,
AlertHandler,
AlertPayload,
KnownErrorRule,
} from './types';
export type { PagerSeverity } from '../../types';
export interface AlarmFields {
error?: Error;
[key: string]: unknown;
// -- Types ------------------------------------------------------------
interface RegisteredHandler {
name: string;
/** Lowest severity this transport accepts. */
minSeverity: PagerSeverity;
/** Highest severity this transport accepts. */
maxSeverity: PagerSeverity;
handler: AlertHandler;
}
interface AlarmOccurrence {
message: string;
fields: AlarmFields;
timestamp: number;
}
/** Severity used when neither the call site nor config picks one. */
const FALLBACK_SEVERITY: PagerSeverity = 'critical';
/**
* How many recent occurrences an alarm keeps. A hot alarm repeats for as long
* as the fault lasts, so retaining every occurrence means retaining every
* message and field set it was ever raised with, request bodies and actors
* included. The last few are what a human reads; the rest is only a count, and
* `count` keeps that.
*/
const OCCURRENCE_HISTORY_LIMIT = 20;
/**
* An alarm that hasn't recurred for this long is forgotten; its next occurrence
* is a new alarm with a fresh count. Kept far past the chat transport's repeat
* throttle so a slowly recurring fault still counts as one alarm.
*/
export const ALARM_IDLE_TTL_MS = 24 * 60 * 60 * 1000;
/**
* Most alarms held at once. Ids that embed a user or key can outrun the idle
* window; past this the least recently raised alarm is dropped.
*/
export const MAX_TRACKED_ALARMS = 1000;
interface Alarm {
id: string;
shortId: string;
message: string;
fields: AlarmFields;
error?: Error;
started: number;
timestamps: number[];
occurrences: AlarmOccurrence[];
severity?: PagerSeverity;
noAlert?: boolean;
}
// A fast-recurring fault repeats far faster than it is useful to report, and
// every report allocates a log line and a payload. Report every occurrence up
// to the burst, then one per interval; `count` still tracks all of them.
const REPEAT_REPORT_BURST = 10;
const REPEAT_REPORT_INTERVAL = 500;
type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
const shouldReportRepeat = (count: number): boolean =>
count <= REPEAT_REPORT_BURST || count % REPEAT_REPORT_INTERVAL === 0;
/** Keeps `info` alarms out of the paging system unless config says otherwise. */
const DEFAULT_PAGERDUTY_MIN_SEVERITY: PagerSeverity = 'warning';
/** Slack's ceiling once a pager exists: chat gets what doesn't page. */
const DEFAULT_SLACK_MAX_SEVERITY_WITH_PAGER: PagerSeverity = 'info';
export interface AlertPayload {
id: string;
message: string;
source: string;
severity: PagerSeverity;
custom?: Record<string, unknown>;
}
type AlertHandler = (alert: AlertPayload) => Promise<void>;
interface KnownErrorRule {
match: {
id: string;
message?: string;
fields?: Record<string, unknown>;
};
action: {
type: 'no-alert' | 'severity';
value?: PagerSeverity;
};
}
// ── Helpers ──────────────────────────────────────────────────────────
// -- Helpers ----------------------------------------------------------
/**
* Deterministic short identifier derived from an alarm ID.
* Produces a readable 3-word slug like "amber-delta-fox".
* Deterministic short identifier derived from an alarm ID. Produces a readable
* 3-word slug like "amber-delta-fox".
*/
const WORD_POOL = [
'alpha',
@@ -166,16 +186,21 @@ function cleanFields(fields: AlarmFields): Record<string, string> {
return out;
}
// ── AlarmClient ──────────────────────────────────────────────────────
// -- AlarmClient ------------------------------------------------------
/**
* Manages system alarms and dispatches alerts to external paging
* services (PagerDuty, or any registered handler).
* Manages system alarms and routes them to alert transports by severity.
*
* Severity is the routing decision: each transport declares the severity window
* it accepts, so `critical` pages on-call while `info` only reaches the chat
* channel. Config can retier or mute any alarm id after the fact — see
* `pager.severityOverrides` in {@link IConfig}.
*/
export class AlarmClient extends PuterClient {
/** Ordered by last occurrence: a repeat moves its alarm to the back. */
private alarms = new Map<string, Alarm>();
private aliases = new Map<string, Alarm>();
private alertHandlers: AlertHandler[] = [];
private alertHandlers: RegisteredHandler[] = [];
private knownErrors: KnownErrorRule[] = [];
private draining = false;
private drainLogged = false;
@@ -184,42 +209,11 @@ export class AlarmClient extends PuterClient {
super(config);
}
// ── Lifecycle ────────────────────────────────────────────────────
// -- Lifecycle ----------------------------------------------------
override async onServerStart(): Promise<void> {
const pagerConf = this.config.pager;
if (!pagerConf?.pagerduty?.enabled) return;
const routingKey = pagerConf.pagerduty.routingKey;
if (!routingKey) {
console.warn(
'[alarm] PagerDuty enabled but no routingKey configured',
);
return;
}
const serverId = this.config.serverId;
this.alertHandlers.push(async (alert) => {
await pdEvent({
data: {
routing_key: routingKey,
event_action: 'trigger',
dedup_key: alert.id,
payload: {
summary: alert.message,
source: alert.source,
severity: alert.severity,
custom_details: {
...alert.custom,
server_id: serverId,
},
},
},
});
});
console.log('[alarm] PagerDuty handler registered');
const paging = this.registerPagerDuty();
this.registerSlack({ paging });
}
override onServerPrepareShutdown(): void {
@@ -228,13 +222,109 @@ export class AlarmClient extends PuterClient {
console.log('[alarm] entering drain mode — suppressing new alarms');
}
// ── Public API ───────────────────────────────────────────────────
/** @returns Whether a working PagerDuty transport was registered. */
private registerPagerDuty(): boolean {
const pagerDutyConf = this.config.pager?.pagerduty;
if (!pagerDutyConf?.enabled) return false;
const routingKey = pagerDutyConf.routingKey;
if (!routingKey) {
console.warn(
'[alarm] PagerDuty enabled but no routingKey configured',
);
return false;
}
const serverId = this.config.serverId;
const minSeverity =
pagerDutyConf.minSeverity ?? DEFAULT_PAGERDUTY_MIN_SEVERITY;
this.addAlertHandler(
async (alert) => {
await pdEvent({
data: {
routing_key: routingKey,
event_action: 'trigger',
dedup_key: alert.dedupKey,
payload: {
summary: alert.message,
source: alert.source,
severity: alert.severity,
custom_details: {
...alert.custom,
server_id: serverId,
},
},
},
});
},
{ name: 'pagerduty', minSeverity },
);
console.log(
`[alarm] PagerDuty handler registered (min severity: ${minSeverity})`,
);
return true;
}
private registerSlack({ paging }: { paging: boolean }): void {
const slackConf = this.config.pager?.slack;
if (!slackConf?.enabled) return;
if (!slackConf.webhookUrl) {
console.warn('[alarm] Slack enabled but no webhookUrl configured');
return;
}
const minSeverity = slackConf.minSeverity ?? 'info';
// With a pager taking everything from `warning` up, chat is where the
// rest is recorded — reposting the paging tiers there only trains
// people to skim the channel. Without one, Slack is the only place an
// alarm can land, so it takes all of them.
const maxSeverity =
slackConf.maxSeverity ??
(paging ? DEFAULT_SLACK_MAX_SEVERITY_WITH_PAGER : 'critical');
this.addAlertHandler(
createSlackAlertHandler(slackConf, {
serverId: this.config.serverId,
}),
{ name: 'slack', minSeverity, maxSeverity },
);
console.log(
`[alarm] Slack handler registered (severity ${minSeverity}..${maxSeverity})`,
);
}
// -- Public API ---------------------------------------------------
/**
* Create or update an alarm. If the alarm ID already exists, the
* occurrence count is incremented and a repeat alert is dispatched.
* Create or update an alarm. If the alarm ID already exists, the occurrence
* count is incremented and a repeat alert is dispatched. An alarm idle for
* {@link ALARM_IDLE_TTL_MS} is forgotten, so its next occurrence is new.
*
* `severity` decides where the alarm lands:
*
* Critical — a real outage; pages on-call. Reserve it for unhandled server
* errors. error — pages on-call as well; prefer `critical` or `warning`.
* warning — worth a look soon, but nobody gets woken up. info — a record in
* the chat channel; never pages.
*
* Omit it to take `pager.defaultSeverity` (itself defaulting to
* 'critical'). Operators can retier or mute any alarm id from config
* afterwards, so the value here is the starting point, not the last word.
*
* Pass `{ dedup: true }` when repeats of this id are one recurring fault
* that should collapse into a single incident — see {@link AlarmOptions}.
*/
create(id: string, message: string, fields: AlarmFields = {}): void {
create(
id: string,
message: string,
fields: AlarmFields = {},
severity?: PagerSeverity,
opts: AlarmOptions = {},
): void {
if (this.draining) {
if (!this.drainLogged) {
this.drainLogged = true;
@@ -243,9 +333,12 @@ export class AlarmClient extends PuterClient {
return;
}
this.evict();
const existing = this.alarms.get(id);
if (existing) {
this.alarms.delete(id);
this.alarms.set(id, existing);
this.recordOccurrence(existing, message, fields);
this.handleRepeat(existing);
return;
@@ -256,8 +349,13 @@ export class AlarmClient extends PuterClient {
shortId: shortId(id),
message,
fields,
severity,
dedup: opts.dedup,
started: Date.now(),
timestamps: [Date.now()],
// `recordOccurrence` below stamps the first occurrence; seeding one
// here too would report every alarm as one occurrence ahead.
count: 0,
timestamps: [],
occurrences: [],
};
if (fields.error) alarm.error = fields.error;
@@ -265,6 +363,7 @@ export class AlarmClient extends PuterClient {
this.alarms.set(id, alarm);
this.aliases.set(alarm.shortId, alarm);
this.recordOccurrence(alarm, message, fields);
this.evict();
this.handleNew(alarm);
}
@@ -273,8 +372,7 @@ export class AlarmClient extends PuterClient {
const alarm = this.alarms.get(id);
if (!alarm) return;
this.alarms.delete(id);
this.aliases.delete(alarm.shortId);
this.forget(alarm);
console.log(`[alarm] CLEAR ${displayId(alarm)} :: ${alarm.message}`);
}
@@ -284,37 +382,84 @@ export class AlarmClient extends PuterClient {
}
/**
* Register an additional alert handler. Handlers are called for
* every alarm that isn't suppressed by a known-error rule.
* Register an additional alert handler. Handlers are called for every alarm
* that isn't suppressed by a known-error rule or muted by config, and whose
* severity falls inside the handler's own `minSeverity`..`maxSeverity`
* window (default: everything).
*/
addAlertHandler(handler: AlertHandler): void {
this.alertHandlers.push(handler);
addAlertHandler(
handler: AlertHandler,
opts: {
name?: string;
minSeverity?: PagerSeverity;
maxSeverity?: PagerSeverity;
} = {},
): void {
this.alertHandlers.push({
name: opts.name ?? `handler-${this.alertHandlers.length}`,
minSeverity: opts.minSeverity ?? 'info',
maxSeverity: opts.maxSeverity ?? 'critical',
handler,
});
}
/**
* Add rules that can suppress or adjust severity of known errors.
*/
/** Add rules that can suppress or adjust severity of known errors. */
setKnownErrors(rules: KnownErrorRule[]): void {
this.knownErrors = rules;
}
// ── Internals ────────────────────────────────────────────────────
// -- Internals ----------------------------------------------------
/**
* Drop idle alarms, then the least recently raised ones past the cap. The
* map's order puts both at the front, so this stops at the first keeper.
*/
private evict(): void {
const now = Date.now();
for (const alarm of this.alarms.values()) {
const lastSeen = alarm.timestamps.at(-1) ?? alarm.started;
if (
this.alarms.size <= MAX_TRACKED_ALARMS &&
now - lastSeen < ALARM_IDLE_TTL_MS
) {
return;
}
this.forget(alarm);
}
}
private forget(alarm: Alarm): void {
this.alarms.delete(alarm.id);
// Short ids can collide; leave an alias that now names another alarm.
if (this.aliases.get(alarm.shortId) === alarm) {
this.aliases.delete(alarm.shortId);
}
}
private recordOccurrence(
alarm: Alarm,
message: string,
fields: AlarmFields,
): void {
const now = Date.now();
alarm.message = message;
alarm.fields = { ...alarm.fields, ...fields };
alarm.timestamps.push(Date.now());
alarm.count++;
if (fields.error) alarm.error = fields.error;
alarm.occurrences.push({
message,
fields,
timestamp: Date.now(),
});
alarm.timestamps.push(now);
alarm.occurrences.push({ message, fields, timestamp: now });
if (alarm.timestamps.length > OCCURRENCE_HISTORY_LIMIT) {
alarm.timestamps.splice(
0,
alarm.timestamps.length - OCCURRENCE_HISTORY_LIMIT,
);
alarm.occurrences.splice(
0,
alarm.occurrences.length - OCCURRENCE_HISTORY_LIMIT,
);
}
}
private applyKnownErrors(alarm: Alarm): void {
@@ -361,8 +506,10 @@ export class AlarmClient extends PuterClient {
private handleRepeat(alarm: Alarm): void {
this.applyKnownErrors(alarm);
if (!shouldReportRepeat(alarm.count)) return;
console.warn(
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.timestamps.length})`,
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.count})`,
);
if (alarm.noAlert) return;
@@ -370,25 +517,88 @@ export class AlarmClient extends PuterClient {
this.dispatchAlert(alarm);
}
/**
* Call-site severity, then any known-error rule (both already on the
* alarm), then the config override — so an operator always has the last
* word over what the code asked for.
*/
private resolveSeverity(alarm: Alarm): SeverityRule {
const base =
alarm.severity ??
this.config.pager?.defaultSeverity ??
FALLBACK_SEVERITY;
return (
resolveSeverityOverride(
alarm.id,
this.config.pager?.severityOverrides,
) ?? base
);
}
private hasHandlerFor(severity: PagerSeverity): boolean {
for (const { minSeverity, maxSeverity } of this.alertHandlers) {
if (
meetsMinSeverity(severity, minSeverity) &&
withinMaxSeverity(severity, maxSeverity)
) {
return true;
}
}
return false;
}
private dispatchAlert(alarm: Alarm): void {
const severity = alarm.severity ?? 'critical';
const resolved = this.resolveSeverity(alarm);
if (resolved === 'mute') {
if (!alarm.muteLogged) {
alarm.muteLogged = true;
console.log(`[alarm] MUTED by config ${displayId(alarm)}`);
}
return;
}
alarm.severity = resolved;
// Build nothing for a severity no transport accepts — the payload
// (cleaned fields plus the stack, twice) is the expensive part.
if (!this.hasHandlerFor(resolved)) return;
const fieldsClean = cleanFields(alarm.fields);
const repeatCount = alarm.count;
const id = alarm.id || 'something-bad';
const payload: AlertPayload = {
id: alarm.id || 'something-bad',
id,
// A de-duplicating transport should fold repeats of a recurring
// fault into one incident, but everything else is a fresh event
// each time it fires — the start time keeps occurrence numbering
// from colliding with an incident left open by an earlier boot.
dedupKey: alarm.dedup
? id
: `${id}#${alarm.started}.${repeatCount}`,
shortId: alarm.shortId,
message: alarm.message || alarm.id || 'something bad happened',
source: 'alarm',
severity,
severity: resolved,
fields: fieldsClean,
trace: alarm.error?.stack,
repeatCount,
isRepeat: repeatCount > 1,
custom: {
fields: fieldsClean,
trace: alarm.error?.stack,
repeat_count: alarm.timestamps.length,
repeat_count: repeatCount,
},
};
for (const handler of this.alertHandlers) {
for (const { name, minSeverity, maxSeverity, handler } of this
.alertHandlers) {
if (!meetsMinSeverity(resolved, minSeverity)) continue;
if (!withinMaxSeverity(resolved, maxSeverity)) continue;
handler(payload).catch((err) => {
console.error(`[alarm] alert handler failed: ${err?.message}`);
console.error(
`[alarm] ${name} alert handler failed: ${err?.message}`,
);
});
}
}
@@ -0,0 +1,76 @@
import { describe, expect, it, vi } from 'vitest';
import { meetsMinSeverity, resolveSeverityOverride } from './severity';
describe('meetsMinSeverity', () => {
it('accepts everything at or above the floor', () => {
expect(meetsMinSeverity('critical', 'warning')).toBe(true);
expect(meetsMinSeverity('error', 'warning')).toBe(true);
expect(meetsMinSeverity('warning', 'warning')).toBe(true);
expect(meetsMinSeverity('info', 'warning')).toBe(false);
});
it('lets an info floor take every severity', () => {
for (const severity of [
'critical',
'error',
'warning',
'info',
] as const) {
expect(meetsMinSeverity(severity, 'info')).toBe(true);
}
});
});
describe('resolveSeverityOverride', () => {
it('returns undefined without overrides or on no match', () => {
expect(resolveSeverityOverride('a:b', undefined)).toBeUndefined();
expect(
resolveSeverityOverride('a:b', { 'c:*': 'info' }),
).toBeUndefined();
});
it('matches an exact id', () => {
expect(
resolveSeverityOverride('cronMonitor:lowSignupRate', {
'cronMonitor:lowSignupRate': 'info',
}),
).toBe('info');
});
it('matches a prefix pattern', () => {
expect(
resolveSeverityOverride('cronMonitor:high_aiLogEntries', {
'cronMonitor:*': 'warning',
}),
).toBe('warning');
});
it('prefers the exact id over a prefix pattern', () => {
expect(
resolveSeverityOverride('cronMonitor:lowSignupRate', {
'cronMonitor:*': 'warning',
'cronMonitor:lowSignupRate': 'mute',
}),
).toBe('mute');
});
it('prefers the longest matching prefix', () => {
expect(
resolveSeverityOverride('abuse:card-verification:setup-failed', {
'abuse:*': 'info',
'abuse:card-verification:*': 'mute',
}),
).toBe('mute');
});
it('drops an unrecognized rule rather than guessing', () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
expect(
resolveSeverityOverride('a:b', {
'a:b': 'silent' as unknown as 'mute',
}),
).toBeUndefined();
expect(warn).toHaveBeenCalled();
warn.mockRestore();
});
});
+94
View File
@@ -0,0 +1,94 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { PagerSeverity, SeverityRule } from '../../types';
/** Ascending urgency. A transport takes everything at or above its floor. */
const RANK: Record<PagerSeverity, number> = {
info: 0,
warning: 1,
error: 2,
critical: 3,
};
const SEVERITIES = Object.keys(RANK) as PagerSeverity[];
export function isPagerSeverity(value: unknown): value is PagerSeverity {
return typeof value === 'string' && value in RANK;
}
/** True when `severity` is urgent enough for a transport whose floor is `min`. */
export function meetsMinSeverity(
severity: PagerSeverity,
min: PagerSeverity,
): boolean {
return RANK[severity] >= RANK[min];
}
/**
* True when `severity` is quiet enough for a transport whose ceiling is `max`.
* A ceiling is what keeps a chat transport from repeating everything the pager
* already delivered.
*/
export function withinMaxSeverity(
severity: PagerSeverity,
max: PagerSeverity,
): boolean {
return RANK[severity] <= RANK[max];
}
/**
* Look up the operator override for an alarm id. Exact ids win over prefix
* patterns (`cronMonitor:*`); among patterns the longest prefix wins, so a
* specific rule can carve an exception out of a broad one.
*/
export function resolveSeverityOverride(
id: string,
overrides: Record<string, SeverityRule> | undefined,
): SeverityRule | undefined {
if (!overrides) return undefined;
const exact = overrides[id];
if (exact !== undefined) return validRule(id, exact);
let bestLength = -1;
let best: SeverityRule | undefined;
for (const [pattern, rule] of Object.entries(overrides)) {
if (!pattern.endsWith('*')) continue;
const prefix = pattern.slice(0, -1);
if (!id.startsWith(prefix)) continue;
if (prefix.length <= bestLength) continue;
bestLength = prefix.length;
best = rule;
}
return best === undefined ? undefined : validRule(id, best);
}
/**
* A typo in the override map would otherwise silently mute or escalate an
* alarm, so an unrecognized value is dropped with a warning instead.
*/
function validRule(id: string, rule: SeverityRule): SeverityRule | undefined {
if (rule === 'mute' || isPagerSeverity(rule)) return rule;
console.warn(
`[alarm] ignoring invalid severity override "${rule}" for ${id} ` +
`(expected mute or one of ${SEVERITIES.join(', ')})`,
);
return undefined;
}
+206
View File
@@ -0,0 +1,206 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { buildSlackMessage, createSlackAlertHandler } from './slack';
import type { AlertPayload } from './types';
const alert = (over: Partial<AlertPayload> = {}): AlertPayload => ({
id: 'cronMonitor:high_aiLogEntries',
dedupKey: 'cronMonitor:high_aiLogEntries#1.1',
shortId: 'amber-delta-fox',
message: 'High AI log entries: 1200 in the last 10 minutes',
source: 'alarm',
severity: 'warning',
fields: { count: '1200', threshold: '1000' },
repeatCount: 1,
isRepeat: false,
...over,
});
describe('buildSlackMessage', () => {
it('renders severity, message and fields', () => {
const msg = buildSlackMessage(alert(), {
channel: '#alerts',
username: 'puter-alarms',
serverId: 'oregon',
});
expect(msg.text).toContain('[WARNING]');
expect(msg.text).toContain('High AI log entries');
expect(msg.channel).toBe('#alerts');
expect(msg.username).toBe('puter-alarms');
expect(msg.attachments[0].fields).toEqual([
{ title: 'count', value: '1200', short: true },
{ title: 'threshold', value: '1000', short: true },
]);
expect(msg.attachments[0].footer).toBe(
'amber-delta-fox • cronMonitor:high_aiLogEntries • oregon',
);
});
it('marks repeats with an occurrence count', () => {
const msg = buildSlackMessage(
alert({ repeatCount: 7, isRepeat: true }),
);
expect(msg.text).toContain('(x7)');
});
it('colours each severity differently', () => {
const colors = (['critical', 'error', 'warning', 'info'] as const).map(
(severity) =>
buildSlackMessage(alert({ severity })).attachments[0].color,
);
expect(new Set(colors).size).toBe(4);
});
it('puts the stack in a code block and keeps it out of the fields', () => {
const msg = buildSlackMessage(
alert({
fields: { error: 'Error: boom', path: '/api/x' },
trace: 'Error: boom\n at handler',
}),
);
expect(msg.attachments[0].text).toBe(
'```Error: boom\n at handler```',
);
expect(msg.attachments[0].fields.map((f) => f.title)).toEqual(['path']);
});
it('truncates long values and caps the field count', () => {
const fields: Record<string, string> = { long: 'x'.repeat(1000) };
for (let i = 0; i < 30; i++) fields[`f${i}`] = String(i);
const msg = buildSlackMessage(alert({ fields }));
expect(msg.attachments[0].fields.length).toBe(12);
expect(msg.attachments[0].fields[0].value).toHaveLength(400);
expect(msg.attachments[0].fields[0].value.endsWith('…')).toBe(true);
});
it('omits channel and username when unset', () => {
const msg = buildSlackMessage(alert());
expect(msg.channel).toBeUndefined();
expect(msg.username).toBeUndefined();
});
});
describe('createSlackAlertHandler', () => {
let fetchMock: ReturnType<typeof vi.fn>;
beforeEach(() => {
vi.useFakeTimers();
fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
});
afterEach(() => {
vi.useRealTimers();
vi.unstubAllGlobals();
});
it('posts the payload to the webhook', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
channel: '#alerts',
});
await handler(alert());
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://hooks.example/abc');
expect(init.method).toBe('POST');
expect(JSON.parse(init.body).channel).toBe('#alerts');
});
it('throttles repeats of the same alarm and lets other ids through', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
await handler(alert());
await handler(alert({ repeatCount: 2, isRepeat: true }));
expect(fetchMock).toHaveBeenCalledTimes(1);
await handler(alert({ id: 'other:alarm' }));
expect(fetchMock).toHaveBeenCalledTimes(2);
vi.advanceTimersByTime(60_000);
await handler(alert({ repeatCount: 3, isRepeat: true }));
expect(fetchMock).toHaveBeenCalledTimes(3);
});
it('posts every occurrence when throttling is disabled', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 0,
});
await handler(alert());
await handler(alert({ repeatCount: 2, isRepeat: true }));
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('does not let a failed post consume the throttle slot', async () => {
fetchMock.mockResolvedValueOnce({ ok: false, status: 500 });
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
await expect(handler(alert())).rejects.toThrow('500');
await handler(alert());
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('releases the throttle slot when the post itself errors', async () => {
fetchMock.mockRejectedValueOnce(new Error('socket hang up'));
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
await expect(handler(alert())).rejects.toThrow('socket hang up');
await handler(alert());
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('evicts throttle entries that have aged out once the map fills', async () => {
const throttleMs = 60_000;
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: throttleMs,
});
// Fill to the high-water mark, then age every entry past the window.
for (let i = 0; i < 5000; i++) {
await handler(alert({ id: `aged:${i}` }));
}
vi.advanceTimersByTime(throttleMs);
await handler(alert({ id: 'fresh' }));
expect(fetchMock).toHaveBeenCalledTimes(5001);
// The aged ids were pruned, so they post again immediately.
await handler(alert({ id: 'aged:0' }));
expect(fetchMock).toHaveBeenCalledTimes(5002);
});
it('drops the oldest entry when the map is full of live ones', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
for (let i = 0; i < 5000; i++) {
await handler(alert({ id: `live:${i}` }));
}
// Nothing has aged out, so the oldest id makes room for the new one.
await handler(alert({ id: 'newcomer' }));
expect(fetchMock).toHaveBeenCalledTimes(5001);
await handler(alert({ id: 'live:0' }));
expect(fetchMock).toHaveBeenCalledTimes(5002);
// The newcomer is still throttled.
await handler(alert({ id: 'newcomer' }));
expect(fetchMock).toHaveBeenCalledTimes(5002);
});
});
+163
View File
@@ -0,0 +1,163 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { ISlackAlertConfig, PagerSeverity } from '../../types';
import { BoundedTtlMap } from '../../util/boundedTtlMap';
import type { AlertHandler, AlertPayload } from './types';
const REQUEST_TIMEOUT_MS = 5000;
export const DEFAULT_REPEAT_THROTTLE_MS = 15 * 60 * 1000;
/** Beyond this many tracked alarm ids, throttle state is pruned. */
const MAX_THROTTLE_ENTRIES = 5000;
const MAX_FIELDS = 12;
const MAX_FIELD_VALUE = 400;
const MAX_TRACE = 1500;
/** Values under this length sit two-per-row in the Slack attachment. */
const SHORT_FIELD_LENGTH = 40;
const STYLE: Record<PagerSeverity, { emoji: string; color: string }> = {
critical: { emoji: ':rotating_light:', color: '#d64545' },
error: { emoji: ':red_circle:', color: '#e08d4c' },
warning: { emoji: ':warning:', color: '#e0b84c' },
info: { emoji: ':information_source:', color: '#4c8de0' },
};
interface SlackField {
title: string;
value: string;
short: boolean;
}
export interface SlackMessage {
text: string;
channel?: string;
username?: string;
icon_emoji?: string;
attachments: Array<{
color: string;
fallback: string;
fields: SlackField[];
footer: string;
mrkdwn_in: string[];
text?: string;
}>;
}
function truncate(value: string, max: number): string {
return value.length <= max ? value : `${value.slice(0, max - 1)}…`;
}
/**
* Render an alert as an incoming-webhook payload: severity-coloured attachment,
* the alarm's fields as a table, and the stack (when there is one) as a code
* block. Split out from the handler so the formatting is testable on its own.
*/
export function buildSlackMessage(
alert: AlertPayload,
opts: { channel?: string; username?: string; serverId?: string } = {},
): SlackMessage {
const style = STYLE[alert.severity] ?? STYLE.info;
const repeat = alert.isRepeat ? ` (x${alert.repeatCount})` : '';
const headline = `${style.emoji} *[${alert.severity.toUpperCase()}]* ${alert.message}${repeat}`;
const fields: SlackField[] = [];
for (const [key, value] of Object.entries(alert.fields)) {
if (fields.length >= MAX_FIELDS) break;
// `error` is already rendered as the trace block below.
if (key === 'error') continue;
const rendered = truncate(value, MAX_FIELD_VALUE);
fields.push({
title: key,
value: rendered,
short: rendered.length <= SHORT_FIELD_LENGTH,
});
}
const footerParts = [alert.shortId, alert.id];
if (opts.serverId) footerParts.push(opts.serverId);
return {
text: headline,
...(opts.channel ? { channel: opts.channel } : {}),
...(opts.username ? { username: opts.username } : {}),
attachments: [
{
color: style.color,
fallback: `[${alert.severity}] ${alert.message}`,
fields,
footer: footerParts.join(' • '),
mrkdwn_in: ['text'],
...(alert.trace
? { text: '```' + truncate(alert.trace, MAX_TRACE) + '```' }
: {}),
},
],
};
}
/**
* Post alerts to a Slack incoming webhook. Repeats of the same alarm id are
* throttled so a hot loop doesn't flood the channel — the occurrence count on
* the next post that gets through tells the reader what they missed.
*/
export function createSlackAlertHandler(
conf: ISlackAlertConfig,
opts: { serverId?: string } = {},
): AlertHandler {
const webhookUrl = conf.webhookUrl as string;
const throttleMs = conf.repeatThrottleMs ?? DEFAULT_REPEAT_THROTTLE_MS;
// A zero or negative throttle expires every entry on arrival.
const lastPosted = new BoundedTtlMap<string, true>({
maxEntries: MAX_THROTTLE_ENTRIES,
ttlMs: throttleMs,
});
return async (alert) => {
if (!lastPosted.shouldEmit(alert.id)) return;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
try {
const res = await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(
buildSlackMessage(alert, {
channel: conf.channel,
username: conf.username,
serverId: opts.serverId,
}),
),
signal: controller.signal,
});
if (!res.ok) {
// A rejected post shouldn't leave the id marked as delivered.
lastPosted.delete(alert.id);
throw new Error(`Slack webhook returned ${res.status}`);
}
} catch (err) {
lastPosted.delete(alert.id);
throw err;
} finally {
clearTimeout(timer);
}
};
}
+105
View File
@@ -0,0 +1,105 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { PagerSeverity } from '../../types';
export interface AlarmFields {
error?: Error;
[key: string]: unknown;
}
export interface AlarmOccurrence {
message: string;
fields: AlarmFields;
timestamp: number;
}
export interface AlarmOptions {
/**
* Collapse repeats of this alarm id into a single incident on transports
* that de-duplicate (PagerDuty). Only right when the id already pins down
* one recurring fault — an uncaught error on a route, say — so that N
* occurrences really are one thing to fix. Off by default: every other
* alarm raises its own incident per occurrence.
*/
dedup?: boolean;
}
export interface Alarm extends AlarmOptions {
id: string;
shortId: string;
message: string;
fields: AlarmFields;
error?: Error;
started: number;
/**
* Every occurrence ever counted, including those aged out of the two lists
* below.
*/
count: number;
/**
* Timestamps of the most recent occurrences only — see
* `OCCURRENCE_HISTORY_LIMIT`.
*/
timestamps: number[];
/** The most recent occurrences only — see `OCCURRENCE_HISTORY_LIMIT`. */
occurrences: AlarmOccurrence[];
severity?: PagerSeverity;
noAlert?: boolean;
/** Set once a config mute has been logged, so it's reported only once. */
muteLogged?: boolean;
}
export interface AlertPayload {
id: string;
/**
* Key a de-duplicating transport groups by. Equal to `id` for alarms raised
* with `dedup`, and unique per occurrence for everything else.
*/
dedupKey: string;
/** Readable slug for the same alarm, for humans quoting it back. */
shortId: string;
message: string;
source: string;
severity: PagerSeverity;
/** Field values rendered as strings, ready to display. */
fields: Record<string, string>;
/** Stack of the attached error, when the alarm carried one. */
trace?: string;
/** Total occurrences of this alarm so far, including this one. */
repeatCount: number;
/** False for the first occurrence of an alarm id. */
isRepeat: boolean;
/** PagerDuty `custom_details` payload. */
custom?: Record<string, unknown>;
}
export type AlertHandler = (alert: AlertPayload) => Promise<void>;
export interface KnownErrorRule {
match: {
id: string;
message?: string;
fields?: Record<string, unknown>;
};
action: {
type: 'no-alert' | 'severity';
value?: PagerSeverity;
};
}
@@ -0,0 +1,35 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
// Optional analytics client, registered by an extension. Only the surface the
// app-stats path consumes is declared; extend it rather than widening to `any`.
export interface ClickhouseQueryResult {
json<T = Record<string, unknown>>(): Promise<T[]>;
}
export interface ClickhouseQueryParams {
query: string;
query_params?: Record<string, unknown>;
format?: string;
}
export interface ClickhouseClient {
query(params: ClickhouseQueryParams): Promise<ClickhouseQueryResult>;
}
@@ -0,0 +1,279 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { DatabaseClientFactory } from './index.js';
import { MySQLDatabaseClient } from './MySQLDatabaseClient.js';
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
const config = (engine: string): IConfig =>
({ port: 0, extensions: [], database: { engine } }) as IConfig;
const sqlite = () => new SqliteDatabaseClient(config('sqlite'));
const mysql = () => new MySQLDatabaseClient(config('mysql'));
const postgres = () => new PostgresDatabaseClient(config('postgres'));
/**
* Minimal concrete client backed by fixture rows — exercises the shared helpers
* on `AbstractDatabaseClient` that the real engines inherit.
*/
class FixtureDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'fixture';
readonly reads: string[] = [];
readonly preads: string[] = [];
readonly writes: { query: string; params: unknown[] }[] = [];
constructor(
private readonly replicaRows: Record<string, unknown>[] | Error,
private readonly primaryRows: Record<string, unknown>[] = [],
private readonly replica = true,
) {
super(config('fixture'));
}
protected override hasReadReplica(): boolean {
return this.replica;
}
override async read(query: string): Promise<Record<string, unknown>[]> {
this.reads.push(query);
if (this.replicaRows instanceof Error) throw this.replicaRows;
return this.replicaRows;
}
override async pread(query: string): Promise<Record<string, unknown>[]> {
this.preads.push(query);
return this.primaryRows;
}
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
this.writes.push({ query, params });
return { insertId: 1, affectedRows: 1, anyRowsAffected: true };
}
}
describe('DatabaseClientFactory', () => {
it('picks sqlite when no engine is configured', () => {
expect(
new DatabaseClientFactory({ port: 0, extensions: [] } as IConfig),
).toBeInstanceOf(SqliteDatabaseClient);
});
it('refuses an unknown engine name', () => {
expect(() => new DatabaseClientFactory(config('cassandra'))).toThrow(
'Unknown database engine: cassandra',
);
});
});
describe('AbstractDatabaseClient — unimplemented surface', () => {
const bare = () => new AbstractDatabaseClient(config('none'));
it('refuses to run queries until a subclass implements them', async () => {
await expect(bare().read('SELECT 1')).rejects.toThrow(
'DatabaseClient.read() not implemented',
);
await expect(bare().pread('SELECT 1')).rejects.toThrow(
'DatabaseClient.pread() not implemented',
);
await expect(bare().write('DELETE FROM x')).rejects.toThrow(
'DatabaseClient.write() not implemented',
);
await expect(bare().batchWrite([])).rejects.toThrow(
'DatabaseClient.batchWrite() not implemented',
);
});
});
describe('AbstractDatabaseClient — replica-aware reads', () => {
it('returns replica rows without waiting on the primary', async () => {
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 1 },
]);
});
it('falls back to the primary when the replica has not caught up', async () => {
const client = new FixtureDatabaseClient([], [{ id: 2 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 2 },
]);
});
it('falls back to the primary when the replica read throws', async () => {
const client = new FixtureDatabaseClient(
new Error('replica unavailable'),
[{ id: 3 }],
);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 3 },
]);
});
it('surfaces the primary error when both reads fail', async () => {
const client = new FixtureDatabaseClient(new Error('replica down'));
client.pread = async () => {
throw new Error('primary down');
};
await expect(client.tryHardRead('SELECT 1')).rejects.toThrow(
'primary down',
);
});
it('issues a single read when there is no replica', async () => {
const client = new FixtureDatabaseClient([], [{ id: 2 }], false);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([]);
expect(client.reads).toEqual(['SELECT 1']);
expect(client.preads).toEqual([]);
});
});
describe('AbstractDatabaseClient — SQL generation', () => {
it('generates an INSERT with quoted columns and placeholders', async () => {
const client = new FixtureDatabaseClient([]);
await client.insert('user', { username: 'ada', email: null });
expect(client.writes).toEqual([
{
query: 'INSERT INTO `user` (`username`, `email`) VALUES (?, ?)',
params: ['ada', null],
},
]);
});
it('quotes each dotted segment and escapes embedded backticks', () => {
const client = sqlite();
expect(client.quoteIdentifier('user')).toBe('`user`');
expect(client.quoteIdentifier('db.user.id')).toBe('`db`.`user`.`id`');
expect(client.quoteIdentifier('user.*')).toBe('`user`.*');
expect(client.quoteIdentifier('we`ird')).toBe('`we``ird`');
});
it('quotes postgres identifiers with double quotes', () => {
const client = postgres();
expect(client.quoteIdentifier('db.user')).toBe('"db"."user"');
expect(client.quoteIdentifier('we"ird')).toBe('"we""ird"');
expect(client.quoteIdentifier('user.*')).toBe('"user".*');
});
it('renders booleans the way each engine expects', () => {
expect(sqlite().booleanLiteral(true)).toBe('1');
expect(sqlite().booleanValue(false)).toBe(0);
expect(postgres().booleanLiteral(true)).toBe('TRUE');
expect(postgres().booleanLiteral(false)).toBe('FALSE');
expect(postgres().booleanValue(true)).toBe(true);
});
it('picks the engine-specific ignore-conflict syntax', () => {
expect(sqlite().insertIgnoreInto('kv')).toBe(
'INSERT OR IGNORE INTO `kv`',
);
expect(sqlite().insertIgnoreSuffix()).toBe('');
expect(postgres().insertIgnoreInto('kv')).toBe('INSERT INTO "kv"');
expect(postgres().insertIgnoreSuffix()).toBe(' ON CONFLICT DO NOTHING');
expect(mysql().insertIgnoreInto('kv')).toBe('INSERT IGNORE INTO `kv`');
expect(mysql().insertIgnoreSuffix()).toBe('');
});
it('builds an upsert clause per engine', () => {
expect(mysql().upsertClause(['user_id'], ['value', 'dt'])).toBe(
'ON DUPLICATE KEY UPDATE `value` = ?, `dt` = ?',
);
expect(sqlite().upsertClause(['user_id', 'app'], ['value'])).toBe(
'ON CONFLICT(`user_id`, `app`) DO UPDATE SET `value` = ?',
);
expect(postgres().upsertClause(['user_id'], ['value'])).toBe(
'ON CONFLICT("user_id") DO UPDATE SET "value" = ?',
);
});
it('rejects an upsert with nothing to update', () => {
expect(() => sqlite().upsertClause(['user_id'], [])).toThrow(
'upsertClause requires at least one update column',
);
});
it('extracts JSON text using each engine dialect', () => {
expect(sqlite().jsonTextExtract('`metadata`', ['a', 'b'])).toBe(
"json_extract(`metadata`, '$.a.b')",
);
expect(mysql().jsonTextExtract('`metadata`', ['a'])).toBe(
"JSON_UNQUOTE(JSON_EXTRACT(`metadata`, '$.a'))",
);
expect(postgres().jsonTextExtract('"metadata"', ['a', 'b'])).toBe(
`"metadata" #>> ARRAY['a', 'b']`,
);
});
it('escapes quotes inside JSON path segments', () => {
expect(sqlite().jsonTextExtract('`m`', ["it's"])).toBe(
"json_extract(`m`, '$.it''s')",
);
});
it('coalesces expressions and rejects an empty list', () => {
expect(sqlite().nullCoalesce('`a`', '`b`', '0')).toBe(
'COALESCE(`a`, `b`, 0)',
);
expect(() => sqlite().nullCoalesce()).toThrow(
'nullCoalesce requires at least one expression',
);
});
it('only postgres needs a RETURNING clause to learn the insert id', () => {
expect(postgres().returningIdClause()).toBe(' RETURNING id');
expect(sqlite().returningIdClause()).toBe('');
expect(mysql().returningIdClause()).toBe('');
});
it('stamps a timestamp from unix seconds in each dialect', () => {
expect(sqlite().timestampFromUnixSql()).toBe(
"datetime(?, 'unixepoch')",
);
expect(mysql().timestampFromUnixSql()).toBe('FROM_UNIXTIME(?)');
expect(postgres().timestampFromUnixSql()).toBe('to_timestamp(?)');
});
it('describes a stored timestamp the way each driver reads it back', () => {
const seconds = Date.UTC(2026, 9, 11, 8, 5, 9) / 1000;
expect(sqlite().timestampValue(seconds)).toBe('2026-10-11 08:05:09');
expect(mysql().timestampValue(seconds)).toEqual(
new Date(seconds * 1000),
);
expect(postgres().timestampValue(seconds)).toEqual(
new Date(seconds * 1000),
);
});
it('falls back to `otherwise` for an engine with no explicit choice', () => {
const client = new FixtureDatabaseClient([]);
expect(
client.case({ sqlite: 'a', mysql: 'b', otherwise: 'fallback' }),
).toBe('fallback');
expect(client.case({ fixture: 'exact', otherwise: 'fallback' })).toBe(
'exact',
);
});
});
+262 -40
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,8 +17,12 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import type { IConfig } from '../../types';
import { Span } from '../../util/span.js';
import { PuterClient } from '../types';
import { compareMigrationFilenames } from './migrationFilenames.js';
export interface WriteResult {
insertId: number | bigint;
@@ -31,12 +35,20 @@ export interface BatchEntry {
values: unknown[];
}
type SqlJsonPath = readonly [string, ...string[]];
/** The slice of a pooled connection that migrations need. */
export interface MigrationConnection {
query(sql: string): Promise<unknown>;
release(): void;
}
/**
* Base database client. Subclasses must override every method that throws here.
*
* Do not instantiate directly — use the factory exported from
* `clients/database/index.ts` which picks the right implementation
* based on `config.database.engine`.
* `clients/database/index.ts` which picks the right implementation based on
* `config.database.engine`.
*/
export class AbstractDatabaseClient extends PuterClient {
/** Short name used by `case()` to pick engine-specific values. */
@@ -50,9 +62,7 @@ export class AbstractDatabaseClient extends PuterClient {
// Abstract interface — subclasses MUST override
// ------------------------------------------------------------------
/**
* Execute a read query. Returns an array of row objects.
*/
/** Execute a read query. Returns an array of row objects. */
async read(
_query: string,
_params: unknown[] = [],
@@ -61,9 +71,9 @@ export class AbstractDatabaseClient extends PuterClient {
}
/**
* Read that prefers the primary database (useful when read-replicas
* may have replication lag). In single-node setups this is identical
* to `read()`.
* Read that prefers the primary database (useful when read-replicas may
* have replication lag). In single-node setups this is identical to
* `read()`.
*/
async pread(
_query: string,
@@ -72,16 +82,12 @@ export class AbstractDatabaseClient extends PuterClient {
throw new Error('DatabaseClient.pread() not implemented');
}
/**
* Execute a write query (INSERT / UPDATE / DELETE).
*/
/** Execute a write query (INSERT / UPDATE / DELETE). */
async write(_query: string, _params: unknown[] = []): Promise<WriteResult> {
throw new Error('DatabaseClient.write() not implemented');
}
/**
* Execute multiple write statements in a single transaction.
*/
/** Execute multiple write statements in a single transaction. */
async batchWrite(_entries: BatchEntry[]): Promise<void> {
throw new Error('DatabaseClient.batchWrite() not implemented');
}
@@ -101,56 +107,152 @@ export class AbstractDatabaseClient extends PuterClient {
const cols = Object.keys(data);
const values = Object.values(data);
const sql =
`INSERT INTO \`${tableName}\` ` +
`(${cols.map((c) => `\`${c}\``).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`;
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
}
/** Whether `read()` goes to a separate replica rather than the primary. */
protected hasReadReplica(): boolean {
return false;
}
/**
* Like `read()` but falls back to the primary when read-replicas are
* in use. Subclasses may override with replica-aware logic; the
* default delegates to `pread()`.
* The replica leg of `tryHardRead`. Override when `read()` falls back to
* the primary by itself, or a replica failure reads the primary twice.
*/
protected replicaRead(
query: string,
params: unknown[],
): Promise<Record<string, unknown>[]> {
return this.read(query, params);
}
/**
* Like `read()`, but an empty or failed replica read falls back to the
* primary, so replication lag can't hide a row that was just written. The
* primary read starts alongside the replica one rather than after it.
* Without a replica this is a single `read()`.
*/
@Span('db.tryHardRead', (query: string) => ({ 'db.statement': query }))
async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const primary = this.pread(query, params);
primary.catch(() => {});
if (!this.hasReadReplica()) return this.read(query, params);
const primary = settle(this.pread(query, params));
try {
const rows = await this.read(query, params);
if (rows.length > 0) {
return rows;
}
const rows = await this.replicaRead(query, params);
if (rows.length > 0) return rows;
} catch {
// replica failed — fall through to primary
}
return primary;
const result = await primary;
if ('error' in result) throw result.error;
return result.value;
}
/**
* Like `tryHardRead()` but throws when the result set is empty.
* Apply the `.sql` files named `<engineName>*` in each configured migration
* directory, in filename order. Files must be idempotent: nothing records
* which ones already ran. A failure aborts startup. `transactional` wraps
* each file in BEGIN/COMMIT, for engines whose DDL can roll back.
*/
async requireRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const rows = await this.tryHardRead(query, params);
if (rows.length === 0) {
throw new Error(`required read returned no rows: ${query}`);
protected async applyMigrationPaths(
connect: () => Promise<MigrationConnection>,
splitStatements: (contents: string) => string[],
{ transactional }: { transactional: boolean },
): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const tag = `[${this.engineName}]`;
const conn = await connect();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) =>
f.endsWith('.sql') &&
f.startsWith(this.engineName),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`${tag} migration path is unreadable: ${dir}`,
{
cause: e,
},
);
}
if (files.length === 0) {
console.log(`${tag} no migrations in ${dir}`);
continue;
}
console.log(
`${tag} running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const contents = readFileSync(
resolvePath(dir, file),
'utf8',
);
const statements = splitStatements(contents);
if (transactional) await conn.query('BEGIN');
try {
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`${tag} failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
if (transactional) await conn.query('COMMIT');
} catch (e) {
if (transactional) {
try {
await conn.query('ROLLBACK');
} catch {
// the original failure is the one worth surfacing
}
}
throw e;
}
console.log(
`${tag} applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
return rows;
}
/**
* Return the value from `choices` that matches the current engine.
*
* Usage:
* ```
* db.case({ sqlite: "datetime('now')", mysql: 'NOW()', otherwise: 'NOW()' })
* ```
*
* db.case({
* sqlite: "datetime('now')",
* mysql: 'NOW()',
* otherwise: 'NOW()',
* });
*
* If the engine name isn't present in `choices`, falls back to
* `choices.otherwise`.
@@ -161,4 +263,124 @@ export class AbstractDatabaseClient extends PuterClient {
}
return choices.otherwise as T;
}
quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `\`${part.replaceAll('`', '``')}\``;
})
.join('.');
}
booleanLiteral(value: boolean): string {
return value ? '1' : '0';
}
booleanValue(value: boolean): boolean | 0 | 1 {
return value ? 1 : 0;
}
insertIgnoreInto(tableName: string): string {
const table = this.quoteIdentifier(tableName);
return this.case({
sqlite: `INSERT OR IGNORE INTO ${table}`,
postgres: `INSERT INTO ${table}`,
otherwise: `INSERT IGNORE INTO ${table}`,
});
}
insertIgnoreSuffix(): string {
return this.case({
postgres: ' ON CONFLICT DO NOTHING',
otherwise: '',
});
}
upsertClause(
conflictColumns: readonly string[],
updateColumns: readonly string[],
): string {
if (updateColumns.length === 0) {
throw new Error('upsertClause requires at least one update column');
}
const updateList = updateColumns
.map((column) => `${this.quoteIdentifier(column)} = ?`)
.join(', ');
return this.case({
mysql: `ON DUPLICATE KEY UPDATE ${updateList}`,
otherwise: `ON CONFLICT(${conflictColumns
.map((column) => this.quoteIdentifier(column))
.join(', ')}) DO UPDATE SET ${updateList}`,
});
}
jsonTextExtract(jsonExpression: string, path: SqlJsonPath): string {
const sqlitePath = `$${path.map((part) => `.${part}`).join('')}`;
return this.case({
sqlite: `json_extract(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)})`,
mysql: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
postgres: `${jsonExpression} #>> ARRAY[${path
.map((part) => this.sqlStringLiteral(part))
.join(', ')}]`,
otherwise: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
});
}
nullCoalesce(...expressions: readonly string[]): string {
if (expressions.length === 0) {
throw new Error('nullCoalesce requires at least one expression');
}
return `COALESCE(${expressions.join(', ')})`;
}
returningIdClause(): string {
return this.case({
postgres: ' RETURNING id',
otherwise: '',
});
}
/**
* A timestamp column value from one bound parameter in unix seconds, stored
* the way `CURRENT_TIMESTAMP` would store that instant.
*/
timestampFromUnixSql(): string {
return this.case({
sqlite: "datetime(?, 'unixepoch')",
postgres: 'to_timestamp(?)',
otherwise: 'FROM_UNIXTIME(?)',
});
}
/**
* What a read of a timestamp column holding `seconds` returns: sqlite gives
* back the UTC text it stored, mysql2 and pg a `Date` read in node's time
* zone (so this matches when node and the database session share one).
*/
timestampValue(seconds: number): Date | string {
const date = new Date(seconds * 1000);
return this.case<Date | string>({
sqlite: date.toISOString().slice(0, 19).replace('T', ' '),
otherwise: date,
});
}
protected sqlStringLiteral(value: string): string {
return `'${value.replaceAll("'", "''")}'`;
}
}
/** Await later without an unhandled rejection in the meantime. */
const settle = async <T>(
promise: Promise<T>,
): Promise<{ value: T } | { error: unknown }> => {
try {
return { value: await promise };
} catch (error) {
return { error };
}
};
@@ -0,0 +1,756 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import {
MySQLDatabaseClient,
compareMigrationFilenames,
} from './MySQLDatabaseClient.js';
interface FakeConnection {
query: ReturnType<typeof vi.fn>;
execute: ReturnType<typeof vi.fn>;
beginTransaction: ReturnType<typeof vi.fn>;
commit: ReturnType<typeof vi.fn>;
rollback: ReturnType<typeof vi.fn>;
release: ReturnType<typeof vi.fn>;
}
interface FakePool {
poolConfig: Record<string, unknown>;
connectionHandlers: ((conn: unknown) => void)[];
connections: FakeConnection[];
calls: { sql: string; values?: unknown[] }[];
ended: boolean;
endError: unknown;
endThrows: unknown;
respond: (sql: string, values?: unknown[]) => unknown;
on: (event: string, handler: (conn: unknown) => void) => FakePool;
promise: () => { getConnection: () => Promise<FakeConnection> };
end: (cb: (err?: unknown) => void) => void;
}
// mysql2 is the wire boundary: stub the driver's pool so we can assert on
// the exact SQL, parameters and connection choreography the adapter emits.
const { createPoolMock, createdPools } = vi.hoisted(() => {
const createdPools: FakePool[] = [];
const createPoolMock = vi.fn((poolConfig: Record<string, unknown>) => {
const pool: FakePool = {
poolConfig,
connectionHandlers: [],
connections: [],
calls: [],
ended: false,
endError: null,
endThrows: null,
respond: () => [[], undefined],
on(event, handler) {
if (event === 'connection') {
pool.connectionHandlers.push(handler);
}
return pool;
},
promise() {
return {
getConnection: async () => {
const run = async (sql: string, values?: unknown[]) => {
pool.calls.push({ sql, values });
return pool.respond(sql, values);
};
const conn: FakeConnection = {
query: vi.fn(run),
execute: vi.fn(run),
beginTransaction: vi.fn(async () => {}),
commit: vi.fn(async () => {}),
rollback: vi.fn(async () => {}),
release: vi.fn(),
};
pool.connections.push(conn);
return conn;
},
};
},
end(cb) {
if (pool.endThrows) throw pool.endThrows;
pool.ended = true;
cb(pool.endError ?? undefined);
},
};
createdPools.push(pool);
return pool;
});
return { createPoolMock, createdPools };
});
vi.mock('mysql2', () => ({ createPool: createPoolMock }));
describe('compareMigrationFilenames', () => {
it('orders numbered migrations numerically, not lexically', () => {
const input = [
'mysql_mig_10.sql',
'mysql_mig_2.sql',
'mysql_mig_1.sql',
'mysql_mig_9.sql',
'mysql_mig_3.sql',
];
const sorted = [...input].sort(compareMigrationFilenames);
expect(sorted).toEqual([
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_3.sql',
'mysql_mig_9.sql',
'mysql_mig_10.sql',
]);
});
it('keeps mig_10 after mig_9 when the real prod set is shuffled', () => {
// Reflects the current migrations/mysql/ listing — guards against
// a future "let's just rename to padded" suggestion accidentally
// re-introducing the lex bug if the rename is incomplete.
const real = [
'mysql_mig_9.sql',
'mysql_mig_3.sql',
'mysql_mig_10.sql',
'mysql_mig_1.sql',
'mysql_mig_7.sql',
'mysql_mig_5.sql',
'mysql_mig_2.sql',
'mysql_mig_4.sql',
'mysql_mig_8.sql',
'mysql_mig_6.sql',
];
const sorted = [...real].sort(compareMigrationFilenames);
for (let i = 1; i <= sorted.length; i += 1) {
expect(sorted[i - 1]).toBe(`mysql_mig_${i}.sql`);
}
});
it('sorts non-numeric filenames after numbered ones, lexically among themselves', () => {
// Numbered files always run first (they're the canonical history);
// unmatched names follow in localeCompare order. Mixing the two
// sets prevents a vendor dump from accidentally wedging itself
// between mig_4 and mig_5 if it happened to lex-sort there.
const mixed = [
'mysql_mig_2.sql',
'mysql_vendor_dump.sql',
'mysql_mig_10.sql',
'mysql_bootstrap.sql',
'mysql_mig_1.sql',
];
const sorted = [...mixed].sort(compareMigrationFilenames);
expect(sorted).toEqual([
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_10.sql',
'mysql_bootstrap.sql',
'mysql_vendor_dump.sql',
]);
});
it('is stable for already-sorted input', () => {
const sorted = [
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_10.sql',
'mysql_mig_11.sql',
];
expect([...sorted].sort(compareMigrationFilenames)).toEqual(sorted);
});
});
// ── Replica read failover ───────────────────────────────────────────
//
// `read()` normally goes to the replica batcher; when the replica side is
// degraded (batcher load-shed or a transient connection error) and a real
// replica is configured, the read retries once on the primary read batcher.
// The write batcher is left unset, so a failover that reached it would throw.
type Batcher = { execute: ReturnType<typeof vi.fn> };
const makeClient = (opts: {
replica: Batcher;
primary: Batcher;
multiNode?: boolean;
}) => {
const client = new MySQLDatabaseClient({
database: { engine: 'mysql' },
} as IConfig);
// Bypass onServerStart (which would connect to a real database) and
// inject the batchers directly. Configuration enum: SINGLE=0, REPLICA=1.
Object.assign(client as unknown as Record<string, unknown>, {
dbReplica: opts.replica,
dbPrimaryRead: opts.primary,
configuration: opts.multiNode === false ? 0 : 1,
});
return client;
};
const codedError = (code: string) => {
const err = new Error(code) as Error & { code: string };
err.code = code;
return err;
};
describe('MySQLDatabaseClient.read — replica failover', () => {
it('fails over to the primary on batcher load-shed errors', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
};
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
const client = makeClient({ replica, primary });
await expect(client.read('SELECT 1')).resolves.toEqual([{ ok: 1 }]);
expect(primary.execute).toHaveBeenCalledTimes(1);
});
it('fails over on transient connection errors', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('ECONNRESET')),
};
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
const client = makeClient({ replica, primary });
await expect(client.read('SELECT 1')).resolves.toEqual([{ ok: 1 }]);
});
it('rethrows deterministic SQL errors without touching the primary', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('ER_PARSE_ERROR')),
};
const primary = { execute: vi.fn() };
const client = makeClient({ replica, primary });
await expect(client.read('SELEC oops')).rejects.toMatchObject({
code: 'ER_PARSE_ERROR',
});
expect(primary.execute).not.toHaveBeenCalled();
});
it('does not fail over in single-node configuration', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
};
const primary = { execute: vi.fn() };
const client = makeClient({ replica, primary, multiNode: false });
await expect(client.read('SELECT 1')).rejects.toMatchObject({
code: 'dbBatchFailed',
});
expect(primary.execute).not.toHaveBeenCalled();
});
});
// -- Driver-level behaviour (stubbed mysql2 pool) ---------------------
const mysqlConfig = (
database: Partial<NonNullable<IConfig['database']>> = {},
): IConfig =>
({
port: 0,
extensions: [],
database: { engine: 'mysql', ...database },
}) as IConfig;
const startClient = async (
database: Partial<NonNullable<IConfig['database']>> = {},
): Promise<MySQLDatabaseClient> => {
const client = new MySQLDatabaseClient(mysqlConfig(database));
await client.onServerStart();
return client;
};
// The batcher coalesces a batch into `stmt1;stmt2; SELECT 1`, so the driver
// must answer with one row-set per statement plus one for the SELECT 1.
const rowSets = (...sets: unknown[][]) => [[...sets, [{ 1: 1 }]], undefined];
// Writes go out one statement at a time, so the driver answers with just the
// result header.
const writeHeader = (header: Record<string, unknown>) => [header, undefined];
beforeEach(() => {
createdPools.length = 0;
createPoolMock.mockClear();
});
describe('MySQLDatabaseClient — pool construction', () => {
it('builds the primary pool from config with multi-statement support', async () => {
await startClient({
host: 'db.internal',
port: 3307,
user: 'puter',
password: 'hunter2',
database: 'puterdb',
});
expect(createPoolMock).toHaveBeenCalledTimes(1);
expect(createdPools[0].poolConfig).toEqual({
maxPreparedStatements: 900,
connectionLimit: 30,
maxIdle: 29,
idleTimeout: 60_000,
gracefulEnd: true,
enableKeepAlive: true,
host: 'db.internal',
port: 3307,
user: 'puter',
password: 'hunter2',
database: 'puterdb',
multipleStatements: true,
});
});
// mysql2 skips its idle reaper unless maxIdle < connectionLimit, and an
// unreaped connection is closed by the server under the pool's feet.
it('keeps the idle reaper on when a pool sets its own connection limit', async () => {
await startClient({
replica: { host: 'replica.internal', connectionLimit: 10 },
});
expect(createdPools[1].poolConfig).toMatchObject({
connectionLimit: 10,
maxIdle: 9,
idleTimeout: 60_000,
});
});
// mysql2 reads 0 as unlimited; a maxIdle of -1 would crash its reaper.
it('leaves maxIdle to mysql2 for an unlimited pool', async () => {
await startClient({
replica: { host: 'replica.internal', connectionLimit: 0 },
});
expect(createdPools[1].poolConfig).toMatchObject({
connectionLimit: 0,
});
expect(createdPools[1].poolConfig).not.toHaveProperty('maxIdle');
});
it('falls back to loopback defaults when the endpoint is unspecified', async () => {
await startClient();
expect(createdPools[0].poolConfig).toMatchObject({
host: '127.0.0.1',
port: 3306,
user: 'root',
password: '',
database: 'puter',
});
});
it('arms a server-side statement timeout on every new connection', async () => {
await startClient({ selectTimeoutMs: 12_000 });
const pool = createdPools[0];
expect(pool.connectionHandlers).toHaveLength(1);
const conn = { query: vi.fn() };
pool.connectionHandlers[0](conn);
expect(conn.query).toHaveBeenCalledWith(
'SET SESSION max_execution_time = 12000',
);
});
it('omits the statement timeout when configured to 0', async () => {
await startClient({ selectTimeoutMs: 0 });
expect(createdPools[0].connectionHandlers).toHaveLength(0);
});
it('shares one pool between reads and writes without a replica', async () => {
await startClient();
expect(createPoolMock).toHaveBeenCalledTimes(1);
});
it('creates a second pool when a read-replica is configured', async () => {
await startClient({
replica: { host: 'replica.internal', port: 3306 },
});
expect(createPoolMock).toHaveBeenCalledTimes(2);
expect(createdPools[1].poolConfig).toMatchObject({
host: 'replica.internal',
multipleStatements: true,
});
});
});
describe('MySQLDatabaseClient — query interface', () => {
it('sends the query and its parameters through to the driver', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => rowSets([{ id: 1, username: 'ada' }]);
await expect(
client.read('SELECT * FROM `user` WHERE `id` = ?', [1]),
).resolves.toEqual([{ id: 1, username: 'ada' }]);
expect(pool.calls).toEqual([
{
sql: 'SELECT * FROM `user` WHERE `id` = ?; SELECT 1',
values: [1],
},
]);
});
it('returns an empty array when the driver yields no row-set', async () => {
const client = await startClient();
createdPools[0].respond = () => [[], undefined];
await expect(client.read('SELECT 1')).resolves.toEqual([]);
await expect(client.pread('SELECT 1')).resolves.toEqual([]);
});
it('reads from the replica pool and primary-reads from the primary', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => rowSets([{ from: 'replica' }]);
await expect(client.read('SELECT 1')).resolves.toEqual([
{ from: 'replica' },
]);
await expect(client.pread('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
});
it('sends a write on its own, outside a transaction', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => writeHeader({ affectedRows: 1 });
await client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
'ada',
1,
]);
expect(pool.calls).toEqual([
{
sql: 'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]);
expect(pool.connections[0].beginTransaction).not.toHaveBeenCalled();
expect(pool.connections[0].commit).not.toHaveBeenCalled();
});
it('maps the mysql result header onto the shared write result', async () => {
const client = await startClient();
createdPools[0].respond = () =>
writeHeader({ insertId: 42, affectedRows: 2 });
await expect(
client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
'ada',
1,
]),
).resolves.toEqual({
insertId: 42,
affectedRows: 2,
anyRowsAffected: true,
});
});
it('reports no rows affected when the header omits the counters', async () => {
const client = await startClient();
createdPools[0].respond = () => writeHeader({});
await expect(client.write('DELETE FROM `user`')).resolves.toEqual({
insertId: 0,
affectedRows: 0,
anyRowsAffected: false,
});
});
it('builds an INSERT with quoted identifiers and positional params', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => writeHeader({ insertId: 7, affectedRows: 1 });
await expect(
client.insert('user', { username: 'ada', email: null }),
).resolves.toMatchObject({ insertId: 7 });
expect(pool.calls[0]).toEqual({
sql: 'INSERT INTO `user` (`username`, `email`) VALUES (?, ?)',
values: ['ada', null],
});
});
});
describe('MySQLDatabaseClient — batchWrite transactions', () => {
it('runs every statement on one connection inside a transaction', async () => {
const client = await startClient();
const pool = createdPools[0];
await client.batchWrite([
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
{
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
values: ['s'],
},
]);
expect(pool.connections).toHaveLength(1);
const conn = pool.connections[0];
expect(conn.beginTransaction).toHaveBeenCalledTimes(1);
expect(conn.execute.mock.calls).toEqual([
['UPDATE `user` SET `x` = ?', [1]],
['DELETE FROM `sessions` WHERE `uuid` = ?', ['s']],
]);
expect(conn.commit).toHaveBeenCalledTimes(1);
expect(conn.rollback).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('rolls back and releases the connection when a statement fails', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = (sql) => {
if (sql.startsWith('DELETE')) throw new Error('constraint blew up');
return [[], undefined];
};
await expect(
client.batchWrite([
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
{ statement: 'DELETE FROM `user`', values: [] },
]),
).rejects.toThrow('constraint blew up');
const conn = pool.connections[0];
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.rollback).toHaveBeenCalledTimes(1);
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('still rethrows the original failure when the rollback also fails', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => {
throw new Error('statement blew up');
};
const original = client.batchWrite([
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
]);
// The rollback is attached after the connection is handed out, so
// patch it once the adapter has acquired one.
await Promise.resolve();
pool.connections[0]?.rollback.mockRejectedValue(
new Error('rollback blew up'),
);
await expect(original).rejects.toThrow('statement blew up');
expect(pool.connections[0].release).toHaveBeenCalledTimes(1);
});
it('never acquires a connection for an empty batch', async () => {
const client = await startClient();
await client.batchWrite([]);
expect(createdPools[0].connections).toHaveLength(0);
});
});
describe('MySQLDatabaseClient — tryHardRead', () => {
it('issues exactly one query in single-node configuration', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => rowSets([{ id: 1 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 1 },
]);
expect(pool.calls).toHaveLength(1);
});
it('prefers replica rows when the replica has them', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => rowSets([{ from: 'replica' }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'replica' },
]);
});
it('falls back to the primary when the replica is behind', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => rowSets([]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
});
it('falls back to the primary when the replica query throws', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => {
throw new Error('replica down');
};
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
});
it('reads the primary once when the replica fails in a way read() fails over on', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
};
const primary = {
execute: vi.fn().mockResolvedValue([[{ from: 'primary' }]]),
};
const client = makeClient({ replica, primary });
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
expect(primary.execute).toHaveBeenCalledTimes(1);
});
});
describe('MySQLDatabaseClient — migrations', () => {
let dir: string;
beforeEach(() => {
dir = mkdtempSync(join(tmpdir(), 'puter-mysql-mig-'));
});
afterEach(() => {
rmSync(dir, { recursive: true, force: true });
});
it('applies mysql migrations in numeric order, one statement at a time', async () => {
writeFileSync(
join(dir, 'mysql_mig_2.sql'),
'CREATE TABLE b (id INT);\nCREATE TABLE c (id INT);\n',
);
writeFileSync(
join(dir, 'mysql_mig_10.sql'),
'CREATE TABLE d (id INT);',
);
// Other engines' files and non-SQL noise must be ignored.
writeFileSync(join(dir, 'postgres_mig_1.sql'), 'CREATE TABLE nope ();');
writeFileSync(join(dir, 'mysql_notes.txt'), 'not sql');
await startClient({ migrationPaths: [dir] });
const pool = createdPools[0];
expect(pool.calls.map((c) => c.sql)).toEqual([
'CREATE TABLE b (id INT)',
'CREATE TABLE c (id INT)',
'CREATE TABLE d (id INT)',
]);
expect(pool.connections[0].release).toHaveBeenCalledTimes(1);
});
it('reports which file and statement index failed', async () => {
writeFileSync(
join(dir, 'mysql_mig_1.sql'),
'CREATE TABLE ok (id INT);\nCREATE TABLE bad (id INT);',
);
const client = new MySQLDatabaseClient(
mysqlConfig({ migrationPaths: [dir] }),
);
const cause = new Error('ER_PARSE_ERROR');
// The pool only exists once onServerStart runs, so arm the failure
// through the factory.
createPoolMock.mockImplementationOnce((poolConfig) => {
const pool = createPoolMock.getMockImplementation()!(poolConfig);
pool.respond = (sql: string) => {
if (sql.includes('bad')) throw cause;
return [[], undefined];
};
return pool;
});
await expect(client.onServerStart()).rejects.toThrow(
'[mysql] failed to apply mysql_mig_1.sql at statement 1',
);
});
it('fails loudly when a configured migration path is unreadable', async () => {
await expect(
startClient({ migrationPaths: [join(dir, 'does-not-exist')] }),
).rejects.toThrow('[mysql] migration path is unreadable');
});
it('skips a configured directory that holds no mysql migrations', async () => {
writeFileSync(join(dir, 'readme.md'), '# nothing here');
await startClient({ migrationPaths: [dir] });
expect(createdPools[0].calls).toEqual([]);
});
});
describe('MySQLDatabaseClient — pool lifecycle', () => {
it('closes both pools on shutdown when a replica is configured', async () => {
const client = await startClient({ replica: { host: 'replica' } });
await client.onServerShutdown();
expect(createdPools[0].ended).toBe(true);
expect(createdPools[1].ended).toBe(true);
});
it('leaves the pools open through prepare-shutdown; only onServerShutdown closes them', async () => {
vi.useFakeTimers();
try {
const client = await startClient();
await client.onServerPrepareShutdown();
expect(createdPools[0].ended).toBe(false);
// No timer closes the pools on its own — the layers above are
// still draining through them for as long as the process drains.
await vi.advanceTimersByTimeAsync(60_000);
expect(createdPools[0].ended).toBe(false);
await client.onServerShutdown();
expect(createdPools[0].ended).toBe(true);
} finally {
vi.useRealTimers();
}
});
it('surfaces a driver error raised while closing the pool', async () => {
const client = await startClient();
createdPools[0].endError = new Error('pool refused to close');
await expect(client.onServerShutdown()).rejects.toThrow(
'pool refused to close',
);
});
it('surfaces a driver error thrown synchronously by end()', async () => {
const client = await startClient();
createdPools[0].endThrows = new Error('end exploded');
await expect(client.onServerShutdown()).rejects.toThrow('end exploded');
});
});
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,32 +17,27 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { createPool, type Pool } from 'mysql2';
import { metrics } from '@opentelemetry/api';
import { createPool, ExecuteValues, type Pool } from 'mysql2';
import { Span } from '../../util/span.js';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { SQLBatcher } from './SQLBatcher.js';
import { isRetriableError } from './retriableErrors.js';
import { splitMysqlStatements } from './splitMysqlStatements.js';
import type { IConfig } from '../../types';
const RETRIABLE_ERROR_CODES = new Set([
'PROTOCOL_CONNECTION_LOST',
'PROTOCOL_SEQUENCE_TIMEOUT',
'PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR',
'ECONNRESET',
'ETIMEDOUT',
'EPIPE',
'ECONNREFUSED',
'EHOSTUNREACH',
'ENETUNREACH',
'EAI_AGAIN',
]);
const DEFAULT_SELECT_TIMEOUT_MS = 30_000;
const DEFAULT_CONNECTION_LIMIT = 30;
const IDLE_TIMEOUT_MS = 60_000;
const RETRIABLE_ERROR_MESSAGES = [
'Connection lost',
'read ECONNRESET',
'ETIMEDOUT',
];
const replicaFailoverCounter = metrics
.getMeter('puter-backend')
.createCounter('db.read.replica_failover', {
description:
'Reads that failed on the replica batcher and were retried on the primary',
});
export { compareMigrationFilenames } from './migrationFilenames.js';
type PoolConfig = Parameters<typeof createPool>[0];
@@ -58,9 +53,10 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
private replicaPool!: Pool;
private db!: SQLBatcher;
private dbReplica!: SQLBatcher;
/** Primary pool, SELECT-only: same rows as `db`, coalesced like the replica. */
private dbPrimaryRead!: SQLBatcher;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(config: IConfig) {
super(config);
@@ -82,7 +78,8 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
});
console.log('[mysql] connected to primary');
this.db = new SQLBatcher(this.primaryPool, 30, 5);
this.db = this.createPrimaryBatcher(this.primaryPool);
this.dbPrimaryRead = this.createPrimaryReadBatcher(this.primaryPool);
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
@@ -93,38 +90,25 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
this.configuration = Configuration.SINGLE;
}
this.dbReplica = new SQLBatcher(this.replicaPool, 10, 5);
this.dbReplica = this.createReplicaBatcher(this.replicaPool);
await this.runMigrations();
await this.applyMigrationPaths(
() => this.primaryPool.promise().getConnection(),
splitMysqlStatements,
{ transactional: false },
);
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
// Allow in-flight queries to drain before closing pools
const drainMs = 60_000;
console.log(
`[mysql] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools('drain').catch((e) =>
console.error('[mysql] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
// Pools stay open; onServerShutdown closes them after the layers
// above have drained.
console.log('[mysql] entering drain mode');
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools('shutdown');
}
@@ -132,24 +116,46 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// Query interface
// ------------------------------------------------------------------
// The db.* spans measure the logical query, including time queued in
// the SQLBatcher — the mysql2 auto-instrumentation only sees the
// coalesced multi-statement flush, so per-query latency lives here.
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.dbReplica.execute(query, params);
let result;
try {
result = await this.dbReplica.execute(query, params);
} catch (error) {
// Replica-side degradation (batcher load-shed or a transient
// connection failure) shouldn't fail reads while the primary is
// healthy. Deterministic errors (bad SQL) are rethrown — they
// would fail identically on the primary.
if (
this.configuration !== Configuration.REPLICA ||
!MySQLDatabaseClient.isFailoverWorthy(error)
) {
throw error;
}
replicaFailoverCounter.add(1);
result = await this.dbPrimaryRead.execute(query, params);
}
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
@Span('db.pread', (query: string) => ({ 'db.statement': query }))
override async pread(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.db.execute(query, params);
const result = await this.dbPrimaryRead.execute(query, params);
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
@@ -167,6 +173,9 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
};
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(
entries: { statement: string; values: unknown[] }[],
): Promise<void> {
@@ -180,7 +189,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
await conn.beginTransaction();
try {
for (const { statement, values } of entries) {
await conn.execute(statement, values);
await conn.execute(statement, values as ExecuteValues);
}
await conn.commit();
} catch (err) {
@@ -192,100 +201,18 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
}
}
override async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
if (this.configuration === Configuration.SINGLE) {
return this.read(query, params);
}
// Run both reads in parallel — prefer replica when it returns rows,
// otherwise fall back to primary to handle replication lag.
const primaryPromise = this.db.execute(query, params);
try {
const replicaResult = await this.dbReplica.execute(query, params);
if (
Array.isArray(replicaResult?.[0]) &&
(replicaResult[0] as unknown[]).length > 0
) {
primaryPromise.catch(() => {}); // suppress unhandled rejection
return replicaResult[0] as Record<string, unknown>[];
}
} catch {
// fall through to primary
}
const primaryResult = await primaryPromise;
return (primaryResult?.[0] as Record<string, unknown>[]) ?? [];
protected override hasReadReplica(): boolean {
return this.configuration === Configuration.REPLICA;
}
// ------------------------------------------------------------------
// Migrations
// ------------------------------------------------------------------
/**
* Apply `.sql` files from each configured migration directory in order.
* Files within a directory are sorted lexically. Files MUST be
* idempotent — there is no per-file applied-state tracking. Failures
* abort startup so operators see schema problems loud.
*/
private async runMigrations(): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const conn = await this.primaryPool.promise().getConnection();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) => f.endsWith('.sql') && f.startsWith('mysql'),
)
.sort();
} catch (e) {
throw new Error(
`[mysql] migration path is unreadable: ${dir}`,
{ cause: e },
);
}
if (files.length === 0) {
console.log(`[mysql] no migrations in ${dir}`);
continue;
}
console.log(
`[mysql] running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const filePath = resolvePath(dir, file);
const contents = readFileSync(filePath, 'utf8');
const statements = splitMysqlStatements(contents);
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`[mysql] failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
console.log(
`[mysql] applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
// No failover: `tryHardRead` already has a primary read in flight.
protected override async replicaRead(
query: string,
params: unknown[],
): Promise<Record<string, unknown>[]> {
const result = await this.dbReplica.execute(query, params);
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
// ------------------------------------------------------------------
@@ -293,112 +220,87 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// ------------------------------------------------------------------
private createPool(poolConfig: PoolConfig): Pool {
return createPool({
const connectionLimit =
poolConfig.connectionLimit ?? DEFAULT_CONNECTION_LIMIT;
const pool = createPool({
maxPreparedStatements: 900,
connectionLimit: 30,
connectionLimit,
// mysql2 only reaps idle connections when maxIdle < connectionLimit.
// Unreaped, they sit until the server's wait_timeout closes them,
// and the next statement sent on one fails. 0 means unlimited,
// and a negative maxIdle would crash the reaper.
...(connectionLimit > 1 && { maxIdle: connectionLimit - 1 }),
idleTimeout: IDLE_TIMEOUT_MS,
// Reaped connections say goodbye instead of dropping the socket.
gracefulEnd: true,
enableKeepAlive: true,
...poolConfig,
multipleStatements: true,
} as PoolConfig);
}
/** Reinitialize the primary pool (e.g. after a health-check failure). */
reinitPrimary(): void {
if (this.shutdownStarted) return;
const dbConf = this.config.database!;
const previous = this.primaryPool;
this.primaryPool = this.createPool({
host: dbConf.host ?? '127.0.0.1',
port: dbConf.port ?? 3306,
user: dbConf.user ?? 'root',
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
});
this.db = new SQLBatcher(this.primaryPool, 30, 5);
if (this.configuration === Configuration.SINGLE) {
this.replicaPool = this.primaryPool;
this.dbReplica = new SQLBatcher(this.primaryPool, 10, 5);
}
if (previous && previous !== this.primaryPool) {
this.closePool(previous, 'reinit:primary').catch(() => {});
}
}
/** Reinitialize the replica pool. */
reinitReplica(): void {
if (this.shutdownStarted || !this.config.database?.replica) return;
const previous = this.replicaPool;
this.replicaPool = this.createPool(this.config.database.replica);
this.dbReplica = new SQLBatcher(this.replicaPool, 10, 5);
if (
previous &&
previous !== this.replicaPool &&
previous !== this.primaryPool
) {
this.closePool(previous, 'reinit:replica').catch(() => {});
}
}
// ------------------------------------------------------------------
// Retry helpers (for health checks or resilient reads)
// ------------------------------------------------------------------
static isRetriableError(error: unknown): boolean {
const code = (error as { code?: string })?.code;
if (code && RETRIABLE_ERROR_CODES.has(code)) return true;
const msg = String((error as Error)?.message ?? '');
return RETRIABLE_ERROR_MESSAGES.some((m) => msg.includes(m));
}
async readWithRetry(
label: string,
operation: () => Promise<unknown[]>,
opts?: {
maxAttempts?: number;
baseBackoffMs?: number;
maxBackoffMs?: number;
jitterRatio?: number;
},
): Promise<unknown[]> {
const maxAttempts = opts?.maxAttempts ?? 3;
const baseBackoffMs = opts?.baseBackoffMs ?? 100;
const maxBackoffMs = opts?.maxBackoffMs ?? 500;
const jitterRatio = opts?.jitterRatio ?? 0.2;
let attempt = 1;
while (true) {
try {
return await operation();
} catch (error) {
if (this.shutdownStarted) throw error;
if (
attempt >= maxAttempts ||
!MySQLDatabaseClient.isRetriableError(error)
)
throw error;
const raw = baseBackoffMs * 2 ** (attempt - 1);
const capped = Math.min(maxBackoffMs, raw);
const window = Math.round(capped * jitterRatio);
const jitter =
window === 0
? 0
: Math.floor(Math.random() * (window * 2 + 1)) - window;
const delay = Math.max(0, capped + jitter);
console.warn(
`[${label}] transient mysql error (${(error as { code?: string })?.code ?? 'unknown'}); retry ${attempt + 1}/${maxAttempts} in ${delay}ms`,
// Server-side kill switch for runaway reads: MySQL applies
// max_execution_time to SELECT statements only, so this is
// write-safe. Without it, a stalled database turns reads into
// indefinite hangs that no client-side timeout ever converts
// into a failure. 0 disables.
const selectTimeoutMs = Math.floor(
Number(
this.config.database?.selectTimeoutMs ??
DEFAULT_SELECT_TIMEOUT_MS,
),
);
if (selectTimeoutMs > 0) {
pool.on('connection', (conn) => {
conn.query(
`SET SESSION max_execution_time = ${selectTimeoutMs}`,
);
await new Promise((r) => setTimeout(r, delay));
attempt++;
}
});
}
return pool;
}
/**
* One write per flush. Coalesced writes share a transaction, so unrelated
* callers hold each other's row locks until COMMIT and deadlock; a lone
* statement needs no BEGIN/COMMIT round trips.
*/
private createPrimaryBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxBatchSize: 1,
poolLabel: 'primary',
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
/** Reads that must see the primary, still coalesced. */
private createPrimaryReadBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 30,
maxBatchSize: 5,
poolLabel: 'primary',
readOnly: true,
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
private createReplicaBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 10,
maxBatchSize: 5,
poolLabel: 'replica',
readOnly: true,
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
/**
* Replica failures worth retrying on the primary: batcher load-shed or
* transient connection errors — never deterministic SQL errors.
*/
private static isFailoverWorthy(error: unknown): boolean {
const code = (error as { code?: string })?.code;
return code === 'dbBatchFailed' || isRetriableError(error);
}
// ------------------------------------------------------------------
@@ -0,0 +1,328 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Pool } from 'pg';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { IConfig } from '../../types';
import type { PuterServer } from '../../server';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import {
createPgMockPostgresDatabaseClient,
POSTGRES_TEST_MIGRATIONS_PATH,
setupTestServer,
} from '../../testUtil.js';
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
const postgresUrl = process.env.PUTER_TEST_POSTGRES_URL;
const postgresMigrationsPath = POSTGRES_TEST_MIGRATIONS_PATH;
const postgresTestSchemaPattern = /^puter_test_[a-f0-9]{32}$/u;
const postgresIntegrationTimeoutMs = 180_000;
let postgresTestSchema: string | undefined;
let postgresTestUrl: string | undefined;
const postgresConfig = (overrides: Partial<IConfig> = {}): IConfig => {
if (postgresUrl && !postgresTestUrl) {
throw new Error('Postgres test schema was not initialized');
}
const { database: databaseOverrides, ...rootOverrides } = overrides;
const database = postgresUrl
? {
engine: 'postgres' as const,
inMemory: false,
connectionString: postgresTestUrl,
migrationPaths: [postgresMigrationsPath],
}
: {
engine: 'postgres' as const,
inMemory: true,
migrationPaths: [postgresMigrationsPath],
};
return {
port: 0,
extensions: [],
database: { ...database, ...(databaseOverrides ?? {}) },
...rootOverrides,
};
};
const quoteTestSchemaIdentifier = (schema: string): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
return `"${schema}"`;
};
const postgresConnectionStringForSchema = (
connectionString: string,
schema: string,
): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
const url = new URL(connectionString);
url.searchParams.set('options', `-c search_path=${schema}`);
return url.toString();
};
const createPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl) return;
const schema = `puter_test_${uuidv4().replaceAll('-', '')}`;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(`CREATE SCHEMA ${quoteTestSchemaIdentifier(schema)}`);
postgresTestSchema = schema;
postgresTestUrl = postgresConnectionStringForSchema(
postgresUrl,
schema,
);
} finally {
await pool.end();
}
};
const dropPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl || !postgresTestSchema) return;
const schema = postgresTestSchema;
postgresTestSchema = undefined;
postgresTestUrl = undefined;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(
`DROP SCHEMA IF EXISTS ${quoteTestSchemaIdentifier(schema)} CASCADE`,
);
} finally {
await pool.end();
}
};
describe('PostgresDatabaseClient integration', () => {
let server: PuterServer | undefined;
beforeEach(async () => {
await createPostgresTestSchema();
});
afterEach(async () => {
try {
await server?.shutdown();
} finally {
server = undefined;
await dropPostgresTestSchema();
}
});
it(
'applies the native migrations idempotently to an empty database',
async () => {
const config = postgresConfig();
const pgMockClient = postgresUrl
? undefined
: await createPgMockPostgresDatabaseClient(config);
try {
const firstClient =
pgMockClient?.client ?? new PostgresDatabaseClient(config);
try {
await firstClient.onServerStart();
} finally {
await firstClient.onServerShutdown();
}
const secondClient =
pgMockClient?.createClient() ??
new PostgresDatabaseClient(config);
try {
await secondClient.onServerStart();
const [systemUser] = await secondClient.read(
'SELECT `id`, `username` FROM `user` WHERE `username` = ?',
['system'],
);
const [devCenter] = await secondClient.read(
'SELECT `name`, `index_url` FROM `apps` WHERE `name` = ?',
['dev-center'],
);
expect(systemUser).toMatchObject({
id: 1,
username: 'system',
});
expect(devCenter).toMatchObject({
name: 'dev-center',
index_url:
'https://builtins.namespaces.puter.com/dev-center',
});
} finally {
await secondClient.onServerShutdown();
}
} finally {
pgMockClient?.destroy();
}
},
postgresIntegrationTimeoutMs,
);
it('starts the server and exercises user, app, fsentry, permission, and session flows', async () => {
const userStorageAllowance = 123_456_789;
server = await setupTestServer(
postgresConfig({
no_default_user: false,
is_storage_limited: true,
}),
);
const admin = await server.stores.user.getByUsername('admin');
expect(admin?.username).toBe('admin');
const username = `pg-${uuidv4().slice(0, 8)}`;
const createdUser = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: userStorageAllowance,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
createdUser,
);
const user = await server.stores.user.getById(createdUser.id);
if (!user) throw new Error('created user was not readable');
expect(user.username).toBe(username);
await expect(
server.stores.fsEntry.getUserStorageAllowance(user.id),
).resolves.toMatchObject({
max: userStorageAllowance,
});
const otherUsername = `pg-other-${uuidv4().slice(0, 8)}`;
const otherUser = await server.stores.user.create({
username: otherUsername,
uuid: uuidv4(),
password: null,
email: `${otherUsername}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const authResult = await server.services.auth.createSessionToken(
user,
{
ip: '127.0.0.1',
user_agent: 'postgres-integration-test',
},
);
const authenticated =
await server.services.auth.authenticateFromToken(authResult.token);
expect(authenticated?.user?.id).toBe(user?.id);
const devCenter = await server.stores.app.getByName('dev-center');
if (!devCenter) throw new Error('dev-center app was not seeded');
expect(devCenter.name).toBe('dev-center');
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
if (!documents) throw new Error('Documents directory was not created');
expect(documents.isDir).toBe(true);
const folder = await server.stores.fsEntry.createNonFileEntry({
userId: user.id,
parent: documents,
name: 'postgres-folder',
kind: 'directory',
});
const renamedFolder = await server.stores.fsEntry.updateEntry(
folder.uuid,
{
name: 'postgres-folder-renamed',
path: `/${username}/Documents/postgres-folder-renamed`,
},
);
expect(renamedFolder.name).toBe('postgres-folder-renamed');
await server.stores.permission.upsertUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
{ ok: true },
);
await expect(
server.stores.permission.hasUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
),
).resolves.toBe(true);
await server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
);
await expect(
server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
),
).resolves.toBeUndefined();
await expect(
server.stores.oidc.link(
otherUser.id,
'postgres-integration-test',
'subject-1',
null,
),
).rejects.toMatchObject({
statusCode: 409,
legacyCode: 'conflict',
});
const session = await server.stores.session.create(user.id, {
meta: { source: 'postgres-integration-test' },
});
const activeSession = await server.stores.session.getByUuid(
session.uuid,
);
expect(activeSession?.uuid).toBe(session.uuid);
const workerName = `worker-${uuidv4().slice(0, 8)}`;
const workerSession = await server.stores.session.getOrCreateWorker(
user.id,
{ workerName },
);
expect(workerSession?.kind).toBe('worker');
expect(workerSession?.meta?.worker_name).toBe(workerName);
await server.stores.session.removeByUuid(session.uuid);
await expect(
server.stores.session.getByUuid(session.uuid),
).resolves.toBeNull();
}, postgresIntegrationTimeoutMs);
});
@@ -0,0 +1,420 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { FieldDef, QueryResult } from 'pg';
import { describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import { DatabaseClientFactory } from './index.js';
import {
mapPostgresWriteResult,
PostgresDatabaseClient,
type PostgresPool,
type PostgresPoolClient,
} from './PostgresDatabaseClient.js';
// Pass-through spy: the rewrite is pure, so its call count shows the memo.
const { rewriteSpy } = vi.hoisted(() => ({ rewriteSpy: vi.fn() }));
vi.mock('./preparePostgresSql.js', async (importOriginal) => {
const actual =
await importOriginal<typeof import('./preparePostgresSql.js')>();
return {
...actual,
preparePostgresSql: (sql: string) => {
rewriteSpy(sql);
return actual.preparePostgresSql(sql);
},
};
});
type QueryCall = {
text: string;
values?: unknown[];
};
const postgresConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
},
});
const postgresReplicaConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
replica: {},
},
});
const field = (name: string, dataTypeID: number): FieldDef => ({
name,
tableID: 0,
columnID: 0,
dataTypeID,
dataTypeSize: -1,
dataTypeModifier: -1,
format: 'text',
});
const int8Field = (name: string): FieldDef => field(name, 20);
const textField = (name: string): FieldDef => field(name, 25);
const queryResult = (
rows: Record<string, unknown>[] = [],
rowCount = rows.length,
fields: FieldDef[] = [],
): QueryResult<Record<string, unknown>> => ({
command: '',
fields,
oid: 0,
rowCount,
rows,
});
class RecordingPoolClient implements PostgresPoolClient {
readonly calls: QueryCall[] = [];
released = false;
constructor(private readonly failOnText?: string) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (this.failOnText && text.includes(this.failOnText)) {
throw new Error(`forced query failure: ${text}`);
}
return queryResult([], text === 'ROLLBACK' ? 0 : 1);
}
release(): void {
this.released = true;
}
}
class RecordingPool implements PostgresPool {
readonly calls: QueryCall[] = [];
constructor(
private readonly client: RecordingPoolClient =
new RecordingPoolClient(),
private readonly nextResult: QueryResult<Record<string, unknown>> =
queryResult(),
) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (text === 'SELECT 1') return queryResult([{ ok: 1 }], 1);
return this.nextResult;
}
async connect(): Promise<PostgresPoolClient> {
return this.client;
}
async end(): Promise<void> {}
}
describe('PostgresDatabaseClient', () => {
it('is selected by the database factory', () => {
const client = new DatabaseClientFactory(postgresConfig());
expect(client).toBeInstanceOf(PostgresDatabaseClient);
});
it('maps pg write results to the shared WriteResult shape', () => {
expect(
mapPostgresWriteResult(queryResult([{ id: '42' }], 1)),
).toEqual({
insertId: 42,
affectedRows: 1,
anyRowsAffected: true,
});
expect(mapPostgresWriteResult(queryResult([], 0))).toEqual({
insertId: 0,
affectedRows: 0,
anyRowsAffected: false,
});
});
it('prepares write SQL at the pg boundary', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ id: '7' }], 1),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
const result = await client.write(
'INSERT INTO `apps` (`name`) VALUES (?) RETURNING id',
['editor'],
);
expect(result.insertId).toBe(7);
expect(pool.calls.at(-1)).toEqual({
text: 'INSERT INTO "apps" ("name") VALUES ($1) RETURNING id',
values: ['editor'],
});
});
it('rewrites a repeated query once', async () => {
const pool = new RecordingPool();
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
const query = 'SELECT `name` FROM `apps` WHERE `id` = ?';
rewriteSpy.mockClear();
await client.read(query, [1]);
await client.pread(query, [2]);
await client.write(query, [3]);
expect(rewriteSpy).toHaveBeenCalledTimes(1);
expect(pool.calls.slice(-3).map((c) => c.text)).toEqual(
Array(3).fill('SELECT "name" FROM "apps" WHERE "id" = $1'),
);
});
it('normalizes int8 fields on read and primary read rows', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[
{
uuid: 'session-1',
created_at: '1710000000',
last_activity: '1710000001',
expires_at: '1710000002',
revoked_at: null,
},
],
1,
[
textField('uuid'),
int8Field('created_at'),
int8Field('last_activity'),
int8Field('expires_at'),
int8Field('revoked_at'),
],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
await expect(client.pread('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
});
it('rejects unsafe int8 values instead of losing precision', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[{ id: '9007199254740992' }],
1,
[int8Field('id')],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT `id` FROM `sessions`')).rejects.toThrow(
'safe integer',
);
});
it('normalizes tryHardRead rows returned by a replica', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '1' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '2' }], 1, [
int8Field('created_at'),
]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 2 }]);
});
it('normalizes tryHardRead rows returned by primary fallback', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '3' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([], 0, [int8Field('created_at')]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 3 }]);
});
it('runs batch writes in order and commits', async () => {
const conn = new RecordingPoolClient();
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
{
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
values: ['session-1'],
},
]);
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{
text: 'DELETE FROM "sessions" WHERE "uuid" = $1',
values: ['session-1'],
},
{ text: 'COMMIT', values: undefined },
]);
expect(conn.released).toBe(true);
});
it('rolls back and releases the pg connection when a batch write fails', async () => {
const conn = new RecordingPoolClient('UPDATE "user"');
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await expect(
client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]),
).rejects.toThrow('forced query failure');
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{ text: 'ROLLBACK', values: undefined },
]);
expect(conn.released).toBe(true);
});
it('leaves the pool open through prepare-shutdown; only onServerShutdown closes it', async () => {
vi.useFakeTimers();
try {
const pool = new RecordingPool();
let ended = false;
pool.end = async () => {
ended = true;
};
const client = new PostgresDatabaseClient(
postgresConfig(),
() => pool,
);
await client.onServerStart();
await client.onServerPrepareShutdown();
expect(ended).toBe(false);
// No timer closes the pool on its own — the layers above are
// still draining through it for as long as the process drains.
await vi.advanceTimersByTimeAsync(60_000);
expect(ended).toBe(false);
await client.onServerShutdown();
expect(ended).toBe(true);
} finally {
vi.useRealTimers();
}
});
});
@@ -0,0 +1,302 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Pool, type PoolConfig, type QueryResult } from 'pg';
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
import { Span } from '../../util/span.js';
import {
AbstractDatabaseClient,
type BatchEntry,
type WriteResult,
} from './DatabaseClient';
import { preparePostgresSql } from './preparePostgresSql.js';
import { splitPostgresStatements } from './splitPostgresStatements.js';
import type { IConfig } from '../../types';
type PostgresEndpointConfig = {
host?: string;
port?: number;
user?: string;
password?: string;
database?: string;
connectionString?: string;
url?: string;
};
export interface PostgresQueryable {
query(query: string, values?: unknown[]): Promise<QueryResult>;
}
export interface PostgresPoolClient extends PostgresQueryable {
release(): void;
}
export interface PostgresPool extends PostgresQueryable {
connect(): Promise<PostgresPoolClient>;
end(): Promise<void>;
}
type PostgresPoolFactory = (poolConfig: PoolConfig) => PostgresPool;
enum Configuration {
SINGLE,
REPLICA,
}
const POSTGRES_INT8_OID = 20;
const PREPARED_SQL_CACHE_SIZE = 1000;
const INTEGER_TEXT_PATTERN = /^-?\d+$/u;
const normalizePostgresInt8 = (
value: unknown,
columnName: string,
): number | null | undefined => {
if (value === null || value === undefined) return value;
const parsed =
typeof value === 'bigint'
? Number(value)
: typeof value === 'number'
? value
: typeof value === 'string' && INTEGER_TEXT_PATTERN.test(value)
? Number(value)
: Number.NaN;
if (!Number.isSafeInteger(parsed)) {
throw new Error(
`[postgres] int8 column ${columnName} is outside JavaScript's safe integer range`,
);
}
return parsed;
};
const normalizePostgresRows = (
result: QueryResult,
): Record<string, unknown>[] => {
const int8Fields = result.fields.filter(
(field) => field.dataTypeID === POSTGRES_INT8_OID,
);
if (int8Fields.length === 0) {
return result.rows as Record<string, unknown>[];
}
return result.rows.map((row) => {
const normalized: Record<string, unknown> = { ...row };
for (const field of int8Fields) {
normalized[field.name] = normalizePostgresInt8(
normalized[field.name],
field.name,
);
}
return normalized;
});
};
export const mapPostgresWriteResult = (result: QueryResult): WriteResult => {
const affectedRows = result.rowCount ?? 0;
const rowId = result.rows[0]?.id;
const insertId =
typeof rowId === 'bigint'
? rowId
: typeof rowId === 'number'
? rowId
: typeof rowId === 'string' && rowId !== ''
? Number(rowId)
: 0;
const normalizedInsertId =
typeof insertId === 'number' && Number.isNaN(insertId) ? 0 : insertId;
return {
insertId: normalizedInsertId,
affectedRows,
anyRowsAffected: affectedRows > 0,
};
};
export class PostgresDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'postgres';
private primaryPool!: PostgresPool;
private replicaPool!: PostgresPool;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
/** Query text as written → its Postgres rewrite, which is pure. */
private readonly preparedSql = new BoundedTtlMap<string, string>({
maxEntries: PREPARED_SQL_CACHE_SIZE,
});
constructor(
config: IConfig,
private readonly poolFactory: PostgresPoolFactory = (poolConfig) =>
new Pool(poolConfig) as unknown as PostgresPool,
) {
super(config);
}
override async onServerStart(): Promise<void> {
const dbConf = this.config.database!;
this.primaryPool = this.createPool(dbConf);
await this.primaryPool.query('SELECT 1');
console.log('[postgres] connected to primary');
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
await this.replicaPool.query('SELECT 1');
this.configuration = Configuration.REPLICA;
console.log('[postgres] connected to read-replica');
} else {
this.replicaPool = this.primaryPool;
this.configuration = Configuration.SINGLE;
}
await this.applyMigrationPaths(
() => this.primaryPool.connect(),
splitPostgresStatements,
{ transactional: true },
);
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
// Pools stay open; onServerShutdown closes them after the layers above
// have drained through them.
console.log('[postgres] entering drain mode');
}
override async onServerShutdown(): Promise<void> {
await this.closeCurrentPools();
}
override quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `"${part.replaceAll('"', '""')}"`;
})
.join('.');
}
override booleanLiteral(value: boolean): string {
return value ? 'TRUE' : 'FALSE';
}
override booleanValue(value: boolean): boolean {
return value;
}
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.replicaPool, query, params);
return normalizePostgresRows(result);
}
@Span('db.pread', (query: string) => ({ 'db.statement': query }))
override async pread(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.primaryPool, query, params);
return normalizePostgresRows(result);
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
const result = await this.query(this.primaryPool, query, params);
return mapPostgresWriteResult(result);
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(entries: BatchEntry[]): Promise<void> {
if (entries.length === 0) return;
const conn = await this.primaryPool.connect();
try {
await conn.query('BEGIN');
try {
for (const { statement, values } of entries) {
await this.query(conn, statement, values);
}
await conn.query('COMMIT');
} catch (err) {
await conn.query('ROLLBACK').catch(() => {});
throw err;
}
} finally {
conn.release();
}
}
protected override hasReadReplica(): boolean {
return this.configuration === Configuration.REPLICA;
}
private createPool(dbConf: PostgresEndpointConfig): PostgresPool {
const connectionString = dbConf.connectionString ?? dbConf.url;
if (connectionString) {
return this.poolFactory({
connectionString,
max: 30,
});
}
return this.poolFactory({
host: dbConf.host ?? '127.0.0.1',
port: dbConf.port ?? 5432,
user: dbConf.user ?? 'postgres',
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
max: 30,
});
}
private async query(
target: PostgresQueryable,
query: string,
params: unknown[] = [],
): Promise<QueryResult> {
let text = this.preparedSql.get(query);
if (text === undefined) {
text = preparePostgresSql(query).text;
this.preparedSql.set(query, text);
}
return target.query(text, params);
}
private async closeCurrentPools(): Promise<void> {
const tasks: Promise<void>[] = [];
if (this.primaryPool) tasks.push(this.primaryPool.end());
if (this.replicaPool && this.replicaPool !== this.primaryPool) {
tasks.push(this.replicaPool.end());
}
await Promise.all(tasks);
}
}
+225 -43
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -18,10 +18,20 @@
*/
import { metrics } from '@opentelemetry/api';
import {
POOL_ACQUIRE_TIMEOUT,
isNeverSentError,
isRetriableError,
isRolledBackError,
} from './retriableErrors.js';
const DEFAULT_MAX_QUEUE_SIZE = 1000;
const DEFAULT_FAILURE_THRESHOLD = 5;
const DEFAULT_COOLDOWN_MS = 5_000;
const DEFAULT_ACQUIRE_TIMEOUT_MS = 5_000;
const ACQUIRE_ATTEMPTS = 3;
const ITEM_RETRY_ATTEMPTS = 2;
const RETRY_BASE_BACKOFF_MS = 100;
const FALLBACK_RETRY_CONCURRENCY = 8;
const meter = metrics.getMeter('puter-backend');
@@ -53,6 +63,15 @@ const fallbackItemFailuresCounter = meter.createCounter(
'Per-item failures observed during SQLBatcher per-item retry',
},
);
const fallbackItemRetriesCounter = meter.createCounter(
'sql_batcher.fallback.item_retries',
{
description:
'Transient per-item failures retried during SQLBatcher fallback',
},
);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
export class SQLBatcher {
dbPool;
@@ -61,18 +80,45 @@ export class SQLBatcher {
maxQueueSize;
failureThreshold;
cooldownMs;
poolLabel;
readOnly;
acquireTimeoutMs;
queue = [];
timeouts = [];
#consecutiveFailures = 0;
#lastFailureAt = 0;
#metricAttrs;
/**
* @param {object} dbPool Mysql2 pool
* @param {object} [opts]
* @param {number} [opts.maxTimeInQueue] Ms an item may wait before flush
* @param {number} [opts.maxBatchSize] Items coalesced per flush
* @param {number} [opts.maxQueueSize] Drop-oldest high-water mark
* @param {number} [opts.failureThreshold] Consecutive failures to open the
* breaker
* @param {number} [opts.cooldownMs] Breaker open duration after last
* failure
* @param {'primary' | 'replica'} [opts.poolLabel] Role label on metrics; in
* single-node setups the 'replica' batcher shares the primary pool, so
* this reflects the read/write role rather than a physical instance
* @param {boolean} [opts.readOnly] This batcher only ever carries SELECTs,
* so any transient failure is safe to retry
* @param {number} [opts.acquireTimeoutMs] Max wait for a pooled connection;
* 0 disables the bound
*/
constructor(
dbPool,
maxTimeInQueue = 20,
maxBatchSize = 50,
maxQueueSize = DEFAULT_MAX_QUEUE_SIZE,
failureThreshold = DEFAULT_FAILURE_THRESHOLD,
cooldownMs = DEFAULT_COOLDOWN_MS,
{
maxTimeInQueue = 20,
maxBatchSize = 50,
maxQueueSize = DEFAULT_MAX_QUEUE_SIZE,
failureThreshold = DEFAULT_FAILURE_THRESHOLD,
cooldownMs = DEFAULT_COOLDOWN_MS,
poolLabel = 'primary',
readOnly = false,
acquireTimeoutMs = DEFAULT_ACQUIRE_TIMEOUT_MS,
} = {},
) {
this.dbPool = dbPool;
this.maxTimeInQueue = maxTimeInQueue;
@@ -80,6 +126,10 @@ export class SQLBatcher {
this.maxQueueSize = maxQueueSize;
this.failureThreshold = failureThreshold;
this.cooldownMs = cooldownMs;
this.poolLabel = poolLabel;
this.readOnly = readOnly;
this.acquireTimeoutMs = acquireTimeoutMs;
this.#metricAttrs = { pool: poolLabel };
}
async execute(sql, values) {
@@ -90,9 +140,13 @@ export class SQLBatcher {
return this;
}
#createPublicBatchError() {
// The public error is deliberately opaque (no SQL, no internals), but
// `reason` distinguishes the load-shed path for logs and callers:
// breakerOpen | queueOverflow | connAcquire.
#createPublicBatchError(reason) {
const error = new Error('Database operation failed');
error.code = 'dbBatchFailed';
error.reason = reason;
return error;
}
@@ -106,8 +160,8 @@ export class SQLBatcher {
async query(sql, values) {
if (this.#isBreakerOpen()) {
enqueueRejectedCounter.add(1);
throw this.#createPublicBatchError();
enqueueRejectedCounter.add(1, this.#metricAttrs);
throw this.#createPublicBatchError('breakerOpen');
}
const { promise, resolve, reject } = Promise.withResolvers();
@@ -117,8 +171,8 @@ export class SQLBatcher {
// to have already exceeded any caller-side timeout anyway.
while (this.queue.length >= this.maxQueueSize) {
const dropped = this.queue.shift();
dropped.reject(this.#createPublicBatchError());
enqueueDroppedCounter.add(1);
dropped.reject(this.#createPublicBatchError('queueOverflow'));
enqueueDroppedCounter.add(1, this.#metricAttrs);
}
this.queue.push({
@@ -142,6 +196,61 @@ export class SQLBatcher {
return promise;
}
// Bounded wait for a pooled connection. Without a bound, a stalled
// database turns every flush into an indefinite hang — nothing fails,
// so neither the breaker nor callers' own timeouts ever engage.
#getConnectionWithTimeout() {
const acquire = this.dbPool.promise().getConnection();
if (!this.acquireTimeoutMs) return acquire;
return new Promise((resolve, reject) => {
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
// A connection that arrives late must go back to the pool.
acquire.then(
(conn) => conn.release(),
() => {},
);
const error = new Error(
'Timed out acquiring database connection',
);
error.code = POOL_ACQUIRE_TIMEOUT;
reject(error);
}, this.acquireTimeoutMs);
acquire.then(
(conn) => {
if (timedOut) return;
clearTimeout(timer);
resolve(conn);
},
(err) => {
if (timedOut) return;
clearTimeout(timer);
reject(err);
},
);
});
}
// Acquisition failures never sent a statement, so retrying is always
// safe regardless of what the batch contains.
async #acquireConnection() {
let lastError;
for (let attempt = 1; attempt <= ACQUIRE_ATTEMPTS; attempt++) {
try {
return await this.#getConnectionWithTimeout();
} catch (error) {
lastError = error;
if (attempt < ACQUIRE_ATTEMPTS) {
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
}
}
}
throw lastError;
}
async flush(batch) {
const timeout = this.timeouts.shift();
if (timeout && !timeout._destroyed) {
@@ -149,37 +258,45 @@ export class SQLBatcher {
}
if (batch.length === 0) return;
const query = `${batch.map((b) => b.sql.replace(/;+\s*$/, '')).join(';')}; SELECT 1`; // SELECT 1 forces mysql2 to return array
const values = batch.map((b) => b.values ?? []).flat();
let connection;
try {
connection = await this.dbPool.promise().getConnection();
connection = await this.#acquireConnection();
} catch (error) {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1);
this.#recordFlushFailure();
console.warn(
'SQLBatcher could not acquire connection for flush:',
error,
);
for (const b of batch) {
b.reject(this.#createPublicBatchError());
b.reject(this.#createPublicBatchError('connAcquire'));
}
return;
}
if (batch.length === 1 && !this.readOnly) {
await this.#flushLoneWrite(batch[0], connection);
return;
}
const query = `${batch.map((b) => b.sql.replace(/;+\s*$/, '')).join(';')}; SELECT 1`; // SELECT 1 forces mysql2 to return array
const values = batch.map((b) => b.values ?? []).flat();
// Run the coalesced multi-statement inside an explicit transaction so
// a single bad statement (e.g. a duplicate-key INSERT) rolls back the
// whole batch atomically, leaving us free to re-run each item
// individually below. Without this, MySQL would commit every
// statement up to the failure point and a per-item retry would
// misreport already-committed inserts as duplicate-key failures.
//
// A read-only batch has nothing to roll back, and the wrapper is not
// free: BEGIN and COMMIT are round trips of their own, which triples
// the cost of a read against a pool in another region.
const wrapInTransaction = !this.readOnly;
let batchSucceeded = false;
try {
await connection.beginTransaction();
if (wrapInTransaction) await connection.beginTransaction();
const [results, fields] = await connection.query(query, values);
await connection.commit();
if (wrapInTransaction) await connection.commit();
batchSucceeded = true;
this.#consecutiveFailures = 0;
for (let i = 0; i < batch.length; i++) {
@@ -187,10 +304,12 @@ export class SQLBatcher {
b.resolve([results[i], fields?.[i]]);
}
} catch (batchError) {
try {
await connection.rollback();
} catch (rollbackError) {
console.warn('SQLBatcher rollback failed:', rollbackError);
if (wrapInTransaction) {
try {
await connection.rollback();
} catch (rollbackError) {
console.warn('SQLBatcher rollback failed:', rollbackError);
}
}
console.warn(
'SQLBatcher batch failed; retrying items individually:',
@@ -202,12 +321,13 @@ export class SQLBatcher {
if (batchSucceeded) return;
// Per-item fallback. The transaction was rolled back so no statement
// committed; re-running each item independently produces clean
// success/failure outcomes for each caller. Concurrency is capped to
// avoid briefly saturating the pool when a large batch fails.
flushFailureCounter.add(1);
fallbackInvocationsCounter.add(1);
// Per-item fallback. Nothing from the batch is live — it was rolled
// back, or carried only SELECTs — so re-running each item
// independently produces clean success/failure outcomes for each
// caller. Concurrency is capped to avoid briefly saturating the pool
// when a large batch fails.
flushFailureCounter.add(1, this.#metricAttrs);
fallbackInvocationsCounter.add(1, this.#metricAttrs);
const settled = new Array(batch.length);
let cursor = 0;
@@ -216,17 +336,7 @@ export class SQLBatcher {
async () => {
while (cursor < batch.length) {
const i = cursor++;
const b = batch[i];
try {
settled[i] = {
ok: true,
value: await this.dbPool
.promise()
.query(b.sql, b.values ?? []),
};
} catch (error) {
settled[i] = { ok: false, error };
}
settled[i] = await this.#runItem(batch[i]);
}
},
);
@@ -246,7 +356,7 @@ export class SQLBatcher {
}
}
if (failureCount > 0) {
fallbackItemFailuresCounter.add(failureCount);
fallbackItemFailuresCounter.add(failureCount, this.#metricAttrs);
}
// Only escalate the breaker when the database itself looks unhealthy
@@ -259,4 +369,76 @@ export class SQLBatcher {
this.#consecutiveFailures++;
}
}
// A single statement is atomic under autocommit, so it skips BEGIN/COMMIT
// and their round trips. With no transaction to roll back, a failure is
// not re-run wholesale; only the per-item retry policy applies.
async #flushLoneWrite(item, connection) {
const r = await this.#runItem(item, connection);
if (r.ok) {
this.#consecutiveFailures = 0;
item.resolve(r.value);
return;
}
// An error the server answered with (duplicate key, bad SQL) means it
// is up; only failing to reach it counts toward the breaker.
if (isRetriableError(r.error)) {
this.#recordFlushFailure();
} else {
this.#consecutiveFailures = 0;
flushFailureCounter.add(1, this.#metricAttrs);
}
item.reject(r.error);
}
#recordFlushFailure() {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1, this.#metricAttrs);
}
// Run one item on its own, retrying transient failures with backoff.
// A read-only batcher may retry anything transient; a batcher that
// carries writes only retries failures where the statement provably
// did not apply — either it never reached the server, or the server
// rolled it back itself. A write that died mid-flight may have
// committed, and re-running that one would double-apply.
//
// Lock contention lands in the second group and is worth retrying rather
// than surfacing: an item is a single statement, so a deadlock victim has
// been fully undone, and the caller sees an unhandled 500 for what the
// database is telling us to just run again.
async #runItem(b, connection) {
let attempt = 0;
while (true) {
if (!connection) {
try {
connection = await this.#acquireConnection();
} catch (error) {
return { ok: false, error };
}
}
let error;
try {
return {
ok: true,
value: await connection.query(b.sql, b.values ?? []),
};
} catch (e) {
error = e;
} finally {
connection.release();
connection = undefined;
}
const canRetry = this.readOnly
? isRetriableError(error) || isRolledBackError(error)
: isNeverSentError(error) || isRolledBackError(error);
if (!canRetry || attempt >= ITEM_RETRY_ATTEMPTS) {
return { ok: false, error };
}
attempt++;
fallbackItemRetriesCounter.add(1, this.#metricAttrs);
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
}
}
}
@@ -0,0 +1,367 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it, vi } from 'vitest';
import { SQLBatcher } from './SQLBatcher.js';
const makeError = (code: string, message = code): Error & { code: string } => {
const error = new Error(message) as Error & { code: string };
error.code = code;
return error;
};
interface FakeConnection {
beginTransaction: ReturnType<typeof vi.fn>;
query: ReturnType<typeof vi.fn>;
commit: ReturnType<typeof vi.fn>;
rollback: ReturnType<typeof vi.fn>;
release: ReturnType<typeof vi.fn>;
}
// The batch flush sends one coalesced multi-statement (always suffixed
// `; SELECT 1`); fallback items send their original single statement.
const isBatchQuery = (sql: string) => sql.endsWith('; SELECT 1');
const makeConnection = (
onQuery: (sql: string, values: unknown[]) => unknown,
): FakeConnection => ({
beginTransaction: vi.fn(async () => {}),
query: vi.fn(async (sql: string, values: unknown[]) => onQuery(sql, values)),
commit: vi.fn(async () => {}),
rollback: vi.fn(async () => {}),
release: vi.fn(),
});
const makePool = (connection: FakeConnection | (() => Promise<FakeConnection>)) => {
const getConnection = vi.fn(async () =>
typeof connection === 'function' ? connection() : connection,
);
return {
pool: { promise: () => ({ getConnection }) },
getConnection,
};
};
// Happy-path onQuery: batch returns one result row-set per statement plus
// the trailing SELECT 1 row-set.
const happyBatch = (sql: string) => {
if (!isBatchQuery(sql)) throw new Error('unexpected fallback query');
const statements = sql.split(';').length - 1;
return [
Array.from({ length: statements + 1 }, (_, i) => [{ n: i }]),
undefined,
];
};
describe('SQLBatcher', () => {
it('resolves each batched item with its own result', async () => {
const conn = makeConnection(happyBatch);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
const [a, b] = await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(a[0]).toEqual([{ n: 0 }]);
expect(b[0]).toEqual([{ n: 1 }]);
expect(conn.beginTransaction).toHaveBeenCalledTimes(1);
expect(conn.commit).toHaveBeenCalledTimes(1);
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('skips the transaction wrapper on a readOnly batcher', async () => {
const conn = makeConnection(happyBatch);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
const [a, b] = await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(a[0]).toEqual([{ n: 0 }]);
expect(b[0]).toEqual([{ n: 1 }]);
// BEGIN and COMMIT are round trips; a SELECT-only batch has nothing to
// roll back, so it must not pay for them.
expect(conn.beginTransaction).not.toHaveBeenCalled();
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('does not roll back a failed readOnly batch', async () => {
const conn = makeConnection((sql: string) => {
if (isBatchQuery(sql)) throw makeError('ER_LOCK_DEADLOCK');
return [[{ n: 0 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(conn.rollback).not.toHaveBeenCalled();
});
it('drops the oldest item with reason queueOverflow at the high-water mark', async () => {
const conn = makeConnection(() => [[{ n: 0 }], undefined]);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
maxQueueSize: 1,
});
const first = batcher.query('SELECT a', []);
const second = batcher.query('SELECT b', []);
await expect(first).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'queueOverflow',
});
await expect(second).resolves.toBeTruthy();
});
it('rejects with reason connAcquire after exhausting acquisition retries', async () => {
const getConnection = vi.fn(async () => {
throw makeError('ECONNREFUSED');
});
const pool = { promise: () => ({ getConnection }) };
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('SELECT a', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'connAcquire',
});
expect(getConnection).toHaveBeenCalledTimes(3);
});
it('bounds connection acquisition and rejects when the pool never answers', async () => {
const getConnection = vi.fn(
() => new Promise(() => {}), // pool never yields a connection
);
const pool = { promise: () => ({ getConnection }) };
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
acquireTimeoutMs: 30,
});
await expect(batcher.query('SELECT a', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'connAcquire',
});
expect(getConnection).toHaveBeenCalledTimes(3);
});
it('opens the breaker after consecutive failures and rejects with reason breakerOpen', async () => {
// Batch and fallback both fail with an ambiguous (non-retriable for
// writes) connection error, so no item gets through.
const conn = makeConnection(() => {
throw makeError('ECONNRESET');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
failureThreshold: 1,
cooldownMs: 60_000,
});
await expect(batcher.query('INSERT x', [])).rejects.toMatchObject({
code: 'ECONNRESET',
});
await expect(batcher.query('INSERT y', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'breakerOpen',
});
});
it('retries transient fallback failures when readOnly', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ECONNRESET');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
const result = await batcher.query('SELECT a', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('does not retry ambiguous failures on a batcher that carries writes', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
throw makeError('ECONNRESET');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('INSERT x', [])).rejects.toMatchObject({
code: 'ECONNRESET',
});
expect(fallbackAttempts).toBe(1);
});
it('retries never-sent failures even on a batcher that carries writes', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ECONNREFUSED');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
const result = await batcher.query('INSERT x', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('retries a deadlocked write instead of surfacing it', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ER_LOCK_DEADLOCK');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ER_LOCK_DEADLOCK');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
// The victim statement was rolled back by the server, so re-running it
// can't double-apply — the caller should never see the deadlock.
const result = await batcher.query('UPDATE notification SET x', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('gives up on a write that deadlocks past the retry budget', async () => {
const conn = makeConnection(() => {
throw makeError('ER_LOCK_DEADLOCK');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('UPDATE hot SET x', [])).rejects.toMatchObject(
{ code: 'ER_LOCK_DEADLOCK' },
);
});
it('sends a lone write as-is, outside a transaction', async () => {
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw new Error('unexpected batch query');
return [{ affectedRows: 1 }, undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxBatchSize: 1 });
const result = await batcher.query('UPDATE t SET x = ?', [1]);
expect(result[0]).toEqual({ affectedRows: 1 });
expect(conn.query).toHaveBeenCalledTimes(1);
expect(conn.query).toHaveBeenCalledWith('UPDATE t SET x = ?', [1]);
expect(conn.beginTransaction).not.toHaveBeenCalled();
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('runs a lone write the server rejects once, without tripping the breaker', async () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
const conn = makeConnection((sql) => {
if (sql === 'INSERT dup') throw makeError('ER_DUP_ENTRY');
return [{ affectedRows: 1 }, undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxBatchSize: 1,
failureThreshold: 1,
cooldownMs: 60_000,
});
await expect(batcher.query('INSERT dup', [])).rejects.toMatchObject({
code: 'ER_DUP_ENTRY',
});
expect(conn.query).toHaveBeenCalledTimes(1);
expect(conn.rollback).not.toHaveBeenCalled();
expect(warnSpy).not.toHaveBeenCalled();
// The server answered, so it is up: the next write goes through.
await expect(batcher.query('INSERT fine', [])).resolves.toBeTruthy();
warnSpy.mockRestore();
});
it('retries a lone write on a connection the server closed for inactivity', async () => {
let attempts = 0;
const conn = makeConnection(() => {
attempts++;
if (attempts === 1) {
throw makeError('ER_CLIENT_INTERACTION_TIMEOUT');
}
return [{ affectedRows: 1 }, undefined];
});
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxBatchSize: 1 });
const result = await batcher.query('INSERT x', []);
expect(result[0]).toEqual({ affectedRows: 1 });
expect(attempts).toBe(2);
expect(getConnection).toHaveBeenCalledTimes(2);
});
it('never retries deterministic row-level errors and does not escalate the breaker', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ER_DUP_ENTRY');
fallbackAttempts++;
if (sql === 'INSERT dup') throw makeError('ER_DUP_ENTRY');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
failureThreshold: 1,
cooldownMs: 60_000,
readOnly: true,
});
const dup = batcher.query('INSERT dup', []);
const fine = batcher.query('INSERT fine', []);
await expect(dup).rejects.toMatchObject({ code: 'ER_DUP_ENTRY' });
await expect(fine).resolves.toBeTruthy();
expect(fallbackAttempts).toBe(2);
// One fallback item succeeded, so the breaker must stay closed.
const conn2 = makeConnection(happyBatch);
// reuse same batcher/pool: next query must not be rejected upfront
conn.query.mockImplementation(conn2.query.getMockImplementation()!);
await expect(batcher.query('SELECT a', [])).resolves.toBeTruthy();
});
});
File diff suppressed because it is too large. Load diff
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,18 +17,24 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { existsSync, readFileSync } from 'fs';
import { basename, extname, join, resolve } from 'path';
import { existsSync, mkdirSync, readFileSync } from 'fs';
import { basename, dirname, extname, join, resolve } from 'path';
import { createContext, runInContext } from 'vm';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import type { IConfig } from '../../types';
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
import { Span } from '../../util/span.js';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
const MIGRATIONS_DIR = resolve(__dirname, './migrations/sqlite');
const STATEMENT_CACHE_SIZE = 1000;
type SqliteDatabase = InstanceType<typeof import('better-sqlite3')>;
type SqliteStatement = import('better-sqlite3').Statement<unknown[]>;
/**
* Ordered list of [threshold_version, files[]] pairs.
* A database whose `user_version` is <= threshold_version will have
* the corresponding files applied.
* Ordered list of [threshold_version, files[]] pairs. A database whose
* `user_version` is <= threshold_version will have the corresponding files
* applied.
*/
const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[-1, ['0001_create-tables.sql', '0002_add-default-apps.sql']],
@@ -76,13 +82,63 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[41, ['0045_user_oidc_providers.sql']],
[42, ['0046_is-private-apps.sql']],
[43, ['0047_app-url-updates.sql']],
[44, ['0048_old-app-names-unique-tuple.sql']],
[45, ['0049_music-player-pdf-player-updates.sql']],
[46, ['0050_add_preamble_version.sql']],
[47, ['0051_sessions_v2.sql']],
[48, ['0052_sessions_v2_lookups.sql']],
[49, ['0053_sessions_access_token_uid.sql']],
[50, ['0054_sessions_workers.sql']],
[50, ['0055_username_nocase_unique.sql']],
[51, ['0056_sessions_kind_worker.sql']],
[52, ['0057_add_user_reputation.sql']],
[53, ['0058_add_phone_verification.sql']],
[54, ['0059_add_card_verification.sql']],
[55, ['0060_add_card_fingerprint.sql']],
[56, ['0061_add_suspended_at.sql']],
[57, ['0062_blocked-app-origins.sql']],
[58, ['0063_add_suspended_reason.sql']],
[59, ['0064_abuse-moderation-events.sql']],
[60, ['0065_app-feedback.sql']],
[61, ['0066_owned-email-unique.sql']],
[62, ['0067_share_entries.sql']],
[63, ['0068_referral-code-unique.sql']],
[64, ['0069_user-block.sql']],
[65, ['0070_drop-orphaned-default-groups.sql']],
[66, ['0071_share_issuer_index.sql']],
[67, ['0072_notification-scope.sql']],
[68, ['0073_notification-created-at.sql']],
[69, ['0074_add_user_home.sql']],
[70, ['0075_event-subscriptions.sql']],
[71, ['0076_event-handlers.sql']],
[72, ['0077_teams.sql']],
[73, ['0078_jct-user-group-pair-unique.sql']],
[74, ['0079_team-audit-and-group-shares.sql']],
[75, ['0080_kv-share-handles.sql']],
[76, ['0081_event-subscriptions-indexes.sql']],
[77, ['0082_temp-password-expiry.sql']],
[78, ['0083_team-directory.sql']],
[79, ['0084_share-anyone-with-link.sql']],
[80, ['0085_event-subscriptions-include-value.sql']],
[81, ['0086_subdomains-app-owner-cascade.sql']],
[82, ['0087_feedback-attachments.sql']],
[83, ['0088_apps-index-url.sql']],
[84, ['0089_team-require-2fa.sql']],
[85, ['0090_user-app-permission-index.sql']],
[86, ['0091_icloud-dot-significance.sql']],
[87, ['0092_share-entry-owner.sql']],
[88, ['0093_fsentries-parent-uid-index.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'sqlite';
// better-sqlite3 instance — set during onServerStart
private db!: InstanceType<typeof import('better-sqlite3')>;
private db!: SqliteDatabase;
/** Prepared statements by query text; SQLite re-prepares on schema change. */
private readonly statements = new BoundedTtlMap<string, SqliteStatement>({
maxEntries: STATEMENT_CACHE_SIZE,
});
constructor(config: IConfig) {
super(config);
@@ -95,15 +151,23 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
override async onServerStart(): Promise<void> {
const Database = (await import('better-sqlite3')).default;
const dbPath = this.config.database?.path ?? ':memory:';
const dbPath = this.config.database?.inMemory
? ':memory:'
: (this.config.database?.path ?? ':memory:');
const isNew = dbPath === ':memory:' || !existsSync(dbPath);
if (dbPath !== ':memory:') {
mkdirSync(dirname(dbPath), { recursive: true });
}
this.statements.clear();
this.db = new Database(dbPath);
await this.runMigrations(isNew);
}
override onServerShutdown(): void {
this.statements.clear();
if (this.db) {
this.db.close();
}
@@ -113,16 +177,14 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
// Query interface
// ------------------------------------------------------------------
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
query = this.transformQuery(query);
params = this.transformParams(params);
return this.db.prepare(query).all(...params) as Record<
string,
unknown
>[];
return this.prepare(query).all(
...this.transformParams(params),
) as Record<string, unknown>[];
}
override async pread(
@@ -133,14 +195,12 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
return this.read(query, params);
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
query = this.transformQuery(query);
params = this.transformParams(params);
const info = this.db.prepare(query).run(...params);
const info = this.prepare(query).run(...this.transformParams(params));
return {
insertId: info.lastInsertRowid,
@@ -149,14 +209,15 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
};
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(
entries: { statement: string; values: unknown[] }[],
): Promise<void> {
this.db.transaction(() => {
for (let { statement, values } of entries) {
statement = this.transformQuery(statement);
values = this.transformParams(values);
this.db.prepare(statement).run(...values);
for (const { statement, values } of entries) {
this.prepare(statement).run(...this.transformParams(values));
}
})();
}
@@ -165,6 +226,15 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
// SQLite-specific transforms
// ------------------------------------------------------------------
private prepare(query: string): SqliteStatement {
let statement = this.statements.get(query);
if (!statement) {
statement = this.db.prepare(this.transformQuery(query));
this.statements.set(query, statement);
}
return statement;
}
private transformQuery(query: string): string {
return query.replace(/now\(\)/gi, "datetime('now')");
}
@@ -453,6 +523,13 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
version: 43,
check: () => this.hasColumn('apps', 'is_private'),
},
{
version: 44,
check: () =>
this.hasRow(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'old_app_names' AND sql LIKE '%UNIQUE%app_uid%name%'",
),
},
];
let inferredUserVersion = 0;
+8 -4
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -24,16 +24,18 @@ export {
} from './DatabaseClient';
export { SqliteDatabaseClient } from './SqliteDatabaseClient';
export { MySQLDatabaseClient } from './MySQLDatabaseClient';
export { PostgresDatabaseClient } from './PostgresDatabaseClient';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient } from './DatabaseClient';
import { MySQLDatabaseClient } from './MySQLDatabaseClient';
import { PostgresDatabaseClient } from './PostgresDatabaseClient';
import { SqliteDatabaseClient } from './SqliteDatabaseClient';
/**
* Factory class registered in `puterClients`. PuterServer calls
* `new DatabaseClientFactory(config)` — the constructor returns the
* concrete subclass selected by `config.database.engine`.
* Factory class registered in `puterClients`. PuterServer calls `new
* DatabaseClientFactory(config)` — the constructor returns the concrete
* subclass selected by `config.database.engine`.
*/
export const DatabaseClientFactory = class DatabaseClientFactory {
constructor(config: IConfig) {
@@ -41,6 +43,8 @@ export const DatabaseClientFactory = class DatabaseClientFactory {
switch (engine) {
case 'mysql':
return new MySQLDatabaseClient(config);
case 'postgres':
return new PostgresDatabaseClient(config);
case 'sqlite':
return new SqliteDatabaseClient(config);
default:
@@ -0,0 +1,43 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Comparator for migration filenames named `<engine>_mig_<N>.sql`.
*
* Existing MySQL files use unpadded integers, so lexical sorting places
* `*_mig_10.sql` before `*_mig_2.sql`. Pull the trailing integer out and sort
* numerically. Anything that does not match the `_<digits>.sql` shape falls
* back to lexical comparison and sorts after numbered files.
*/
export const compareMigrationFilenames = (a: string, b: string): number => {
const numericIndex = (name: string): number => {
const m = /_(\d+)\.sql$/.exec(name);
return m ? Number.parseInt(m[1], 10) : Number.NaN;
};
const na = numericIndex(a);
const nb = numericIndex(b);
if (Number.isFinite(na) && Number.isFinite(nb)) {
if (na !== nb) return na - nb;
} else if (Number.isFinite(na)) {
return -1;
} else if (Number.isFinite(nb)) {
return 1;
}
return a.localeCompare(b);
};
@@ -24,7 +24,9 @@
--
--
-- Idempotent column-ensure helper (used by CALLs below; dropped at end of file)
-- Idempotent column-ensure helper. Used by the CALLs below AND by later
-- migration files, which CALL it as a one-line idempotent column add. Left
-- resident (not dropped at end of file) so it outlives this migration.
--
DROP PROCEDURE IF EXISTS _puter_add_col;
@@ -60,7 +62,7 @@ CREATE TABLE IF NOT EXISTS `access_token_permissions` (
`extra` json DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=43870 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('access_token_permissions', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -100,7 +102,7 @@ CREATE TABLE IF NOT EXISTS `ai_usage` (
KEY `idx_ai_usage_user_timestamp` (`user_id`,`created_at`),
CONSTRAINT `ai_usage_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `ai_usage_ibfk_2` FOREIGN KEY (`app_id`) REFERENCES `apps` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=935038 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('ai_usage', 'id', '`id` int NOT NULL AUTO_INCREMENT');
@@ -131,7 +133,7 @@ CREATE TABLE IF NOT EXISTS `app_filetype_association` (
KEY `app_id` (`app_id`),
KEY `type` (`type`),
CONSTRAINT `app_filetype_association_ibfk_1` FOREIGN KEY (`app_id`) REFERENCES `apps` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=28897 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('app_filetype_association', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -157,7 +159,7 @@ CREATE TABLE IF NOT EXISTS `app_opens` (
KEY `idx_app_opens_app_user` (`app_uid`,`user_id`),
CONSTRAINT `app_opens_ibfk_2` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `app_opens_ibfk_3` FOREIGN KEY (`app_uid`) REFERENCES `apps` (`uid`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=14510891 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('app_opens', '_id', '`_id` bigint unsigned NOT NULL AUTO_INCREMENT');
@@ -231,7 +233,7 @@ CREATE TABLE IF NOT EXISTS `apps` (
KEY `idx_apps_listing_timestamp` (`approved_for_listing`,`timestamp` DESC),
CONSTRAINT `apps_ibfk_1` FOREIGN KEY (`owner_user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_apps_app_owner` FOREIGN KEY (`app_owner`) REFERENCES `apps` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=169455 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('apps', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -278,7 +280,7 @@ CREATE TABLE IF NOT EXISTS `audit_dev_to_app_permissions` (
KEY `fk_audit_dev_to_app_permissions_app_id` (`app_id`),
CONSTRAINT `fk_audit_dev_to_app_permissions_app_id` FOREIGN KEY (`app_id`) REFERENCES `apps` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_audit_dev_to_app_permissions_user_id` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=34 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('audit_dev_to_app_permissions', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -314,7 +316,7 @@ CREATE TABLE IF NOT EXISTS `audit_user_to_app_permissions` (
KEY `fk_audit_user_to_app_permissions_app_id` (`app_id`),
CONSTRAINT `fk_audit_user_to_app_permissions_app_id` FOREIGN KEY (`app_id`) REFERENCES `apps` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_audit_user_to_app_permissions_user_id` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=7352860 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('audit_user_to_app_permissions', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -350,7 +352,7 @@ CREATE TABLE IF NOT EXISTS `audit_user_to_group_permissions` (
KEY `group_id` (`group_id`),
CONSTRAINT `audit_user_to_group_permissions_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `audit_user_to_group_permissions_ibfk_2` FOREIGN KEY (`group_id`) REFERENCES `group` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=17 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('audit_user_to_group_permissions', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -384,7 +386,7 @@ CREATE TABLE IF NOT EXISTS `audit_user_to_user_permissions` (
PRIMARY KEY (`id`),
KEY `fk_audit_user_to_user_permissions_issuer_user_id` (`issuer_user_id`),
KEY `fk_audit_user_to_user_permissions_holder_user_id` (`holder_user_id`)
) ENGINE=InnoDB AUTO_INCREMENT=981 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('audit_user_to_user_permissions', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -437,7 +439,7 @@ CREATE TABLE IF NOT EXISTS `feedback` (
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
CONSTRAINT `feedback_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=815 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('feedback', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -503,7 +505,7 @@ CREATE TABLE IF NOT EXISTS `fsentries` (
CONSTRAINT `fsentries_ibfk_2` FOREIGN KEY (`parent_id`) REFERENCES `fsentries` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fsentries_ibfk_3` FOREIGN KEY (`shortcut_to`) REFERENCES `fsentries` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fsentries_ibfk_4` FOREIGN KEY (`associated_app_id`) REFERENCES `apps` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=37893498 DEFAULT CHARSET=latin1;
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('fsentries', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -555,7 +557,7 @@ CREATE TABLE IF NOT EXISTS `fsentry_versions` (
KEY `user_id` (`user_id`),
CONSTRAINT `fsentry_versions_ibfk_1` FOREIGN KEY (`fsentry_id`) REFERENCES `fsentries` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fsentry_versions_ibfk_2` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=24662498 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('fsentry_versions', 'id', '`id` bigint NOT NULL AUTO_INCREMENT');
@@ -623,7 +625,7 @@ CREATE TABLE IF NOT EXISTS `group` (
UNIQUE KEY `uid` (`uid`),
KEY `owner_user_id` (`owner_user_id`),
CONSTRAINT `group_ibfk_1` FOREIGN KEY (`owner_user_id`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=27 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('group', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -651,7 +653,7 @@ CREATE TABLE IF NOT EXISTS `jct_user_group` (
KEY `group_id` (`group_id`),
CONSTRAINT `jct_user_group_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `jct_user_group_ibfk_2` FOREIGN KEY (`group_id`) REFERENCES `group` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=3513197 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('jct_user_group', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -681,7 +683,7 @@ CREATE TABLE IF NOT EXISTS `kv` (
KEY `user_id` (`user_id`),
KEY `kkey_hash` (`kkey_hash`),
CONSTRAINT `kv_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=101649 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('kv', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -744,7 +746,7 @@ CREATE TABLE IF NOT EXISTS `notification` (
UNIQUE KEY `uid` (`uid`),
KEY `user_id` (`user_id`),
CONSTRAINT `notification_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=164238 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('notification', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -770,7 +772,7 @@ CREATE TABLE IF NOT EXISTS `old_app_names` (
KEY `app_uid` (`app_uid`),
KEY `old_app_names_app_name` (`name`),
CONSTRAINT `old_app_names_ibfk_1` FOREIGN KEY (`app_uid`) REFERENCES `apps` (`uid`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=45405 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('old_app_names', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -792,7 +794,7 @@ CREATE TABLE IF NOT EXISTS `per_user_credit` (
PRIMARY KEY (`id`),
UNIQUE KEY `user_id` (`user_id`),
CONSTRAINT `per_user_credit_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=388003 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('per_user_credit', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -847,7 +849,7 @@ CREATE TABLE IF NOT EXISTS `sessions` (
KEY `fk_sessions_user_id` (`user_id`),
KEY `uuid` (`uuid`),
CONSTRAINT `fk_sessions_user_id` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=2293466 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('sessions', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -875,7 +877,7 @@ CREATE TABLE IF NOT EXISTS `share` (
KEY `issuer_user_id` (`issuer_user_id`),
KEY `recipient_email` (`recipient_email`),
CONSTRAINT `share_ibfk_1` FOREIGN KEY (`issuer_user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=40 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('share', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -904,7 +906,7 @@ CREATE TABLE IF NOT EXISTS `storage_audit` (
PRIMARY KEY (`id`),
KEY `fk_storage_audit_user_id` (`user_id`),
CONSTRAINT `fk_storage_audit_user_id` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=22080 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('storage_audit', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -948,7 +950,7 @@ CREATE TABLE IF NOT EXISTS `subdomains` (
CONSTRAINT `subdomains_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `subdomains_ibfk_2` FOREIGN KEY (`root_dir_id`) REFERENCES `fsentries` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `subdomains_ibfk_3` FOREIGN KEY (`associated_app_id`) REFERENCES `apps` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=171443 DEFAULT CHARSET=utf8mb3;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('subdomains', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -983,7 +985,7 @@ CREATE TABLE IF NOT EXISTS `thread` (
KEY `idx_thread_uid` (`uid`),
CONSTRAINT `thread_ibfk_1` FOREIGN KEY (`parent_uid`) REFERENCES `thread` (`uid`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `thread_ibfk_2` FOREIGN KEY (`owner_user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=1987 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('thread', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -1058,6 +1060,7 @@ CREATE TABLE IF NOT EXISTS `user` (
`signup_server` varchar(255) DEFAULT NULL,
`metadata` json DEFAULT (json_object()),
`reputation` smallint DEFAULT '100',
`home` varchar(64) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uid` (`uuid`),
UNIQUE KEY `username` (`username`),
@@ -1090,7 +1093,7 @@ CREATE TABLE IF NOT EXISTS `user` (
KEY `idx_user_signup_origin` (`signup_origin`),
KEY `idx_user_signup_server` (`signup_server`),
CONSTRAINT `fk_user_referred_by` FOREIGN KEY (`referred_by`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=2987224 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('user', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -1149,6 +1152,7 @@ CALL _puter_add_col('user', 'signup_user_agent', '`signup_user_agent` varchar(51
CALL _puter_add_col('user', 'signup_origin', '`signup_origin` varchar(255) DEFAULT NULL');
CALL _puter_add_col('user', 'signup_server', '`signup_server` varchar(255) DEFAULT NULL');
CALL _puter_add_col('user', 'metadata', '`metadata` json DEFAULT (json_object())');
CALL _puter_add_col('user', 'home', '`home` varchar(64) DEFAULT NULL');
CALL _puter_add_col('user', 'reputation', '`reputation` smallint DEFAULT ''100''');
--
@@ -1218,7 +1222,7 @@ CREATE TABLE IF NOT EXISTS `user_oidc_providers` (
KEY `user_id` (`user_id`),
KEY `idx_user_oidc_providers_provider` (`provider`),
CONSTRAINT `user_oidc_providers_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=51584 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('user_oidc_providers', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -1319,7 +1323,7 @@ CREATE TABLE IF NOT EXISTS `user_update_audit` (
PRIMARY KEY (`id`),
KEY `fk_user_update_audit_user_id` (`user_id`),
CONSTRAINT `fk_user_update_audit_user_id` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=8078 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
CALL _puter_add_col('user_update_audit', 'id', '`id` int unsigned NOT NULL AUTO_INCREMENT');
@@ -1333,7 +1337,10 @@ CALL _puter_add_col('user_update_audit', 'new_username', '`new_username` varchar
CALL _puter_add_col('user_update_audit', 'reason', '`reason` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_col;
-- _puter_add_col is intentionally NOT dropped here: later migration files CALL
-- it as a one-line idempotent column add, and migrations replay on every boot
-- with no applied-state tracking, so the helper must outlive this file. The
-- DROP-before-CREATE at the top keeps this migration itself replay-safe.
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
@@ -0,0 +1,53 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0053. Adds the
-- `access_token_uid` reverse-lookup column on `sessions` so raw-uuid
-- revoke can find the matching session row when only the v2 token_uid
-- (no JWT) is presented.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_access_token_uid;
DELIMITER //
CREATE PROCEDURE _puter_sessions_access_token_uid()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'access_token_uid'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `access_token_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_access_token_uid'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_access_token_uid` (`access_token_uid`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_access_token_uid();
DROP PROCEDURE IF EXISTS _puter_sessions_access_token_uid;
@@ -0,0 +1,70 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Worker session uniqueness. Each Puter worker is a separately-deployed
-- code unit (its own subdomain), so one user can have many workers under
-- the same app — distinguished by `meta.worker_name`. The natural unique
-- key for an active worker session is therefore
-- (user_id, app_uid, worker_name), and app_uid is allowed NULL for
-- user-scoped workers that aren't bound to any specific app.
--
-- Implemented the same way as `app_unique_key` from mig_9: a VIRTUAL
-- generated column that's non-NULL only for the rows under the rule,
-- then a UNIQUE INDEX on the column. NULLs don't conflict in MySQL
-- UNIQUE indexes, so soft-revoked / non-worker rows fall out
-- automatically. IFNULL normalises NULL `app_uid` so two user-scoped
-- workers with the same name still dedupe. JSON_UNQUOTE strips the
-- JSON value quoting from JSON_EXTRACT so the concatenated key is a
-- plain string that matches the SELECT path's binding.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_worker_unique;
DELIMITER //
CREATE PROCEDURE _puter_sessions_worker_unique()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'worker_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `worker_unique_key` VARCHAR(550)
GENERATED ALWAYS AS (
IF(`kind` = 'worker' AND `revoked_at` IS NULL,
CONCAT(`user_id`, '|', IFNULL(`app_uid`, ''), '|',
IFNULL(JSON_UNQUOTE(JSON_EXTRACT(`meta`, '$.worker_name')), '')),
NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_worker_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_user_worker_active` (`worker_unique_key`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_worker_unique();
DROP PROCEDURE IF EXISTS _puter_sessions_worker_unique;
@@ -0,0 +1,50 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Add 'worker' to the `sessions.kind` ENUM. mig_11 already added the
-- worker uniqueness index and the application code in SessionStore
-- inserts rows with kind='worker', but the ENUM defined in mig_8 never
-- listed 'worker' as a permitted value. Under STRICT_TRANS_TABLES the
-- INSERT fails outright; under a relaxed sql_mode the value is coerced
-- to '' (and the worker SELECT-by-kind path then misses the row).
-- Mirrors SQLite migration 0056.
--
-- Idempotent: guarded against COLUMN_TYPE so re-running the directory is
-- a no-op once 'worker' is in the ENUM.
DROP PROCEDURE IF EXISTS _puter_sessions_kind_worker;
DELIMITER //
CREATE PROCEDURE _puter_sessions_kind_worker()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND COLUMN_NAME = 'kind'
AND FIND_IN_SET('worker', REPLACE(REPLACE(REPLACE(COLUMN_TYPE, 'enum(', ''), ')', ''), '''', '')) > 0
) THEN
ALTER TABLE `sessions`
MODIFY COLUMN `kind`
ENUM('web', 'app', 'access_token', 'asset', 'worker')
NOT NULL DEFAULT 'web';
END IF;
END//
DELIMITER ;
CALL _puter_sessions_kind_worker();
DROP PROCEDURE IF EXISTS _puter_sessions_kind_worker;
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
-- E.164 number collected during verification (indexed like `email`);
-- `requires_phone_verification` gates account use for low-reputation signups
-- (not indexed, mirroring `requires_email_confirmation`).
--
-- Idempotent: column adds use _puter_add_col (defined in mig_1, which leaves it
-- resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'phone', '`phone` varchar(20) DEFAULT NULL');
CALL _puter_add_col('user', 'requires_phone_verification', '`requires_phone_verification` tinyint(1) NOT NULL DEFAULT ''0''');
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_phone_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_phone'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_phone` (`phone`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_phone_index();
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification column. Mirrors SQLite migration 0059.
-- `requires_card_verification` gates account use for low-reputation signups;
-- the card itself never touches our DB, so this is the only column (not
-- indexed, mirroring `requires_phone_verification`).
--
-- Idempotent: the column add uses _puter_add_col (from mig_1) so the
-- directory replays safely.
CALL _puter_add_col('user', 'requires_card_verification', '`requires_card_verification` tinyint(1) NOT NULL DEFAULT ''0''');
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
-- is the Stripe card fingerprint (stable per card number) recorded when a user
-- clears card verification — the card sibling of `phone`, indexed like it so
-- admin tooling can find the accounts that verified with a given card. The card
-- itself never touches our DB, only Stripe's fingerprint for it.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'card_fingerprint', '`card_fingerprint` varchar(128) DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_card_fingerprint_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_card_fingerprint'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_card_fingerprint` (`card_fingerprint`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_card_fingerprint_index();
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
@@ -0,0 +1,46 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspended-at column. Mirrors SQLite migration 0061. `suspended_at` is when
-- the account was suspended, as unix seconds (NULL while not suspended) — the
-- timestamp sibling of the boolean `suspended` flag, indexed so the
-- signup-abuse harness can count an IP's recently-suspended accounts.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'suspended_at', '`suspended_at` bigint DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_user_suspended_at_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_suspended_at_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_suspended_at'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_suspended_at` (`suspended_at`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_suspended_at_index();
DROP PROCEDURE IF EXISTS _puter_add_user_suspended_at_index;
@@ -0,0 +1,36 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Admin-managed blocklist of app origins. Mirrors SQLite migration 0062.
-- An app whose `index_url` host (or a request origin) matches an entry is
-- denied access to Puter resources: it cannot obtain an app token and
-- already-issued app tokens are rejected on each request. `include_subdomains
-- = 1` also blocks every subdomain of `domain`. Enforced in AuthService via
-- AppOriginBlocklistService.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` lets the directory replay safely.
CREATE TABLE IF NOT EXISTS `blocked_app_origins` (
`id` INT NOT NULL AUTO_INCREMENT,
`domain` VARCHAR(255) NOT NULL,
`include_subdomains` TINYINT(1) NOT NULL DEFAULT 0,
`reason` TEXT DEFAULT NULL,
`created_by` VARCHAR(255) DEFAULT NULL,
`created_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_blocked_app_origins_domain` (`domain`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspension reason column. Mirrors SQLite migration 0063. Why an account was
-- suspended (NULL while not suspended) — companion to the boolean `suspended`
-- flag and `suspended_at` timestamp. Constrained at the application layer to a
-- fixed set of reasons (see extensions/admin suspension_reasons.js).
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations).
CALL _puter_add_col('user', 'suspended_reason', '`suspended_reason` VARCHAR(64) DEFAULT NULL');
@@ -0,0 +1,38 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Append-only log of admin moderation actions. Mirrors SQLite migration 0064.
-- Used to measure the abuse system's false-positive rate: `unsuspend` (admin
-- unblock) and `admin_create_user` are the false-positive signals; `suspend`
-- gives the denominator. Preserves history the `user` suspension columns lose
-- on unsuspend. `created_at` is unix seconds (matching `user.suspended_at`).
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` lets the directory replay safely.
CREATE TABLE IF NOT EXISTS `abuse_moderation_events` (
`id` INT NOT NULL AUTO_INCREMENT,
`action` VARCHAR(32) NOT NULL,
`target_user_id` BIGINT DEFAULT NULL,
`target_username` VARCHAR(255) DEFAULT NULL,
`admin_username` VARCHAR(255) DEFAULT NULL,
`reason` TEXT DEFAULT NULL,
`source` VARCHAR(64) DEFAULT NULL,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_abuse_moderation_events_created_at` (`created_at`),
KEY `idx_abuse_moderation_events_action` (`action`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,48 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- User-to-developer feedback for apps that opt in. Mirrors SQLite migration
-- 0065. Opt-in is the new `apps.feedback_enabled` column (developer-writable
-- through the regular `puter.apps.update` path). Each row of `app_feedback`
-- is one message a signed-in user submitted through the GUI feedback dialog;
-- a copy is emailed to the app owner unless the per-app daily email cap
-- suppressed it (`email_sent` records which). `app_uid` is denormalized
-- alongside `app_id` so rows stay attributable after an app is deleted.
-- `created_at` is unix seconds.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- always runs first) and the table uses `CREATE TABLE IF NOT EXISTS`, so the
-- directory can replay safely.
CALL _puter_add_col('apps', 'feedback_enabled', '`feedback_enabled` tinyint(1) DEFAULT ''0''');
CREATE TABLE IF NOT EXISTS `app_feedback` (
`id` INT NOT NULL AUTO_INCREMENT,
`uid` CHAR(36) NOT NULL,
`app_id` INT NOT NULL,
`app_uid` CHAR(40) NOT NULL,
`user_id` INT NOT NULL,
`message` TEXT NOT NULL,
`source_env` VARCHAR(16) DEFAULT NULL,
`source_origin` VARCHAR(2048) DEFAULT NULL,
`email_sent` TINYINT(1) NOT NULL DEFAULT 0,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uq_app_feedback_uid` (`uid`),
KEY `idx_app_feedback_app_created` (`app_id`, `created_at`),
KEY `idx_app_feedback_user_created` (`user_id`, `created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,128 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce "at most one account owns an email address". Mirrors SQLite
-- migration 0066.
--
-- `user.email` is deliberately not UNIQUE: several rows may legitimately hold
-- the same address while unconfirmed (admin-provisioned placeholders, signups
-- that were never confirmed, temp accounts on their way to becoming real). What
-- must never happen is two rows both *owning* an address — owning meaning the
-- row is confirmed, or holds a password and so can drive password recovery for
-- that inbox.
--
-- Signup, save-account, change-email, OIDC and admin provisioning each check for
-- an owner before writing, but a check and an insert are not one operation: two
-- requests can both read "free" and both write. This index is what actually
-- holds the invariant; the application checks just produce a nicer error most of
-- the time.
--
-- SQLite expresses that with a partial index. MySQL has none, so the predicate
-- lives in a generated column that evaluates to NULL for every row that does not
-- own its address — and NULLs do not collide in an InnoDB unique index, which is
-- exactly the "unlimited unconfirmed placeholders" behaviour we need.
--
-- The column is VIRTUAL, not STORED, on purpose: adding a stored generated
-- column rebuilds the table, while a virtual one is a metadata-only change and
-- the index that follows builds INPLACE. On a `user` table of any size with
-- read replicas attached, that is the difference between a routine change and an
-- outage. The ALGORITHM/LOCK clauses are spelled out so a server that cannot
-- honour them refuses the statement instead of quietly copying the table.
--
-- Matching is on the canonical address so provider aliases
-- (`[email protected]` vs `[email protected]`) collide. `clean_email` is
-- written on every modern write path; the COALESCE covers rows old enough to
-- predate the column. Run the `clean_email` backfill before this migration or
-- alias collisions among those rows go unnoticed.
--
-- Idempotent: both steps are guarded on INFORMATION_SCHEMA so the directory
-- replays safely.
--
-- If the index creation fails with ER_DUP_ENTRY, the DB already contains
-- duplicate owners. Collapse them first (admin → One-off Jobs → Collapse
-- Duplicate Emails) — there is no safe automatic merge of two accounts.
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
DELIMITER //
CREATE PROCEDURE _puter_add_owned_email()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND COLUMN_NAME = 'owned_email'
) THEN
ALTER TABLE `user`
ADD COLUMN `owned_email` VARCHAR(256)
CHARACTER SET latin1 COLLATE latin1_swedish_ci
GENERATED ALWAYS AS (
CASE
WHEN `email` IS NOT NULL
AND (`email_confirmed` = 1 OR `password` IS NOT NULL)
THEN COALESCE(`clean_email`, LOWER(`email`))
ELSE NULL
END
) VIRTUAL,
ALGORITHM=INSTANT;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_owned_email'
) THEN
ALTER TABLE `user`
ADD UNIQUE KEY `idx_user_owned_email` (`owned_email`),
ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_owned_email();
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
-- One Puter account per external identity. OIDCStore.link already assumes this
-- constraint exists — it catches the unique violation to tell "re-linking the
-- same account" apart from "this sub belongs to someone else" — but the table
-- never actually had it, so two concurrent first-time logins could each create
-- an account and each link the same sub. Subsequent logins then resolved to
-- whichever row came back first.
--
-- Dedupe `user_oidc_providers` before applying this: keep the lowest `id` per
-- (provider, provider_sub) and point it at the account the collapse job kept.
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
DELIMITER //
CREATE PROCEDURE _puter_add_oidc_sub_unique()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_oidc_providers'
AND INDEX_NAME = 'idx_user_oidc_provider_sub'
) THEN
ALTER TABLE `user_oidc_providers`
ADD UNIQUE KEY `idx_user_oidc_provider_sub` (`provider`, `provider_sub`);
END IF;
END//
DELIMITER ;
CALL _puter_add_oidc_sub_unique();
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
@@ -0,0 +1,106 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Grow `share` from a pending-email-invite table into the index of active
-- shares. See sqlite/0067_share_entries.sql for the column rationale.
--
-- Idempotent: columns go through _puter_add_col (from mig_1), indexes and
-- foreign keys through the guarded procedure below. There is no per-file
-- applied-state tracking, so every statement must tolerate a re-run.
CALL _puter_add_col('share', 'holder_user_id', '`holder_user_id` int unsigned DEFAULT NULL');
CALL _puter_add_col('share', 'fsentry_id', '`fsentry_id` int unsigned DEFAULT NULL');
CALL _puter_add_col('share', 'mode', '`mode` varchar(20) DEFAULT NULL');
CALL _puter_add_col('share', 'applied_at', '`applied_at` timestamp NULL DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
DELIMITER //
CREATE PROCEDURE _puter_add_share_index_constraints()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_holder` (`holder_user_id`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_fsentry'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_fsentry` (`fsentry_id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_entry_issuer'
) THEN
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_holder_entry_issuer`
(`holder_user_id`, `fsentry_id`, `issuer_user_id`);
END IF;
-- Retiring a deleted node's grants looks them up by permission text
-- (`fs:<uuid>` and `fs:<uuid>:%`); the subject lives in that text rather
-- than a column, so no foreign key can cascade it. The primary key is
-- (issuer, holder, permission), which never puts `permission` first, so
-- without this both the equality and the left-anchored LIKE degrade to a
-- full scan on the delete path. The column is ascii varchar(255), well
-- inside InnoDB's key limit, so it is indexed whole.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_to_user_permissions'
AND INDEX_NAME = 'idx_user_to_user_permissions_permission'
) THEN
ALTER TABLE `user_to_user_permissions`
ADD INDEX `idx_user_to_user_permissions_permission` (`permission`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_holder_user_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_holder_user_fk`
FOREIGN KEY (`holder_user_id`) REFERENCES `user` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
-- The cascade that retires a share with its file.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_fsentry_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_fsentry_fk`
FOREIGN KEY (`fsentry_id`) REFERENCES `fsentries` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END//
DELIMITER ;
CALL _puter_add_share_index_constraints();
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- One user refusing contact from another. See sqlite/0068_user-block.sql for
-- the rationale. `created_at` is unix seconds.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- so the directory can replay safely.
CREATE TABLE IF NOT EXISTS `user_block` (
`id` INT NOT NULL AUTO_INCREMENT,
`blocker_user_id` INT UNSIGNED NOT NULL,
`blocked_user_id` INT UNSIGNED NOT NULL,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_user_block_pair` (`blocker_user_id`, `blocked_user_id`),
CONSTRAINT `fk_user_block_blocker` FOREIGN KEY (`blocker_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_user_block_blocked` FOREIGN KEY (`blocked_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,61 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Widen `notification.shown` and `notification.acknowledged` to hold the unix
-- second the store has been writing to them since the backend rework.
--
-- Both arrived from the v1 schema as `tinyint(1)`, where they were flags. The
-- rework changed the writes to timestamps and sqlite (`INTEGER`) and postgres
-- (`bigint`) took them; only mysql was left too narrow, so every `markShown`
-- and `markAcknowledged` there fails with ER_WARN_DATA_OUT_OF_RANGE and the
-- column stays NULL. Dismissing a notification therefore never sticks, and one
-- already delivered is re-sent on every reconnect.
--
-- No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a legacy
-- `1` keeps meaning "yes" once widened.
--
-- Guarded on the current type rather than run unconditionally. Changing a
-- column type copies the table, and this one grows without bound — a migration
-- directory that replays on every boot must not rebuild it every time.
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
DELIMITER //
CREATE PROCEDURE _puter_widen_notification_stamps()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND COLUMN_NAME = 'shown'
AND DATA_TYPE = 'tinyint'
) THEN
ALTER TABLE `notification` MODIFY `shown` BIGINT DEFAULT NULL;
END IF;
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND COLUMN_NAME = 'acknowledged'
AND DATA_TYPE = 'tinyint'
) THEN
ALTER TABLE `notification` MODIFY `acknowledged` BIGINT DEFAULT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_notification_stamps();
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
@@ -0,0 +1,22 @@
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
-- `dangerous`. No migration creates them — they were added by hand back when
-- hardcoded permissions were keyed by group name. That map is now a flat
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
-- so a group nothing looks up grants nothing, and the rows are dead weight.
--
-- Guarded, and deliberately so: `user_to_group_permissions.group_id` and
-- `jct_user_group.group_id` both cascade on delete, so removing a group that
-- still carries permissions or members would silently revoke them from every
-- member. Only a group with neither is dropped. A group that survives this
-- migration has dependents and needs a deliberate decision, not a sweep --
-- query `user_to_group_permissions` by `group_id` to see what it holds.
--
-- The joins rather than `NOT IN` subqueries: MySQL will not read from the table
-- it is deleting from inside a subquery on the same statement.
DELETE g FROM `group` g
LEFT JOIN `user_to_group_permissions` p ON p.`group_id` = g.`id`
LEFT JOIN `jct_user_group` j ON j.`group_id` = g.`id`
WHERE JSON_UNQUOTE(JSON_EXTRACT(g.`extra`, '$.name'))
IN ('freeai', 'experimental', 'dangerous')
AND p.`group_id` IS NULL
AND j.`group_id` IS NULL;
@@ -0,0 +1,80 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Give `notification` a scope tuple: what the row is (`type`), the app it is
-- about (`app_uid`, NULL for platform), and who the recipient is in relation
-- to it (`audience`). Attribution lived inside the `value` JSON as two
-- free-form markers, `source` and `template`, that could disagree; these
-- columns replace them and are what a scoped list can be indexed on.
--
-- `app_uid` is char(40) to match `apps`.`uid`, which is `app-` plus a uuid.
--
-- Idempotent: columns go through `_puter_add_col` (mig_1), the index through
-- the guarded procedure. There is no per-file applied-state tracking, so every
-- statement here tolerates a re-run.
CALL _puter_add_col('notification', 'app_uid', '`app_uid` char(40) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
CALL _puter_add_col('notification', 'audience', '`audience` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT ''account''');
CALL _puter_add_col('notification', 'type', '`type` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT ''''');
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_scope_index()
BEGIN
-- Serves both the scoped list and the unacknowledged count.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_scope'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_scope`
(`user_id`, `audience`, `app_uid`, `acknowledged`);
END IF;
END //
DELIMITER ;
CALL _puter_add_notification_scope_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_index;
-- Backfill from the markers the payload actually carried. `type = ''` is the
-- not-yet-classified state, so re-running matches nothing the first pass
-- claimed, and a row matching no marker keeps the defaults (`''` / 'account')
-- and reads as legacy.
UPDATE `notification`
SET `type` = 'share.received', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.template')) = 'file-shared-with-you';
UPDATE `notification`
SET `type` = 'share.claimed', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.template')) = 'file-shared-before-you-joined';
-- Worker rows carried only `source: 'worker'` and a title, so the app they
-- belong to is unrecoverable and they stay unattributed. The title prefix is
-- the sole surviving signal of which way the deploy went.
UPDATE `notification`
SET `audience` = 'developer',
`app_uid` = NULL,
`type` = CASE
WHEN JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.title')) LIKE 'Successfully deployed %'
THEN 'app.worker.deployed'
ELSE 'app.worker.deployFailed'
END
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.source')) = 'worker';
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index `notification`.`created_at`. The retention sweep selects by age alone
-- and by nothing else, so no existing index on the table narrows it — every
-- pass would otherwise scan the whole table to find the oldest few hundred
-- rows.
--
-- Idempotent: the guarded procedure, as mig_26. There is no per-file
-- applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_add_notification_created_at_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_created_at_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_created_at'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_created_at`
(`created_at`);
END IF;
END //
DELIMITER ;
CALL _puter_add_notification_created_at_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_created_at_index;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Home region column. Mirrors SQLite migration 0074. Home region this
-- account's data belongs in — a node id, not a storage region. NULL for
-- accounts that predate the column, which fall back to `signup_server`.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations).
CALL _puter_add_col('user', 'home', '`home` VARCHAR(64) DEFAULT NULL');
@@ -0,0 +1,59 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Subscriptions that outlive the connection that made them. See
-- sqlite/0075_event-subscriptions.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_23. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `event_subscriptions` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`sub_id` varchar(80) NOT NULL,
`token` varchar(255) NOT NULL,
`owner_user_id` int unsigned NOT NULL,
`holder_user_id` int unsigned NOT NULL,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion. No foreign key: a row outlives the
-- app that made it, which is what keeps it revocable afterwards.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci
DEFAULT NULL,
`subject` varchar(4096) NOT NULL,
`anchor_uid` char(36) NOT NULL,
`anchor_path` varchar(4096) NOT NULL,
`match` varchar(1024) DEFAULT NULL,
`delivery` varchar(16) NOT NULL,
`ops` varchar(64) DEFAULT NULL,
`handler_name` varchar(128) DEFAULT NULL,
`targets` json NOT NULL,
`context` text,
`permission` varchar(32) NOT NULL,
`suspended_at` bigint DEFAULT NULL,
`suspended_reason` varchar(64) DEFAULT NULL,
`expires_at` bigint DEFAULT NULL,
`created_at` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_subscriptions_sub_id` (`sub_id`),
KEY `idx_event_subscriptions_token` (`token`),
KEY `idx_event_subscriptions_holder` (`holder_user_id`, `app_uid`),
KEY `idx_event_subscriptions_owner` (`owner_user_id`),
CONSTRAINT `fk_event_subscriptions_owner` FOREIGN KEY (`owner_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_event_subscriptions_holder` FOREIGN KEY (`holder_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,43 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions`.`anchor_uid` was sized for a filesystem node uuid.
-- A key-value subscription anchors on an app instead, and `apps`.`uid` is
-- `char(40)`, so the column has to hold one.
--
-- Idempotent: the guarded procedure, as mig_27. There is no per-file
-- applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_widen_event_subscriptions_anchor_uid;
DELIMITER //
CREATE PROCEDURE _puter_widen_event_subscriptions_anchor_uid()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'anchor_uid'
AND CHARACTER_MAXIMUM_LENGTH < 40
) THEN
ALTER TABLE `event_subscriptions`
MODIFY COLUMN `anchor_uid` varchar(40) NOT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_event_subscriptions_anchor_uid();
DROP PROCEDURE IF EXISTS _puter_widen_event_subscriptions_anchor_uid;
Loaded 100 of 2161 files, more files were not shown because too many files have changed in this diff. Show more