Compare commits

...
64 Commits
Author SHA1 Message Date
Daniel Salazar 0405de30bd update taskbar apps (#3286) 2026-06-22 14:26:35 -07:00
Nariman Jelveh 40c2799b06 Add 'builder' to recommended apps list
Include 'builder' in the RECOMMENDED_APP_NAMES array in src/backend/services/apps/RecommendedAppsService.ts so the RecommendedAppsService treats the builder app as a recommended application and surfaces it accordingly.
2026-06-22 14:22:27 -07:00
Daniel Salazar 4bcdeefe68 chore: bump up deps (#3285) 2026-06-22 13:58:45 -07:00
Daniel Salazar b6ba6dd90e fix: [PUT-1199] more prelude signals (#3282) 2026-06-22 08:16:59 -07:00
Daniel Salazar 954f8b7b8d fix: puter self host guides + default bug (#3281)
* fix: oss dev center html_encode hardening

* fix: puter self host guides + default bug
2026-06-21 22:32:43 -07:00
Neal Shah 24d700673c Compaction support for OpenAI and Anthropic (#3279)
* Alpha: compaction support for OpenAI and Anthropic

* update lock

* fix billing for anthropic compactions

* Fix max_tokens bug in together provider

* Fix responses compaction
2026-06-21 21:57:56 -04:00
c6d64029e0 feat: meter Gemini thinking tokens and grounding requests (#3178)
* feat: meter Gemini thinking tokens and grounding requests

- Thinking tokens: Extracted from standard completion tokens to ensure they are billed accurately at the correct model-specific rate.
- Grounding requests: Added flat-fee metering for Google Search by tracking grounding_metadata across both streaming and non-streaming responses.
- Pricing updates: Corrected stale rates for Gemini 2.5 Flash output, cached tokens, thinking tokens, and grounding requests.

* fix: correct Gemini 2.x metering rates and harden grounding capture

Pricing corrections (verified against ai.google.dev/gemini-api/docs/pricing):
- gemini-2.5-flash output is $2.50/M, not $1.00/M: restore
  completion_tokens to 250 and bill thinking_tokens at the same output
  rate (250). The previous 100 under-billed output ~60%, and this is the
  provider's default model.
- gemini-2.5-flash cache read is $0.03/M: restore cached_tokens to 3
  (the 7.5 value over-billed).
- Grounding with Google Search is $35 / 1,000 requests for Gemini 2.x
  models and $14 / 1,000 for 3.x. Set grounding_requests to 3_500_000
  for gemini-2.0-flash, gemini-2.5-flash, gemini-2.5-flash-lite and
  gemini-2.5-pro; 3.x models keep 1_400_000.

Streaming robustness:
- In create_chat_stream_handler, don't let a later extra_content chunk
  without grounding_metadata overwrite an earlier one that carried it,
  so grounding requests are still metered.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-20 15:08:38 -07:00
jelveh 40ff866598 Add 'butler' to recommended apps
Include 'butler' in the RECOMMENDED_APP_NAMES array in RecommendedAppsService so it appears in the recommended apps list. Updates src/backend/services/apps/RecommendedAppsService.ts.
2026-06-19 15:50:57 -07:00
Nariman Jelveh 36888e35d3 Add phone verification UI improvements and helpers (#3274)
Revamps the phone verification dialog and adds reusable phone helper modules.

- Heavily refactors UIWindowPhoneVerificationRequired.js to add a searchable inline country picker, national-number live formatting, country autodetect for "+" input, WebOTP auto-fill support, resend cooldown timer, accessibility improvements, proper teardown/cleanup, and last-used country persistence.
- Introduces helpers/src/helpers/country_codes.js (get_country_list, flag emoji) which builds a localized list of dialable countries from libphonenumber-js and Intl.DisplayNames.
- Introduces helpers/src/helpers/phone.js with thin wrappers around libphonenumber-js: format_phone_as_you_type, inspect_phone, phone_example_for, detect_country_from_input.
- Adds English i18n strings for the new UI in src/gui/src/i18n/translations/en.js.

These changes keep submission canonicalization using libphonenumber-js (E.164), improve UX for selecting/formatting numbers, and make the country list and phone utilities reusable across the app.
2026-06-19 10:28:30 -07:00
Reynaldi Chernando 30432b931a Add PUT and DELETE to preflight allowed methods (#3278) 2026-06-19 12:35:29 -04:00
Reynaldi Chernando e2174a14cf Update router cors preflight docs (#3277) 2026-06-19 20:50:41 +07:00
Daniel Salazar d3cb89d8c9 fix: import in extensions (#3276) 2026-06-18 21:30:00 -07:00
Reynaldi Chernando c8625b3396 Release puterjs npm (#3275)
* Release puterjs npm

* add package lock
2026-06-18 18:26:39 -07:00
Reynaldi Chernando b576f01d12 Sync Puter.js types and docs (#3194)
* AI chat types

* ai voice types

* update tts docs and jsdoc

* stt types and docs

* txt2img docs and types

* txt2vid and img2txt

* apps

* signin

* perms

* fs

* kv

* hosting

* workers

* net

* ui

* jsdoc
2026-06-18 09:34:13 -07:00
Vyasdev a87cafee3c Revise bug report and feature request instructions (#3220)
Updated the wording for reporting bugs and feature requests in Japanese.
2026-06-18 09:27:55 -07:00
Reynaldi Chernando 1c51fde163 Add minimax m3 direct integration + cleanup model list (#3263) 2026-06-18 10:12:44 -04:00
Daniel Salazar b05699db77 fix: AGENTS.md cleanup (#3273) 2026-06-17 17:07:20 -07:00
Daniel Salazar 163e48ab27 feat: viz over user verification types (#3271) 2026-06-17 14:21:55 -07:00
Reynaldi Chernando 0b41d992a4 Add glm 5.2 direct integration (#3269) 2026-06-17 09:40:10 -04:00
Miika Kuisma c2ddfd6867 Merge pull request #3270 from HeyPuter/fix-file-dialog
puter.js - fix issue where content of file dialog wasn't visible
2026-06-17 13:08:43 +03:00
Miika Kuisma 81b9dc1f06 fix issue where content of file dialog wasn't visible 2026-06-17 13:07:13 +03:00
Daniel Salazar 5506ddb2c2 feat: fp cleanup and better devex (#3268) 2026-06-16 22:48:54 -07:00
Neal Shah c74469b01a prelude RCS (#3267) 2026-06-15 19:08:30 -04:00
Daniel Salazar b706693f82 wip: hardening (#3266) 2026-06-15 16:07:44 -07:00
Daniel SalazarandNeal Shah cd800148d5 feat: optional additional verification gates (#3262)
* phone number verificiation initial

* finish phone verification

* feat: add card gate for signups

* chore: npm

* fix: modal order and wording

* fix:wording

---------

Co-authored-by: Neal Shah <[email protected]>
2026-06-15 09:16:00 -07:00
Ricardo Sawir 900b0d7aa6 fix: correct puter container healthcheck URL (#3255)
The healthcheck was using 'puter.localhost:4100/test' which fails for
two reasons:
- 'puter.localhost' does not resolve inside the container; the service
  listens on all interfaces at port 4100, so 'localhost' works
- '/test' endpoint does not exist; use '/' which serves the shell

This caused the container to be perpetually marked as unhealthy.

Fixes #3191
2026-06-13 23:17:44 -07:00
Daniel Salazar 7e21c1f888 fix: fs metadata sanitation (#3257) 2026-06-12 22:56:06 -07:00
Daniel Salazar 00d6f9c230 fix: disallow unverified outlook emails (#3256) 2026-06-12 18:52:24 -07:00
jelveh 4c03080efa Update mcp.md 2026-06-12 16:55:37 +02:00
jelveh 432a11d289 Add Cursor MCP docs and icons 2026-06-12 16:37:53 +02:00
jelveh e9e233404b Add icons for Claude, Codex and OpenCode 2026-06-12 13:20:15 +02:00
jelveh 8101d42604 Capitalize OpenCode in MCP docs 2026-06-12 13:16:41 +02:00
jelveh 88cedd43bc Add opencode MCP example to docs 2026-06-12 13:11:24 +02:00
Daniel Salazar d3a2934d9b feat: fingerprinting for signup checks (#3254) 2026-06-12 00:59:21 -07:00
Daniel Salazar 13a36b6199 fix: secure http calls (#3252) 2026-06-11 13:43:58 -07:00
Daniel Salazar a155464d46 fix: more data for puter site serve event (#3251) 2026-06-10 23:27:13 -07:00
Daniel Salazar 40d1c998bb feat: pass args to all events (#3248)
* feat: pass args to all events

* fix: alias app joining

* fix: actor in event
2026-06-10 14:13:04 -07:00
Daniel Salazar b4837d49b6 perf: some perf gains from fable (#3249) 2026-06-10 13:26:35 -07:00
ProgrammerIn-wonderland e23042b42e add preperation for reputation score persistence (#3250) 2026-06-10 16:09:42 -04:00
Daniel Salazar 240a733285 sec: misc fable hardening (#3244)
* sec: misc fable hardening

* more fixes

* more fixes

* fix: cors issue
2026-06-10 11:19:41 -07:00
Reynaldi Chernando bb202be12c Minor clarification local webserver in deployment (#3247) 2026-06-10 23:04:42 +07:00
Reynaldi Chernando 8c4cb601c9 Add cli docs (#3246) 2026-06-10 20:53:53 +07:00
Reynaldi Chernando 64a7c7b7c4 Add mcp docs (#3245) 2026-06-10 20:31:31 +07:00
wwb1942 2dd2887ff7 ci: add docs build workflow (#3235)
* ci: add docs build workflow

Add CI workflow that runs npm ci and npm run build in src/docs

whenever files under src/docs change in a pull request.

Closes #3225

* ci: also trigger docs build when workflow file changes

* ci: address docs build workflow review
2026-06-10 10:53:40 +07:00
ProgrammerIn-wonderland 4bbe8a2789 add protections for PUT-1094 (#3243) 2026-06-09 22:29:57 -04:00
Dilan Melvin T 3e9ceb673b fix: coerce Mistral image_url parts from object to string (#3177)
* fix: coerce Mistral image_url parts from object to string

Mistral's API expects image_url content parts to be a plain string URL,
not the OpenAI-style { url: string } object that process_input_messages
produces. Without this coercion, sending an image to any Mistral model
that supports vision (mistral-small, mistral-medium, mistral-large,
ministral-*) results in a request error from the Mistral SDK.

Add #coerceImageUrls() as a private method on MistralAIProvider that
maps { type: 'image_url', image_url: { url } } -> { type: 'image_url',
image_url: url } for every content part in every message. Messages with
plain string content are left untouched, as are parts whose image_url is
already a string.

Add four unit tests covering: object-to-string coercion, already-flat
strings, plain string message content, and mixed text+image content.

* style: use plain ASCII dashes in comment section divider
2026-06-09 20:12:44 -04:00
Daniel Salazar 9c4d1ef535 fix: misc bugs + new middleware for before/after routes (#3242) 2026-06-09 17:05:56 -07:00
ProgrammerIn-wonderland bf57518262 MCP: assert worker_exec ends with puter.work (#3241) 2026-06-09 17:43:34 -04:00
Reynaldi Chernando d48fb11de6 Add claude fable 5 (#3238) 2026-06-09 14:05:31 -04:00
ProgrammerIn-wonderland 7ed87f760b make max tokens less lenient (#3240) 2026-06-09 14:03:13 -04:00
Reynaldi Chernando ee37be16f0 Add cli deployment docs (#3239) 2026-06-10 00:40:19 +07:00
Reynaldi Chernando a8cfaeefbd Add subdomain and workers github actions deployment (#3226)
* Add subdomain and workers github actions deployment

* minor adjust
2026-06-09 23:35:34 +07:00
Badri467 58cb8ee048 skip github actions backend for docs only changes (#3236) 2026-06-09 23:28:48 +07:00
Daniel Salazar a72fbd1d47 fix: inform acl of full access flag (#3237) 2026-06-09 08:04:36 -07:00
Daniel Salazar f6408db74e fix: access tokens with full accesses allowed through with new middleware gate (#3234) 2026-06-08 18:47:07 -07:00
ProgrammerIn-wonderland df20993d8b set azure openai as the default (#3233) 2026-06-08 20:33:41 -04:00
ProgrammerIn-wonderland 708fe1bc4e azure ai provider (#3232) 2026-06-08 18:31:49 -04:00
Daniel Salazar e782fa832e fix: kv costs (#3231) 2026-06-08 13:32:36 -07:00
ProgrammerIn-wonderland c947c2c989 fix mcp server cloudfront cache bug (#3230) 2026-06-08 15:42:49 -04:00
Daniel Salazar 3de4613b7d Revert "feat (PUT-1091): have gui tokens labeled and be different than reques…" (#3229)
This reverts commit 1335b46404.
2026-06-08 11:15:24 -07:00
Daniel Salazar def7290c48 feat: emit user.deleted event on user deletion (#3228) 2026-06-08 10:30:24 -07:00
Daniel Salazar 1335b46404 feat (PUT-1091): have gui tokens labeled and be different than requestor ones (#3222)
* feat (PUT-1091): have gui tokens labeled and be different than creator ones

* fix: errors with auth token
2026-06-08 09:03:02 -07:00
Reynaldi Chernando 0e15ac98a9 Update cli readme and set version (#3227) 2026-06-08 22:23:32 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI f127b2509d Add docs fixes for deployment and workers pages (#3223)
* Add docs fixes for deployment and workers pages

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* handle router edge case

* explicit info about wildcard requiring name

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-08 17:43:00 +07:00
237 changed files with 21361 additions and 4048 deletions

No files matched your search

+2
View File
@@ -3,6 +3,8 @@ name: Backend Tests
on:
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- 'src/docs/**'
permissions:
contents: read
+27
View File
@@ -0,0 +1,27 @@
name: Docs Build
on:
pull_request:
paths:
- 'src/docs/**'
jobs:
build-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build docs
run: npm run build
working-directory: src/docs
+2 -2
View File
@@ -64,9 +64,9 @@ Keep comments light. Prefer self-documenting code — clear names, small functio
- The _why_ is non-obvious (a hidden constraint, a workaround, a subtle invariant).
- A non-trivial usage detail would otherwise trip the next reader.
Don't restate what the code already says. Don't write comments that reference the current task or PR — those rot.
Don't restate what the code already says. Don't write comments that reference the current task or PR or version of something — those rot.
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the *why* in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the _why_ in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
**Use plain ASCII `-` in comment section dividers, not box-drawing characters.** Write `// -- Section name --`, not `// ── Section name ──`. The `─` (U+2500) character renders inconsistently in some editors and diff tools and adds nothing over `-`.
+4
View File
@@ -260,6 +260,10 @@
// ─ Chat / completion ─
"claude": { "apiKey": "" },
"openai-completion": { "apiKey": "" },
"azure-openai": {
"apiKey": "",
"apiURL": ""
},
"gemini": { "apiKey": "" },
"groq": { "apiKey": "" },
"deepseek": { "apiKey": "" },
+1 -1
View File
@@ -104,7 +104,7 @@ Puterは[**puter.com**](https://puter.com)でホストサービスとして利
メンテナーやコミュニティと以下のチャンネルを通じてつながりましょう:
- バグ報告や機能リクエストがありますか? [issueを開く](https://github.com/HeyPuter/puter/issues/new/choose) してください。
- バグ報告や機能リクエストは [新規Issue](https://github.com/HeyPuter/puter/issues/new/choose) まで
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
+26
View File
@@ -199,6 +199,32 @@ Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.local", ... } }
```
## Running behind your own reverse proxy
The bundled `puter-nginx` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Caddy, Traefik, HAProxy, a cloud load balancer, another nginx). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
You can keep the bundled nginx as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
**The rules that actually matter** (getting any of these wrong is what causes the redirect loops and "Invalid Host header" failures people hit):
1. **Don't rewrite the `Host` header.** Puter routes entirely on Host — `api.<domain>`, `site.<domain>`, `app.<domain>` are all distinguished by the incoming host. Forward the original host through unchanged. Do **not** rewrite external hostnames to an internal name like `puter.local`; that forces you to remap every subdomain by hand and still breaks signed-URL and CORS checks. Most proxies preserve Host by default — just don't override it.
2. **The external domain must equal `domain` in your `config.json`.** If users reach the box at `puter.example.com`, then `domain` must be `puter.example.com` and the hosting domains must be its real subdomains (`site.puter.example.com`, `app.puter.example.com`, …) — exactly as the installer/`config.json` lays them out. Puter rejects hosts it doesn't recognize with `Invalid Host header`.
3. **Set `protocol` to match the public scheme.** When your proxy terminates TLS, set `"protocol": "https"` (installer: `PUTER_PROTOCOL=https`). Puter builds its origins, redirects, signed S3 URLs, and OIDC callback URLs from this — leave it `http` behind an HTTPS proxy and you get redirect loops, broken logins, and mixed-content errors.
4. **Forward the standard proxy headers.** Puter needs:
- `Host` — the original request host (rule 1).
- `X-Forwarded-Proto` — the **external** scheme (`https`), so Puter knows TLS was terminated upstream.
- `X-Forwarded-For` — the real client IP (rate limiting + audit logs).
- `Upgrade` / `Connection` — passed through for WebSocket / socket.io upgrades, or the realtime connection silently fails.
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled nginx if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
6. **Route the wildcard to Puter.** Your proxy must forward `*.<domain>` (covering `api.*`, `app.*`, and `s3.<domain>`) and `*.site.<domain>` (and any other hosting domains) to Puter — same wildcard DNS as Step 2. Puter does the per-subdomain routing internally; the proxy just needs to hand it the traffic with the Host intact.
With those in place the rest of the stack is unchanged — `docker compose up -d` as below.
## Step 4 — Bring it up
```bash
+1 -1
View File
@@ -264,7 +264,7 @@ services:
# Persistent runtime data (anything your config points at /var/puter).
- ./puter/data/puter:/var/puter:z
healthcheck:
test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
test: wget --no-verbose --tries=1 --spider http://localhost:4100/ || exit 1
interval: 30s
timeout: 3s
retries: 3
+1 -1
View File
@@ -71,6 +71,6 @@ export const handleInstalledApps = async (
extension.get(
'/installedApps',
{ subdomain: 'api', requireUserActor: true },
{ subdomain: 'api', requireUserActor: true, allowFullAccessToken: true },
handleInstalledApps,
);
+1 -1
View File
@@ -1,4 +1,3 @@
import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import {
@@ -6,6 +5,7 @@ import {
driversContainers,
} from '@heyputer/backend/src/exports';
import { extension } from '@heyputer/backend/src/extensions';
import type { Request, Response } from 'express';
const services = extension.import('service');
const clients = extension.import('client');
+3
View File
@@ -72,6 +72,9 @@ export const handleWhoami = async (
unconfirmed_email: user.email,
email_confirmed: user.email_confirmed || user.username === 'admin',
requires_email_confirmation: user.requires_email_confirmation,
phone: user.phone,
requires_phone_verification: user.requires_phone_verification,
requires_card_verification: user.requires_card_verification,
desktop_bg_url: user.desktop_bg_url,
desktop_bg_color: user.desktop_bg_color,
desktop_bg_fit: user.desktop_bg_fit,
+22 -10
View File
@@ -16,11 +16,20 @@
# the compose file + brings the stack up. Set PUTER_FORCE=1 to overwrite.
#
# Tunable env vars:
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_PROTOCOL public scheme: http | https (default: http)
# Set https when a TLS-terminating reverse proxy (Caddy,
# Traefik, nginx, a cloud LB) sits in front — Puter then
# builds https:// origins, S3 URLs, and redirects. See
# "Running behind your own reverse proxy" in doc/self-hosting.md.
# PUTER_TRUST_PROXY number of reverse-proxy hops in front (default: 1)
# 1 = the bundled nginx (or a single external proxy);
# 2 = two hops (e.g. Cloudflare → your proxy → Puter).
# PUTER_ENV prod | dev (default: prod)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
set -eu
@@ -28,6 +37,9 @@ PUTER_DIR="${PUTER_DIR:-puter-selfhosted}"
PUTER_URL="${PUTER_URL:-https://raw.githubusercontent.com/HeyPuter/puter/main}"
PUTER_DOMAIN="${PUTER_DOMAIN:-puter.localhost}"
PUTER_PORT="${PUTER_PORT:-80}"
PUTER_PROTOCOL="${PUTER_PROTOCOL:-http}"
PUTER_TRUST_PROXY="${PUTER_TRUST_PROXY:-1}"
PUTER_ENV="${PUTER_ENV:-prod}"
PUTER_FORCE="${PUTER_FORCE:-0}"
log() { printf '\033[1;36m[puter-install]\033[0m %s\n' "$*"; }
@@ -117,9 +129,9 @@ EOF
cat > puter/config/config.json <<EOF
{
"domain": "$PUTER_DOMAIN",
"protocol": "http",
"protocol": "$PUTER_PROTOCOL",
"pub_port": $PUTER_PORT,
"env": "prod",
"env": "$PUTER_ENV",
"static_hosting_domain": "site.$PUTER_DOMAIN",
"static_hosting_domain_alt": "host.$PUTER_DOMAIN",
@@ -159,7 +171,7 @@ EOF
"s3": {
"s3Config": {
"endpoint": "http://s3:9000",
"publicEndpoint": "http://s3.$PUTER_DOMAIN",
"publicEndpoint": "$PUTER_PROTOCOL://s3.$PUTER_DOMAIN",
"accessKeyId": "puter",
"secretAccessKey": "$S3_SECRET_KEY",
"region": "us-east-1",
@@ -173,7 +185,7 @@ EOF
"ollama": { "enabled": false }
},
"trust_proxy": 1
"trust_proxy": $PUTER_TRUST_PROXY
}
EOF
fi
@@ -187,6 +199,6 @@ log "stack starting. first boot takes ~30s while MariaDB initialises."
log "follow puter logs:"
log " cd $PUTER_DIR && docker compose logs -f puter"
log ""
log "open http://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
log "open $PUTER_PROTOCOL://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
log "first-boot admin password is logged once — grab it with:"
log " cd $PUTER_DIR && docker compose logs puter | grep password"
+3900 -3262
View File
File diff suppressed because it is too large. Load diff
+12 -5
View File
@@ -44,18 +44,18 @@
"yaml": "^2.8.1"
},
"scripts": {
"test:backend": "vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"test:backend": "npm run setupExtensions && vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "npm run setupExtensions && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run build:ts",
"prestart": "npm run setupExtensions && npm run build:ts",
"dev": "npm start",
"build": "npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
"check-translations": "node tools/check-translations.js",
"prepare": "husky",
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
"postinstall": "node ./tools/extensionSetup.mjs"
"setupExtensions": "node ./tools/extensionSetup.mjs"
},
"workspaces": [
"src/*",
@@ -77,9 +77,16 @@
"ai": "^6.0.73",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.6",
"open": "^10.1.0"
},
"engines": {
"node": ">=24.0.0"
},
"allowScripts": {
"[email protected]": true,
"[email protected]": true,
"@google/genai": false,
"[email protected]": true
}
}
@@ -85,6 +85,9 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[50, ['0054_sessions_workers.sql']],
[50, ['0055_username_nocase_unique.sql']],
[51, ['0056_sessions_kind_worker.sql']],
[52, ['0057_add_user_reputation.sql']],
[53, ['0058_add_phone_verification.sql']],
[54, ['0059_add_card_verification.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -24,7 +24,9 @@
--
--
-- Idempotent column-ensure helper (used by CALLs below; dropped at end of file)
-- Idempotent column-ensure helper. Used by the CALLs below AND by later
-- migration files, which CALL it as a one-line idempotent column add. Left
-- resident (not dropped at end of file) so it outlives this migration.
--
DROP PROCEDURE IF EXISTS _puter_add_col;
@@ -1333,7 +1335,10 @@ CALL _puter_add_col('user_update_audit', 'new_username', '`new_username` varchar
CALL _puter_add_col('user_update_audit', 'reason', '`reason` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_col;
-- _puter_add_col is intentionally NOT dropped here: later migration files CALL
-- it as a one-line idempotent column add, and migrations replay on every boot
-- with no applied-state tracking, so the helper must outlive this file. The
-- DROP-before-CREATE at the top keeps this migration itself replay-safe.
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
-- E.164 number collected during verification (indexed like `email`);
-- `requires_phone_verification` gates account use for low-reputation signups
-- (not indexed, mirroring `requires_email_confirmation`).
--
-- Idempotent: column adds use _puter_add_col (defined in mig_1, which leaves it
-- resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'phone', '`phone` varchar(20) DEFAULT NULL');
CALL _puter_add_col('user', 'requires_phone_verification', '`requires_phone_verification` tinyint(1) NOT NULL DEFAULT ''0''');
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_phone_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_phone'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_phone` (`phone`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_phone_index();
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification column. Mirrors SQLite migration 0059.
-- `requires_card_verification` gates account use for low-reputation signups;
-- the card itself never touches our DB, so this is the only column (not
-- indexed, mirroring `requires_phone_verification`).
--
-- Idempotent: the column add uses _puter_add_col (from mig_1) so the
-- directory replays safely.
CALL _puter_add_col('user', 'requires_card_verification', '`requires_card_verification` tinyint(1) NOT NULL DEFAULT ''0''');
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
-- is the Stripe card fingerprint (stable per card number) recorded when a user
-- clears card verification — the card sibling of `phone`, indexed like it so
-- admin tooling can find the accounts that verified with a given card. The card
-- itself never touches our DB, only Stripe's fingerprint for it.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'card_fingerprint', '`card_fingerprint` varchar(128) DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_card_fingerprint_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_card_fingerprint'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_card_fingerprint` (`card_fingerprint`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_card_fingerprint_index();
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
-- E.164 number collected during verification (indexed like `email`);
-- `requires_phone_verification` gates account use for low-reputation signups
-- (not indexed, mirroring `requires_email_confirmation`).
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS phone varchar(20);
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_phone_verification boolean NOT NULL DEFAULT FALSE;
CREATE INDEX IF NOT EXISTS idx_user_phone ON "user" (phone);
@@ -0,0 +1,24 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification column. Mirrors SQLite migration 0059.
-- `requires_card_verification` gates account use for low-reputation signups;
-- the card itself never touches our DB, so this is the only column (not
-- indexed, mirroring `requires_phone_verification`).
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_card_verification boolean NOT NULL DEFAULT FALSE;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
-- is the Stripe card fingerprint (stable per card number) recorded when a user
-- clears card verification — the card sibling of `phone`, indexed like it so
-- admin tooling can find the accounts that verified with a given card. The card
-- itself never touches our DB, only Stripe's fingerprint for it.
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS card_fingerprint varchar(128);
CREATE INDEX IF NOT EXISTS idx_user_card_fingerprint ON "user" (card_fingerprint);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Abuse v2 reputation score (0–100+) recorded at signup. Mirrors the
-- `reputation smallint DEFAULT 100` column already present in the mysql and
-- postgres baseline migrations.
ALTER TABLE `user` ADD COLUMN `reputation` SMALLINT DEFAULT 100;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification. `phone` holds the E.164 number collected during
-- verification (indexed like `email`). `requires_phone_verification` gates
-- account use until verified — the abuse v2 harness sets it for low-reputation
-- signups instead of blocking them. (Not indexed, mirroring
-- `requires_email_confirmation`.)
ALTER TABLE `user` ADD COLUMN `phone` varchar(20) DEFAULT NULL;
ALTER TABLE `user` ADD COLUMN `requires_phone_verification` tinyint(1) NOT NULL DEFAULT 0;
CREATE INDEX IF NOT EXISTS idx_user_phone ON `user` (`phone`);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification ($0 authorization handled by a payments
-- extension). `requires_card_verification` gates account use until verified —
-- the abuse v2 harness sets it for low-reputation signups instead of blocking
-- them. The card itself never touches our DB, so this is the only column.
-- (Not indexed, mirroring `requires_phone_verification`.)
ALTER TABLE `user` ADD COLUMN `requires_card_verification` tinyint(1) NOT NULL DEFAULT 0;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. `card_fingerprint` is the Stripe card fingerprint
-- (stable per card number) recorded when a user clears card verification — the
-- card sibling of `phone`. Indexed like `phone` so admin tooling can find the
-- accounts that verified with a given card. The card itself never touches our
-- DB, only Stripe's fingerprint for it. Server-only (not on the whoami
-- allowlist), so it never reaches clients.
ALTER TABLE `user` ADD COLUMN `card_fingerprint` varchar(128) DEFAULT NULL;
CREATE INDEX IF NOT EXISTS idx_user_card_fingerprint ON `user` (`card_fingerprint`);
+192 -1
View File
@@ -14,6 +14,10 @@
* the SocketService fans out to user-scoped channels.
*/
import type {
Request as ExpressRequest,
Response as ExpressResponse,
} from 'express';
import { Actor } from '../../core';
import { FSEntry } from '../../stores/fs/FSEntry';
@@ -126,7 +130,20 @@ export type EventMap = {
req?: unknown;
data?: unknown;
abuse?: unknown;
custom?: unknown;
trail?: Array<string>;
/**
* Set by the abuse harness for flagged signups — the id under which the
* decision trail is persisted to KV (`abuse:trail:<id>`), shared back on
* the request for log / support correlation.
*/
trail_id?: string;
/** Device signals forwarded verbatim from the signup request body. */
fingerprint?: string | null;
dfp_telemetry_id?: string | null;
/** Set by the abuse harness — require SMS phone verification post-signup. */
requires_phone_verification?: boolean;
/** Set by the abuse harness — require card verification post-signup. */
requires_card_verification?: boolean;
[key: string]: unknown;
};
'puter.signup.success': {
@@ -135,6 +152,9 @@ export type EventMap = {
email: string;
username: string;
ip?: string | null;
fingerprint?: string | null;
/** True when the created account is a temp user (no email/password). */
is_temp?: boolean;
[key: string]: unknown;
};
'email.validate': {
@@ -154,12 +174,93 @@ export type EventMap = {
user_uid: string;
email: string;
};
// Phone-reuse cap is pure mechanism here — the abuse extension answers
// (emitted via `emitAndWait`) by counting how many OTHER accounts have
// already verified this number and flipping `allowed` to false when the
// cross-account limit is hit. No extension listening → `allowed` stays
// true and the send proceeds.
'puter.phone-verification.check': {
user_id: number;
user_uid: string;
phone: string;
// Client-supplied device fingerprint for this request, or null. Lets
// the abuse extension cap sends per device (across accounts) so a
// device farm can't dodge the per-account cap with fresh signups.
device_fingerprint: string | null;
allowed: boolean;
reason: string | null;
[key: string]: unknown;
};
// Fire-and-forget signal that a code was actually sent — the abuse
// extension bumps its per-number / per-account / per-device send-velocity
// counters off this. No-op with no extension listening.
'puter.phone-verification.sent': {
user_id: number;
user_uid: string;
phone: string;
device_fingerprint: string | null;
};
'user.phone-verified': {
user_id: number;
user_uid: string;
phone: string;
};
// Card verification is pure mechanism here — a payments extension fills
// these in (emitted via `emitAndWait`). `enabled` stays null when no
// extension is installed; the extension always sets it (true/false) so
// the endpoints can distinguish "disabled" from "not installed".
'puter.card-verification.setup': {
user_id: number;
user_uid: string;
ip?: string | null;
// Client-supplied device fingerprint for this request, or null. Lets
// the abuse extension cap card-verification setups per device (across
// accounts) before any Stripe SetupIntent is created.
device_fingerprint: string | null;
enabled: boolean | null;
// Set false by the extension to refuse this setup (e.g. the per-device
// setup-velocity cap); `reason` carries the opaque code. Stays true
// when allowed or with no extension listening.
allowed: boolean;
reason: string | null;
client_secret: string | null;
publishable_key: string | null;
[key: string]: unknown;
};
'puter.card-verification.confirm': {
user_id: number;
user_uid: string;
setup_intent_id: string;
enabled: boolean | null;
verified: boolean;
reason: string | null;
fingerprint: string | null;
funding: string | null;
country: string | null;
[key: string]: unknown;
};
'user.card-verified': {
user_id: number;
user_uid: string;
fingerprint: string | null;
funding: string | null;
country: string | null;
};
'user.username-changed': {
user_id: number;
old_username: string;
new_username: string;
};
'user.email-changed': { user_id: number; new_email: string };
// Fired after an account is torn down (self-serve, admin, or temp-user
// logout cleanup). Listeners purge external state tied to the account —
// e.g. the marketplace extension cancels the user's Stripe subscriptions.
// The row is already gone by emit time, so identifiers ride the payload.
'user.delete': {
user_id: number;
user_uuid?: string;
stripe_customer_id?: string | null;
};
// ---- Filesystem ----
'fs.copy.node': {
@@ -211,6 +312,8 @@ export type EventMap = {
entry: unknown;
host: string;
requestPath: string;
requestUrl?: string;
requestHash?: string;
mime: string;
};
@@ -257,6 +360,94 @@ export type EventMap = {
// SocketService re-emits each fanout-eligible event under
// `sent-to-user.<wireName>` so per-user channels can subscribe by wire name.
[K in `sent-to-user.${string}`]: { user_id: number; response: unknown };
} & {
// Generic per-driver-method lifecycle, emitted by DriverController for
// EVERY driver call under a key scoped to the interface + method:
// `driver.<iface>.<method>.before|after|error|reject`. Wildcard
// subscribers can listen to `driver.*` for everything, `driver.<iface>.*`
// for one interface, or the exact key for one method. The `.before` phase
// is emitted via `emitAndWait`, so a listener may set `allow = false`
// (with an optional `rejectReason`) to veto the call before it runs — a
// vetoed call emits `.reject` instead of running.
[K in `driver.${string}`]: DriverMethodLifecycleEvent;
} & {
// Generic per-route-endpoint lifecycle, emitted by the route materializer
// for EVERY non-middleware route under a key scoped to the HTTP method +
// normalized path: `route.<method>.<path>.before|after|error|reject`. Same
// wildcard + veto semantics as the driver lifecycle above.
[K in `route.${string}`]: RouteLifecycleEvent;
};
/**
* Phase of a request/method lifecycle. `reject` is emitted when a `before`
* listener vetoes the call (sets `allow = false`); the call never runs and no
* `after`/`error` follows.
*/
export type LifecyclePhase = 'before' | 'after' | 'error' | 'reject';
/**
* Payload for `driver.<iface>.<method>.<phase>` events.
*
* One shape across all phases; read `phase` (or the key suffix) to
* branch. `allow`/`rejectReason` are only meaningful on the `before` phase
* (emitted via `emitAndWait`).
*/
export type DriverMethodLifecycleEvent = {
phase: LifecyclePhase;
iface: string;
method: string;
/** Resolved concrete driver name. */
driver: string;
/** Full actor object, if the request is authenticated. */
actor?: Actor;
/** Stable actor id (see `actorUid`), if the request is authenticated. */
actorUid?: string;
/** Call arguments. Present on every phase. */
args?: unknown;
/** Return value. Present on `after`. */
result?: unknown;
/** Thrown error. Present on `error`. */
error?: unknown;
/** Wall-clock duration of the invocation. Present on `after`/`error`. */
durationMs?: number;
/** Veto channel for `before`: set `false` to block the call. */
allow?: boolean;
/** Optional human-readable reason surfaced to the caller when vetoed. */
rejectReason?: string;
};
/**
* Payload for `route.<method>.<path>.<phase>` events. Mirrors
* {@link DriverMethodLifecycleEvent} for HTTP endpoints.
*/
export type RouteLifecycleEvent = {
phase: LifecyclePhase;
/** HTTP method, lowercased (`get`, `post`, ...). */
method: string;
/** Full route path including the controller prefix. */
path: string;
/**
* The live express request/response for this call. Present on every phase.
* On `before` a listener can read the parsed body/headers or write its own
* response; on terminal phases they're useful for logging. These are real
* in-process objects — never serialize or forward them across nodes.
*/
req: ExpressRequest;
res: ExpressResponse;
/** Full actor object, if the request is authenticated. */
actor?: Actor;
/** Stable actor id (see `actorUid`), if the request is authenticated. */
actorUid?: string;
/** Response status code. Present on `after`/`error`. */
statusCode?: number;
/** Wall-clock duration from `before` to terminal phase. */
durationMs?: number;
/** Set when the request did not complete normally (abort / >=500). */
error?: unknown;
/** Veto channel for `before`: set `false` to block the request. */
allow?: boolean;
/** Optional human-readable reason surfaced to the caller when vetoed. */
rejectReason?: string;
};
export type EventKey = keyof EventMap & string;
+2
View File
@@ -24,6 +24,7 @@ import { EventClient } from './event/EventClient';
import { DDBClient } from './dynamodb/DDBClient';
import { RedisClient } from './redis/RedisClient';
import { S3Client } from './s3/S3Client';
import { PreludeClient } from './prelude/PreludeClient';
import type { IPuterClientRegistry } from './types';
export const puterClients = {
@@ -34,4 +35,5 @@ export const puterClients = {
dynamo: DDBClient,
redis: RedisClient,
s3: S3Client,
prelude: PreludeClient,
} satisfies IPuterClientRegistry;
@@ -0,0 +1,173 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { PreludeClient } from './PreludeClient';
import type { IConfig } from '../../types';
const makeClient = (apiKey?: string) =>
new PreludeClient({
prelude: apiKey ? { apiKey } : undefined,
} as unknown as IConfig);
const okJson = (body: unknown) =>
({
ok: true,
status: 200,
json: async () => body,
}) as unknown as Response;
describe('PreludeClient', () => {
let fetchMock: ReturnType<typeof vi.fn>;
beforeEach(() => {
fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
});
afterEach(() => {
vi.unstubAllGlobals();
});
it('isConfigured reflects whether an apiKey is set', () => {
expect(makeClient('sk_test').isConfigured()).toBe(true);
expect(makeClient().isConfigured()).toBe(false);
});
describe('isCountrySupported (€0.07 cap)', () => {
const client = makeClient('sk_test');
it('allows revenue markets up to the cap (incl. the priciest)', () => {
expect(client.isCountrySupported('US')).toBe(true); // €0.0043
expect(client.isCountrySupported('DE')).toBe(true); // €0.0598
expect(client.isCountrySupported('SA')).toBe(true); // €0.0638
expect(client.isCountrySupported('us')).toBe(true); // case-insensitive
});
it('rejects countries above the cap, with no SMS, or unknown', () => {
expect(client.isCountrySupported('PK')).toBe(false); // €0.3548
expect(client.isCountrySupported('ID')).toBe(false); // €0.2430
expect(client.isCountrySupported('LI')).toBe(false); // null (no SMS)
expect(client.isCountrySupported('ZZ')).toBe(false); // unknown
expect(client.isCountrySupported(undefined)).toBe(false);
});
it('honors a configured maxSmsCostEur override', () => {
const strict = new PreludeClient({
prelude: { apiKey: 'sk', maxSmsCostEur: 0.01 },
} as unknown as IConfig);
expect(strict.isCountrySupported('US')).toBe(true); // €0.0043
expect(strict.isCountrySupported('DE')).toBe(false); // €0.0598 > 0.01
const loose = new PreludeClient({
prelude: { apiKey: 'sk', maxSmsCostEur: 0.5 },
} as unknown as IConfig);
expect(loose.isCountrySupported('PK')).toBe(true); // €0.3548 <= 0.5
});
});
it('createVerification POSTs the phone target + ip signal with bearer auth', async () => {
fetchMock.mockResolvedValue(
okJson({ id: 'vrf_1', status: 'success' }),
);
const client = makeClient('sk_test');
const res = await client.createVerification('+14155550123', {
ip: '203.0.113.7',
});
expect(res).toEqual({ id: 'vrf_1', status: 'success' });
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://api.prelude.dev/v2/verification');
expect(init.method).toBe('POST');
expect(init.headers.Authorization).toBe('Bearer sk_test');
expect(JSON.parse(init.body)).toEqual({
target: { type: 'phone_number', value: '+14155550123' },
// Defaults to RCS (cheaper); Prelude falls back to SMS.
options: { code_size: 6, preferred_channel: 'rcs' },
signals: { ip: '203.0.113.7' },
});
});
it('forwards device_id and user_agent signals when supplied', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = makeClient('sk_test');
await client.createVerification('+14155550123', {
ip: '203.0.113.7',
device_id: 'thumb_abc123',
user_agent: 'Mozilla/5.0',
});
const [, init] = fetchMock.mock.calls[0];
expect(JSON.parse(init.body).signals).toEqual({
ip: '203.0.113.7',
device_id: 'thumb_abc123',
user_agent: 'Mozilla/5.0',
});
});
it('omits the signals object entirely when none are supplied', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = makeClient('sk_test');
await client.createVerification('+14155550123');
expect(JSON.parse(fetchMock.mock.calls[0][1].body)).not.toHaveProperty(
'signals',
);
});
it('includes a configured template_id + sender_id + preferred channel', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = new PreludeClient({
prelude: {
apiKey: 'sk_test',
templateId: 'tmpl_puter',
senderId: 'Puter',
preferredChannel: 'sms',
},
} as unknown as IConfig);
await client.createVerification('+14155550123');
const [, init] = fetchMock.mock.calls[0];
expect(JSON.parse(init.body).options).toEqual({
code_size: 6,
preferred_channel: 'sms', // config override beats the rcs default
template_id: 'tmpl_puter',
sender_id: 'Puter',
});
});
it('checkVerification POSTs target + code and returns the status', async () => {
fetchMock.mockResolvedValue(okJson({ status: 'success' }));
const client = makeClient('sk_test');
const res = await client.checkVerification('+14155550123', '123456');
expect(res).toEqual({ status: 'success' });
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://api.prelude.dev/v2/verification/check');
expect(JSON.parse(init.body)).toEqual({
target: { type: 'phone_number', value: '+14155550123' },
code: '123456',
});
});
it('throws (does not call fetch) when not configured', async () => {
const client = makeClient();
await expect(
client.createVerification('+14155550123'),
).rejects.toThrow(/not configured/i);
expect(fetchMock).not.toHaveBeenCalled();
});
it('throws on a non-2xx Prelude response', async () => {
fetchMock.mockResolvedValue({
ok: false,
status: 400,
json: async () => ({ message: 'bad target' }),
} as unknown as Response);
await expect(
makeClient('sk_test').checkVerification('+1', 'x'),
).rejects.toThrow(/Prelude .* failed: 400/);
});
});
@@ -0,0 +1,222 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { IConfig } from '../../types';
import { PuterClient } from '../types';
import { COUNTRY_SMS_PRICES } from './countries.js';
const PRELUDE_API_BASE = 'https://api.prelude.dev/v2';
const REQUEST_TIMEOUT_MS = 8000;
/** OTP length — matches the 6-box code UI. Prelude allows 4–8. */
const PRELUDE_CODE_SIZE = 6;
/**
* Channel Prelude prioritizes for delivery. RCS is far cheaper than SMS, so we
* prefer it by default; Prelude routes to the next reachable channel (SMS) when
* RCS isn't available on the destination. (Requires an RCS agent provisioned in
* the Prelude account to actually use RCS — otherwise it just falls through to
* SMS.) Override with `config.prelude.preferredChannel`.
*/
const DEFAULT_PREFERRED_CHANNEL: PreludeChannel = 'rcs';
/** Delivery channels Prelude can prioritize via `options.preferred_channel`. */
export type PreludeChannel =
| 'sms'
| 'rcs'
| 'whatsapp'
| 'viber'
| 'zalo'
| 'telegram';
/**
* Default per-SMS cost ceiling (EUR). Countries whose Prelude SMS rate exceeds
* this — or that have no SMS channel — are not offered phone verification. The
* cap covers every realistic revenue market (priciest are Germany €0.0598 and
* Saudi Arabia €0.0638) while excluding the expensive, high-fraud long tail.
* Override per-deployment with `config.prelude.maxSmsCostEur`.
*/
const DEFAULT_MAX_SMS_COST_EUR = 0.07;
/** Status returned by Prelude when creating/retrying a verification. */
export type PreludeCreateStatus =
| 'success'
| 'retry'
| 'challenged'
| 'blocked'
| 'shadow_blocked';
/** Status returned by Prelude when checking a code. */
export type PreludeCheckStatus =
| 'success'
| 'failure'
| 'expired_or_not_found'
| 'transaction_missing'
| 'transaction_mismatch';
/**
* Prelude Verify v2 client (https://docs.prelude.so/verify/v2). Sends and checks
* SMS one-time codes — Prelude generates, delivers, and validates the code, so
* we never store one ourselves. No-ops with a warning when no API key is
* configured (so dev environments don't crash); `isConfigured()` lets callers
* surface a clean "phone verification unavailable" error instead.
*/
export class PreludeClient extends PuterClient {
private apiKey: string | null = null;
constructor(config: IConfig) {
super(config);
this.apiKey = config.prelude?.apiKey ?? null;
}
override onServerStart(): void {
if (!this.apiKey) {
console.warn(
'[prelude] no apiKey configured — SMS phone verification is disabled',
);
}
}
/** True when an API key is configured and verification can be attempted. */
isConfigured(): boolean {
return !!this.apiKey;
}
/** ISO region used to parse local-format numbers (config-driven). */
get defaultCountry(): string | undefined {
return this.config.prelude?.defaultCountry;
}
/** Per-SMS cost ceiling in EUR (config override or the default cap). */
get maxSmsCostEur(): number {
return this.config.prelude?.maxSmsCostEur ?? DEFAULT_MAX_SMS_COST_EUR;
}
/**
* Whether SMS verification should be offered for a country. False when the
* country is unknown, has no SMS channel, or its rate exceeds the cost cap.
* @param iso ISO-3166 alpha-2 (e.g. 'US') — from the parsed phone number.
*/
isCountrySupported(iso: string | undefined): boolean {
if (!iso) return false; // couldn't determine country → can't price it
const price = COUNTRY_SMS_PRICES[iso.toUpperCase()];
if (!price || price.sms == null) return false;
return price.sms <= this.maxSmsCostEur;
}
/**
* Create (or retry) a verification: Prelude sends an OTP to `target`.
* @param target E.164 phone number, e.g. "+14155550123".
* @param signals Optional anti-fraud signals. The more we forward, the
* better Prelude's fraud model scores the attempt (per their docs IP +
* device + platform are the highest-value). We pass what the backend
* already has: the request `ip`, the client `device_id` (ThumbmarkJS
* hash), and the `user_agent` (Prelude infers platform/model/OS from it).
* Richer client-collected signals require Prelude's frontend SDK, which
* produces a `dispatch_id` the backend would forward here.
*/
async createVerification(
target: string,
signals: {
ip?: string;
device_id?: string;
user_agent?: string;
} = {},
): Promise<{ id?: string; status: PreludeCreateStatus }> {
// Match the 6-box code UI (UIWindowPhoneVerificationRequired). Without
// code_size Prelude uses the dashboard default (4). preferred_channel
// prioritizes RCS (cheaper); Prelude falls back to SMS when unavailable.
const options: Record<string, unknown> = {
code_size: PRELUDE_CODE_SIZE,
preferred_channel:
this.config.prelude?.preferredChannel ??
DEFAULT_PREFERRED_CHANNEL,
};
// Branding lives in the Prelude dashboard (the message text is a
// template); these just select a Puter-branded template / sender when
// configured. See IPreludeConfig.
const { templateId, senderId } = this.config.prelude ?? {};
if (templateId) options.template_id = templateId;
if (senderId) options.sender_id = senderId;
const body: Record<string, unknown> = {
target: { type: 'phone_number', value: target },
options,
};
// Only attach signals we actually have — Prelude treats the object as
// optional and an empty one adds nothing.
const sig: Record<string, string> = {};
if (signals.ip) sig.ip = signals.ip;
if (signals.device_id) sig.device_id = signals.device_id;
if (signals.user_agent) sig.user_agent = signals.user_agent;
if (Object.keys(sig).length > 0) body.signals = sig;
return this.#post('/verification', body) as Promise<{
id?: string;
status: PreludeCreateStatus;
}>;
}
/**
* Check a code the user entered against the active verification for `target`.
* @returns `{ status }` — `'success'` means verified.
*/
async checkVerification(
target: string,
code: string,
): Promise<{ status: PreludeCheckStatus }> {
return this.#post('/verification/check', {
target: { type: 'phone_number', value: target },
code,
}) as Promise<{ status: PreludeCheckStatus }>;
}
async #post(
path: string,
body: Record<string, unknown>,
): Promise<Record<string, unknown>> {
if (!this.apiKey) {
throw new Error('Prelude is not configured');
}
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
try {
const res = await fetch(`${PRELUDE_API_BASE}${path}`, {
method: 'POST',
headers: {
Authorization: `Bearer ${this.apiKey}`,
'Content-Type': 'application/json',
Accept: 'application/json',
},
body: JSON.stringify(body),
signal: controller.signal,
});
const json = (await res.json().catch(() => ({}))) as Record<
string,
unknown
>;
if (!res.ok) {
throw new Error(
`Prelude ${path} failed: ${res.status} ${
(json as { message?: string })?.message ?? ''
}`.trim(),
);
}
return json;
} finally {
clearTimeout(timer);
}
}
}
+952
View File
@@ -0,0 +1,952 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/** Per-country Prelude SMS pricing, hardcoded from the Prelude global price
* sheet. `sms` is the average cost in EUR, or null when SMS is unavailable
* there. Used to cap phone verification to affordable countries (see
* PreludeClient.isCountrySupported). Regenerate from the price sheet when
* Prelude updates rates. */
export interface CountryPrice {
name: string;
/** Prelude SMS average in EUR, or null when SMS is not available. */
sms: number | null;
}
export const COUNTRY_SMS_PRICES: Record<string, CountryPrice> = {
AD: {
name: 'Andorra',
sms: 0.0515,
},
AE: {
name: 'United Arab Emirates',
sms: 0.0145,
},
AF: {
name: 'Afghanistan',
sms: 0.2409,
},
AG: {
name: 'Antigua and Barbuda',
sms: 0.0755,
},
AI: {
name: 'Anguilla',
sms: 0.0765,
},
AL: {
name: 'Albania',
sms: 0.0345,
},
AM: {
name: 'Armenia',
sms: 0.118,
},
AN: {
name: 'Netherlands Antilles',
sms: null,
},
AO: {
name: 'Angola',
sms: 0.0347,
},
AR: {
name: 'Argentina',
sms: 0.0614,
},
AS: {
name: 'American Samoa',
sms: 0.077,
},
AT: {
name: 'Austria',
sms: 0.014,
},
AU: {
name: 'Australia',
sms: 0.0091,
},
AW: {
name: 'Aruba',
sms: 0.0617,
},
AZ: {
name: 'Azerbaijan',
sms: 0.0969,
},
BA: {
name: 'Bosnia and Herzegovina',
sms: 0.0801,
},
BB: {
name: 'Barbados',
sms: 0.2021,
},
BD: {
name: 'Bangladesh',
sms: 0.1844,
},
BE: {
name: 'Belgium',
sms: 0.0297,
},
BF: {
name: 'Burkina Faso',
sms: 0.063,
},
BG: {
name: 'Bulgaria',
sms: 0.0799,
},
BH: {
name: 'Bahrain',
sms: 0.014,
},
BI: {
name: 'Burundi',
sms: 0.217,
},
BJ: {
name: 'Benin',
sms: 0.13,
},
BM: {
name: 'Bermuda',
sms: 0.21,
},
BN: {
name: 'Brunei',
sms: 0.035,
},
BO: {
name: 'Bolivia',
sms: 0.0708,
},
BQ: {
name: 'Caribbean Netherlands',
sms: 0.0565,
},
BR: {
name: 'Brazil',
sms: 0.0086,
},
BS: {
name: 'Bahamas',
sms: 0.0672,
},
BT: {
name: 'Bhutan',
sms: null,
},
BW: {
name: 'Botswana',
sms: 0.0235,
},
BY: {
name: 'Belarus',
sms: 0.118,
},
BZ: {
name: 'Belize',
sms: 0.1529,
},
CA: {
name: 'Canada',
sms: 0.0052,
},
CD: {
name: 'Congo RDC',
sms: 0.1112,
},
CF: {
name: 'Central African Republic',
sms: 0.2,
},
CG: {
name: 'Congo',
sms: 0.2507,
},
CH: {
name: 'Switzerland',
sms: 0.0163,
},
CI: {
name: "Cote d'Ivoire",
sms: 0.19,
},
CK: {
name: 'Cook Islands',
sms: 0.1041,
},
CL: {
name: 'Chile',
sms: 0.0027,
},
CM: {
name: 'Cameroon',
sms: 0.1504,
},
CN: {
name: 'China',
sms: 0.0053,
},
CO: {
name: 'Colombia',
sms: 0.0008,
},
CR: {
name: 'Costa Rica',
sms: 0.0045,
},
CU: {
name: 'Cuba',
sms: 0.0512,
},
CV: {
name: 'Cabo Verde',
sms: 0.2031,
},
CW: {
name: 'Curacao',
sms: 0.0138,
},
CY: {
name: 'Northern Cyprus',
sms: 0.0065,
},
CZ: {
name: 'Czech Republic',
sms: 0.0299,
},
DE: {
name: 'Germany',
sms: 0.0598,
},
DJ: {
name: 'Djibouti',
sms: 0.0897,
},
DK: {
name: 'Denmark',
sms: 0.0301,
},
DM: {
name: 'Dominica',
sms: 0.0824,
},
DO: {
name: 'Dominican Republic',
sms: 0.0351,
},
DZ: {
name: 'Algeria',
sms: 0.196,
},
EC: {
name: 'Ecuador',
sms: 0.1041,
},
EE: {
name: 'Estonia',
sms: 0.0234,
},
EG: {
name: 'Egypt',
sms: 0.1561,
},
ER: {
name: 'Eritrea',
sms: 0.0659,
},
ES: {
name: 'Spain',
sms: 0.0193,
},
ET: {
name: 'Ethiopia',
sms: 0.2741,
},
FI: {
name: 'Finland',
sms: 0.043,
},
FJ: {
name: 'Fiji',
sms: 0.069,
},
FK: {
name: 'Falkland Islands',
sms: 0.0713,
},
FM: {
name: 'Micronesia',
sms: 0.0118,
},
FO: {
name: 'Faroe Islands',
sms: 0.0319,
},
FR: {
name: 'France',
sms: 0.03,
},
GA: {
name: 'Gabon',
sms: 0.15,
},
GB: {
name: 'United Kingdom',
sms: 0.026,
},
GD: {
name: 'Grenada',
sms: null,
},
GE: {
name: 'Georgia',
sms: 0.0788,
},
GF: {
name: 'French Guiana',
sms: 0.05,
},
GG: {
name: 'Guernsey',
sms: 0.025,
},
GH: {
name: 'Ghana',
sms: 0.15,
},
GI: {
name: 'Gibraltar',
sms: 0.0134,
},
GL: {
name: 'Greenland',
sms: 0.0048,
},
GM: {
name: 'Gambia',
sms: 0.1283,
},
GN: {
name: 'Guinea',
sms: 0.2,
},
GP: {
name: 'Guadeloupe',
sms: 0.0455,
},
GQ: {
name: 'Equatorial Guinea',
sms: 0.099,
},
GR: {
name: 'Greece',
sms: 0.0296,
},
GT: {
name: 'Guatemala',
sms: 0.1202,
},
GU: {
name: 'Guam',
sms: 0.02,
},
GW: {
name: 'Guinea-Bissau',
sms: 0.1566,
},
GY: {
name: 'Guyana',
sms: 0.2178,
},
HK: {
name: 'Hong Kong',
sms: 0.038,
},
HN: {
name: 'Honduras',
sms: 0.162,
},
HR: {
name: 'Croatia',
sms: 0.03,
},
HT: {
name: 'Haiti',
sms: 0.09,
},
HU: {
name: 'Hungary',
sms: 0.029,
},
ID: {
name: 'Indonesia',
sms: 0.243,
},
IE: {
name: 'Ireland',
sms: 0.0305,
},
IL: {
name: 'Israel',
sms: 0.01,
},
IM: {
name: 'Isle of Man',
sms: 0.0385,
},
IN: {
name: 'India',
sms: 0.0375,
},
IQ: {
name: 'Iraq',
sms: 0.151,
},
IR: {
name: 'Iran',
sms: 0.14,
},
IS: {
name: 'Iceland',
sms: 0.0493,
},
IT: {
name: 'Italy',
sms: 0.0245,
},
JE: {
name: 'Jersey',
sms: 0.025,
},
JM: {
name: 'Jamaica',
sms: 0.1525,
},
JO: {
name: 'Jordan',
sms: 0.2094,
},
JP: {
name: 'Japan',
sms: 0.017,
},
KE: {
name: 'Kenya',
sms: 0.125,
},
KG: {
name: 'Kyrgyzstan',
sms: 0.15,
},
KH: {
name: 'Cambodia',
sms: 0.1529,
},
KI: {
name: 'Kiribati',
sms: 0.025,
},
KM: {
name: 'Comoros',
sms: 0.15,
},
KN: {
name: 'Saint Kitts and Nevis',
sms: 0.1295,
},
KR: {
name: 'South Korea',
sms: 0.006,
},
KW: {
name: 'Kuwait',
sms: 0.145,
},
KY: {
name: 'Cayman Islands',
sms: 0.2172,
},
KZ: {
name: 'Kazakhstan',
sms: 0.202,
},
LA: {
name: 'Laos',
sms: 0.15,
},
LB: {
name: 'Lebanon',
sms: 0.153,
},
LC: {
name: 'Saint Lucia',
sms: 0.0892,
},
LI: {
name: 'Liechtenstein',
sms: null,
},
LK: {
name: 'Sri Lanka',
sms: 0.3593,
},
LR: {
name: 'Liberia',
sms: 0.1448,
},
LS: {
name: 'Lesotho',
sms: 0.0301,
},
LT: {
name: 'Lithuania',
sms: 0.0261,
},
LU: {
name: 'Luxembourg',
sms: 0.034,
},
LV: {
name: 'Latvia',
sms: 0.028,
},
LY: {
name: 'Libya',
sms: 0.1898,
},
MA: {
name: 'Morocco',
sms: 0.105,
},
MC: {
name: 'Monaco',
sms: 0.1219,
},
MD: {
name: 'Moldova',
sms: 0.065,
},
ME: {
name: 'Montenegro',
sms: 0.07,
},
MG: {
name: 'Madagascar',
sms: 0.24,
},
MH: {
name: '',
sms: 0.032,
},
MK: {
name: 'Macedonia',
sms: 0.0096,
},
ML: {
name: 'Mali',
sms: 0.136,
},
MM: {
name: 'Myanmar',
sms: 0.3037,
},
MN: {
name: 'Mongolia',
sms: 0.183,
},
MO: {
name: 'Macao',
sms: 0.005,
},
MP: {
name: 'Northern Mariana Islands',
sms: 0.0669,
},
MQ: {
name: 'Martinique',
sms: 0.0455,
},
MR: {
name: 'Mauritania',
sms: 0.161,
},
MS: {
name: 'Montserrat',
sms: 0.0725,
},
MT: {
name: 'Malta',
sms: 0.0343,
},
MU: {
name: 'Mauritius',
sms: 0.1431,
},
MV: {
name: 'Maldives',
sms: 0.145,
},
MW: {
name: 'Malawi',
sms: 0.19,
},
MX: {
name: 'Mexico',
sms: 0.0021,
},
MY: {
name: 'Malaysia',
sms: 0.08,
},
MZ: {
name: 'Mozambique',
sms: 0.2266,
},
NA: {
name: 'Namibia',
sms: 0.0173,
},
NC: {
name: 'New Caledonia',
sms: 0.052,
},
NE: {
name: 'Niger',
sms: 0.143,
},
NG: {
name: 'Nigeria',
sms: 0.198,
},
NI: {
name: 'Nicaragua',
sms: 0.1033,
},
NL: {
name: 'Netherlands',
sms: 0.046,
},
NO: {
name: 'Norway',
sms: 0.03,
},
NP: {
name: 'Nepal',
sms: 0.1645,
},
NR: {
name: 'Nauru',
sms: null,
},
NU: {
name: 'Niue',
sms: null,
},
NZ: {
name: 'New Zealand',
sms: 0.0372,
},
OM: {
name: 'Oman',
sms: 0.0713,
},
PA: {
name: 'Panama',
sms: 0.07,
},
PE: {
name: 'Peru',
sms: 0.13,
},
PF: {
name: 'French Polynesia',
sms: 0.0518,
},
PG: {
name: 'Papua New Guinea',
sms: 0.15,
},
PH: {
name: 'Philippines',
sms: 0.1237,
},
PK: {
name: 'Pakistan',
sms: 0.3548,
},
PL: {
name: 'Poland',
sms: 0.011,
},
PM: {
name: 'Saint Pierre and Miquelon',
sms: 0.0905,
},
PR: {
name: 'Puerto Rico',
sms: 0.02,
},
PS: {
name: 'Palestine',
sms: 0.2541,
},
PT: {
name: 'Portugal',
sms: 0.009,
},
PW: {
name: 'Palau',
sms: null,
},
PY: {
name: 'Paraguay',
sms: 0.0236,
},
QA: {
name: 'Qatar',
sms: 0.1393,
},
RE: {
name: 'Reunion',
sms: 0.034,
},
RO: {
name: 'Romania',
sms: 0.0255,
},
RS: {
name: 'Serbia',
sms: 0.1935,
},
RU: {
name: 'Russia',
sms: 0.2028,
},
RW: {
name: 'Rwanda',
sms: 0.1177,
},
SA: {
name: 'Saudi Arabia',
sms: 0.0638,
},
SB: {
name: 'Solomon Islands',
sms: 0.0357,
},
SC: {
name: 'Seychelles',
sms: 0.0404,
},
SD: {
name: 'Sudan',
sms: 0.224,
},
SE: {
name: 'Sweden',
sms: 0.026,
},
SG: {
name: 'Singapore',
sms: 0.0256,
},
SI: {
name: 'Slovenia',
sms: 0.1,
},
SK: {
name: 'Slovakia',
sms: 0.0245,
},
SL: {
name: 'Sierra Leone',
sms: 0.2322,
},
SM: {
name: 'San Marino',
sms: null,
},
SN: {
name: 'Senegal',
sms: 0.1151,
},
SO: {
name: 'Somalia',
sms: 0.06,
},
SR: {
name: 'Suriname',
sms: 0.119,
},
SS: {
name: 'South Sudan',
sms: 0.15,
},
ST: {
name: 'Sao Tome and Principe',
sms: 0.0133,
},
SV: {
name: 'El Salvador',
sms: 0.06,
},
SX: {
name: 'Sint Maarten',
sms: 0.0623,
},
SY: {
name: 'Syria',
sms: 0.221,
},
SZ: {
name: 'Eswatini',
sms: 0.12,
},
TC: {
name: 'Turks and Caicos Islands',
sms: null,
},
TD: {
name: 'Chad',
sms: 0.1711,
},
TG: {
name: 'Togo',
sms: 0.1806,
},
TH: {
name: 'Thailand',
sms: 0.003,
},
TJ: {
name: 'Tajikistan',
sms: 0.2626,
},
TL: {
name: 'Timor-Leste',
sms: 0.0675,
},
TM: {
name: 'Turkmenistan',
sms: 0.1677,
},
TN: {
name: 'Tunisia',
sms: 0.225,
},
TO: {
name: 'Tonga',
sms: 0.0902,
},
TR: {
name: 'Turkey',
sms: 0.0008,
},
TT: {
name: 'Trinidad and Tobago',
sms: 0.1556,
},
TW: {
name: 'Taiwan',
sms: 0.0165,
},
TZ: {
name: 'Tanzania',
sms: 0.2753,
},
UA: {
name: 'Ukraine',
sms: 0.094,
},
UG: {
name: 'Uganda',
sms: 0.1707,
},
US: {
name: 'United States',
sms: 0.0043,
},
UY: {
name: 'Uruguay',
sms: 0.0174,
},
UZ: {
name: 'Uzbekistan',
sms: 0.292,
},
VC: {
name: 'Saint Vincent and The Grenadines',
sms: 0.1216,
},
VE: {
name: 'Venezuela',
sms: 0.0427,
},
VG: {
name: 'British Virgin Islands',
sms: 0.083,
},
VI: {
name: 'Virgin Island',
sms: 0.0017,
},
VN: {
name: 'Vietnam',
sms: 0.0885,
},
VU: {
name: 'Vanuatu',
sms: 0.1347,
},
WF: {
name: 'Wallis and Futuna',
sms: 0.09,
},
WS: {
name: 'Samoa',
sms: 0.2,
},
XK: {
name: 'Kosovo',
sms: 0.1394,
},
YE: {
name: 'Yemen',
sms: 0.1486,
},
YT: {
name: 'Mayotte',
sms: 0.06,
},
ZA: {
name: 'South Africa',
sms: 0.039,
},
ZM: {
name: 'Zambia',
sms: 0.21,
},
ZW: {
name: 'Zimbabwe',
sms: 0.13,
},
};
@@ -56,6 +56,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const apps = await this.appDriver.select({
@@ -71,6 +72,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const name = req.query?.name;
@@ -185,6 +187,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const raw = req.params.name;
File diff suppressed because it is too large. Load diff
+720 -31
View File
@@ -47,6 +47,7 @@ import {
} from '../../services/auth/OTPUtil.js';
import { sessionCookieFlags } from '../../util/cookieFlags.js';
import { cleanEmail, isBlockedEmail } from '../../util/email.js';
import { parsePhone } from '../../util/phone.js';
import { generate_identifier } from '../../util/identifier.js';
import { getTaskbarItems } from '../../util/taskbarItems.js';
import {
@@ -57,6 +58,8 @@ import { PuterController } from '../types.js';
const USERNAME_REGEX = /^\w{1,}$/;
const USERNAME_MAX_LENGTH = 45;
const FINGERPRINT_MAX_LENGTH = 128;
const DFP_TELEMETRY_ID_MAX_LENGTH = 64;
const RESERVED_USERNAMES = new Set([
'admin',
'administrator',
@@ -99,7 +102,16 @@ export class AuthController extends PuterController {
@Post('/login', {
captcha: true,
rateLimit: { scope: 'login', limit: 10, window: 15 * 60_000 },
// Two limits: per-fingerprint keeps users behind a shared IP
// (offices, campuses) from throttling each other, while the
// coarser per-IP backstop stops an attacker from minting fresh
// fingerprint buckets by rotating client-controlled headers
// (User-Agent etc.). Same pattern on the other unauthenticated
// credential endpoints below.
rateLimit: [
{ scope: 'login', limit: 10, window: 15 * 60_000 },
{ scope: 'login-ip', limit: 50, window: 15 * 60_000, key: 'ip' },
],
})
async handleLogin(req: Request, res: Response): Promise<void> {
const { username, email, password } = req.body;
@@ -206,11 +218,15 @@ export class AuthController extends PuterController {
@Post('/login/otp', {
captcha: true,
rateLimit: {
scope: 'login-otp',
limit: 15,
window: 30 * 60_000,
},
rateLimit: [
{ scope: 'login-otp', limit: 15, window: 30 * 60_000 },
{
scope: 'login-otp-ip',
limit: 60,
window: 30 * 60_000,
key: 'ip',
},
],
})
async handleLoginOtp(req: Request, res: Response): Promise<void> {
const { token, code } = req.body;
@@ -266,11 +282,15 @@ export class AuthController extends PuterController {
@Post('/login/recovery-code', {
captcha: true,
rateLimit: {
scope: 'login-recovery',
limit: 10,
window: 60 * 60_000,
},
rateLimit: [
{ scope: 'login-recovery', limit: 10, window: 60 * 60_000 },
{
scope: 'login-recovery-ip',
limit: 40,
window: 60 * 60_000,
key: 'ip',
},
],
})
async handleLoginRecoveryCode(req: Request, res: Response): Promise<void> {
const { token, code } = req.body;
@@ -339,7 +359,10 @@ export class AuthController extends PuterController {
@Post('/signup', {
captcha: true,
rateLimit: { scope: 'signup', limit: 10, window: 15 * 60_000 },
rateLimit: [
{ scope: 'signup', limit: 10, window: 15 * 60_000 },
{ scope: 'signup-ip', limit: 50, window: 15 * 60_000, key: 'ip' },
],
})
async handleSignup(req: Request, res: Response): Promise<void> {
const body = req.body ?? {};
@@ -355,6 +378,43 @@ export class AuthController extends PuterController {
return;
}
// Optional device signals (browser fingerprint hash, DFP telemetry
// id). Core only enforces shape and forwards the values verbatim —
// signup-abuse policy built on them lives in extensions. Checked
// before the reauth short-circuit so a malformed value is rejected
// on every /signup path.
if (body.fingerprint !== undefined && body.fingerprint !== null) {
if (typeof body.fingerprint !== 'string')
throw new HttpError(400, 'fingerprint must be a string.', {
legacyCode: 'bad_request',
});
if (body.fingerprint.length > FINGERPRINT_MAX_LENGTH)
throw new HttpError(
400,
`fingerprint cannot be longer than ${FINGERPRINT_MAX_LENGTH} characters.`,
{ legacyCode: 'bad_request' },
);
}
if (
body.dfp_telemetry_id !== undefined &&
body.dfp_telemetry_id !== null
) {
if (typeof body.dfp_telemetry_id !== 'string')
throw new HttpError(400, 'dfp_telemetry_id must be a string.', {
legacyCode: 'bad_request',
});
if (body.dfp_telemetry_id.length > DFP_TELEMETRY_ID_MAX_LENGTH)
throw new HttpError(
400,
`dfp_telemetry_id cannot be longer than ${DFP_TELEMETRY_ID_MAX_LENGTH} characters.`,
{ legacyCode: 'bad_request' },
);
}
// Empty strings are treated as absent — a signal that wasn't
// collected, not a malformed request.
const fingerprint: string | null = body.fingerprint || null;
const dfp_telemetry_id: string | null = body.dfp_telemetry_id || null;
// Temp-user reauth short-circuit: when an existing temp user is
// forced through the reauth flow, the GUI re-submits /signup with
// is_temp=true plus the server-signed reauth_token from the 401.
@@ -534,23 +594,45 @@ export class AuthController extends PuterController {
allow: true,
no_temp_user: false,
requires_email_confirmation: false,
// Set by the abuse harness for low-reputation signups: the account is
// created + logged in but gated behind SMS phone verification (in
// addition to email confirmation) instead of being blocked.
requires_phone_verification: false,
// Same idea, one rung up the ladder: gate the account behind
// credit-card verification (a $0 auth handled by an extension).
requires_card_verification: false,
message: null,
code: null,
user_agent: req?.headers?.['user-agent'] ?? null,
fingerprint,
dfp_telemetry_id,
// Populated by the abuse extension's v2 harness; persisted to the
// user row below so the signup-time reputation is referable later.
reputation: null as number | null,
// Stamped by the abuse harness for flagged signups — the id keying
// the `abuse:trail:<id>` decision trail (carrying both the live and
// shadow trails). Surfaced to a blocked user as the Request Code so
// the code they quote support leads straight to their trail.
trail_id: undefined as string | undefined,
};
try {
await this.clients.event?.emitAndWait(
'puter.signup.validate' as never,
validateEvent as never,
'puter.signup.validate',
validateEvent,
{},
);
} catch (e) {
console.warn('[signup] validate hook failed:', e);
}
if (!validateEvent.allow) {
// Pass the trail id back to a blocked user as the Request Code (when
// the harness stamped one), embedded in the message so the existing
// signup-block UI surfaces it without a GUI change.
const requestCode = validateEvent.trail_id;
throw new HttpError(
403,
validateEvent.message ?? 'Signup blocked',
(validateEvent.message ?? 'Signup blocked') +
(requestCode ? ` Request Code: ${requestCode}` : ''),
{
...(validateEvent.code
? { legacyCode: validateEvent.code as never }
@@ -573,6 +655,17 @@ export class AuthController extends PuterController {
const force_email_confirmation = Boolean(
validateEvent.requires_email_confirmation,
);
const force_phone_verification =
Boolean(validateEvent.requires_phone_verification) ||
// Test/QA switch: force the SMS gate on every signup regardless of
// reputation (see config.always_require_phone_verification).
Boolean(this.config.always_require_phone_verification);
const force_card_verification = Boolean(
validateEvent.requires_card_verification ||
// Test/QA switch: force the card gate on every signup regardless of
// reputation (see config.always_require_card_verification).
this.config.always_require_card_verification,
);
// Prepare shared fields
const user_uuid = uuidv4();
@@ -597,11 +690,13 @@ export class AuthController extends PuterController {
email_confirm_code,
email_confirm_token,
email_confirmed: 0,
// Pseudo claims always require email confirmation — the
// validate hook can only tighten, not loosen, so `1`
// stays hardcoded here.
requires_email_confirmation: 1,
last_activity_ts: signupSqlTs,
...(validateEvent.reputation != null
? { reputation: validateEvent.reputation }
: {}),
requires_phone_verification: force_phone_verification ? 1 : 0,
requires_card_verification: force_card_verification ? 1 : 0,
});
// Move from temp group to regular user group
@@ -650,6 +745,8 @@ export class AuthController extends PuterController {
ip_fwd: proxyIpChain,
user_agent: req.headers?.['user-agent'],
origin: req.headers?.origin,
fingerprint,
dfp_telemetry_id,
},
signup_ip: clientIp,
signup_ip_forwarded: proxyIpChain,
@@ -658,6 +755,11 @@ export class AuthController extends PuterController {
signup_server: (this.config as { serverId?: string }).serverId,
referrer: req.body.referrer ?? null,
last_activity_ts: signupSqlTs,
reputation: validateEvent.reputation,
// Phone collected later in the verification dialog (null now).
phone: null,
requires_phone_verification: force_phone_verification,
requires_card_verification: force_card_verification,
} as never);
// Add to default group
@@ -728,6 +830,11 @@ export class AuthController extends PuterController {
user_uuid: user!.uuid,
email: user!.email,
username: user!.username,
fingerprint,
// Reflects the row that was actually created/claimed —
// a pseudo-user claim ends up with credentials, so it
// reports false here. Same signal completeLogin uses.
is_temp: user!.password === null && user!.email === null,
ip:
(req?.headers?.['x-forwarded-for'] as
| string
@@ -928,6 +1035,507 @@ export class AuthController extends PuterController {
res.json({ email_confirmed: true, original_client_socket_id });
}
// -- Phone verification (SMS via Prelude) ------------------------
@Post('/send-confirm-phone', {
subdomain: ['api', ''],
requireUserActor: true,
allowUnconfirmed: true,
rateLimit: {
scope: 'send-confirm-phone',
limit: 10,
window: 60 * 60_000,
key: 'user',
},
})
async handleSendConfirmPhone(req: Request, res: Response): Promise<void> {
const user = await this.stores.user.getById(req.actor!.user.id!, {
force: true,
});
if (!user)
throw new HttpError(404, 'User not found.', {
legacyCode: 'user_not_found' as never,
});
if (user.suspended)
throw new HttpError(403, 'Account suspended.', {
legacyCode: 'account_suspended',
});
if (!this.clients.prelude?.isConfigured())
throw new HttpError(503, 'Phone verification is unavailable.', {
legacyCode: 'service_unavailable' as never,
});
// Parse to E.164 (Prelude's required form + the stored form) and the
// country, so we can apply the per-country cost cap.
const parsed = parsePhone(
req.body?.phone,
this.clients.prelude.defaultCountry,
);
if (!parsed)
throw new HttpError(400, 'Invalid phone number.', {
legacyCode: 'bad_request',
});
// Cost cap: skip countries with no SMS channel or rates above the cap
// (see PreludeClient / countries.ts). Avoids paying exorbitant per-SMS
// rates in low-revenue, high-fraud geographies.
if (!this.clients.prelude.isCountrySupported(parsed.country))
throw new HttpError(
400,
'Phone verification is not available for this country.',
{ legacyCode: 'phone_country_not_supported' as never },
);
// Abuse caps live ENTIRELY in a listening abuse extension, consulted
// via `puter.phone-verification.check`. The backend ships no thresholds
// or detection of its own (so none of it is readable in the open-source
// repo): it forwards the user / number / ip, and the extension decides
// `allowed` plus an opaque `reason` (per-account + per-number send
// velocity, cross-account reuse, …). With no extension listening
// `allowed` stays true. Fail-open on a hook error — this is abuse/cost
// control, not a security boundary (the route rate limit and the country
// cost cap remain), so a flaky hook must not lock signups out.
const abuseCheck = {
user_id: user.id,
user_uid: user.uuid,
phone: parsed.e164,
device_fingerprint: req.deviceFingerprint ?? null,
allowed: true,
reason: null as string | null,
};
try {
await this.clients.event?.emitAndWait(
'puter.phone-verification.check',
abuseCheck,
{},
);
} catch (e) {
console.warn('[send-confirm-phone] abuse-check hook failed:', e);
}
// Forward the verdict verbatim: a generic 429 plus the opaque reason for
// the client to message on. The backend never interprets the reason —
// its meaning lives in the extension (which sets it) and the GUI (which
// displays it), so no abuse semantics leak into the OSS repo.
if (abuseCheck.allowed === false)
throw new HttpError(
429,
'Phone verification is unavailable for this number right now.',
{
legacyCode: 'phone_verification_unavailable' as never,
fields: abuseCheck.reason
? { reason: abuseCheck.reason }
: {},
},
);
// Stage the parsed number as pending in KV (NOT on the user row) so a
// never-confirmed number is never written to the indexed `phone`
// column. /confirm-phone reads it back and persists it to the row only
// once Prelude confirms the code. ~1h TTL covers the code's lifetime.
// Stored before the send so we never dispatch an SMS we couldn't later
// confirm against.
const pendingPhoneKey = `phone-verify-pending:${user.id}`;
try {
await this.stores.kv.set({
key: pendingPhoneKey,
value: parsed.e164,
expireAt: Math.floor(Date.now() / 1000) + 60 * 60,
});
} catch (e) {
console.warn('[send-confirm-phone] pending-store failed:', e);
throw new HttpError(503, 'Could not start phone verification.', {
legacyCode: 'service_unavailable' as never,
});
}
const ip = req.ip || req.socket?.remoteAddress || undefined;
const userAgent =
typeof req.headers['user-agent'] === 'string'
? req.headers['user-agent']
: undefined;
try {
const result = await this.clients.prelude.createVerification(
parsed.e164,
{
ip,
device_id: req.deviceFingerprint ?? undefined,
user_agent: userAgent,
},
);
// Prelude rejected the attempt as abusive — surface as rate-limit.
if (
result.status === 'blocked' ||
result.status === 'shadow_blocked'
) {
throw new HttpError(
429,
'Phone verification is temporarily unavailable for this number.',
{ legacyCode: 'too_many_requests' as never },
);
}
} catch (e) {
if (e instanceof HttpError) throw e;
console.warn('[send-confirm-phone] createVerification failed:', e);
throw new HttpError(502, 'Could not send verification code.', {
legacyCode: 'upstream_error' as never,
});
}
// Tell the abuse extension a code was actually sent, so it can bump its
// send-velocity counters (per number + per account). Fire-and-forget;
// the backend keeps no send counts of its own. Only reached after a
// successful send, so an upstream error never burns quota.
try {
this.clients.event?.emit(
'puter.phone-verification.sent' as never,
{
user_id: user.id,
user_uid: user.uuid,
phone: parsed.e164,
device_fingerprint: req.deviceFingerprint ?? null,
} as never,
{},
);
} catch {
// ignore — best-effort velocity signal
}
res.json({});
}
@Post('/confirm-phone', {
subdomain: ['api', ''],
requireUserActor: true,
allowUnconfirmed: true,
rateLimit: {
scope: 'confirm-phone',
limit: 10,
window: 10 * 60_000,
key: 'user',
},
})
async handleConfirmPhone(req: Request, res: Response): Promise<void> {
const { code, original_client_socket_id } = req.body ?? {};
if (!code)
throw new HttpError(400, 'Missing `code`.', {
legacyCode: 'bad_request',
});
const user = await this.stores.user.getById(req.actor!.user.id!, {
force: true,
});
if (!user)
throw new HttpError(404, 'User not found.', {
legacyCode: 'not_found',
});
if (!user.requires_phone_verification) {
res.json({ phone_verified: true, original_client_socket_id });
return;
}
// The number being verified is the one staged at send time (KV
// pending), not the user row — we don't persist an unverified number.
// Fall back to a row value for accounts that already have one on file
// (legacy / a number persisted by a prior verified flow).
const pendingPhoneKey = `phone-verify-pending:${user.id}`;
let pendingPhone: string | null = null;
try {
const { res: staged } = await this.stores.kv.get({
key: pendingPhoneKey,
});
if (typeof staged === 'string' && staged) pendingPhone = staged;
} catch (e) {
console.warn('[confirm-phone] pending read failed:', e);
}
if (!pendingPhone) pendingPhone = user.phone ?? null;
if (!pendingPhone)
throw new HttpError(
400,
'No phone number on file. Request a code first.',
{ legacyCode: 'bad_request' },
);
if (!this.clients.prelude?.isConfigured())
throw new HttpError(503, 'Phone verification is unavailable.', {
legacyCode: 'service_unavailable' as never,
});
let status;
try {
({ status } = await this.clients.prelude.checkVerification(
pendingPhone,
String(code),
));
} catch (e) {
console.warn('[confirm-phone] checkVerification failed:', e);
throw new HttpError(502, 'Could not verify code.', {
legacyCode: 'upstream_error' as never,
});
}
if (status !== 'success') {
res.json({ phone_verified: false, original_client_socket_id });
return;
}
// Verified — persist the number now (and only now) and clear the gate.
await this.stores.user.update(user.id, {
requires_phone_verification: 0,
phone: pendingPhone,
});
try {
this.clients.event?.emit(
'user.phone-verified' as never,
{
user_id: user.id,
user_uid: user.uuid,
phone: pendingPhone,
} as never,
{},
);
} catch {
// ignore — event is a side-channel signal, not load-bearing
}
// Notify other tabs/devices for this user so they refresh + drop the gate.
try {
await this.services.socket?.send(
{ room: user.id },
'user.phone_verified',
{ original_client_socket_id },
);
} catch {
// ignore — best-effort
}
res.json({ phone_verified: true, original_client_socket_id });
}
// -- Card verification ($0 auth via a payments extension) --------
/**
* Start card verification for the calling user. Pure mechanism: the
* endpoint emits `puter.card-verification.setup` and a payments
* extension fills in the client credentials — the OSS backend holds
* no provider knowledge or config. Phone verification (when required)
* must be completed first; the ordering is enforced here so a client
* can't skip the cheaper gate.
*/
@Post('/card-verification/setup', {
subdomain: ['api', ''],
requireUserActor: true,
allowUnconfirmed: true,
rateLimit: {
scope: 'card-verification-setup',
limit: 5,
window: 60 * 60_000,
key: 'user',
},
})
async handleCardVerificationSetup(
req: Request,
res: Response,
): Promise<void> {
const user = await this.stores.user.getById(req.actor!.user.id!, {
force: true,
});
if (!user)
throw new HttpError(404, 'User not found.', {
legacyCode: 'user_not_found' as never,
});
if (user.suspended)
throw new HttpError(403, 'Account suspended.', {
legacyCode: 'account_suspended',
});
if (!user.requires_card_verification) {
res.json({ card_verified: true });
return;
}
if (user.requires_phone_verification)
throw new HttpError(
409,
'Phone verification must be completed first.',
{ legacyCode: 'conflict' },
);
// `enabled` stays null when no extension is listening; an installed
// extension always sets it (true/false) before doing any work.
const setupEvent = {
user_id: user.id,
user_uid: user.uuid,
ip: (req.ip || req.socket?.remoteAddress || null) as string | null,
device_fingerprint: req.deviceFingerprint ?? null,
enabled: null as boolean | null,
allowed: true,
reason: null as string | null,
client_secret: null as string | null,
publishable_key: null as string | null,
};
try {
await this.clients.event?.emitAndWait(
'puter.card-verification.setup',
setupEvent,
{},
);
} catch (e) {
console.warn('[card-verification/setup] setup hook failed:', e);
}
// Abuse veto (e.g. per-device setup-velocity cap): the extension refused
// before any Stripe work. Forward the opaque reason verbatim as a 429,
// same as the phone gate — the backend never interprets it.
if (setupEvent.allowed === false)
throw new HttpError(
429,
'Card verification is unavailable right now.',
{
legacyCode: 'too_many_requests' as never,
fields: setupEvent.reason
? { reason: setupEvent.reason }
: {},
},
);
// Kill switch: the extension reports the feature disabled — unstick
// any user still carrying the flag instead of dead-ending them.
if (setupEvent.enabled === false) {
await this.stores.user.update(user.id, {
requires_card_verification: 0,
});
res.json({ card_verified: true, disabled: true });
return;
}
if (!setupEvent.client_secret || !setupEvent.publishable_key)
throw new HttpError(503, 'Card verification is not available.', {
legacyCode: 'service_unavailable' as never,
});
res.json({
client_secret: setupEvent.client_secret,
publishable_key: setupEvent.publishable_key,
});
}
/**
* Complete card verification. The client confirms the setup intent with
* the payment provider directly, then posts the resulting id here; the
* payments extension checks it (and applies its own abuse limits) via
* `puter.card-verification.confirm`. On success the gate clears exactly
* like `/confirm-phone` clears the phone gate.
*/
@Post('/card-verification/confirm', {
subdomain: ['api', ''],
requireUserActor: true,
allowUnconfirmed: true,
rateLimit: {
scope: 'card-verification-confirm',
limit: 10,
window: 10 * 60_000,
key: 'user',
},
})
async handleCardVerificationConfirm(
req: Request,
res: Response,
): Promise<void> {
const { setup_intent_id, original_client_socket_id } = req.body ?? {};
if (
typeof setup_intent_id !== 'string' ||
setup_intent_id.length === 0 ||
setup_intent_id.length > 255
)
throw new HttpError(400, 'Invalid `setup_intent_id`.', {
legacyCode: 'bad_request',
});
const user = await this.stores.user.getById(req.actor!.user.id!, {
force: true,
});
if (!user)
throw new HttpError(404, 'User not found.', {
legacyCode: 'not_found',
});
if (!user.requires_card_verification) {
res.json({ card_verified: true });
return;
}
if (user.requires_phone_verification)
throw new HttpError(
409,
'Phone verification must be completed first.',
{ legacyCode: 'conflict' },
);
const confirmEvent = {
user_id: user.id,
user_uid: user.uuid,
setup_intent_id,
enabled: null as boolean | null,
verified: false,
reason: null as string | null,
fingerprint: null as string | null,
funding: null as string | null,
country: null as string | null,
};
try {
await this.clients.event?.emitAndWait(
'puter.card-verification.confirm',
confirmEvent,
{},
);
} catch (e) {
console.warn('[card-verification/confirm] confirm hook failed:', e);
}
// Kill switch — same semantics as /card-verification/setup.
if (confirmEvent.enabled === false) {
await this.stores.user.update(user.id, {
requires_card_verification: 0,
});
res.json({ card_verified: true, disabled: true });
return;
}
// No extension listening — nothing could have verified anything.
if (confirmEvent.enabled === null)
throw new HttpError(503, 'Card verification is not available.', {
legacyCode: 'service_unavailable' as never,
});
if (confirmEvent.verified !== true) {
res.json({ card_verified: false, reason: confirmEvent.reason });
return;
}
await this.stores.user.update(user.id, {
requires_card_verification: 0,
});
try {
this.clients.event?.emit(
'user.card-verified' as never,
{
user_id: user.id,
user_uid: user.uuid,
fingerprint: confirmEvent.fingerprint,
funding: confirmEvent.funding,
country: confirmEvent.country,
} as never,
{},
);
} catch {
// ignore — event is a side-channel signal, not load-bearing
}
// Notify other tabs/devices for this user so they refresh + drop the gate.
try {
await this.services.socket?.send(
{ room: user.id },
'user.card_verified',
{ original_client_socket_id },
);
} catch {
// ignore — best-effort
}
res.json({ card_verified: true });
}
// -- Password recovery -------------------------------------------
@Post('/send-pass-recovery-email', {
@@ -1129,6 +1737,12 @@ export class AuthController extends PuterController {
}
await this.stores.user.invalidateById(user.id);
// A password reset is the "I think someone else has access" flow —
// evict every interactive session so a hijacked one doesn't survive.
await this.services.auth.revokeInteractiveSessionsForUserId(
user.id as number,
);
res.send('Password successfully updated.');
}
@@ -1165,6 +1779,10 @@ export class AuthController extends PuterController {
change_email_confirm_token: null,
});
// Sign out every other web session (cascading to their derived
// rows); only the session that changed the password survives.
await this.services.auth.revokeAllSessions(req.actor!);
if (this.clients.email && user.email) {
try {
await this.clients.email.send(
@@ -2140,12 +2758,15 @@ export class AuthController extends PuterController {
// -- Access tokens -----------------------------------------------
async handleMigrateToken(req: Request, res: Response): Promise<void> {
// 1. Origin lock. Reject anything that isn't same-origin to
// `config.origin` or in the explicit per-deployment allowlist.
// No Origin header → reject (this endpoint is browser-only by
// design; server-side callers should re-auth properly).
// 1. Browser-only gate. No Origin header → reject (server-side
// callers should re-auth properly). The exchange itself is open
// to any browser origin: puter.js apps live on arbitrary
// third-party domains, the v1 bearer token is the credential,
// and cookies are never read here — so a hostile page learns
// nothing it doesn't already hold. Origin *trust* only decides
// whether the `puter_token_v2` companion cookie is set below.
const reqOrigin = req.headers.origin;
if (!reqOrigin || !this.#isMigrateTokenOriginAllowed(reqOrigin)) {
if (!reqOrigin) {
throw new HttpError(403, 'Origin not allowed', {
legacyCode: 'forbidden',
});
@@ -2175,12 +2796,19 @@ export class AuthController extends PuterController {
: undefined,
});
// `puter_token_v2` is the cookie companion to v2 app tokens —
// the app runs in-browser (we already gated on Origin above) so
// the GUI's cookie-only middleware can authenticate subsequent
// calls from the same iframe without the client having to
// forward Authorization headers. Access tokens are programmatic
// (no browser cookie surface) so we deliberately skip them here.
if (result.kind === 'app') {
// it lets the GUI's cookie-only middleware authenticate
// subsequent same-origin calls without the client forwarding
// Authorization headers. Issuing it is gated on the trusted
// origin allowlist: an arbitrary attacker page must not be able
// to plant a session cookie on the GUI origin (login CSRF).
// Untrusted origins still get the token in the JSON body — the
// cookie is only consumed same-origin (see authProbe) so they
// lose nothing. Access tokens are programmatic (no browser
// cookie surface) so we deliberately skip them here.
if (
result.kind === 'app' &&
this.#isMigrateTokenOriginAllowed(reqOrigin)
) {
res.cookie('puter_token_v2', result.token, {
...sessionCookieFlags(this.config),
httpOnly: true,
@@ -2212,13 +2840,25 @@ export class AuthController extends PuterController {
requireAuth: true,
})
async handleCreateAccessToken(req: Request, res: Response): Promise<void> {
const { permissions, expiresIn } = req.body;
const { permissions, expiresIn, label } = req.body;
if (!Array.isArray(permissions) || permissions.length === 0) {
throw new HttpError(400, 'Missing or empty `permissions` array', {
legacyCode: 'bad_request',
});
}
// Optional user-facing name for the manage-sessions UI. Trim and clamp
// to the same 64-char limit the rename endpoint enforces.
let normalizedLabel: string | null = null;
if (label !== undefined && label !== null) {
if (typeof label !== 'string') {
throw new HttpError(400, '`label` must be a string', {
legacyCode: 'bad_request',
});
}
normalizedLabel = label.trim().slice(0, 64) || null;
}
// Normalize specs: string → [string], [string] → [string, {}], [string, extra] → as-is
const normalized = permissions.map((spec) => {
if (typeof spec === 'string') return [spec];
@@ -2233,7 +2873,10 @@ export class AuthController extends PuterController {
const token = await this.services.auth.createAccessToken(
req.actor!,
normalized as never,
expiresIn ? { expiresIn } : {},
{
...(expiresIn ? { expiresIn } : {}),
...(normalizedLabel ? { label: normalizedLabel } : {}),
},
);
res.json({ token });
}
@@ -2470,6 +3113,9 @@ export class AuthController extends PuterController {
});
await this.stores.group.addUsers(uid, users);
// New members inherit the group's permissions immediately, not
// after the permission-cache TTL.
await this.services.permission.bumpPermissionCacheForUsernames(users);
res.json({});
}
@@ -2499,6 +3145,9 @@ export class AuthController extends PuterController {
});
await this.stores.group.removeUsers(uid, users);
// Removed members must lose the group's permissions immediately,
// not after the permission-cache TTL.
await this.services.permission.bumpPermissionCacheForUsernames(users);
res.json({});
}
@@ -2805,6 +3454,22 @@ export class AuthController extends PuterController {
// -- Private helpers ----------------------------------------------
async #cascadeDeleteUser(userId: number): Promise<void> {
// Capture the identifiers downstream teardown needs before the row is
// gone — the marketplace extension cancels the user's Stripe
// subscriptions off `user.delete`, keyed by uuid / customer id.
let userUuid: string | undefined;
let stripeCustomerId: string | null = null;
try {
const rows = (await this.clients.db.read(
'SELECT `uuid`, `stripe_customer_id` FROM `user` WHERE `id` = ?',
[userId],
)) as Array<{ uuid?: string; stripe_customer_id?: string | null }>;
userUuid = rows[0]?.uuid;
stripeCustomerId = rows[0]?.stripe_customer_id ?? null;
} catch (e) {
console.warn('[cascade-delete-user] identifier lookup failed:', e);
}
try {
await this.services.fs.removeAllForUser(userId);
} catch (e) {
@@ -2822,6 +3487,24 @@ export class AuthController extends PuterController {
userId,
]);
await this.stores.user.invalidateById(userId);
// Fire-and-forget: let listeners purge external state tied to the
// account (Stripe subscriptions are cancelled immediately, without
// proration). Emitted after the row delete — listeners key off the
// payload, not the DB row.
try {
this.clients.event?.emit(
'user.delete',
{
user_id: userId,
user_uuid: userUuid,
stripe_customer_id: stripeCustomerId,
},
{},
);
} catch {
// ignore — event emission shouldn't block deletion
}
}
async #generateRandomUsername(): Promise<string> {
@@ -2981,6 +3664,9 @@ export class AuthController extends PuterController {
password?: string | null;
email_confirmed?: number | boolean;
requires_email_confirmation?: number | boolean;
phone?: string | null;
requires_phone_verification?: number | boolean;
requires_card_verification?: number | boolean;
},
): Promise<void> {
const meta = {
@@ -3029,6 +3715,9 @@ export class AuthController extends PuterController {
email: user.email,
email_confirmed: user.email_confirmed,
requires_email_confirmation: user.requires_email_confirmation,
phone: user.phone,
requires_phone_verification: user.requires_phone_verification,
requires_card_verification: user.requires_card_verification,
is_temp: user.password === null && user.email === null,
taskbar_items,
},
@@ -51,6 +51,7 @@ export class DesktopController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { url, color, fit } = req.body ?? {};
@@ -105,6 +106,7 @@ export class DesktopController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { items } = req.body ?? {};
@@ -130,6 +132,7 @@ export class DesktopController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { item_uid, item_path, layout } = req.body ?? {};
@@ -156,6 +159,7 @@ export class DesktopController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { item_uid, item_path, sort_by, sort_order } =
@@ -21,7 +21,8 @@ import { Readable } from 'node:stream';
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { actorUid, type Actor } from '../../core/actor.js';
import type { DriverMethodLifecycleEvent } from '../../clients/event/types.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -460,6 +461,129 @@ describe('DriverController.#handleCall (via captured router)', () => {
});
});
describe('DriverController driver-method lifecycle events', () => {
let routes: Captured;
beforeAll(() => {
routes = captureRoutes(controller);
});
const grantKvPerm = async (actor: Actor) => {
await server.stores.permission.setFlatUserPerm(
actor.user!.id!,
'service:puter-kvstore:ii:puter-kvstore',
{
permission: 'service:puter-kvstore:ii:puter-kvstore',
deleted: false,
issuer_user_id: actor.user!.id!,
} as never,
);
};
it('emits before then after around a successful driver method', async () => {
const actor = await makeUserActor();
await grantKvPerm(actor);
// Correlate on the actor: each test creates a fresh user, so this
// isolates our call from any other traffic on the shared event bus.
const who = actorUid(actor);
const before: DriverMethodLifecycleEvent[] = [];
const after: DriverMethodLifecycleEvent[] = [];
server.clients.event.on(
'driver.puter-kvstore.set.before',
(_k, data) => {
if (data.actorUid === who) before.push(data);
},
);
server.clients.event.on(
'driver.puter-kvstore.set.after',
(_k, data) => {
if (data.actorUid === who) after.push(data);
},
);
const key = `lc-${uuidv4()}`;
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key, value: 'v' },
},
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
);
expect(before).toHaveLength(1);
expect(before[0]).toMatchObject({
phase: 'before',
iface: 'puter-kvstore',
method: 'set',
actor,
actorUid: who,
});
expect(after).toHaveLength(1);
expect(after[0].phase).toBe('after');
expect(after[0].args).toMatchObject({ key, value: 'v' });
expect(typeof after[0].durationMs).toBe('number');
});
it('emits reject and answers 403 when a before listener vetoes', async () => {
const actor = await makeUserActor();
await grantKvPerm(actor);
// Scope the veto to a sentinel key so it doesn't affect other calls
// sharing this server's event bus.
const key = `veto-${uuidv4()}`;
const reject: DriverMethodLifecycleEvent[] = [];
server.clients.event.on(
'driver.puter-kvstore.set.before',
(_k, data) => {
if ((data.args as { key?: string })?.key === key) {
data.allow = false;
data.rejectReason = 'blocked in test';
}
},
);
server.clients.event.on(
'driver.puter-kvstore.set.reject',
(_k, data) => {
if (data.rejectReason === 'blocked in test') reject.push(data);
},
);
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key, value: 'v' },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({ statusCode: 403 });
expect(reject).toHaveLength(1);
expect(reject[0]).toMatchObject({
phase: 'reject',
method: 'set',
args: { key, value: 'v' },
rejectReason: 'blocked in test',
});
});
});
describe('DriverController.#handleListInterfaces', () => {
let routes: Captured;
beforeAll(() => {
@@ -18,6 +18,7 @@
*/
import type { Request, Response } from 'express';
import { actorUid } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { Controller } from '../../core/http/decorators.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
@@ -34,6 +35,7 @@ import {
resolveDriverMethodRateLimit,
} from '../../drivers/meta.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import { PermissionUtil } from '../../services/permission/permissionUtil.js';
import type { WithLifecycle } from '../../types';
import { PuterController } from '../types.js';
@@ -221,7 +223,16 @@ export class DriverController extends PuterController {
| PermissionService
| undefined;
if (permService) {
const permKey = `service:${resolvedDriverName}:ii:${ifaceName}`;
// Build via PermissionUtil.join so any `:` in a driver or
// interface name is escaped — raw interpolation would let a
// crafted name shift permission-segment boundaries and match
// a broader/narrower parent than intended in the scan logic.
const permKey = PermissionUtil.join(
'service',
String(resolvedDriverName),
'ii',
ifaceName,
);
const hasPermission = await permService.check(
req.actor,
permKey,
@@ -324,14 +335,91 @@ export class DriverController extends PuterController {
// a stale value from a prior call.
Context.set('driverName', requestedDriver);
// Per-method lifecycle events, scoped to `driver.<iface>.<method>`.
// Subscribers can listen on `driver.*`, `driver.<iface>.*`, or the
// exact key. `before` is emitted via `emitAndWait` so a listener may
// veto the call by setting `allow = false` (emits `reject`, throws
// 403); otherwise `after`/`error` carry the result/error + duration.
const actor = req.actor ? actorUid(req.actor) : undefined;
const resolved = String(resolvedDriverName);
const beforeEvent = {
phase: 'before' as const,
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
allow: true as boolean,
rejectReason: undefined as string | undefined,
};
await this.clients.event?.emitAndWait(
`driver.${ifaceName}.${method}.before`,
beforeEvent,
{},
);
if (beforeEvent.allow === false) {
this.clients.event?.emit(
`driver.${ifaceName}.${method}.reject`,
{
phase: 'reject',
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
rejectReason: beforeEvent.rejectReason,
},
{},
);
throw new HttpError(
403,
beforeEvent.rejectReason ??
`Blocked by policy: ${ifaceName}:${method}`,
{ legacyCode: 'forbidden' },
);
}
// Drivers read actor/context via the Context API — no drilled args.
const startedAt = Date.now();
let result;
try {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
result = await (fn as (...x: unknown[]) => any).call(driver, args);
} catch (e) {
this.clients.event?.emit(
`driver.${ifaceName}.${method}.error`,
{
phase: 'error',
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
error: e,
durationMs: Date.now() - startedAt,
},
{},
);
throw translateProviderError(e);
}
this.clients.event?.emit(
`driver.${ifaceName}.${method}.after`,
{
phase: 'after',
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
result,
durationMs: Date.now() - startedAt,
},
{},
);
if (isDriverStreamResult(result)) {
res.setHeader('Content-Type', result.content_type);
+233 -15
View File
@@ -18,6 +18,7 @@
*/
import type { Request, Response } from 'express';
import type { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
@@ -124,6 +125,14 @@ const makeRes = () => {
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
const streamToString = async (stream: Readable): Promise<string> => {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.from(chunk as Buffer));
}
return Buffer.concat(chunks).toString('utf8');
};
// ── /startBatchWrite ────────────────────────────────────────────────
describe('FSController.startBatchWrites', () => {
@@ -434,6 +443,65 @@ describe('FSController.completeBatchWrites', () => {
expect(finalized?.wasOverwrite).toBe(true);
});
// Regression: a signed (direct-to-S3) upload could declare a tiny size
// and PUT far more — the presigned URL doesn't bound the body. The
// completion path must reconcile the recorded size against the object's
// true size, or storage accounting is permanently understated and the
// quota is bypassable.
it('reconciles the recorded size to the real uploaded bytes (ignores under-declared size)', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/under-declared.bin`;
// Declare 1 byte.
const start = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path: target, size: 1 } }],
actor,
}),
start.res,
),
);
const [started] = start.captured.body as SignedWriteResponse[];
// Actually upload 4096 bytes to the session's object key (simulating
// a client that PUTs more than it declared via the signed URL).
const realBytes = Buffer.alloc(4096, 0x41);
await server.stores.s3Object.uploadFromServer(
{
bucket: started!.bucket,
objectKey: started!.objectKey,
contentType: 'application/octet-stream',
body: realBytes,
contentLength: realBytes.byteLength,
},
started!.bucketRegion,
);
const complete = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: started!.sessionId }],
actor,
}),
complete.res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
expect(entry).not.toBeNull();
// Recorded size is the true 4096 bytes, not the declared 1.
expect(entry?.size).toBe(4096);
// And the user's accounted usage reflects the real bytes.
const allowance =
await server.stores.fsEntry.getUserStorageAllowance(userId);
expect(allowance.curr).toBeGreaterThanOrEqual(4096);
});
it('emits updated events with GUI metadata when overwriting via batch completion', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
@@ -490,9 +558,7 @@ describe('FSController.completeBatchWrites', () => {
.body as SignedWriteResponse[];
const emitSpy = vi.spyOn(server.clients.event, 'emit');
let updatedCall:
| (typeof emitSpy.mock.calls)[number]
| undefined;
let updatedCall: (typeof emitSpy.mock.calls)[number] | undefined;
try {
await withActor(actor, () =>
controller.completeBatchWrites(
@@ -667,9 +733,9 @@ describe('FSController.statEntry', () => {
...makeReq({ body: { path: '/x' }, actor }),
actor: undefined,
} as unknown as Request;
await expect(
controller.statEntry(req, res),
).rejects.toMatchObject({ statusCode: 401 });
await expect(controller.statEntry(req, res)).rejects.toMatchObject({
statusCode: 401,
});
});
});
@@ -839,9 +905,7 @@ describe('FSController.searchEntries', () => {
).toBe(true);
}
expect(
results.some(
(r) => r.path === `/${username}/Documents/${needle}`,
),
results.some((r) => r.path === `/${username}/Documents/${needle}`),
).toBe(false);
});
@@ -1259,10 +1323,6 @@ describe('FSController.copyEntry', () => {
// ── /read (readEntry, full read) ────────────────────────────────────
describe('FSController.readEntry (file streaming)', () => {
// makeRes here adds a real Writable surface so that
// `pipeline(download.body, res)` inside the controller can pipe
// the in-memory S3 stream into the test's response and capture
// bytes for assertions.
const makeStreamingRes = () => {
const captured = {
statusCode: 200,
@@ -1270,7 +1330,8 @@ describe('FSController.readEntry (file streaming)', () => {
bodyChunks: [] as Buffer[],
};
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { Writable } = require('node:stream') as typeof import('node:stream');
const { Writable } =
require('node:stream') as typeof import('node:stream');
const writable = new Writable({
write(chunk: Buffer, _enc, cb) {
captured.bodyChunks.push(chunk);
@@ -1330,7 +1391,9 @@ describe('FSController.readEntry (file streaming)', () => {
// Pipeline awaits the stream-end on success.
expect(captured.statusCode).toBe(200);
expect(captured.headers['Content-Type']).toMatch(/text\/plain/);
expect(captured.headers['Content-Length']).toBe(String(body.byteLength));
expect(captured.headers['Content-Length']).toBe(
String(body.byteLength),
);
expect(captured.headers['Content-Disposition']).toMatch(
/inline; filename=/,
);
@@ -1774,3 +1837,158 @@ describe('FSController.mkshortcutEntry', () => {
expect(body.isShortcut).toBe(true);
});
});
describe('FSController metadata.objectKey injection', () => {
const writeFile = async (
actor: Actor,
path: string,
content: string,
metadata?: Record<string, unknown>,
) => {
await withActor(actor, () =>
controller.write(
makeReq({
body: {
fileMetadata: {
path,
size: Buffer.byteLength(content),
contentType: 'text/plain',
overwrite: true,
...(metadata ? { metadata } : {}),
},
fileContent: content,
encoding: 'utf8',
},
actor,
}) as unknown as Request<
Record<string, never>,
null,
import('./requestTypes.js').WriteRequest
>,
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(path, {
skipCache: true,
});
if (!entry) throw new Error(`entry not found after write: ${path}`);
return entry;
};
it("does not stream another user's file when a client injects metadata.objectKey on write", async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimSecret = 'VICTIM-TOP-SECRET-PAYLOAD';
const attackerDecoy = 'attacker-own-decoy-bytes';
const victimEntry = await writeFile(
victim.actor,
`/${victim.actor.user!.username}/Documents/secret.txt`,
victimSecret,
);
const victimRead = await server.services.fs.readContent(victimEntry);
expect(await streamToString(victimRead.body)).toBe(victimSecret);
const attackerEntry = await writeFile(
attacker.actor,
`/${attacker.actor.user!.username}/Documents/loot.txt`,
attackerDecoy,
{ objectKey: victimEntry.uuid },
);
const persisted = attackerEntry.metadata
? (JSON.parse(attackerEntry.metadata) as Record<string, unknown>)
: {};
expect(persisted.objectKey).toBeUndefined();
const attackerRead =
await server.services.fs.readContent(attackerEntry);
const got = await streamToString(attackerRead.body);
expect(got).toBe(attackerDecoy);
expect(got).not.toBe(victimSecret);
});
it('read path ignores a divergent metadata.objectKey on an already-poisoned row', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimSecret = 'VICTIM-SECRET-FOR-POISON-TEST';
const attackerDecoy = 'attacker-decoy-for-poison-test';
const victimEntry = await writeFile(
victim.actor,
`/${victim.actor.user!.username}/Documents/secret2.txt`,
victimSecret,
);
const attackerEntry = await writeFile(
attacker.actor,
`/${attacker.actor.user!.username}/Documents/loot2.txt`,
attackerDecoy,
);
await server.stores.fsEntry.updateEntry(attackerEntry.uuid, {
metadata: JSON.stringify({ objectKey: victimEntry.uuid }),
});
const poisoned = await server.stores.fsEntry.getEntryByPath(
attackerEntry.path,
{ skipCache: true },
);
if (!poisoned) throw new Error('poisoned entry not found');
expect(
(JSON.parse(poisoned.metadata!) as { objectKey: string }).objectKey,
).toBe(victimEntry.uuid);
const read = await server.services.fs.readContent(poisoned);
expect(await streamToString(read.body)).toBe(attackerDecoy);
});
it('scrubs objectKey from move newMetadata while preserving legit trash metadata', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimSecret = 'VICTIM-SECRET-FOR-MOVE-TEST';
const attackerDecoy = 'attacker-decoy-for-move-test';
const username = attacker.actor.user!.username!;
const victimEntry = await writeFile(
victim.actor,
`/${victim.actor.user!.username}/Documents/secret3.txt`,
victimSecret,
);
const attackerEntry = await writeFile(
attacker.actor,
`/${username}/Documents/loot3.txt`,
attackerDecoy,
);
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
{ skipCache: true },
);
if (!documents) throw new Error('Documents dir not found');
const moved = await withActor(attacker.actor, () =>
server.services.fs.move(attacker.userId, {
source: attackerEntry,
destinationParent: documents,
newName: 'loot3-moved.txt',
newMetadata: {
original_path: `/${username}/Documents/loot3.txt`,
trashed_ts: 1700000000,
objectKey: victimEntry.uuid,
},
}),
);
const persisted = JSON.parse(moved.metadata!) as Record<
string,
unknown
>;
expect(persisted.objectKey).toBeUndefined();
expect(persisted.original_path).toBe(
`/${username}/Documents/loot3.txt`,
);
expect(persisted.trashed_ts).toBe(1700000000);
const read = await server.services.fs.readContent(moved);
expect(await streamToString(read.body)).toBe(attackerDecoy);
});
});
+7 -1
View File
@@ -37,6 +37,7 @@ import {
runWithConcurrencyLimit,
runWithConcurrencyLimitSettled,
} from '../../util/concurrency.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import type {
@@ -964,8 +965,13 @@ export class FSController extends PuterController {
range,
});
if (download.contentType)
if (download.contentType) {
res.setHeader('Content-Type', download.contentType);
// Stored type is uploader-controlled and served inline on the
// api origin — sandbox active-document types (HTML/SVG/XML) so
// embedded scripts can't run here. Inert types are untouched.
applyInlineContentSecurity(res, download.contentType);
}
if (download.contentLength !== null)
res.setHeader('Content-Length', String(download.contentLength));
if (download.contentRange)
@@ -29,6 +29,7 @@ import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { ACLService } from '../../services/acl/ACLService.js';
import type { SignedFile } from '../../util/fileSigning.js';
import { verifySignature } from '../../util/fileSigning.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import {
@@ -45,14 +46,6 @@ import {
} from './legacyFsHelpers.js';
import { RouteOptions } from '../../core/http/index.js';
/**
* Legacy FS routes, implemented as thin shims over `FSService`.
*
* Each shim parses the request shape (FSNodeParam-style `{ path, uid, id }`
* or `{ parent, name }`), invokes the service method, and returns the
* snake_case response clients expect.
*/
type RouterCache = Map<string, RequestHandler | null>;
const additionalRoutePaths: Record<string, string> = {};
@@ -146,6 +139,11 @@ export class LegacyFSController extends PuterController {
{
subdomain: ['api', ''],
requireUserActor: true,
// The user's own credential may download their files; apps and
// scoped tokens stay blocked. antiCsrf still protects the
// cookie-authed GUI path — a bearer-token PAT is exempt below
// (CSRF can't forge a header-credentialed request).
allowFullAccessToken: true,
requireVerified: true,
antiCsrf: true,
},
@@ -1352,8 +1350,15 @@ export class LegacyFSController extends PuterController {
query.download === '1' ||
query.download === true;
if (download.contentType)
if (download.contentType) {
res.setHeader('Content-Type', download.contentType);
// Uploader-controlled type served inline on the file origin —
// sandbox active-document types so an uploaded HTML/SVG can't
// execute scripts here. Mirrors FSController /fs/read.
if (!wantsAttachment) {
applyInlineContentSecurity(res, download.contentType);
}
}
if (download.contentLength !== null)
res.setHeader('Content-Length', String(download.contentLength));
if (download.contentRange)
@@ -1618,17 +1623,6 @@ export class LegacyFSController extends PuterController {
download.body.pipe(res);
};
// Helpers for writeFile
// -- GUI event emission -------------------------------------------
//
// Fire-and-forget `outer.gui.item.*` events so SocketService,
// BroadcastService, WorkerDriver (hot-reload), and cache-invalidation
// listeners pick up mutations made through the legacy (bare-path) routes.
// FSController (v2-native /fs/* routes) emits these from its own handlers;
// LegacyFSController delegates to the same FSService but needs its
// own emissions because the service layer deliberately doesn't emit GUI
// events (that's a controller concern).
async #emitGuiEvent(
eventName:
| 'outer.gui.item.added'
@@ -44,6 +44,7 @@ export class HostingController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
requireVerified: true,
},
async (req, res) => {
@@ -38,7 +38,11 @@ export class NotificationController extends PuterController {
* POST /notif/mark-ack — user dismissed a notification.
* Sets `acknowledged` timestamp; pushes ack event to sockets.
*/
@Post('/mark-ack', { subdomain: 'api', requireUserActor: true })
@Post('/mark-ack', {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
})
async markAck(req: Request, res: Response): Promise<void> {
const uid = req.body?.uid;
if (typeof uid !== 'string' || uid.length === 0) {
@@ -77,7 +81,11 @@ export class NotificationController extends PuterController {
* POST /notif/mark-read — user saw a notification.
* Sets `shown` timestamp; pushes ack event to sockets.
*/
@Post('/mark-read', { subdomain: 'api', requireUserActor: true })
@Post('/mark-read', {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
})
async markRead(req: Request, res: Response): Promise<void> {
const uid = req.body?.uid;
if (typeof uid !== 'string' || uid.length === 0) {
+21 -1
View File
@@ -17,6 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
import type { Request, Response } from 'express';
import type { Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
@@ -24,6 +25,21 @@ import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterController } from '../types.js';
import { PEER_COSTS } from './costs.js';
/**
* Constant-time secret comparison for the internal-auth header. HMAC both
* sides under a random per-process key to a fixed 32-byte digest first:
* this avoids leaking length via an early-return and sidesteps
* `timingSafeEqual`'s equal-length requirement for arbitrary-length
* inputs. The key need not persist — it only has to be unknown to the
* attacker for the duration of the comparison.
*/
const COMPARE_KEY = randomBytes(32);
const secretsEqual = (a: string, b: string): boolean => {
const ha = createHmac('sha256', COMPARE_KEY).update(a).digest();
const hb = createHmac('sha256', COMPARE_KEY).update(b).digest();
return timingSafeEqual(ha, hb);
};
/**
* Encode a UUID (or `app-<uuid>` UID) as base64url with no padding.
* Strips an `app-` prefix and dashes, then reinterprets the hex bytes.
@@ -170,7 +186,11 @@ export class PeerController extends PuterController {
}
const expectedSecret = cfg.internal_auth_secret;
const header = req.headers['x-puter-internal-auth'];
if (!expectedSecret || header !== expectedSecret) {
if (
!expectedSecret ||
typeof header !== 'string' ||
!secretsEqual(header, expectedSecret)
) {
throw new HttpError(403, 'Forbidden', { legacyCode: 'forbidden' });
}
@@ -188,10 +188,11 @@ describe('PuterAIController.registerRoutes', () => {
]),
);
// The four upstream-proxy routes are user-only: app actors must
// call puter-chat-completion directly. The `requireUserActor`
// route option wires up the gate that enforces this, so each
// proxy route gets the assertion (not just chat/completions).
// The four upstream-proxy routes reject third-party apps (they must
// call puter-chat-completion directly), but admit the user's own
// full-access personal access token: `requireUserActor` keeps apps out
// and `allowFullAccessToken` opens the gate to a full-access PAT. Each
// proxy route carries both flags.
const userOnlyPaths = [
'/puterai/openai/v1/chat/completions',
'/puterai/openai/v1/completions',
@@ -203,6 +204,7 @@ describe('PuterAIController.registerRoutes', () => {
expect(route?.opts).toEqual({
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
});
}
const modelsRoute = calls.find(
@@ -1504,6 +1506,212 @@ describe('PuterAIController.anthropicMessages streaming + helpers', () => {
});
});
// ── Inline compaction ───────────────────────────────────────────────
describe('PuterAIController inline compaction', () => {
// Extract the single `event: compaction\ndata: {...}` SSE frame.
const compactionFrame = (out: string): string | null => {
const m = out.match(/event: compaction\ndata: [^\n]*\n\n/);
return m ? m[0] : null;
};
it('forwards the compaction opt-in to the driver (/responses)', async () => {
const spy = stubChatComplete({
message: { role: 'assistant', content: 'ok' },
finish_reason: 'stop',
});
await controller.openaiResponses(
makeReq({
body: { model: 'gpt-test', input: 'hi', compaction: true },
actor: makeUserActor(),
}),
makeRes().res,
);
expect(spy.mock.calls[0]![0].compaction).toBe(true);
});
it('round-trips a compaction `input` item into a messages compaction item', async () => {
const spy = stubChatComplete({
message: { role: 'assistant', content: 'ack' },
finish_reason: 'stop',
});
await controller.openaiResponses(
makeReq({
body: {
model: 'gpt-test',
input: [
{
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
],
},
actor: makeUserActor(),
}),
makeRes().res,
);
expect(spy.mock.calls[0]![0].messages).toContainEqual({
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
});
});
it('emits a canonical compaction SSE event and output item (/responses stream)', async () => {
stubChatComplete({
dataType: 'stream',
content_type: 'application/x-ndjson',
stream: ndjsonStreamFrom([
{ type: 'text', text: 'hi' },
{ type: 'compaction', id: 'cmpct_9', encrypted_content: 'ENC9' },
{ type: 'usage', usage: { prompt_tokens: 1, completion_tokens: 1 } },
]),
});
const { res, captured } = makeRes();
await controller.openaiResponses(
makeReq({
body: {
model: 'gpt-test',
input: 'hi',
stream: true,
compaction: true,
},
actor: makeUserActor(),
}),
res,
);
await new Promise<void>((resolve) => setImmediate(resolve));
const out = captured.written.join('');
expect(compactionFrame(out)).toBe(
'event: compaction\ndata: {"type":"compaction","id":"cmpct_9","encrypted_content":"ENC9"}\n\n',
);
// Native shape also lands in the final response.completed output[].
const completed = out
.split('event: response.completed\n')[1]
?.split('\n\n')[0];
expect(completed).toContain('"type":"compaction"');
expect(completed).toContain('"encrypted_content":"ENC9"');
});
it('emits the compaction item in non-streaming /responses output', async () => {
stubChatComplete({
message: { role: 'assistant', content: 'done' },
finish_reason: 'stop',
compaction: { id: 'cmpct_n', encrypted_content: 'ENCN' },
});
const { res, captured } = makeRes();
await controller.openaiResponses(
makeReq({
body: { model: 'gpt-test', input: 'hi', compaction: true },
actor: makeUserActor(),
}),
res,
);
const body = captured.body as { output: Array<Record<string, unknown>> };
expect(body.output).toContainEqual(
expect.objectContaining({
type: 'compaction',
encrypted_content: 'ENCN',
}),
);
});
it('emits an identical canonical compaction SSE event on the Anthropic surface', async () => {
// /responses frame
stubChatComplete({
dataType: 'stream',
content_type: 'application/x-ndjson',
stream: ndjsonStreamFrom([
{ type: 'compaction', id: 'cmpct_x', encrypted_content: 'ENCX' },
]),
});
const r1 = makeRes();
await controller.openaiResponses(
makeReq({
body: { model: 'gpt-test', input: 'hi', stream: true },
actor: makeUserActor(),
}),
r1.res,
);
await new Promise<void>((resolve) => setImmediate(resolve));
// /anthropic/v1/messages frame
stubChatComplete({
dataType: 'stream',
content_type: 'application/x-ndjson',
stream: ndjsonStreamFrom([
{ type: 'compaction', id: 'cmpct_x', encrypted_content: 'ENCX' },
]),
});
const r2 = makeRes();
await controller.anthropicMessages(
makeReq({
body: {
model: 'claude-test',
messages: [{ role: 'user', content: 'hi' }],
stream: true,
},
actor: makeUserActor(),
}),
r2.res,
);
await new Promise<void>((resolve) => setImmediate(resolve));
const a = compactionFrame(r1.captured.written.join(''));
const b = compactionFrame(r2.captured.written.join(''));
expect(a).not.toBeNull();
expect(a).toBe(b); // byte-identical streaming shape across providers
});
it('renders a native compaction content block in non-streaming /messages', async () => {
stubChatComplete({
message: { role: 'assistant', content: 'done' },
finish_reason: 'stop',
compaction: { id: 'cmpct_m', encrypted_content: 'ENCM' },
});
const { res, captured } = makeRes();
await controller.anthropicMessages(
makeReq({
body: {
model: 'claude-test',
messages: [{ role: 'user', content: 'hi' }],
},
actor: makeUserActor(),
}),
res,
);
const body = captured.body as { content: Array<Record<string, unknown>> };
expect(body.content).toContainEqual({
type: 'compaction',
id: 'cmpct_m',
encrypted_content: 'ENCM',
});
});
it('does not emit compaction frames for a normal stream (regression)', async () => {
stubChatComplete({
dataType: 'stream',
content_type: 'application/x-ndjson',
stream: ndjsonStreamFrom([
{ type: 'text', text: 'hello' },
{ type: 'usage', usage: { prompt_tokens: 1, completion_tokens: 1 } },
]),
});
const { res, captured } = makeRes();
await controller.openaiResponses(
makeReq({
body: { model: 'gpt-test', input: 'hi', stream: true },
actor: makeUserActor(),
}),
res,
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(captured.written.join('')).not.toContain('event: compaction');
});
});
// ── Model details listing ───────────────────────────────────────────
describe('PuterAIController model details', () => {
@@ -52,6 +52,12 @@ export class PuterAIController extends PuterController {
const apiAuthOpts = {
subdomain: 'api',
requireUserActor: true,
// These are the user's own programmatic AI surface (OpenAI/Anthropic
// wire). `requireUserActor` here exists to keep third-party apps
// out, not to block the user's own credential — so admit full-access
// personal access tokens (CLI/MCP/scripts). Apps and scoped tokens
// stay blocked.
allowFullAccessToken: true,
} as RouteOptions;
const publicOpts = { subdomain: 'api', requireAuth: false } as const;
@@ -581,6 +587,15 @@ export class PuterAIController extends PuterController {
? { metadata: body.metadata as Record<string, string> }
: {}),
...(body.conversation ? { conversation: body.conversation } : {}),
...(body.context_management !== undefined
? { context_management: body.context_management }
: {}),
...(body.compaction !== undefined
? {
compaction:
body.compaction as ICompleteArguments['compaction'],
}
: {}),
...(body.previous_response_id
? { previous_response_id: String(body.previous_response_id) }
: {}),
@@ -744,6 +759,30 @@ export class PuterAIController extends PuterController {
output_index: outputIndex,
item,
});
} else if (ev.type === 'compaction') {
// Native shape in the final `output[]`, plus the
// canonical SSE event shared with the Anthropic surface.
const item = {
type: 'compaction',
...(ev.id !== undefined ? { id: ev.id } : {}),
encrypted_content: ev.encrypted_content,
};
const outputIndex = output.length;
output.push(item);
sendEvent({
type: 'response.output_item.added',
output_index: outputIndex,
item,
});
sendEvent({
type: 'response.output_item.done',
output_index: outputIndex,
item,
});
writeCompactionEvent(res, {
id: ev.id,
encrypted_content: ev.encrypted_content,
});
} else if (ev.type === 'usage') {
usage = buildResponsesUsage(
ev.usage as Record<string, unknown>,
@@ -863,6 +902,15 @@ export class PuterAIController extends PuterController {
...(body.max_tokens !== undefined
? { max_tokens: Number(body.max_tokens) }
: {}),
...(body.context_management !== undefined
? { context_management: body.context_management }
: {}),
...(body.compaction !== undefined
? {
compaction:
body.compaction as ICompleteArguments['compaction'],
}
: {}),
...(body.provider
? { provider: toStringOrEmpty(body.provider) }
: { provider: DEFAULTS.anthropic }),
@@ -962,6 +1010,14 @@ export class PuterAIController extends PuterController {
},
});
closeBlock();
} else if (ev.type === 'compaction') {
// Close any open content block, then emit the canonical
// compaction SSE event — byte-identical to /responses.
closeBlock();
writeCompactionEvent(res, {
id: ev.id,
encrypted_content: ev.encrypted_content,
});
} else if (ev.type === 'usage') {
usage = ev.usage as Record<string, unknown>;
}
@@ -1020,6 +1076,18 @@ export class PuterAIController extends PuterController {
if (textContent)
contentBlocks.push({ type: 'text', text: textContent });
contentBlocks.push(...toolUseBlocks);
// Native Anthropic-shaped compaction block (non-streaming bodies stay
// provider-native, unlike the unified streaming event).
const compaction = (
messageResult as { compaction?: Record<string, unknown> }
).compaction;
if (compaction) {
contentBlocks.push({
type: 'compaction',
...(compaction.id !== undefined ? { id: compaction.id } : {}),
encrypted_content: compaction.encrypted_content,
});
}
if (contentBlocks.length === 0)
contentBlocks.push({ type: 'text', text: '' });
@@ -1076,6 +1144,25 @@ const setSseHeaders = (res: Response): void => {
res.setHeader('Connection', 'keep-alive');
};
/**
* Inline-compaction is emitted in a single canonical SSE shape that is
* byte-identical across the `/responses` and `/anthropic/v1/messages` streaming
* surfaces, so a streaming client parses compaction the same way regardless of
* which upstream served the request. (Non-streaming bodies stay provider-native.)
*/
const writeCompactionEvent = (
res: Response,
compaction: { id?: unknown; encrypted_content?: unknown },
): void => {
const payload = {
type: 'compaction',
...(compaction.id !== undefined ? { id: compaction.id } : {}),
encrypted_content: compaction.encrypted_content,
};
res.write(`event: compaction\n`);
res.write(`data: ${JSON.stringify(payload)}\n\n`);
};
/**
* The chat driver returns either a stream-result envelope or a plain
* message result. Proxy routes invoked with `stream: true` expect the
@@ -1319,6 +1406,17 @@ const responseInputToMessages = (input: unknown): unknown[] => {
if (!item || typeof item !== 'object') continue;
const it = item as Record<string, unknown>;
if (it.type === 'compaction') {
// Round-tripped compaction artifact: preserve it as a bare item so
// `normalize_single_message` wraps it into an internal compaction
// content block (providers map it back to their native input shape).
messages.push({
type: 'compaction',
...(it.id !== undefined ? { id: it.id } : {}),
encrypted_content: it.encrypted_content,
});
continue;
}
if (it.type === 'function_call_output') {
messages.push({
role: 'tool',
@@ -1410,6 +1508,16 @@ const responseOutputFromResult = (
});
}
const compaction = (result as { compaction?: Record<string, unknown> })
.compaction;
if (compaction) {
output.push({
id: (compaction.id as string | undefined) || generateId('cmpct'),
type: 'compaction',
encrypted_content: compaction.encrypted_content,
});
}
return output;
};
@@ -77,6 +77,7 @@ export class SystemController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: {
scope: 'contact-us',
limit: 10,
@@ -4,6 +4,7 @@ import type { Request, Response } from 'express';
import { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { hash as bcryptHash } from 'bcrypt';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -228,6 +229,112 @@ describe('WebDAVController', () => {
});
});
describe('pending-verification gate', () => {
// WebDAV must enforce the same gate every other authenticated route
// gets from requireVerifiedAccount — it dispatches off a single use()
// with no route options, so the middleware is never wired in and it
// has to call assertVerifiedAccount itself. Without it, an account
// still pending email/phone/card verification could read/write its
// whole filesystem over the `dav` subdomain.
const gatedActor = (flags: Record<string, unknown>) => ({
user: {
id: 1,
uuid: 'gated-uuid',
username: 'gated',
...flags,
},
});
it('rejects a session actor pending phone verification with 403', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'PROPFIND',
actor: gatedActor({ requires_phone_verification: true }),
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
it('rejects a session actor pending card verification with 403', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'GET',
actor: gatedActor({ requires_card_verification: true }),
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
it('rejects a session actor with an unconfirmed email with 403', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'PROPFIND',
actor: gatedActor({
requires_email_confirmation: true,
email_confirmed: false,
}),
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
it('enforces the gate on the Basic-auth path (flags carried onto the built actor)', async () => {
const username = `webdav-gated-${Math.random()
.toString(36)
.slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: await bcryptHash('correct-horse', 4),
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await server.stores.user.update(created.id, {
requires_phone_verification: 1,
});
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'PROPFIND',
headers: {
authorization: basicAuth(username, 'correct-horse'),
},
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
it('lets a fully-verified session actor through the gate', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'OPTIONS',
actor: gatedActor({
requires_phone_verification: false,
requires_card_verification: false,
requires_email_confirmation: false,
}),
}),
res,
noop,
);
expect(captured.statusCode).toBe(200);
});
});
describe('unsupported methods', () => {
it('returns 405 for unknown HTTP methods', async () => {
const { res, captured } = makeRes();
@@ -23,6 +23,7 @@ import { posix as pathPosix } from 'node:path';
import { EventMap } from '../../clients/event/types.js';
import type { Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { assertVerifiedAccount } from '../../core/http/middleware/gates.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
@@ -89,6 +90,16 @@ export class WebDAVController extends PuterController {
const actor = await this.#resolveActor(req, res);
if (!actor) return; // 401 already sent
// Apply the same pending-verification gate every other authenticated
// route gets from `requireVerifiedAccount`. WebDAV dispatches every
// method off a single `router.use` with no route options, so that
// middleware is never inserted into its chain — without this call an
// account still pending email / phone / card verification could read,
// write, and delete its entire filesystem over the `dav` subdomain,
// bypassing the gate. Throws a 403 HttpError, surfaced by the catch in
// registerRoutes.
assertVerifiedAccount(actor.user);
// Expand `~`/`~/...` against the authenticated actor's username.
// WebDAV doesn't standardize `~`, but some clients do — and the
// pre-existing behaviour silently expanded it via the FS store.
@@ -233,6 +244,15 @@ export class WebDAVController extends PuterController {
email_confirmed: user.email_confirmed ?? false,
requires_email_confirmation:
user.requires_email_confirmation ?? false,
// Carry the signup-time verification flags so the
// assertVerifiedAccount() gate in #dispatch can see them on
// the Basic-auth path too (the cookie / `-token` paths get
// them from AuthService#actorUserFromRow). Omitting these is
// what let phone/card-gated accounts through WebDAV.
requires_phone_verification:
user.requires_phone_verification ?? false,
requires_card_verification:
user.requires_card_verification ?? false,
},
};
}
+9
View File
@@ -33,6 +33,15 @@ export interface ActorAccessToken {
uid: string;
issuer: Actor;
authorized?: Actor | null;
/**
* Full-API-access ("personal access token") flag, set from the signed
* `full_access` JWT claim. Such a token may exercise everything its issuing
* user can do via the API and is admitted past `requireNonAccessTokenGate`
* — but is still rejected by `requireUserActor` / web-session gates, so it
* can never manage the account. Normal (scoped) access tokens leave this
* false and remain blocked from non-`allowAccessToken` routes.
*/
fullAccess?: boolean;
}
export interface Actor {
@@ -66,6 +66,21 @@ declare global {
/** True when the request's Host is a custom domain (not one of the configured Puter domains). */
is_custom_domain?: boolean;
/**
* Coarse server-derived request fingerprint (IP + UA + accept
* headers), populated by the global fingerprint middleware. Always
* present; the same value the rate limiter keys on.
*/
networkFingerprint?: string;
/**
* Client-supplied device fingerprint (ThumbmarkJS hash) from the
* body or `x-puter-device-fingerprint` header, populated by the
* global fingerprint middleware. Present only when the client sent a
* well-shaped value; spoofable but stable per device across IPs.
*/
deviceFingerprint?: string;
/** Parsed cookies, populated by the global `cookie-parser` middleware. */
cookies?: Record<string, string>;
}
@@ -73,6 +73,19 @@ export const antiCsrf = {
export function requireAntiCsrf() {
return async (req, _res, next) => {
try {
// CSRF only applies to ambient credentials the browser attaches
// automatically (the session cookie). A full-access personal access
// token travels in the Authorization header — it can't be forged
// cross-origin — so it needs no anti-CSRF token, which is what makes
// routes like `/fs/down` reachable by a PAT. We deliberately scope
// this to full-access tokens (the only access tokens routed onto
// antiCsrf-protected endpoints): anything else — cookie-authed user
// actors, app actors, or a scoped token that somehow reaches here —
// still goes through the check below and fails closed.
if (req.actor?.accessToken?.fullAccess) {
return next();
}
const sessionId = req.actor?.user?.uuid;
if (!sessionId) {
return next(
@@ -118,6 +118,51 @@ describe('requireAntiCsrf middleware', () => {
expect(arg).toBeUndefined();
});
it('exempts a FULL-ACCESS access token (no anti_csrf token needed)', async () => {
// A full-access PAT is header-credentialed (bearer), so it can't be
// CSRF-forged — the middleware lets it through without a token. This is
// what makes antiCsrf routes like /fs/down reachable by a PAT.
const arg = await runMiddleware({
actor: {
user: { uuid: 'user-uuid-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'user-uuid-1' } },
fullAccess: true,
},
},
body: {},
});
expect(arg).toBeUndefined();
});
it('does NOT exempt a scoped (non-full-access) access token — fails closed', async () => {
// Scoped tokens are never deliberately routed onto antiCsrf endpoints;
// if one reaches here it must still present a token (which it can't get)
// rather than getting a free pass.
const arg = await runMiddleware({
actor: {
user: { uuid: 'user-uuid-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'user-uuid-1' } },
},
},
body: {},
});
expect(isHttpError(arg)).toBe(true);
expect(arg.statusCode).toBe(400);
});
it('still enforces the token for cookie-authed user actors (no accessToken)', async () => {
const arg = await runMiddleware({
actor: { user: { uuid: 'user-uuid-1' } },
body: {},
});
expect(isHttpError(arg)).toBe(true);
expect(arg.statusCode).toBe(400);
});
it('returns 401 unauthorized when no actor is attached', async () => {
const arg = await runMiddleware({ body: { anti_csrf: 'whatever' } });
expect(isHttpError(arg)).toBe(true);
@@ -0,0 +1,93 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { describe, expect, it, vi } from 'vitest';
import { createFingerprintMiddleware } from './fingerprint';
const VALID_FP = 'dev-fingerprint-abc123';
const run = (reqPartial: Partial<Request> & Record<string, unknown>) => {
const middleware = createFingerprintMiddleware();
const req = {
headers: {},
...reqPartial,
} as unknown as Request;
const next = vi.fn();
middleware(req, {} as Response, next);
return { req, next };
};
describe('createFingerprintMiddleware', () => {
it('always sets a well-formed networkFingerprint and calls next', () => {
const { req, next } = run({ ip: '203.0.113.1' });
expect(req.networkFingerprint).toMatch(/^[A-Za-z0-9_-]{16}$/);
expect(next).toHaveBeenCalledOnce();
});
it('networkFingerprint is stable for identical inputs and varies by UA', () => {
const headers = { 'user-agent': 'UA/1.0' };
const a = run({ ip: '203.0.113.1', headers }).req.networkFingerprint;
const b = run({ ip: '203.0.113.1', headers }).req.networkFingerprint;
const c = run({
ip: '203.0.113.1',
headers: { 'user-agent': 'UA/2.0' },
}).req.networkFingerprint;
expect(a).toBe(b);
expect(a).not.toBe(c);
});
it('reads a well-shaped device fingerprint from the body', () => {
const { req } = run({ body: { fingerprint: VALID_FP } });
expect(req.deviceFingerprint).toBe(VALID_FP);
});
it('falls back to the x-puter-device-fingerprint header', () => {
const { req } = run({
headers: { 'x-puter-device-fingerprint': VALID_FP },
});
expect(req.deviceFingerprint).toBe(VALID_FP);
});
it('prefers the body fingerprint over the header', () => {
const { req } = run({
body: { fingerprint: VALID_FP },
headers: { 'x-puter-device-fingerprint': 'other-fingerprint-xyz' },
});
expect(req.deviceFingerprint).toBe(VALID_FP);
});
it('drops a malformed fingerprint to undefined', () => {
expect(run({ body: { fingerprint: 'bad fp!' } }).req.deviceFingerprint)
.toBeUndefined();
expect(run({ body: { fingerprint: 'short' } }).req.deviceFingerprint)
.toBeUndefined();
expect(
run({ body: { fingerprint: 123 as unknown as string } }).req
.deviceFingerprint,
).toBeUndefined();
});
it('leaves deviceFingerprint undefined when nothing was supplied', () => {
const { req } = run({ ip: '203.0.113.1' });
expect(req.deviceFingerprint).toBeUndefined();
});
});
@@ -0,0 +1,76 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { RequestHandler } from 'express';
import { computeNetworkFingerprint } from './rateLimit.js';
import '../expressAugmentation';
/**
* Conservative charset/length for the client-supplied device fingerprint. It
* becomes part of KV keys downstream (abuse extension), so anything outside
* this shape is dropped to `undefined` rather than trusted. Kept in sync with
* the abuse extension's `DEVICE_FINGERPRINT_SHAPE`; the core only enforces the
* shape, never any abuse policy built on the value.
*/
const DEVICE_FINGERPRINT_SHAPE = /^[A-Za-z0-9._-]{8,128}$/;
/** Header the GUI may send the device fingerprint on for non-signup requests. */
const DEVICE_FINGERPRINT_HEADER = 'x-puter-device-fingerprint';
function readDeviceFingerprint(req: {
body?: unknown;
headers?: Record<string, unknown>;
}): string | undefined {
// Body first (what /signup already sends), then a header fallback so
// authenticated, bodyless-or-different-shape requests can still carry it.
const body = req.body as { fingerprint?: unknown } | undefined;
const candidate =
(typeof body?.fingerprint === 'string'
? body.fingerprint
: undefined) ??
(typeof req.headers?.[DEVICE_FINGERPRINT_HEADER] === 'string'
? (req.headers[DEVICE_FINGERPRINT_HEADER] as string)
: undefined);
if (candidate && DEVICE_FINGERPRINT_SHAPE.test(candidate)) return candidate;
return undefined;
}
/**
* Stamp request-scoped fingerprints onto `req` so any downstream gate, handler,
* or service (via the ALS `Context.get('req')`) can read them without recomputing:
*
* - `req.networkFingerprint` — always set; a coarse IP+headers hash (the same
* value the rate limiter keys on). Server-derived, so it can't be forged
* away, but it's coarse (shared behind NAT/VPN, rotates with UA).
* - `req.deviceFingerprint` — set only when the client supplied a well-shaped
* device fingerprint (ThumbmarkJS hash) in the body or the
* `x-puter-device-fingerprint` header; `undefined` otherwise. Client-supplied
* and spoofable, but stable per real device across IP rotation.
*
* Install AFTER the body parsers (so the body fingerprint is readable) and
* before `requestContext` (so the snapshot into ALS already carries them).
* Never rejects — a missing/invalid device fingerprint is simply absent.
*/
export const createFingerprintMiddleware = (): RequestHandler => {
return (req, _res, next) => {
req.networkFingerprint = computeNetworkFingerprint(req);
req.deviceFingerprint = readDeviceFingerprint(req);
next();
};
};
+155 -7
View File
@@ -25,7 +25,8 @@ import {
adminOnlyGate,
allowedAppIdsGate,
requireAuthGate,
requireEmailConfirmedGate,
requireVerifiedAccount,
requireNonAccessTokenGate,
requireUserActorGate,
requireVerifiedGate,
subdomainGate,
@@ -224,6 +225,110 @@ describe('requireUserActorGate', () => {
const got = runGate(requireUserActorGate(), {});
expectHttpError(got, 401, 'token_missing');
});
it('rejects a FULL-ACCESS access token too — account routes stay closed', () => {
// The account wall is actor-type based: even a full-access PAT (which
// the resource wall lets through) is rejected here, so it can never
// reach change-password/email/2FA/token-minting/etc.
const got = runGate(requireUserActorGate(), {
actor: {
user: { uuid: 'u-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1' } },
fullAccess: true,
},
},
});
expectHttpError(got, 403, 'forbidden');
});
// allowFullAccess opt-in: relaxes ONLY the access-token half, for
// user-resource/inference routes (e.g. the AI proxy).
it('admits a full-access PAT when allowFullAccess is set', () => {
const got = runGate(requireUserActorGate({ allowFullAccess: true }), {
actor: {
user: { uuid: 'u-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1' } },
fullAccess: true,
},
},
});
expect(got).toBeUndefined();
});
it('still rejects a SCOPED access token even when allowFullAccess is set', () => {
const got = runGate(requireUserActorGate({ allowFullAccess: true }), {
actor: {
user: { uuid: 'u-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1' } },
// no fullAccess
},
},
});
expectHttpError(got, 403, 'forbidden');
});
it('still rejects a third-party app even when allowFullAccess is set', () => {
const got = runGate(requireUserActorGate({ allowFullAccess: true }), {
actor: { user: { uuid: 'u-1' }, app: { uid: 'app-1' } },
});
expectHttpError(got, 403, 'forbidden');
});
});
// -- requireNonAccessTokenGate --
describe('requireNonAccessTokenGate', () => {
it('passes through for plain user actors', () => {
const got = runGate(requireNonAccessTokenGate(), {
actor: { user: { uuid: 'u-1' } },
});
expect(got).toBeUndefined();
});
it('passes through for app-under-user actors (only access tokens are gated)', () => {
const got = runGate(requireNonAccessTokenGate(), {
actor: { user: { uuid: 'u-1' }, app: { uid: 'app-1' } },
});
expect(got).toBeUndefined();
});
it('rejects a normal (scoped) access token with 403', () => {
const got = runGate(requireNonAccessTokenGate(), {
actor: {
user: { uuid: 'u-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1' } },
},
},
});
expectHttpError(got, 403, 'forbidden');
});
it('admits a FULL-ACCESS access token (the resource-wall carve-out)', () => {
const got = runGate(requireNonAccessTokenGate(), {
actor: {
user: { uuid: 'u-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1' } },
fullAccess: true,
},
},
});
expect(got).toBeUndefined();
});
it("falls back to 401 if there's no actor at all (defensive)", () => {
const got = runGate(requireNonAccessTokenGate(), {});
expectHttpError(got, 401, 'token_missing');
});
});
// ── adminOnlyGate ───────────────────────────────────────────────────
@@ -335,11 +440,11 @@ describe('requireVerifiedGate', () => {
});
});
// ── requireEmailConfirmedGate ───────────────────────────────────────
// ── requireVerifiedAccount ──────────────────────────────────────────
describe('requireEmailConfirmedGate', () => {
describe('requireVerifiedAccount', () => {
it('passes through users that do not require confirmation (e.g. legacy/temp)', () => {
const got = runGate(requireEmailConfirmedGate(), {
const got = runGate(requireVerifiedAccount(), {
actor: {
user: {
uuid: 'u-1',
@@ -352,7 +457,7 @@ describe('requireEmailConfirmedGate', () => {
});
it('passes through confirmed users even when confirmation is required', () => {
const got = runGate(requireEmailConfirmedGate(), {
const got = runGate(requireVerifiedAccount(), {
actor: {
user: {
uuid: 'u-1',
@@ -365,7 +470,7 @@ describe('requireEmailConfirmedGate', () => {
});
it('returns 403 email_confirmation_required for pending-confirmation users', () => {
const got = runGate(requireEmailConfirmedGate(), {
const got = runGate(requireVerifiedAccount(), {
actor: {
user: {
uuid: 'u-1',
@@ -377,8 +482,51 @@ describe('requireEmailConfirmedGate', () => {
expectHttpError(got, 403, 'email_confirmation_required');
});
it('returns 403 phone_verification_required while the phone gate is set', () => {
const got = runGate(requireVerifiedAccount(), {
actor: {
user: {
uuid: 'u-1',
requires_email_confirmation: false,
email_confirmed: true,
requires_phone_verification: true,
},
},
});
expectHttpError(got, 403, 'phone_verification_required');
});
it('returns 403 card_verification_required while the card gate is set', () => {
const got = runGate(requireVerifiedAccount(), {
actor: {
user: {
uuid: 'u-1',
requires_email_confirmation: false,
email_confirmed: true,
requires_card_verification: true,
},
},
});
expectHttpError(got, 403, 'card_verification_required');
});
it('passes through once every gate is cleared', () => {
const got = runGate(requireVerifiedAccount(), {
actor: {
user: {
uuid: 'u-1',
requires_email_confirmation: true,
email_confirmed: true,
requires_phone_verification: false,
requires_card_verification: false,
},
},
});
expect(got).toBeUndefined();
});
it("passes through when there's no actor (auth gate handled it)", () => {
const got = runGate(requireEmailConfirmedGate(), {});
const got = runGate(requireVerifiedAccount(), {});
expect(got).toBeUndefined();
});
});
+95 -17
View File
@@ -99,8 +99,18 @@ export const requireAuthGate = (): RequestHandler => {
* Reject app-under-user and access-token actors with 403. Use on endpoints
* that should only be exercised by a human session — settings changes,
* admin-style actions on the user's own account.
*
* `allowFullAccess` (set per-route via the `allowFullAccessToken` route option)
* relaxes ONLY the access-token half: a full-access ("personal access token")
* actor is admitted, because it represents the user's own full API reach.
* Third-party apps are ALWAYS rejected, and scoped access tokens are always
* rejected. This opt-in is for user-resource / inference endpoints (AI proxy,
* etc.) that use this gate purely to keep apps out — NEVER for account or
* security management, which must stay closed to every access token.
*/
export const requireUserActorGate = (): RequestHandler => {
export const requireUserActorGate = (
opts: { allowFullAccess?: boolean } = {},
): RequestHandler => {
return (req, _res, next) => {
const actor = req.actor;
// requireAuth runs first; this gate just narrows the actor type.
@@ -108,7 +118,14 @@ export const requireUserActorGate = (): RequestHandler => {
next(rejectAuth(req));
return;
}
if (actor.app || actor.accessToken) {
// Third-party apps are never allowed through this gate.
const appBlocked = !!actor.app;
// Access tokens are blocked unless the route opted in AND this is a
// full-access PAT (the user's own credential). Scoped tokens: blocked.
const tokenBlocked =
!!actor.accessToken &&
!(opts.allowFullAccess && actor.accessToken.fullAccess);
if (appBlocked || tokenBlocked) {
next(
new HttpError(
403,
@@ -129,7 +146,13 @@ export const requireNonAccessTokenGate = (): RequestHandler => {
next(rejectAuth(req));
return;
}
if (actor.accessToken) {
// Full-access ("personal access token") access tokens are admitted here:
// they carry the user's full API reach by design. They remain blocked
// from account management because those routes also use
// `requireUserActorGate`, which rejects ALL access tokens. Normal
// (scoped) access tokens stay blocked from non-`allowAccessToken`
// routes.
if (actor.accessToken && !actor.accessToken.fullAccess) {
next(
new HttpError(
403,
@@ -201,29 +224,84 @@ export const requireVerifiedGate = (strictFlag: boolean): RequestHandler => {
};
/**
* Reject authenticated users whose account is pending email confirmation
* (`requires_email_confirmation && !email_confirmed`). Runs on every
* authenticated route by default; routes that set `allowUnconfirmed: true`
* skip this gate.
* Reject authenticated users whose account is still pending any signup-time
* verification — email confirmation, SMS phone verification, or credit-card
* verification. The abuse harness sets the phone/card flags on low-reputation
* signups (in place of a hard block), and this gate is what actually keeps
* those accounts out of the product until the flag clears: the flags live on
* `req.actor.user`, so every authenticated route enforces them, not just the
* GUI modal.
*
* Returns 403 with `email_confirmation_required` so clients can show the
* confirmation prompt instead of a generic error.
* Runs on every authenticated route by default; routes that set
* `allowUnconfirmed: true` opt out (the verification endpoints themselves,
* plus essential flows like whoami / logout / save-account so a pending
* account can still reach the screens that clear the gate).
*
* Returns 403 with a per-gate legacy code (`email_confirmation_required` /
* `phone_verification_required` / `card_verification_required`) so clients can
* show the right prompt instead of a generic error. There is no state where a
* user should be allowed in with one verification pending, so any pending gate
* rejects.
*/
export const requireEmailConfirmedGate = (): RequestHandler => {
export const requireVerifiedAccount = (): RequestHandler => {
return (req, _res, next) => {
const user = req.actor?.user;
if (user?.requires_email_confirmation && !user?.email_confirmed) {
next(
new HttpError(403, 'Please confirm your email to continue', {
legacyCode: 'email_confirmation_required',
}),
);
try {
assertVerifiedAccount(req.actor?.user);
} catch (err) {
next(err);
return;
}
next();
};
};
/**
* The pending-verification check, factored out of {@link requireVerifiedAccount}
* so auth paths that build their own actor outside the route-option machinery
* can enforce the exact same gate. The WebDAV controller is the motivating
* case: it dispatches every method off a single `router.use`, so
* `requireVerifiedAccount` is never wired into its chain — it has to call this
* directly. Keeping one implementation is the point: a verification gate added
* here is picked up by every caller, so the paths can't drift (which is how
* WebDAV came to bypass the phone/card gate to begin with).
*
* Throws 403 with a per-gate legacy code (`email_confirmation_required` /
* `phone_verification_required` / `card_verification_required`) so clients can
* show the right prompt instead of a generic error. There is no state where a
* user should be let in with any verification pending, so the first pending
* gate rejects.
*/
export const assertVerifiedAccount = (
user:
| {
requires_email_confirmation?: unknown;
email_confirmed?: unknown;
requires_phone_verification?: unknown;
requires_card_verification?: unknown;
}
| undefined,
): void => {
if (user?.requires_email_confirmation && !user?.email_confirmed) {
throw new HttpError(403, 'Please confirm your email to continue', {
legacyCode: 'email_confirmation_required',
});
}
if (user?.requires_phone_verification) {
throw new HttpError(
403,
'Please verify your phone number to continue',
{
legacyCode: 'phone_verification_required' as never,
},
);
}
if (user?.requires_card_verification) {
throw new HttpError(403, 'Please verify your card to continue', {
legacyCode: 'card_verification_required' as never,
});
}
};
/**
* Reject unless the actor is acting through one of the named apps.
* App-under-user actors are permitted iff `actor.app.uid` is in the allowList;
@@ -264,6 +264,79 @@ describe('createUserSubdomainRedirect', () => {
]);
});
const selfHosted = {
domain: 'puter.localhost',
static_hosting_domain: 'site.puter.localhost',
static_hosting_domain_alt: 'host.puter.localhost',
private_app_hosting_domain: 'app.puter.localhost',
private_app_hosting_domain_alt: 'dev.puter.localhost',
} as IConfig;
it('still redirects a bare subdomain on the main domain to the hosting domain (self-hosted)', () => {
const { out, next } = run(
createUserSubdomainRedirect(selfHosted),
makeReq({
subdomains: ['localhost', 'puter', 'foo'],
host: 'foo.puter.localhost',
originalUrl: '/bar?x=1',
protocol: 'http',
}),
);
expect(out.redirectArgs).toEqual([
302,
'http://foo.site.puter.localhost/bar?x=1',
]);
expect(next).not.toHaveBeenCalled();
});
it('passes through hosts already on the static hosting domain (no redirect loop)', () => {
const { out, next } = run(
createUserSubdomainRedirect(selfHosted),
makeReq({
subdomains: ['localhost', 'puter', 'site', 'foo'],
host: 'foo.site.puter.localhost',
originalUrl: '/',
}),
);
expect(out.redirectArgs).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
});
it('passes through hosts on the alt / private-app hosting domains too', () => {
for (const host of [
'foo.host.puter.localhost',
'foo.app.puter.localhost',
'foo.dev.puter.localhost',
]) {
const { out, next } = run(
createUserSubdomainRedirect(selfHosted),
makeReq({
subdomains: [
'localhost',
'puter',
host.split('.')[1],
'foo',
],
host,
}),
);
expect(out.redirectArgs).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
}
});
it('passes through the hosting-domain root itself (exact match, no loop)', () => {
const { out, next } = run(
createUserSubdomainRedirect(selfHosted),
makeReq({
subdomains: ['localhost', 'puter', 'site'],
host: 'site.puter.localhost',
}),
);
expect(out.redirectArgs).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
});
it("passes through when the request host has a port the configured domain doesn't", () => {
// Edge case worth pinning: the suffix check is exact-`endsWith`,
// so a port mismatch silently bypasses the redirect. Documenting
@@ -88,6 +88,15 @@ export const createUserSubdomainRedirect = (
if (!domain || !target) {
return (_req, _res, next) => next();
}
const hostingDomains = [
config.static_hosting_domain,
config.static_hosting_domain_alt,
config.private_app_hosting_domain,
config.private_app_hosting_domain_alt,
]
.map((d) => (d ?? '').toLowerCase().split(':')[0])
.filter((d) => d.length > 0);
return (req, res, next) => {
const active = (
req.subdomains?.[req.subdomains.length - 1] ?? ''
@@ -95,6 +104,14 @@ export const createUserSubdomainRedirect = (
if (active === '' || RESERVED_SUBDOMAINS.has(active)) return next();
const host = (req.headers.host ?? '').toLowerCase();
const hostName = host.split(':')[0];
if (
hostingDomains.some(
(d) => hostName === d || hostName.endsWith(`.${d}`),
)
) {
return next();
}
if (!host.endsWith(domain)) return next();
// host ends in domain — swap the domain suffix for the hosting one,
@@ -534,6 +534,51 @@ describe('createPuterSiteMiddleware — file serving', () => {
expect(piped!.equals(body)).toBe(true);
});
it('emits site.htmlServed with the original URL target including query string', async () => {
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
const sub = `serveq-${Math.random().toString(36).slice(2, 8)}`;
await server.stores.subdomain.create({
userId: owner.id,
subdomain: sub,
rootDirId: homeEntry!.id,
});
await writeFile(
owner.id,
`${homePath}/index.html`,
Buffer.from('<html>hi</html>'),
'text/html',
);
const emitSpy = vi.spyOn(server.clients.event, 'emit');
try {
const mw = buildMiddleware();
const { res } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/index.html',
originalUrl: '/index.html?brand=paypal',
}),
res,
vi.fn(),
);
const htmlServedCall = emitSpy.mock.calls.find(
(call) => call[0] === 'site.htmlServed',
);
expect(htmlServedCall?.[1]).toMatchObject({
subdomain: sub,
requestPath: '/index.html',
requestUrl: '/index.html?brand=paypal',
mime: 'text/html',
});
} finally {
emitSpy.mockRestore();
}
});
it('returns 206 status when a Range header is supplied', async () => {
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
@@ -555,8 +555,12 @@ export const createPuterSiteMiddleware = (
res.setHeader('Content-Type', mime);
// Fire-and-forget signal for downstream extensions
if (mime === 'text/html' || mime === 'application/xhtml+xml') {
const mimeBase = mime.split(';', 1)[0].trim().toLowerCase();
if (mimeBase === 'text/html' || mimeBase === 'application/xhtml+xml') {
try {
const requestUrl = (req.originalUrl || '/').startsWith('/')
? req.originalUrl || '/'
: `/${req.originalUrl}`;
layers.clients.event.emit(
'site.htmlServed',
{
@@ -564,7 +568,8 @@ export const createPuterSiteMiddleware = (
entry,
host,
requestPath: req.path,
mime,
requestUrl,
mime: mimeBase,
},
{},
);
+12 -1
View File
@@ -392,7 +392,14 @@ function ip(req) {
return req.ip || req.socket?.remoteAddress || 'unknown';
}
function fingerprint(req) {
/**
* A coarse network fingerprint for a request: a short hash of the (proxy-aware)
* IP plus the headers a client can't trivially vary per-request without also
* changing how the request looks. Used as the default rate-limit key here, and
* exported so the global fingerprint middleware can stamp the identical value
* on `req.networkFingerprint` (one key space shared by both).
*/
export function computeNetworkFingerprint(req) {
const parts = [
ip(req),
req.headers?.['user-agent'] || '',
@@ -406,6 +413,10 @@ function fingerprint(req) {
.slice(0, 16);
}
function fingerprint(req) {
return computeNetworkFingerprint(req);
}
// -- Route middleware ------------------------------------------------
/**
@@ -140,6 +140,67 @@ describe('rateLimitGate — memory backend (default)', () => {
expect(isHttpError(re)).toBe(true);
});
// Stacked gates, mirroring the materializer's handling of a
// `rateLimit: [...]` array (each entry is its own gate; a rejection
// short-circuits the chain). This is the credential-endpoint pattern:
// per-fingerprint budget for fairness on shared IPs, per-IP backstop
// against header rotation.
const runStack = async (stack, req) => {
for (const opts of stack) {
const err = await runGate(opts, req);
if (err) return err;
}
return undefined;
};
it('stacked gates: per-IP backstop catches header rotation that escapes the fingerprint key', async () => {
const stack = [
{ limit: 3, window: 60_000, scope: 'stack-fp' },
{ limit: 6, window: 60_000, key: 'ip', scope: 'stack-ip' },
];
// Rotating the User-Agent mints a fresh fingerprint bucket every
// request, so the fingerprint gate admits all of these — the IP
// backstop must cap the total anyway.
for (let i = 0; i < 6; i++) {
const arg = await runStack(
stack,
makeReq({ headers: { 'user-agent': `rotated-ua-${i}` } }),
);
expect(arg).toBeUndefined();
}
const rejected = await runStack(
stack,
makeReq({ headers: { 'user-agent': 'rotated-ua-final' } }),
);
expect(isHttpError(rejected)).toBe(true);
expect(rejected.statusCode).toBe(429);
});
it('stacked gates: clients sharing an IP each get their own fingerprint budget under the IP cap', async () => {
const stack = [
{ limit: 2, window: 60_000, scope: 'stack2-fp' },
{ limit: 10, window: 60_000, key: 'ip', scope: 'stack2-ip' },
];
// Two clients (distinct UAs) behind one NAT IP: each gets the
// full per-fingerprint budget, and each is cut off by its own
// fingerprint bucket — not starved by the other's traffic.
for (const ua of ['alice-browser', 'bob-browser']) {
for (let i = 0; i < 2; i++) {
expect(
await runStack(
stack,
makeReq({ headers: { 'user-agent': ua } }),
),
).toBeUndefined();
}
const rejected = await runStack(
stack,
makeReq({ headers: { 'user-agent': ua } }),
);
expect(isHttpError(rejected)).toBe(true);
}
});
it("falls back to fingerprint when 'user' strategy is selected but no actor present", async () => {
// The fallback prevents anonymous traffic from sharing a single
// bucket (which would invite trivial DoS via global rate-limit).
@@ -0,0 +1,251 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { EventEmitter } from 'node:events';
import type { Request, Response } from 'express';
import { describe, expect, it } from 'vitest';
import { EventClient } from '../../clients/event/EventClient';
import type {
EventMap,
RouteLifecycleEvent,
} from '../../clients/event/types';
import type { IConfig } from '../../types';
import {
createRouteLifecycleMiddleware,
pathKeySegment,
routeEventKeyBase,
} from './routeLifecycle';
// -- Fakes --
//
// EventClient is a pure in-memory bus (no external deps), so we use the real
// one and register real listeners. The request/response are the boundary we
// fake: a minimal `res` that is an EventEmitter (for `finish`/`close`) and
// records status/body.
const makeEvents = () => new EventClient({} as IConfig);
class FakeRes extends EventEmitter {
statusCode = 200;
writableFinished = false;
headersSent = false;
body: unknown;
status(code: number) {
this.statusCode = code;
return this;
}
json(b: unknown) {
this.body = b;
return this;
}
}
const makeReq = (actorUuid?: string): Request =>
({
actor: actorUuid ? { user: { uuid: actorUuid } } : undefined,
}) as unknown as Request;
// Collect every emit on a given key into an array for assertions.
const record = (events: EventClient, key: keyof EventMap) => {
const seen: RouteLifecycleEvent[] = [];
events.on(key as never, (_k, data) =>
seen.push(data as RouteLifecycleEvent),
);
return seen;
};
const POST = 'post' as const;
const PATH = '/fs/completeBatchWrite';
const BASE = 'route.post.fs.completeBatchWrite';
describe('pathKeySegment', () => {
it('joins path parts with dots and drops slashes', () => {
expect(pathKeySegment('/fs/completeBatchWrite')).toBe(
'fs.completeBatchWrite',
);
});
it('strips the leading colon from path params', () => {
expect(pathKeySegment('/foo/:id')).toBe('foo.id');
});
it('maps the root path and non-string paths to placeholders', () => {
expect(pathKeySegment('/')).toBe('root');
expect(pathKeySegment(/^\/x/)).toBe('_');
});
});
describe('routeEventKeyBase', () => {
it('scopes the key to method + normalized path', () => {
expect(routeEventKeyBase(POST, PATH)).toBe(BASE);
});
});
describe('createRouteLifecycleMiddleware', () => {
it('emits before, then after on a clean finish', async () => {
const events = makeEvents();
const before = record(events, `${BASE}.before` as keyof EventMap);
const after = record(events, `${BASE}.after` as keyof EventMap);
const error = record(events, `${BASE}.error` as keyof EventMap);
const mw = createRouteLifecycleMiddleware(events, POST, PATH);
const req = makeReq('u-1');
const res = new FakeRes();
let nextCalled = false;
await mw(req, res as unknown as Response, () => {
nextCalled = true;
});
expect(nextCalled).toBe(true);
expect(before).toHaveLength(1);
expect(before[0]).toMatchObject({
phase: 'before',
method: 'post',
path: PATH,
actor: { user: { uuid: 'u-1' } },
actorUid: 'user:u-1',
});
// The live req/res are exposed so listeners can read the body or
// respond themselves.
expect(before[0].req).toBe(req);
expect(before[0].res).toBe(res);
res.statusCode = 200;
res.writableFinished = true;
res.emit('finish');
expect(after).toHaveLength(1);
expect(after[0]).toMatchObject({ phase: 'after', statusCode: 200 });
expect(typeof after[0].durationMs).toBe('number');
expect(error).toHaveLength(0);
});
it('emits error (not after) when the response is a 5xx', async () => {
const events = makeEvents();
const after = record(events, `${BASE}.after` as keyof EventMap);
const error = record(events, `${BASE}.error` as keyof EventMap);
const mw = createRouteLifecycleMiddleware(events, POST, PATH);
const res = new FakeRes();
await mw(makeReq(), res as unknown as Response, () => {});
res.statusCode = 500;
res.writableFinished = true;
res.emit('finish');
expect(after).toHaveLength(0);
expect(error).toHaveLength(1);
expect(error[0]).toMatchObject({ phase: 'error', statusCode: 500 });
});
it('emits error when the connection closes without finishing (abort)', async () => {
const events = makeEvents();
const error = record(events, `${BASE}.error` as keyof EventMap);
const mw = createRouteLifecycleMiddleware(events, POST, PATH);
const res = new FakeRes();
await mw(makeReq(), res as unknown as Response, () => {});
res.writableFinished = false;
res.emit('close');
expect(error).toHaveLength(1);
expect(error[0].phase).toBe('error');
expect(error[0].error).toBeInstanceOf(Error);
});
it('emits a terminal event only once across finish + close', async () => {
const events = makeEvents();
const after = record(events, `${BASE}.after` as keyof EventMap);
const error = record(events, `${BASE}.error` as keyof EventMap);
const mw = createRouteLifecycleMiddleware(events, POST, PATH);
const res = new FakeRes();
await mw(makeReq(), res as unknown as Response, () => {});
res.statusCode = 200;
res.writableFinished = true;
res.emit('finish');
res.emit('close');
expect(after).toHaveLength(1);
expect(error).toHaveLength(0);
});
it('vetoes the request when a before listener sets allow=false', async () => {
const events = makeEvents();
events.on(`${BASE}.before` as never, (_k, data) => {
(data as RouteLifecycleEvent).allow = false;
(data as RouteLifecycleEvent).rejectReason = 'quota exceeded';
});
const reject = record(events, `${BASE}.reject` as keyof EventMap);
const after = record(events, `${BASE}.after` as keyof EventMap);
const mw = createRouteLifecycleMiddleware(events, POST, PATH);
const res = new FakeRes();
let nextCalled = false;
await mw(makeReq('u-9'), res as unknown as Response, () => {
nextCalled = true;
});
expect(nextCalled).toBe(false);
expect(res.statusCode).toBe(403);
expect(res.body).toMatchObject({
error: { code: 'forbidden', message: 'quota exceeded' },
});
expect(reject).toHaveLength(1);
expect(reject[0]).toMatchObject({
phase: 'reject',
statusCode: 403,
rejectReason: 'quota exceeded',
});
// A vetoed request never runs, so no terminal after fires.
res.emit('finish');
expect(after).toHaveLength(0);
});
it('treats a listener-sent response as a terminal after (not a reject) with the real status', async () => {
const events = makeEvents();
events.on(`${BASE}.before` as never, (_k, data) => {
// Listener answers the request itself without vetoing.
const e = data as RouteLifecycleEvent;
(e.res as unknown as FakeRes).headersSent = true;
e.res.status(418).json({ teapot: true });
});
const reject = record(events, `${BASE}.reject` as keyof EventMap);
const after = record(events, `${BASE}.after` as keyof EventMap);
const mw = createRouteLifecycleMiddleware(events, POST, PATH);
const res = new FakeRes();
let nextCalled = false;
await mw(makeReq(), res as unknown as Response, () => {
nextCalled = true;
});
expect(nextCalled).toBe(false);
// Listener's own 418 stands; the middleware doesn't clobber it with 403.
expect(res.statusCode).toBe(418);
expect(res.body).toEqual({ teapot: true });
// Not a veto, so no reject — a terminal `after` keyed off the real 418.
expect(reject).toHaveLength(0);
expect(after).toHaveLength(1);
expect(after[0]).toMatchObject({ phase: 'after', statusCode: 418 });
});
});
+162
View File
@@ -0,0 +1,162 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler } from 'express';
import { actorUid } from '../actor';
import type { EventClient } from '../../clients/event/EventClient';
import type { RouteMethod } from './types';
type RoutePathValue = string | RegExp | Array<string | RegExp> | undefined;
/**
* Normalize a route path into a dot-separated key segment so lifecycle events
* can be scoped per endpoint. Strips slashes and param markers so the key is
* stable (`/drivers/call` -> `drivers.call`, `/foo/:id` -> `foo.id`).
* Non-string paths (RegExp / arrays) collapse to a single `_` placeholder.
*/
export const pathKeySegment = (fullPath: RoutePathValue): string => {
if (typeof fullPath !== 'string') return '_';
const seg = fullPath
.split('/')
.filter(Boolean)
.map((p) => p.replace(/^:/, '').replace(/[^A-Za-z0-9_-]/g, '_'))
.join('.');
return seg || 'root';
};
/**
* Scoped event-key base for a route: `route.<method>.<normalized-path>`.
* Callers append `.before` / `.after` / `.error` / `.reject`.
*/
export const routeEventKeyBase = (
method: RouteMethod,
fullPath: RoutePathValue,
): `route.${string}` => `route.${method}.${pathKeySegment(fullPath)}`;
/**
* Build the per-endpoint lifecycle middleware for one route.
*
* On each request it: emits `before` (awaited, so a listener may veto by
* setting `allow = false`), times the handler, and emits a single terminal
* event when the response finishes or the connection closes — `after` on a
* clean response, `error` on an abort or a >=500, `reject` only when a
* listener vetoes (it then answers 403 and the handler never runs). A listener
* that writes its own response without vetoing terminates as `after`/`error`
* keyed off the real status, not `reject`.
*
* Subscribers can listen on `route.*`, `route.<method>.*`, or the exact key.
*/
export const createRouteLifecycleMiddleware = (
events: EventClient,
method: RouteMethod,
fullPath: RoutePathValue,
): RequestHandler => {
const pathLabel =
typeof fullPath === 'string' ? fullPath : String(fullPath);
const keyBase = routeEventKeyBase(method, fullPath);
return async (req: Request, res, next) => {
const actor = req.actor ? actorUid(req.actor) : undefined;
const startedAt = Date.now();
const base = {
method,
path: pathLabel,
req,
res,
actor: req.actor,
actorUid: actor,
};
const beforeEvent = {
phase: 'before' as const,
...base,
allow: true as boolean,
rejectReason: undefined as string | undefined,
};
await events.emitAndWait(`${keyBase}.before`, beforeEvent, {});
// Explicit veto: a listener set `allow = false`. Emit `reject` and
// answer 403 — unless the listener already wrote its own response, in
// which case we report that status rather than a misleading 403.
if (beforeEvent.allow === false) {
events.emit(
`${keyBase}.reject`,
{
phase: 'reject',
...base,
statusCode: res.headersSent ? res.statusCode : 403,
durationMs: Date.now() - startedAt,
rejectReason: beforeEvent.rejectReason,
},
{},
);
if (!res.headersSent) {
res.status(403).json({
error: {
code: 'forbidden',
message:
beforeEvent.rejectReason ?? 'Blocked by policy',
},
});
}
return;
}
// A listener answered the request itself without vetoing (it wrote a
// response in the `before` hook). That's a normal terminal outcome,
// not a reject — emit `after`/`error` keyed off the real status.
if (res.headersSent) {
const statusCode = res.statusCode;
const isError = statusCode >= 500;
events.emit(
`${keyBase}.${isError ? 'error' : 'after'}`,
{
phase: isError ? 'error' : 'after',
...base,
statusCode,
durationMs: Date.now() - startedAt,
},
{},
);
return;
}
let settled = false;
const settle = (aborted: boolean) => {
if (settled) return;
settled = true;
const statusCode = res.statusCode;
const isError = aborted || statusCode >= 500;
events.emit(
`${keyBase}.${isError ? 'error' : 'after'}`,
{
phase: isError ? 'error' : 'after',
...base,
statusCode,
durationMs: Date.now() - startedAt,
...(aborted ? { error: new Error('request aborted') } : {}),
},
{},
);
};
res.once('finish', () => settle(false));
res.once('close', () => settle(!res.writableFinished));
next();
};
};
+35 -14
View File
@@ -66,6 +66,22 @@ export type RoutePath = string | RegExp | Array<string | RegExp>;
* in the chain. Commented-out slots are reserved for the next chunks
* (body parsing, post-auth gates, timing).
*/
/** One rate-limit window. See `RouteOptions.rateLimit` for semantics. */
export interface RouteRateLimit {
limit: number;
window: number;
/**
* Per-subscription overrides for `limit` (`SubscriptionPolicy.id`
* → cap). Same mechanic as `concurrent.bySubscription`: resolved
* via the metering service per request; falls back to the base
* `limit` when there's no actor / no metering / no match.
*/
bySubscription?: Record<string, number>;
key?: 'fingerprint' | 'ip' | 'user' | ((req: Request) => string);
scope?: string;
backend?: 'memory' | 'redis' | 'kv';
}
export interface RouteOptions {
/** Extra per-route middleware. Applied after built-in gates, before the handler. */
middleware?: RequestHandler[];
@@ -86,6 +102,17 @@ export interface RouteOptions {
/** Reject app/access-token actors. Implies `requireAuth`. */
requireUserActor?: boolean;
/**
* Only meaningful alongside `requireUserActor`. Relaxes the access-token
* half of that gate so a FULL-ACCESS personal access token (the user's own
* credential) is admitted — third-party apps and scoped tokens stay
* blocked. Use ONLY on user-resource / inference endpoints that gate with
* `requireUserActor` purely to keep apps out (e.g. the AI proxy). NEVER set
* on account/security/credential routes — those must stay closed to every
* access token. Default-deny: omitting this keeps PATs blocked.
*/
allowFullAccessToken?: boolean;
/** Allows access-tokens */
allowAccessToken?: boolean;
@@ -198,21 +225,15 @@ export interface RouteOptions {
* All registered backends stay co-resident at runtime, so different
* routes can pick whichever fits their access pattern. Omitting
* `backend` uses the server-wide default (`config.rate_limit.backend`).
*
* An array applies every limit independently (a request must pass all
* of them). Use this to pair a per-client budget with a coarser
* backstop on a different key — e.g. per-fingerprint for fairness on
* shared IPs, plus per-IP so rotating headers can't mint fresh
* fingerprint buckets indefinitely. Give each entry its own `scope`
* so the counters don't collide.
*/
rateLimit?: {
limit: number;
window: number;
/**
* Per-subscription overrides for `limit` (`SubscriptionPolicy.id`
* → cap). Same mechanic as `concurrent.bySubscription`: resolved
* via the metering service per request; falls back to the base
* `limit` when there's no actor / no metering / no match.
*/
bySubscription?: Record<string, number>;
key?: 'fingerprint' | 'ip' | 'user' | ((req: Request) => string);
scope?: string;
backend?: 'memory' | 'redis' | 'kv';
};
rateLimit?: RouteRateLimit | RouteRateLimit[];
/**
* Concurrent in-flight limiting. Caps how many requests for this
@@ -547,6 +547,46 @@ describe('ChatCompletionDriver.complete credit gate and max_tokens cap', () => {
expect(passed.max_tokens!).toBeGreaterThan(0);
});
it('throws 402 instead of leaving `max_tokens` unset when credits cannot afford one output token', async () => {
// Regression: previously a sub-1 cap set max_tokens to `undefined`,
// which let the provider run to the model's full output limit and
// overdraw the account. It must reject instead.
vi.spyOn(FakeChatProvider.prototype, 'models').mockResolvedValueOnce([
{
id: 'capme',
aliases: [],
costs_currency: 'usd-cents',
costs: { input_tokens: 1000, output_tokens: 2000 },
max_tokens: 8192,
},
]);
const d = await makeDriver();
// Pass the cheap pre-flight gate but leave a balance too small to
// afford a single 2000-microcent output token.
vi.spyOn(server.services.metering, 'hasEnoughCredits').mockResolvedValue(
true,
);
vi.spyOn(server.services.metering, 'getRemainingUsage').mockResolvedValue(
100,
);
const completeSpy = vi.spyOn(FakeChatProvider.prototype, 'complete');
await expect(
withTestActor(() =>
d.complete({
model: 'capme',
messages: [{ role: 'user', content: 'hi' }],
}),
),
).rejects.toMatchObject({
statusCode: 402,
legacyCode: 'insufficient_funds',
});
expect(completeSpy).not.toHaveBeenCalled();
});
it('rejects subscriber-only models for the default free subscription', async () => {
vi.spyOn(FakeChatProvider.prototype, 'models').mockResolvedValueOnce([
{
@@ -17,8 +17,9 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { PassThrough } from 'node:stream';
import crypto from 'node:crypto';
import { PassThrough } from 'node:stream';
import { EventMap } from '../../clients/event/types.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import {
@@ -28,12 +29,17 @@ import {
import type { DriverStreamResult } from '../meta.js';
import { PuterDriver } from '../types.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../util/aiLimits.js';
import { AlibabaProvider } from './providers/alibaba/AlibabaProvider.js';
import { AzureChatProvider } from './providers/azure/AzureChatProvider.js';
import { AzureResponsesProvider } from './providers/azure/AzureResponsesProvider.js';
import { ClaudeProvider } from './providers/claude/ClaudeProvider.js';
import { DeepSeekProvider } from './providers/deepseek/DeepSeekProvider.js';
import { FakeChatProvider } from './providers/FakeChatProvider.js';
import { GeminiChatProvider } from './providers/gemini/GeminiChatProvider.js';
import { GroqAIProvider } from './providers/groq/GroqAIProvider.js';
import { MiniMaxProvider } from './providers/minimax/MiniMaxProvider.js';
import { MistralAIProvider } from './providers/mistral/MistralAiProvider.js';
import { MoonshotProvider } from './providers/moonshot/MoonshotProvider.js';
import { OllamaChatProvider } from './providers/ollama/OllamaProvider.js';
import { OpenAiChatProvider } from './providers/openai/OpenAiChatCompletionsProvider.js';
import { OpenAiResponsesChatProvider } from './providers/openai/OpenAiChatResponsesProvider.js';
@@ -41,9 +47,6 @@ import { OpenRouterProvider } from './providers/openrouter/OpenRouterProvider.js
import { TogetherAIProvider } from './providers/together/TogetherAIProvider.js';
import { XAIProvider } from './providers/xai/XAIProvider.js';
import { ZAIProvider } from './providers/zai/ZAIProvider.js';
import { AlibabaProvider } from './providers/alibaba/AlibabaProvider.js';
import { MoonshotProvider } from './providers/moonshot/MoonshotProvider.js';
import { MiniMaxProvider } from './providers/minimax/MiniMaxProvider.js';
import type {
IChatCompleteResult,
IChatModel,
@@ -57,7 +60,6 @@ import {
normalize_single_message,
} from './utils/Messages.js';
import { AIChatStream } from './utils/Streaming.js';
import { EventMap } from '../../clients/event/types.js';
const MAX_FALLBACKS = 4; // includes first attempt
@@ -277,7 +279,7 @@ export class ChatCompletionDriver extends PuterDriver {
let intendedProvider = args.provider || '';
if (!args.model && !intendedProvider) {
intendedProvider = 'claude'; // default provider
intendedProvider = 'azure-openai'; // default provider
}
if (
!args.model &&
@@ -398,16 +400,25 @@ export class ChatCompletionDriver extends PuterDriver {
remainingCredits - approximateInputCost;
const maxAllowedOutputTokens =
maxAllowedOutputUcents / outputTokenCost;
if (maxAllowedOutputTokens) {
const cap = Math.floor(
Math.min(
args.max_tokens ?? Number.POSITIVE_INFINITY,
maxAllowedOutputTokens,
model.max_tokens - approximateTokenCount,
),
);
args.max_tokens = cap < 1 ? undefined : cap;
const cap = Math.floor(
Math.min(
args.max_tokens ?? Number.POSITIVE_INFINITY,
maxAllowedOutputTokens,
model.max_tokens - approximateTokenCount,
),
);
// `cap` is the credit-bounded ceiling on output tokens. When
// it drops below 1 the user can't afford even a single output
// token, so reject the request. Crucially we must NOT leave
// `max_tokens` unset here: an undefined max_tokens lets the
// provider run to the model's full output limit (e.g. 128k for
// Claude), billing far past the user's remaining balance.
if (cap < 1) {
throw new HttpError(402, 'No usage left for request.', {
legacyCode: 'insufficient_funds',
});
}
args.max_tokens = cap;
}
}
@@ -796,6 +807,43 @@ export class ChatCompletionDriver extends PuterDriver {
);
}
// Azure AI Foundry (OpenAI + xAI Grok). Registered before the regular
// OpenAI/xAI providers so that since its costs mirror theirs but
// Azure is preferred for us, it takes precedence in the per-model
// bucket
const azureOpenai = providers['azure-openai'];
const azureOpenaiKey = readKey(azureOpenai);
const azureOpenaiURL = azureOpenai?.apiURL as string | undefined;
if (azureOpenaiKey && azureOpenaiURL) {
const azureStores = {
fsEntry: this.stores.fsEntry,
s3Object: this.stores.s3Object,
};
const azureConfig = {
apiKey: azureOpenaiKey,
apiURL: azureOpenaiURL,
};
const azureCompletions = new AzureChatProvider(
metering,
azureStores,
this.services.fs,
azureConfig,
);
// Codex / Responses-API-only models can't use Chat Completions, so
// they route through a sibling Responses provider pointed at the
// same Azure endpoint. web_search (also Responses-only) delegates
// here too.
const azureResponses = new AzureResponsesProvider(
metering,
azureStores,
this.services.fs,
azureConfig,
);
azureCompletions.setResponsesProvider(azureResponses);
this.#providers['azure-openai'] = azureCompletions;
this.#providers['azure-openai-responses'] = azureResponses;
}
const openaiKey = readKey(providers['openai-completion']);
if (openaiKey) {
const openaiStores = {
@@ -0,0 +1,101 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Azure AI Foundry chat-completions provider.
*
* Hits the real Azure endpoint. Exercises both flavours of model the
* provider fronts:
* - an OpenAI model (`gpt-4o`, non-reasoning so `max_tokens=16` returns
* visible text), and
* - an xAI Grok model (`grok-4-20-non-reasoning`), which regression-tests
* the `safety_identifier` param being stripped for Grok (Azure's Grok
* deployments 400 on that OpenAI-only argument).
*
* Skipped unless both `PUTER_TEST_AI_AZURE_OPENAI_API_KEY` and
* `PUTER_TEST_AI_AZURE_OPENAI_API_URL` are set.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { AzureChatProvider } from './AzureChatProvider.js';
const KEY_ENV = 'PUTER_TEST_AI_AZURE_OPENAI_API_KEY';
const URL_ENV = 'PUTER_TEST_AI_AZURE_OPENAI_API_URL';
describe.skipIf(skipUnlessEnv(KEY_ENV) || skipUnlessEnv(URL_ENV))(
'AzureChatProvider (integration)',
() => {
const buildProvider = () =>
new AzureChatProvider(
makeMeteringStub(),
{ fsEntry: undefined as never, s3Object: undefined as never },
undefined as never,
{ apiKey: optionalEnv(KEY_ENV)!, apiURL: optionalEnv(URL_ENV)! },
);
const expectNonEmptyText = (result: unknown) => {
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
};
it(
'returns a non-empty completion from gpt-4o',
{ timeout: INTEGRATION_TEST_TIMEOUT_MS },
async () => {
const provider = buildProvider();
const result = await withTestActor(() =>
provider.complete({
model: 'gpt-4o',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
expectNonEmptyText(result);
},
);
it(
'returns a non-empty completion from grok-4-20-non-reasoning (no safety_identifier 400)',
{ timeout: INTEGRATION_TEST_TIMEOUT_MS },
async () => {
const provider = buildProvider();
const result = await withTestActor(() =>
provider.complete({
model: 'grok-4-20-non-reasoning',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
expectNonEmptyText(result);
},
);
},
);
@@ -0,0 +1,277 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { HttpError } from '@heyputer/backend/src/core/http/HttpError.js';
import { OpenAI } from 'openai';
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
import { Context } from '../../../../core/context.js';
import type { FSService } from '../../../../services/fs/FSService.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
import type { IChatProvider, ICompleteArguments } from '../../types.js';
import {
messagesHaveCompaction,
wantsCompaction,
} from '../../utils/compaction.js';
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import { processPuterPathUploads } from '../openai/fileUpload.js';
import { AZURE_MODELS } from './models.js';
/**
* AzureChatProvider exposes the models we serve through Azure AI Foundry.
* Despite the name, this is not OpenAI-only — Azure AI also fronts xAI's Grok
* models — so it carries its own {@link AZURE_MODELS} list instead of reusing
* the OpenAI one. It speaks the OpenAI-compatible Chat Completions API,
* pointing the client at a configurable Azure endpoint authenticated with an
* Azure-issued API key.
*
* Billing note: the model `costs` are the standard public OpenAI / xAI list
* prices, NOT Azure's. Azure is subsidised for us, so routing through it is
* cheaper while we still bill users at the normal model price.
*
* Implements the puter-chat-completion interface and handles usage tracking,
* spending records, and content moderation.
*/
export class AzureChatProvider implements IChatProvider {
/**
* @type {import('openai').OpenAI}
*/
#openAi: OpenAI;
#defaultModel = 'gpt-5.4-nano';
#meteringService: MeteringService;
#stores: { fsEntry: FSEntryStore; s3Object: S3ObjectStore };
#fsService: FSService;
// Sibling Responses-API provider (Azure or OpenAI) used to handle
// Responses-only features like web_search. Typed loosely since we only
// ever forward `complete()` to it.
#responsesProvider: IChatProvider | null = null;
constructor(
meteringService: MeteringService,
stores: { fsEntry: FSEntryStore; s3Object: S3ObjectStore },
fsService: FSService,
config: { apiKey: string; apiURL: string },
) {
this.#meteringService = meteringService;
this.#stores = stores;
this.#fsService = fsService;
this.#openAi = new OpenAI({
apiKey: config.apiKey,
baseURL: config.apiURL,
});
}
checkModeration(_text: string): { flagged: boolean; categories: string[] } {
throw new Error('Method not implemented.');
}
// Wired up by the driver after the OpenAI providers are built, so the
// Chat Completions path can delegate `web_search` tool calls (Responses-only)
// to the OpenAI Responses provider without a circular constructor dependency.
setResponsesProvider(provider: IChatProvider): void {
this.#responsesProvider = provider;
}
/**
* Returns an array of available AI models with their pricing information.
* Each model object includes an ID and cost details (currency, tokens, input/output rates).
*/
models() {
return AZURE_MODELS.filter((e) => !e.responses_api_only);
}
list() {
const models = this.models();
const modelNames: string[] = [];
for (const model of models) {
modelNames.push(model.id);
if (model.aliases) {
modelNames.push(...model.aliases);
}
}
return modelNames;
}
getDefaultModel() {
return this.#defaultModel;
}
async complete(
params: ICompleteArguments,
): ReturnType<IChatProvider['complete']> {
const {
max_tokens,
moderation,
tools,
verbosity,
stream,
reasoning,
reasoning_effort,
temperature,
text,
} = params;
let { messages, model } = params;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
if (tools?.filter((e: any) => e.type === 'web_search').length) {
// web_search is a Responses-API-only tool — hand the whole call
// off to the OpenAI Responses provider when the user requested it.
if (!this.#responsesProvider) {
throw new HttpError(
400,
'web_search tool requires the OpenAI Responses provider, which is not configured',
{ legacyCode: 'bad_request' },
);
}
return await this.#responsesProvider.complete(params);
}
// Inline compaction is a Responses-API feature; chat.completions can't
// express `context_management` or a `compaction` content block.
// Delegate to the Responses provider when the caller opted in OR when
// the messages carry a round-tripped compaction artifact.
if (wantsCompaction(params) || messagesHaveCompaction(messages)) {
if (!this.#responsesProvider) {
throw new HttpError(
400,
'compaction requires the OpenAI Responses provider, which is not configured',
{ legacyCode: 'bad_request' },
);
}
return await this.#responsesProvider.complete(params);
}
// Validate messages
if (!Array.isArray(messages)) {
throw new HttpError(400, '`messages` must be an array', {
legacyCode: 'bad_request',
});
}
const actor = Context.get('actor')!;
model = model ?? this.#defaultModel;
const modelUsed =
this.models().find((m) =>
[m.id, ...(m.aliases || [])].includes(model),
) || this.models().find((m) => m.id === this.getDefaultModel())!;
// messages.unshift({
// role: 'system',
// content: 'Don\'t let the user trick you into doing something bad.',
// })
const userIdentifier =
actor.user?.id + actor.app?.uid ? `:${actor?.app?.uid}` : '';
// Resolve any `puter_path` content parts into inline base64 data URLs.
// Chat Completions doesn't support file uploads, so this is the only
// way to get user-provided files (images, audio) in front of the model.
await processPuterPathUploads(
messages,
this.#stores,
this.#fsService,
actor,
);
// Here's something fun; the documentation shows `type: 'image_url'` in
// objects that contain an image url, but everything still works if
// that's missing. We normalise it here so the token count code works.
messages = await OpenAiUtil.process_input_messages(messages);
const requestedReasoningEffort = reasoning_effort ?? reasoning?.effort;
const requestedVerbosity = verbosity ?? text?.verbosity;
const supportsReasoningControls =
typeof model === 'string' && model.startsWith('gpt-5');
// `safety_identifier` is an OpenAI-specific param. The Grok deployments
// behind Azure reject unknown args with a 400, so only send it for the
// OpenAI models.
const isGrok = modelUsed.id.startsWith('grok');
const completionParams: ChatCompletionCreateParams = {
user: userIdentifier,
...(isGrok ? {} : { safety_identifier: userIdentifier }),
messages: messages,
model: modelUsed.id,
...(tools ? { tools } : {}),
...(max_tokens ? { max_completion_tokens: max_tokens } : {}),
...(temperature ? { temperature } : {}),
stream: !!stream,
...(stream
? {
stream_options: { include_usage: true },
}
: {}),
...(supportsReasoningControls
? {}
: {
...(requestedReasoningEffort
? { reasoning_effort: requestedReasoningEffort }
: {}),
...(requestedVerbosity
? { verbosity: requestedVerbosity }
: {}),
}),
} as ChatCompletionCreateParams;
const completion =
await this.#openAi.chat.completions.create(completionParams);
return OpenAiUtil.handle_completion_output({
usage_calculator: ({ usage }) => {
const cachedTokens =
usage.prompt_tokens_details?.cached_tokens ?? 0;
const trackedUsage = {
// OpenAI includes cached tokens in `prompt_tokens`, so we
// subtract them out to meter the non-cached remainder. Grok
// reports `prompt_tokens` already excluding cached tokens
// (they're additive, not a subset) — subtracting there
// underflows to a negative count, which bills a negative
// (crediting) cost. Match xAI's `extractMeteredUsage` and
// take Grok's `prompt_tokens` as-is.
prompt_tokens: isGrok
? (usage.prompt_tokens ?? 0)
: (usage.prompt_tokens ?? 0) - cachedTokens,
completion_tokens: usage.completion_tokens ?? 0,
cached_tokens: cachedTokens,
};
const costsOverrideFromModel = Object.fromEntries(
Object.entries(trackedUsage).map(([k, v]) => {
return [k, v * modelUsed.costs[k]];
}),
);
this.#meteringService.utilRecordUsageObject(
trackedUsage,
actor!,
`azure-openai:${modelUsed?.id}`,
costsOverrideFromModel,
);
return trackedUsage;
},
stream,
completion,
moderate: moderation ? this.checkModeration.bind(this) : undefined,
});
}
}
@@ -0,0 +1,83 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Azure AI Foundry Responses provider.
*
* Hits the real Azure endpoint with `gpt-5-codex` — a Responses-API-only
* (Codex) model that the Chat Completions endpoint rejects. This verifies
* the completions/responses split actually routes Codex correctly. Codex is
* a reasoning model, so we give it a generous `max_tokens` and low reasoning
* effort to leave room for visible output.
*
* Skipped unless both `PUTER_TEST_AI_AZURE_OPENAI_API_KEY` and
* `PUTER_TEST_AI_AZURE_OPENAI_API_URL` are set.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { AzureResponsesProvider } from './AzureResponsesProvider.js';
const KEY_ENV = 'PUTER_TEST_AI_AZURE_OPENAI_API_KEY';
const URL_ENV = 'PUTER_TEST_AI_AZURE_OPENAI_API_URL';
describe.skipIf(skipUnlessEnv(KEY_ENV) || skipUnlessEnv(URL_ENV))(
'AzureResponsesProvider (integration)',
() => {
it(
'returns a non-empty completion from gpt-5-codex',
{ timeout: INTEGRATION_TEST_TIMEOUT_MS },
async () => {
const provider = new AzureResponsesProvider(
makeMeteringStub(),
{
fsEntry: undefined as never,
s3Object: undefined as never,
},
undefined as never,
{
apiKey: optionalEnv(KEY_ENV)!,
apiURL: optionalEnv(URL_ENV)!,
},
);
const result = await withTestActor(() =>
provider.complete({
model: 'gpt-5-codex',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 2048,
reasoning: { effort: 'low' },
}),
);
const text = (result as { message?: { content?: string } })
.message?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
},
);
},
);
@@ -0,0 +1,307 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { OpenAI } from 'openai';
import { ResponseCreateParams } from 'openai/resources/responses/responses.mjs';
import { Context } from '../../../../core/context.js';
import type { FSService } from '../../../../services/fs/FSService.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
import type { IChatProvider, ICompleteArguments } from '../../types.js';
import { toOpenAiContextManagement } from '../../utils/compaction.js';
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import { processPuterPathUploads } from '../openai/fileUpload.js';
import { AZURE_MODELS } from './models.js';
import { HttpError } from '@heyputer/backend/src/core/http/HttpError.js';
/**
* AzureResponsesProvider serves the Responses-API-only models we expose through
* Azure AI Foundry (the Codex family and similar). It mirrors
* {@link OpenAiResponsesChatProvider}, but points the OpenAI client at the
* configurable Azure endpoint and draws from {@link AZURE_MODELS}.
*
* Codex / `responses_api_only` models reject the Chat Completions endpoint, so
* the sibling {@link AzureChatProvider} (Chat Completions) filters them out and
* the driver routes them here instead.
*
* Billing note: the model `costs` are the standard public OpenAI list prices,
* NOT Azure's — Azure is subsidised for us.
*/
export class AzureResponsesProvider implements IChatProvider {
/**
* @type {import('openai').OpenAI}
*/
#openAi: OpenAI;
#defaultModel = 'gpt-5-codex';
#meteringService: MeteringService;
#stores: { fsEntry: FSEntryStore; s3Object: S3ObjectStore };
#fsService: FSService;
constructor(
meteringService: MeteringService,
stores: { fsEntry: FSEntryStore; s3Object: S3ObjectStore },
fsService: FSService,
config: { apiKey: string; apiURL: string },
) {
this.#meteringService = meteringService;
this.#stores = stores;
this.#fsService = fsService;
this.#openAi = new OpenAI({
apiKey: config.apiKey,
baseURL: config.apiURL,
});
}
/**
* Returns an array of available AI models with their pricing information.
* Each model object includes an ID and cost details (currency, tokens, input/output rates).
*/
models(extra_params?: { no_restrictions?: boolean }) {
if (extra_params?.no_restrictions) {
return AZURE_MODELS;
}
return AZURE_MODELS.filter((e) => e.responses_api_only === true);
}
list() {
const models = this.models({ no_restrictions: false });
const modelNames: string[] = [];
for (const model of models) {
modelNames.push(model.id);
if (model.aliases) {
modelNames.push(...model.aliases);
}
}
return modelNames;
}
getDefaultModel() {
return this.#defaultModel;
}
async complete({
messages,
model,
max_tokens,
moderation,
tools,
tool_choice,
parallel_tool_calls,
include,
conversation,
compaction,
context_management,
previous_response_id,
instructions,
metadata,
prompt,
prompt_cache_key,
prompt_cache_retention,
store,
top_p,
truncation,
background,
service_tier,
verbosity,
stream,
reasoning,
reasoning_effort,
temperature,
text,
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
// Validate messages
if (!Array.isArray(messages)) {
throw new HttpError(400, '`messages` must be an array', {
legacyCode: 'bad_request',
});
}
const actor = Context.get('actor');
model = model ?? this.#defaultModel;
const modelUsed =
this.models({ no_restrictions: true }).find((m) =>
[m.id, ...(m.aliases || [])].includes(model),
) ||
this.models({ no_restrictions: true }).find(
(m) => m.id === this.getDefaultModel(),
)!;
const userIdentifier =
actor?.user.id + actor?.app?.uid ? `:${actor?.app?.uid}` : '';
// Resolve any `puter_path` content parts into inline base64 data URLs
// before the Responses API sees them.
await processPuterPathUploads(
messages,
this.#stores,
this.#fsService,
actor,
);
if (tools) {
// Unravel tools to OpenAI Responses API format
// eslint-disable-next-line @typescript-eslint/no-explicit-any
tools = (tools as any).map((e) => {
if (e.type === 'function') {
const tool = e.function;
tool.type = 'function';
return tool;
} else {
return e;
}
});
}
// Here's something fun; the documentation shows `type: 'image_url'` in
// objects that contain an image url, but everything still works if
// that's missing. We normalise it here so the token count code works.
messages =
await OpenAiUtil.process_input_messages_responses_api(messages);
const requestedReasoningEffort = reasoning_effort ?? reasoning?.effort;
const requestedVerbosity = verbosity ?? text?.verbosity;
const supportsReasoningControls =
typeof model === 'string' && model.startsWith('gpt-5');
// Translate the neutral compaction opt-in (or pass a raw
// `context_management` payload through) to OpenAI's Responses shape.
const contextManagement = toOpenAiContextManagement({
compaction,
context_management,
});
const completionParams: ResponseCreateParams = {
user: userIdentifier,
safety_identifier: userIdentifier,
input: messages,
model: modelUsed.id,
...(tools ? { tools } : {}),
...(tool_choice !== undefined ? { tool_choice } : {}),
...(parallel_tool_calls !== undefined
? { parallel_tool_calls }
: {}),
...(include !== undefined ? { include } : {}),
...(contextManagement !== undefined
? { context_management: contextManagement }
: {}),
...(conversation !== undefined ? { conversation } : {}),
...(previous_response_id !== undefined
? { previous_response_id }
: {}),
...(instructions !== undefined ? { instructions } : {}),
...(metadata !== undefined ? { metadata } : {}),
...(prompt !== undefined ? { prompt } : {}),
...(prompt_cache_key !== undefined ? { prompt_cache_key } : {}),
...(prompt_cache_retention !== undefined
? { prompt_cache_retention }
: {}),
...(store !== undefined ? { store } : {}),
...(max_tokens !== undefined
? { max_output_tokens: max_tokens }
: {}),
...(temperature !== undefined ? { temperature } : {}),
...(top_p !== undefined ? { top_p } : {}),
...(truncation !== undefined ? { truncation } : {}),
...(background !== undefined ? { background } : {}),
...(service_tier !== undefined ? { service_tier } : {}),
...(stream !== undefined ? { stream: !!stream } : {}),
...(text !== undefined ? { text } : {}),
...(supportsReasoningControls
? {}
: {
...(requestedReasoningEffort
? { reasoning_effort: requestedReasoningEffort }
: {}),
...(requestedVerbosity
? { verbosity: requestedVerbosity }
: {}),
}),
...(supportsReasoningControls && reasoning ? { reasoning } : {}),
} as ResponseCreateParams;
const completion =
await this.#openAi.responses.create(completionParams);
return OpenAiUtil.handle_completion_output_responses_api({
usage_calculator: ({ usage }) => {
const trackedUsage = {
prompt_tokens:
// eslint-disable-next-line @typescript-eslint/no-explicit-any
((usage as any).input_tokens ?? 0) -
// eslint-disable-next-line @typescript-eslint/no-explicit-any
((usage as any).input_tokens_details?.cached_tokens ??
0),
// eslint-disable-next-line @typescript-eslint/no-explicit-any
completion_tokens: (usage as any).output_tokens ?? 0,
cached_tokens:
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(usage as any).input_tokens_details?.cached_tokens ?? 0,
};
const costsOverrideFromModel = Object.fromEntries(
Object.entries(trackedUsage).map(([k, v]) => {
return [k, v * modelUsed.costs[k]];
}),
);
this.#meteringService.utilRecordUsageObject(
trackedUsage,
actor,
`azure-openai:${modelUsed?.id}`,
costsOverrideFromModel,
);
return trackedUsage;
},
stream,
completion,
moderate: moderation ? this.checkModeration.bind(this) : undefined,
});
}
async checkModeration(text: string) {
// create moderation
const results = await this.#openAi.moderations.create({
model: 'omni-moderation-latest',
input: text,
});
let flagged = false;
for (const result of results?.results ?? []) {
// OpenAI does a crazy amount of false positives. We filter by their 80% interval
const veryFlaggedEntries = Object.entries(
result.category_scores,
).filter((e) => e[1] > 0.8);
if (veryFlaggedEntries.length > 0) {
flagged = true;
break;
}
}
return {
flagged,
results,
};
}
}
@@ -0,0 +1,464 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { IChatModel } from '../../types.js';
// Models served through our Azure AI Foundry deployment. This is NOT just
// OpenAI — Azure AI also fronts xAI's Grok models — so the list lives in its
// own provider folder rather than sharing the OpenAI list.
//
// IMPORTANT: the `costs` below intentionally mirror the public list prices of
// the equivalent OpenAI / xAI models (see `../openai/models.ts` and
// `../xai/models.ts`). Azure is subsidised for us, so our actual spend is
// lower — but we bill users at the standard model price, which is the whole
// reason we route through Azure. Do NOT replace these with Azure's own rates.
//
// `id` is the Azure deployment name and is what we send upstream.
export const AZURE_MODELS: IChatModel[] = [
// -- xAI Grok (via Azure AI Foundry) -----------------------------------
{
// Costs mirror xai grok-4-1-fast-non-reasoning.
puterId: 'azure:x-ai/grok-4-1-fast-non-reasoning',
id: 'grok-4-1-fast-non-reasoning',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-07',
release_date: '2025-11-19',
name: 'Grok 4.1 Fast (Non-Reasoning)',
aliases: ['x-ai/grok-4-1-fast-non-reasoning'],
context: 2_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 20,
completion_tokens: 50,
cached_tokens: 5,
},
max_tokens: 2_000_000,
},
{
// Costs mirror xai grok-4-1-fast (alias grok-4-1-fast-reasoning).
puterId: 'azure:x-ai/grok-4-1-fast-reasoning',
id: 'grok-4-1-fast-reasoning',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-07',
release_date: '2025-11-19',
name: 'Grok 4.1 Fast (Reasoning)',
aliases: ['x-ai/grok-4-1-fast-reasoning'],
context: 2_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 20,
completion_tokens: 50,
cached_tokens: 5,
},
max_tokens: 2_000_000,
},
{
// Costs mirror xai grok-4.3.
puterId: 'azure:x-ai/grok-4.3',
id: 'grok-4.3',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
release_date: '2026-05-01',
name: 'Grok 4.3',
aliases: ['x-ai/grok-4.3'],
context: 1_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
completion_tokens: 250,
cached_tokens: 20,
},
max_tokens: 30_000,
},
{
// Costs mirror xai grok-4.20 (grok-4.20-0309-non-reasoning).
puterId: 'azure:x-ai/grok-4-20-non-reasoning',
id: 'grok-4-20-non-reasoning',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-07',
release_date: '2026-03-09',
name: 'Grok 4.20 (Non-Reasoning)',
aliases: ['x-ai/grok-4-20-non-reasoning'],
context: 2_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
completion_tokens: 250,
cached_tokens: 20,
},
max_tokens: 30_000,
},
{
// Costs mirror xai grok-4.20 (grok-4.20-0309-reasoning).
puterId: 'azure:x-ai/grok-4-20-reasoning',
id: 'grok-4-20-reasoning',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-07',
release_date: '2026-03-09',
name: 'Grok 4.20 (Reasoning)',
aliases: ['x-ai/grok-4-20-reasoning'],
context: 2_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
completion_tokens: 250,
cached_tokens: 20,
},
max_tokens: 30_000,
},
// -- OpenAI (via Azure AI Foundry) -------------------------------------
{
// Costs mirror openai gpt-5.
puterId: 'azure:openai/gpt-5',
id: 'gpt-5',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-09-30',
release_date: '2025-08-07',
aliases: ['openai/gpt-5'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
cached_tokens: 13,
completion_tokens: 1000,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5-codex (same list price as gpt-5).
puterId: 'azure:openai/gpt-5-codex',
id: 'gpt-5-codex',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-09-30',
release_date: '2025-08-07',
aliases: ['openai/gpt-5-codex'],
// Codex models are Responses-API only on Azure, same as OpenAI.
responses_api_only: true,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
cached_tokens: 13,
completion_tokens: 1000,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5-nano.
puterId: 'azure:openai/gpt-5-nano',
id: 'gpt-5-nano',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-05-30',
release_date: '2025-08-07',
aliases: ['openai/gpt-5-nano'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 5,
cached_tokens: 1,
completion_tokens: 40,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5-mini.
puterId: 'azure:openai/gpt-5-mini',
id: 'gpt-5-mini',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-05-30',
release_date: '2025-08-07',
aliases: ['openai/gpt-5-mini'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 25,
cached_tokens: 3,
completion_tokens: 200,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-4o.
puterId: 'azure:openai/gpt-4o',
id: 'gpt-4o',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2023-09',
release_date: '2024-05-13',
aliases: ['openai/gpt-4o'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 250,
cached_tokens: 125,
completion_tokens: 1000,
},
context: 128_000,
max_tokens: 16384,
},
{
// Costs mirror openai gpt-5.1-codex-mini.
puterId: 'azure:openai/gpt-5.1-codex-mini',
id: 'gpt-5.1-codex-mini',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-09-30',
release_date: '2025-11-13',
aliases: ['openai/gpt-5.1-codex-mini'],
responses_api_only: true,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 25,
cached_tokens: 3,
completion_tokens: 200,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5.1.
puterId: 'azure:openai/gpt-5.1',
id: 'gpt-5.1',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-09-30',
release_date: '2025-11-13',
aliases: ['openai/gpt-5.1'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
cached_tokens: 13,
completion_tokens: 1000,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5.1-codex.
puterId: 'azure:openai/gpt-5.1-codex',
id: 'gpt-5.1-codex',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-09-30',
release_date: '2025-11-13',
aliases: ['openai/gpt-5.1-codex'],
responses_api_only: true,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 125,
cached_tokens: 13,
completion_tokens: 1000,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5.2.
puterId: 'azure:openai/gpt-5.2',
id: 'gpt-5.2',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-08-31',
release_date: '2025-12-11',
aliases: ['openai/gpt-5.2'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 175,
cached_tokens: 17.5,
completion_tokens: 1400,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5.2-codex.
puterId: 'azure:openai/gpt-5.2-codex',
id: 'gpt-5.2-codex',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-08-31',
release_date: '2025-12-11',
aliases: ['openai/gpt-5.2-codex'],
responses_api_only: true,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 175,
cached_tokens: 18,
completion_tokens: 1400,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5.4-nano.
puterId: 'azure:openai/gpt-5.4-nano',
id: 'gpt-5.4-nano',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-08-31',
release_date: '2026-03-19',
aliases: ['openai/gpt-5.4-nano'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 20,
cached_tokens: 2,
completion_tokens: 125,
},
context: 400_000,
max_tokens: 128_000,
},
{
// Costs mirror openai gpt-5.4-mini.
puterId: 'azure:openai/gpt-5.4-mini',
id: 'gpt-5.4-mini',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-08-31',
aliases: ['openai/gpt-5.4-mini'],
responses_api_only: true,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 75,
cached_tokens: 7.5,
completion_tokens: 450,
},
context: 400_000,
max_tokens: 128_000,
},
{
// Costs mirror openai gpt-5.3-codex.
puterId: 'azure:openai/gpt-5.3-codex',
id: 'gpt-5.3-codex',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-08-31',
aliases: ['openai/gpt-5.3-codex'],
responses_api_only: true,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 175,
cached_tokens: 17.5,
completion_tokens: 1400,
},
context: 128_000,
max_tokens: 128000,
},
{
// Costs mirror openai gpt-5.4.
puterId: 'azure:openai/gpt-5.4',
id: 'gpt-5.4',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-08-31',
release_date: '2026-03-05',
aliases: ['openai/gpt-5.4'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 250,
cached_tokens: 25,
completion_tokens: 1500,
},
context: 1_050_000,
max_tokens: 1_050_000,
},
];
@@ -486,6 +486,52 @@ describe('ClaudeProvider.complete non-stream output', () => {
10 * Number(haiku.costs.cache_read_input_tokens),
);
});
it('bills the compaction pass by summing usage.iterations', async () => {
const { provider } = makeProvider();
// Per Anthropic: top-level input/output reflect only the message pass;
// the compaction pass lives in `iterations` and must be summed to bill.
const msg = {
content: [{ type: 'text', text: 'done' }],
usage: {
input_tokens: 23000, // message pass only (NOT the total)
output_tokens: 1000,
iterations: [
{
type: 'compaction',
input_tokens: 180000,
output_tokens: 3500,
},
{ type: 'message', input_tokens: 23000, output_tokens: 1000 },
],
},
};
messagesCreateMock.mockResolvedValueOnce(msg);
const result = (await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [{ role: 'user', content: 'hi' }],
compaction: true,
}),
)) as { usage: Record<string, number> };
// Totals are the SUM across iterations, not the top-level fields.
expect(result.usage.input_tokens).toBe(203000); // 180000 + 23000
expect(result.usage.output_tokens).toBe(4500); // 3500 + 1000
const haiku = CLAUDE_MODELS.find(
(m) => m.id === 'claude-haiku-4-5-20251001',
)!;
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
expect(usage.input_tokens).toBe(203000);
expect(overrides.input_tokens).toBe(
203000 * Number(haiku.costs.input_tokens),
);
expect(overrides.output_tokens).toBe(
4500 * Number(haiku.costs.output_tokens),
);
});
});
// ── Streaming deltas ────────────────────────────────────────────────
@@ -618,6 +664,220 @@ describe('ClaudeProvider.complete streaming', () => {
});
});
// ── Inline compaction ───────────────────────────────────────────────
describe('ClaudeProvider.complete compaction', () => {
it('translates the neutral opt-in to context_management + the compaction beta', async () => {
const { provider } = makeProvider();
messagesCreateMock.mockResolvedValueOnce({
content: [{ type: 'text', text: 'hi' }],
usage: { input_tokens: 1, output_tokens: 1 },
});
await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [{ role: 'user', content: 'hi' }],
compaction: { trigger_tokens: 50000 },
}),
);
const [args] = messagesCreateMock.mock.calls[0]!;
expect(args.context_management).toEqual({
edits: [
{
type: 'compact_20260112',
trigger: { type: 'input_tokens', value: 50000 },
},
],
});
expect(args.betas).toContain('compact-2026-01-12');
});
it('emits a canonical compaction event from a streamed compaction block', async () => {
const { provider } = makeProvider();
messagesStreamMock.mockReturnValueOnce(
makeStreamLike(
[
{ type: 'message_start' },
{
type: 'content_block_start',
content_block: {
type: 'compaction',
id: 'cmpct_1',
content: 'ENC', // Anthropic carries the summary here
},
},
{ type: 'content_block_stop' },
{
type: 'message_delta',
usage: { input_tokens: 1, output_tokens: 1 },
},
{ type: 'message_stop' },
],
{ input_tokens: 1, output_tokens: 1 },
),
);
const result = await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [{ role: 'user', content: 'hi' }],
stream: true,
compaction: true,
}),
);
const harness = makeCapturingChatStream();
await (
result as {
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
}
).init_chat_stream({ chatStream: harness.chatStream });
const compaction = harness
.events()
.find((e) => e.type === 'compaction');
expect(compaction).toEqual({
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
});
});
it('enables the compaction beta when a round-tripped compaction block is resent (no opt-in)', async () => {
const { provider } = makeProvider();
messagesCreateMock.mockResolvedValueOnce({
content: [{ type: 'text', text: 'ok' }],
usage: { input_tokens: 1, output_tokens: 1 },
});
await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [
{ role: 'user', content: 'continue' },
{
role: 'assistant',
content: [
{
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
],
},
],
// note: no `compaction`/`context_management` opt-in
}),
);
const [args] = messagesCreateMock.mock.calls[0]!;
expect(args.betas).toContain('compact-2026-01-12');
// No new compaction was requested, so no context_management is sent.
expect(args.context_management).toBeUndefined();
});
it('surfaces a compaction block from a non-streaming response as result.compaction', async () => {
const { provider } = makeProvider();
messagesCreateMock.mockResolvedValueOnce({
content: [
{ type: 'text', text: 'done' },
{
type: 'compaction',
id: 'cmpct_2',
content: 'ENC2', // Anthropic carries the summary here
},
],
usage: { input_tokens: 1, output_tokens: 1 },
});
const result = (await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [{ role: 'user', content: 'hi' }],
compaction: true,
}),
)) as { compaction?: { id?: string; encrypted_content: string } };
// Anthropic's `content` is surfaced under the unified `encrypted_content`.
expect(result.compaction).toEqual({
type: 'compaction',
id: 'cmpct_2',
encrypted_content: 'ENC2',
});
});
it('maps a round-tripped compaction block back to Anthropic `content` on input', async () => {
const { provider } = makeProvider();
messagesCreateMock.mockResolvedValueOnce({
content: [{ type: 'text', text: 'ok' }],
usage: { input_tokens: 1, output_tokens: 1 },
});
await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [
{ role: 'user', content: 'continue' },
{
role: 'assistant',
content: [
{ type: 'compaction', encrypted_content: 'SUMMARY' },
],
},
],
}),
);
const [args] = messagesCreateMock.mock.calls[0]!;
const compactionBlock = args.messages
.flatMap((m: { content?: unknown[] }) =>
Array.isArray(m.content) ? m.content : [],
)
.find((c: { type?: string }) => c?.type === 'compaction');
// Internal `encrypted_content` carrier → Anthropic native `content`.
expect(compactionBlock).toEqual({
type: 'compaction',
content: 'SUMMARY',
});
});
it('preserves the reply text when an assistant turn carries compaction + text', async () => {
const { provider } = makeProvider();
messagesCreateMock.mockResolvedValueOnce({
content: [{ type: 'text', text: 'ok' }],
usage: { input_tokens: 1, output_tokens: 1 },
});
await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [
{
role: 'assistant',
content: [
{ type: 'compaction', encrypted_content: 'SUMMARY' },
{ type: 'text', text: 'earlier reply' },
],
},
{ role: 'user', content: 'continue' },
],
}),
);
const [args] = messagesCreateMock.mock.calls[0]!;
const blocks = args.messages.flatMap((m: { content?: unknown[] }) =>
Array.isArray(m.content) ? m.content : [],
);
// Compaction mapped to Anthropic `content`, AND the reply text kept.
expect(blocks).toContainEqual({
type: 'compaction',
content: 'SUMMARY',
});
expect(blocks).toContainEqual({ type: 'text', text: 'earlier reply' });
});
});
// ── Moderation ──────────────────────────────────────────────────────
describe('ClaudeProvider.checkModeration', () => {
@@ -34,6 +34,10 @@ import type {
ICompleteArguments,
IChatCompleteResult,
} from '../../types.js';
import {
messagesHaveCompaction,
toAnthropicContextManagement,
} from '../../utils/compaction.js';
import { make_claude_tools } from '../../utils/FunctionCalling.js';
import { extract_and_remove_system_messages } from '../../utils/Messages.js';
import type {
@@ -44,6 +48,11 @@ import type {
import { FILES_API_BETA, processPuterPathUploads } from './fileUpload.js';
import { CLAUDE_MODELS } from './models.js';
// Anthropic inline-compaction beta. The vendored SDK (0.68.0) doesn't type the
// `compact_20260112` edit or the `compaction` content block, so the request
// params and streamed/returned blocks are handled with `as any` casts.
const COMPACTION_BETA = 'compact-2026-01-12';
export class ClaudeProvider implements IChatProvider {
anthropic: Anthropic;
@@ -97,9 +106,18 @@ export class ClaudeProvider implements IChatProvider {
temperature,
reasoning,
reasoning_effort,
compaction,
context_management,
}: ICompleteArguments): Promise<IChatCompleteResult> {
tools = make_claude_tools(tools);
// Translate the neutral compaction opt-in (or pass a raw
// `context_management` payload through) to Anthropic's beta shape.
const contextManagement = toAnthropicContextManagement({
compaction,
context_management,
});
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let system_prompts: string | any[];
[system_prompts, messages] =
@@ -218,6 +236,23 @@ export class ClaudeProvider implements IChatProvider {
return message;
});
// Map round-tripped compaction blocks back to Anthropic's native shape.
// The internal/unified carrier field is `encrypted_content`; Anthropic's
// compaction block uses `content` (a plaintext summary).
// eslint-disable-next-line @typescript-eslint/no-explicit-any
messages = messages.map((message: any) => {
if (!Array.isArray(message.content)) return message;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
message.content = message.content.map((part: any) => {
if (part?.type !== 'compaction') return part;
return {
type: 'compaction',
content: part.content ?? part.encrypted_content ?? '',
};
});
return message;
});
const modelUsed =
this.models().find((m) =>
[m.id, ...(m.aliases || [])].includes(model),
@@ -229,9 +264,10 @@ export class ClaudeProvider implements IChatProvider {
reasoningEffort: requestedReasoningEffort,
maxTokens: max_tokens,
});
// Opus 4.7 and 4.8 error on non-default sampling params; omit temperature entirely.
// Fable 5 and Opus 4.7/4.8 error on non-default sampling params; omit temperature entirely.
// Other models require temperature=1 when thinking is enabled.
const omitsTemperature = [
'claude-fable-5',
'claude-opus-4-7',
'claude-opus-4-8',
].includes(modelUsed.id);
@@ -241,6 +277,8 @@ export class ClaudeProvider implements IChatProvider {
? 1
: (temperature ?? 0);
const supportsEffort = [
'claude-fable-5',
'claude-opus-4-8',
'claude-opus-4-7',
'claude-opus-4-6',
'claude-sonnet-4-6',
@@ -259,6 +297,18 @@ export class ClaudeProvider implements IChatProvider {
actor,
);
const usesBetaFiles = uploadedFileIds.length > 0;
// The compaction beta is needed both to *request* compaction
// (contextManagement) and to *accept a round-tripped* compaction block
// back as input (messagesHaveCompaction).
const usesCompaction =
!!contextManagement || messagesHaveCompaction(messages);
// Compaction and Files API both require the beta endpoint; combine their
// beta headers and route through `beta.messages.*` if either is active.
const betas = [
...(usesBetaFiles ? [FILES_API_BETA] : []),
...(usesCompaction ? [COMPACTION_BETA] : []),
];
const usesBeta = betas.length > 0;
const sdkParams: MessageCreateParams & {
betas?: string[];
@@ -288,7 +338,11 @@ export class ClaudeProvider implements IChatProvider {
...(supportsEffort && requestedReasoningEffort
? { output_config: { effort: requestedReasoningEffort } }
: {}),
...(usesBetaFiles ? { betas: [FILES_API_BETA] } : {}),
// Cast: `context_management` compaction edits aren't typed in SDK 0.68.0.
...(contextManagement
? { context_management: contextManagement as any }
: {}),
...(usesBeta ? { betas } : {}),
} as MessageCreateParams & { betas?: string[] };
const cleanupUploads = async () => {
@@ -312,13 +366,23 @@ export class ClaudeProvider implements IChatProvider {
}: {
chatStream: AIChatStream;
}) => {
const completion = usesBetaFiles
const completion = usesBeta
? this.anthropic.beta.messages.stream(sdkParams)
: this.anthropic.messages.stream(sdkParams);
const usageSum: Record<string, number> = {};
let message, contentBlock;
let currentContentBlockType: string | null = null;
// Inline-compaction block is an untyped beta block; capture its
// artifact across start/delta and emit on stop. Anthropic carries
// the summary in `content` (plaintext) — unlike OpenAI's
// `encrypted_content` — so read `content` first.
let compactionData: {
id?: string;
payload: string;
buffer: string;
} | null = null;
let emittedCompaction = false;
for await (const event of completion) {
if (event.type === 'message_delta') {
const meteredData = this.#usageFormatterUtil(
@@ -342,6 +406,24 @@ export class ClaudeProvider implements IChatProvider {
}
if (event.type === 'content_block_start') {
currentContentBlockType = event.content_block.type;
if (
(event.content_block.type as string) ===
'compaction'
) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const block = event.content_block as any;
compactionData = {
id: block.id,
// Anthropic uses `content`; fall back to
// `encrypted_content` in case the field varies.
payload:
block.content ??
block.encrypted_content ??
'',
buffer: '',
};
continue;
}
if (event.content_block.type === 'tool_use') {
contentBlock = message!.contentBlock({
type: event.content_block.type,
@@ -360,12 +442,42 @@ export class ClaudeProvider implements IChatProvider {
continue;
}
if (event.type === 'content_block_stop') {
if (currentContentBlockType === 'compaction') {
const encrypted_content =
compactionData?.payload ||
compactionData?.buffer ||
'';
// Only emit (and mark done) if we actually captured
// the summary; otherwise let the finalMessage
// fallback recover it from the complete block.
if (encrypted_content) {
chatStream.compaction({
id: compactionData?.id,
encrypted_content,
});
emittedCompaction = true;
}
compactionData = null;
currentContentBlockType = null;
continue;
}
contentBlock!.end();
contentBlock = null;
currentContentBlockType = null;
continue;
}
if (event.type === 'content_block_delta') {
if (currentContentBlockType === 'compaction') {
// Capture any streamed payload for the artifact.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const d = event.delta as any;
const chunk =
d.partial_json ?? d.text ?? d.data ?? '';
if (typeof chunk === 'string' && compactionData) {
compactionData.buffer += chunk;
}
continue;
}
if (event.delta.type === 'input_json_delta') {
(contentBlock as AIChatToolUseStream)!.addPartialJSON(
event.delta.partial_json,
@@ -388,18 +500,37 @@ export class ClaudeProvider implements IChatProvider {
// signature_delta — ignored
}
}
const finalUsage = await completion
const finalMessage = await completion
.finalMessage()
.then((msg) =>
this.#usageFormatterUtil(
msg.usage as Usage | BetaUsage,
),
)
.catch(() => null);
if (finalUsage) {
if (finalMessage) {
const finalUsage = this.#usageFormatterUtil(
finalMessage.usage as Usage | BetaUsage,
);
for (const [key, value] of Object.entries(finalUsage)) {
usageSum[key] = value;
}
// Fallback: some SDK versions surface the compaction block
// only in the final message, not as a streamed block.
if (!emittedCompaction) {
const block = (
(finalMessage.content as unknown[]) ?? []
).find(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(c: any) => c?.type === 'compaction',
// eslint-disable-next-line @typescript-eslint/no-explicit-any
) as any;
if (block) {
chatStream.compaction({
id: block.id,
encrypted_content:
block.content ??
block.encrypted_content ??
'',
});
emittedCompaction = true;
}
}
}
chatStream.end(usageSum);
const costsOverrideFromModel =
@@ -420,7 +551,7 @@ export class ClaudeProvider implements IChatProvider {
}
try {
const msg = await (usesBetaFiles
const msg = await (usesBeta
? this.anthropic.beta.messages.create(sdkParams)
: this.anthropic.messages.create(sdkParams));
const usage = this.#usageFormatterUtil(
@@ -437,7 +568,37 @@ export class ClaudeProvider implements IChatProvider {
costsOverrideFromModel,
);
return { message: msg, usage, finish_reason: 'stop' };
// Surface any inline-compaction artifact for stateless round-trip.
const compactionBlock = (
((msg as Message).content as unknown[]) ?? []
).find(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(c: any) => c?.type === 'compaction',
// eslint-disable-next-line @typescript-eslint/no-explicit-any
) as any;
return {
message: msg,
usage,
finish_reason: 'stop',
...(compactionBlock
? {
compaction: {
// `type` makes the artifact a drop-in `messages`
// item for the round-trip (symmetric with the
// streaming compaction chunk).
type: 'compaction' as const,
...(compactionBlock.id !== undefined
? { id: compactionBlock.id }
: {}),
encrypted_content:
compactionBlock.content ??
compactionBlock.encrypted_content ??
'',
},
}
: {}),
};
} finally {
await cleanupUploads();
}
@@ -445,8 +606,31 @@ export class ClaudeProvider implements IChatProvider {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
#usageFormatterUtil(usage: any) {
// Compaction responses report per-pass usage in `usage.iterations` (the
// compaction pass + the message pass). Per Anthropic's docs the
// top-level `input_tokens`/`output_tokens` reflect ONLY the
// non-compaction passes, so to bill the (often large) compaction pass we
// must sum across all iterations. Compaction is billed at normal token
// rates, and iterations don't break out cache fields — those stay
// top-level (the message pass's), which bills the compaction input at
// the full input rate.
const iterations = Array.isArray(usage?.iterations)
? usage.iterations
: null;
const inputTokens = iterations
? iterations.reduce(
(sum: number, it: any) => sum + (it?.input_tokens || 0),
0,
)
: usage?.input_tokens || 0;
const outputTokens = iterations
? iterations.reduce(
(sum: number, it: any) => sum + (it?.output_tokens || 0),
0,
)
: usage?.output_tokens || 0;
return {
input_tokens: usage?.input_tokens || 0,
input_tokens: inputTokens,
ephemeral_5m_input_tokens:
usage?.cache_creation?.ephemeral_5m_input_tokens ||
usage?.cache_creation_input_tokens ||
@@ -454,7 +638,7 @@ export class ClaudeProvider implements IChatProvider {
ephemeral_1h_input_tokens:
usage?.cache_creation?.ephemeral_1h_input_tokens || 0,
cache_read_input_tokens: usage?.cache_read_input_tokens || 0,
output_tokens: usage?.output_tokens || 0,
output_tokens: outputTokens,
thinking_tokens:
usage?.thinking_tokens ||
usage?.output_tokens_details?.thinking_tokens ||
@@ -489,11 +673,16 @@ export class ClaudeProvider implements IChatProvider {
}) {
if (!reasoningEffort) return undefined;
// Opus 4.7, 4.6, and Sonnet 4.6 use adaptive thinking
// (`budget_tokens` is deprecated on 4.6/Sonnet 4.6, removed on 4.7).
// Opus 4.7 omits thinking content by default; `display: 'summarized'`
// restores visible reasoning in the stream.
if (modelId === 'claude-opus-4-7') {
// Fable 5, Opus 4.7/4.8, 4.6, and Sonnet 4.6 use adaptive thinking
// (`budget_tokens` is deprecated on 4.6/Sonnet 4.6, removed on
// Fable 5 and 4.7+). Fable 5 and Opus 4.7/4.8 omit thinking content
// by default; `display: 'summarized'` restores visible reasoning in
// the stream.
if (
modelId === 'claude-fable-5' ||
modelId === 'claude-opus-4-8' ||
modelId === 'claude-opus-4-7'
) {
return {
type: 'adaptive' as const,
display: 'summarized' as const,
@@ -21,6 +21,35 @@ import type { IChatModel } from '../../types.js';
// Hardcoded from https://models.dev/api.json
export const CLAUDE_MODELS: IChatModel[] = [
{
puterId: 'anthropic:anthropic/claude-fable-5',
id: 'claude-fable-5',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
release_date: '2026-06-09',
aliases: [
'claude-fable',
'claude-fable-latest',
'claude-fable-5-latest',
'claude-fable-5',
'anthropic/claude-fable-5',
],
name: 'Claude Fable 5',
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 1000,
ephemeral_5m_input_tokens: 1000 * 1.25,
ephemeral_1h_input_tokens: 1000 * 2,
cache_read_input_tokens: 1000 * 0.1,
output_tokens: 5000,
},
context: 1000000,
max_tokens: 128000,
},
{
puterId: 'anthropic:anthropic/claude-opus-4-8',
id: 'claude-opus-4-8',
@@ -355,6 +355,8 @@ describe('GeminiChatProvider.complete non-stream output', () => {
prompt_tokens: 90,
completion_tokens: 50,
cached_tokens: 10,
thinking_tokens: 0,
grounding_requests: 0,
});
// gemini-2.5-flash costs: prompt=30, completion=250, cached=3.
@@ -367,11 +369,15 @@ describe('GeminiChatProvider.complete non-stream output', () => {
prompt_tokens: 90,
completion_tokens: 50,
cached_tokens: 10,
thinking_tokens: 0,
grounding_requests: 0,
});
expect(overrides).toEqual({
prompt_tokens: 90 * Number(flash.costs.prompt_tokens),
completion_tokens: 50 * Number(flash.costs.completion_tokens),
cached_tokens: 10 * Number(flash.costs.cached_tokens ?? 0),
thinking_tokens: 0,
grounding_requests: 0,
});
});
@@ -446,6 +452,8 @@ describe('GeminiChatProvider.complete streaming', () => {
prompt_tokens: 3, // 4 - 1 cached
completion_tokens: 2,
cached_tokens: 1,
thinking_tokens: 0,
grounding_requests: 0,
});
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
@@ -456,10 +464,192 @@ describe('GeminiChatProvider.complete streaming', () => {
prompt_tokens: 3 * Number(flash.costs.prompt_tokens),
completion_tokens: 2 * Number(flash.costs.completion_tokens),
cached_tokens: 1 * Number(flash.costs.cached_tokens ?? 0),
thinking_tokens: 0,
grounding_requests: 0,
});
});
});
// ── Thinking-token metering ─────────────────────────────────────────
describe('GeminiChatProvider.complete thinking token metering', () => {
it('splits thinking tokens out of completion_tokens and bills each at its own rate', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: { content: 'result', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: {
prompt_tokens: 100,
completion_tokens: 60,
completion_tokens_details: { reasoning_tokens: 20 },
prompt_tokens_details: { cached_tokens: 10 },
},
});
const result = await withTestActor(() =>
provider.complete({
model: 'gemini-2.5-flash',
messages: [{ role: 'user', content: 'think hard' }],
}),
);
expect((result as { usage: unknown }).usage).toEqual({
prompt_tokens: 90, // 100 - 10 cached
completion_tokens: 40, // 60 - 20 thinking
cached_tokens: 10,
thinking_tokens: 20,
grounding_requests: 0,
});
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
expect(recordSpy).toHaveBeenCalledTimes(1);
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
expect(usage.thinking_tokens).toBe(20);
expect(usage.completion_tokens).toBe(40);
expect(overrides).toMatchObject({
thinking_tokens: 20 * Number(flash.costs.thinking_tokens),
completion_tokens: 40 * Number(flash.costs.completion_tokens),
});
});
it('sets thinking_tokens to 0 when completion_tokens_details is absent', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: { content: 'ok', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 5, completion_tokens: 3 },
});
await withTestActor(() =>
provider.complete({
model: 'gemini-2.5-flash',
messages: [{ role: 'user', content: 'hi' }],
}),
);
const [usage] = recordSpy.mock.calls[0]!;
expect(usage.thinking_tokens).toBe(0);
expect(usage.completion_tokens).toBe(3);
});
});
// ── Grounding-request metering ───────────────────────────────────────
describe('GeminiChatProvider.complete grounding request metering', () => {
it('charges one grounding request when non-stream response contains grounding_metadata', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: {
content: 'result',
role: 'assistant',
extra_content: {
grounding_metadata: { web_search_queries: ['foo'] },
},
},
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 10, completion_tokens: 5 },
});
await withTestActor(() =>
provider.complete({
model: 'gemini-2.5-flash',
messages: [{ role: 'user', content: 'search for foo' }],
}),
);
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
expect(usage.grounding_requests).toBe(1);
expect(overrides!.grounding_requests).toBe(
1 * Number(flash.costs.grounding_requests),
);
});
it('does not charge a grounding request when no grounding_metadata is present', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: { content: 'ok', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 5, completion_tokens: 3 },
});
await withTestActor(() =>
provider.complete({
model: 'gemini-2.5-flash',
messages: [{ role: 'user', content: 'hi' }],
}),
);
const [usage] = recordSpy.mock.calls[0]!;
expect(usage.grounding_requests).toBe(0);
});
it('charges one grounding request when streaming response contains grounding extra_content', async () => {
const { provider } = makeProvider();
createMock.mockReturnValueOnce(
asAsyncIterable([
{ choices: [{ delta: { content: 'ans' } }] },
{
choices: [
{
delta: {
extra_content: {
grounding_metadata: {
web_search_queries: ['bar'],
},
},
},
},
],
},
{
choices: [{ delta: {} }],
usage: { prompt_tokens: 8, completion_tokens: 4 },
},
]),
);
const result = await withTestActor(() =>
provider.complete({
model: 'gemini-2.5-flash',
messages: [{ role: 'user', content: 'search bar' }],
stream: true,
}),
);
const harness = makeCapturingChatStream();
await (
result as {
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
}
).init_chat_stream({ chatStream: harness.chatStream });
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
expect(recordSpy).toHaveBeenCalledTimes(1);
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
expect(usage.grounding_requests).toBe(1);
expect(overrides!.grounding_requests).toBe(
1 * Number(flash.costs.grounding_requests),
);
});
});
// ── Error mapping ───────────────────────────────────────────────────
describe('GeminiChatProvider.complete error mapping', () => {
@@ -102,24 +102,52 @@ export class GeminiChatProvider implements IChatProvider {
}
return handle_completion_output({
usage_calculator: ({ usage }) => {
usage_calculator: (args) => {
// Cast to access Gemini-specific extras passed alongside usage:
// - choices: non-stream grounding metadata lives in choices[0].message.extra_content
// - extra_content: streaming grounding metadata accumulated by the stream handler
const { usage, choices, extra_content } = args as {
usage: typeof args.usage;
choices?: Array<{
message?: {
extra_content?: { grounding_metadata?: unknown };
};
}>;
extra_content?: { grounding_metadata?: unknown };
};
const cached_tokens =
usage?.prompt_tokens_details?.cached_tokens ?? 0;
// Thinking tokens are a subset of completion_tokens billed at a different rate
const thinking_tokens =
usage?.completion_tokens_details?.reasoning_tokens ?? 0;
const trackedUsage = {
prompt_tokens:
(usage.prompt_tokens ?? 0) -
(usage.prompt_tokens_details?.cached_tokens ?? 0),
completion_tokens: usage.completion_tokens ?? 0,
cached_tokens:
usage.prompt_tokens_details?.cached_tokens ?? 0,
prompt_tokens: (usage?.prompt_tokens ?? 0) - cached_tokens,
completion_tokens: Math.max(
0,
(usage?.completion_tokens ?? 0) - thinking_tokens,
),
cached_tokens,
thinking_tokens,
// Grounding search is a per-request fee not reflected in token counts
grounding_requests:
(choices?.[0]?.message?.extra_content
?.grounding_metadata ??
extra_content?.grounding_metadata)
? 1
: 0,
};
const costsOverrideFromModel = Object.fromEntries(
Object.entries(trackedUsage).map(([k, v]) => {
return [k, v * modelUsed.costs[k]];
return [k, v * (modelUsed.costs[k] ?? 0)];
}),
);
this.meteringService.utilRecordUsageObject(
trackedUsage,
actor,
actor!,
`gemini:${modelUsed?.id}`,
costsOverrideFromModel,
);
@@ -43,7 +43,10 @@ export const GEMINI_MODELS: IChatModel[] = [
tokens: 1_000_000,
prompt_tokens: 150,
completion_tokens: 900,
thinking_tokens: 900,
cached_tokens: 15,
// Gemini 3.x grounding is $14 / 1,000 requests
grounding_requests: 1_400_000,
},
},
{
@@ -68,6 +71,8 @@ export const GEMINI_MODELS: IChatModel[] = [
prompt_tokens: 10,
completion_tokens: 40,
cached_tokens: 3,
// Gemini 2.x grounding is $35 / 1,000 requests
grounding_requests: 3_500_000,
},
max_tokens: 8192,
},
@@ -115,8 +120,13 @@ export const GEMINI_MODELS: IChatModel[] = [
costs: {
tokens: 1_000_000,
prompt_tokens: 30,
// Output is $2.50/M; thinking tokens bill at the same output rate
completion_tokens: 250,
thinking_tokens: 250,
// Cache read is $0.03/M (10% of input)
cached_tokens: 3,
// Gemini 2.x grounding is $35 / 1,000 requests
grounding_requests: 3_500_000,
},
max_tokens: 65536,
},
@@ -141,7 +151,10 @@ export const GEMINI_MODELS: IChatModel[] = [
tokens: 1_000_000,
prompt_tokens: 10,
completion_tokens: 40,
thinking_tokens: 40,
cached_tokens: 1,
// Gemini 2.x grounding is $35 / 1,000 requests
grounding_requests: 3_500_000,
},
max_tokens: 65536,
},
@@ -166,7 +179,10 @@ export const GEMINI_MODELS: IChatModel[] = [
tokens: 1_000_000,
prompt_tokens: 125,
completion_tokens: 1000,
cached_tokens: 13,
thinking_tokens: 1000,
cached_tokens: 31,
// Gemini 2.x grounding is $35 / 1,000 requests
grounding_requests: 3_500_000,
},
max_tokens: 200_000,
},
@@ -191,7 +207,9 @@ export const GEMINI_MODELS: IChatModel[] = [
tokens: 1_000_000,
prompt_tokens: 200,
completion_tokens: 1200,
thinking_tokens: 1200,
cached_tokens: 20,
grounding_requests: 1_400_000,
},
max_tokens: 65536,
},
@@ -216,7 +234,9 @@ export const GEMINI_MODELS: IChatModel[] = [
tokens: 1_000_000,
prompt_tokens: 50,
completion_tokens: 300,
thinking_tokens: 300,
cached_tokens: 5,
grounding_requests: 1_400_000,
},
max_tokens: 65536,
},
@@ -245,7 +265,9 @@ export const GEMINI_MODELS: IChatModel[] = [
tokens: 1_000_000,
prompt_tokens: 25,
completion_tokens: 150,
thinking_tokens: 150,
cached_tokens: 2.5,
grounding_requests: 1_400_000,
},
max_tokens: 65536,
},
@@ -143,6 +143,11 @@ describe('MiniMaxProvider model catalog', () => {
it('uses MiniMax completion-token limits instead of the context window', () => {
for (const model of MINIMAX_MODELS) {
if (model.id === 'minimax-m3') {
expect(model.context).toBe(1_048_576);
expect(model.max_tokens).toBe(512_000);
continue;
}
expect(model.context).toBe(204_800);
expect(model.max_tokens).toBe(196_608);
}
@@ -18,60 +18,215 @@
*/
import type { IChatModel } from '../../types.js';
import { usdPerMToken } from '../../utils/pricing.js';
const CONTEXT_WINDOW = 204_800;
const MAX_OUTPUT_TOKENS = 196_608;
type MiniMaxChatModel = IChatModel & {
apiModel: string;
};
const minimaxModel = (
apiModel: string,
name: string,
costs: IChatModel['costs'],
): MiniMaxChatModel => {
const id = apiModel.toLowerCase();
return {
puterId: `minimax:minimax/${id}`,
id,
apiModel,
name,
aliases: [`minimax/${id}`, apiModel, `minimax/${apiModel}`],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: CONTEXT_WINDOW,
max_tokens: MAX_OUTPUT_TOKENS,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs,
};
};
// Hardcoded from MiniMax OpenAI-compatible API docs and pay-as-you-go pricing:
// https://platform.minimax.io/docs/api-reference/text-openai-api
// https://platform.minimax.io/docs/guides/pricing-paygo
export const MINIMAX_MODELS: MiniMaxChatModel[] = [
minimaxModel('MiniMax-M2.7', 'MiniMax M2.7', usdPerMToken(0.3, 1.2, 0.06)),
minimaxModel(
'MiniMax-M2.7-highspeed',
'MiniMax M2.7 Highspeed',
usdPerMToken(0.6, 2.4, 0.06),
),
minimaxModel('MiniMax-M2.5', 'MiniMax M2.5', usdPerMToken(0.3, 1.2, 0.03)),
minimaxModel(
'MiniMax-M2.5-highspeed',
'MiniMax M2.5 Highspeed',
usdPerMToken(0.6, 2.4, 0.03),
),
minimaxModel('MiniMax-M2.1', 'MiniMax M2.1', usdPerMToken(0.3, 1.2, 0.03)),
minimaxModel(
'MiniMax-M2.1-highspeed',
'MiniMax M2.1 Highspeed',
usdPerMToken(0.6, 2.4, 0.03),
),
minimaxModel('MiniMax-M2', 'MiniMax M2', usdPerMToken(0.3, 1.2, 0.03)),
// -- MiniMax M3 (Flagship) --------------------------------------
// MiniMax Sparse Attention, 1M context, native multimodal input.
{
puterId: 'minimax:minimax/minimax-m3',
id: 'minimax-m3',
apiModel: 'MiniMax-M3',
name: 'MiniMax M3',
aliases: ['minimax/minimax-m3', 'MiniMax-M3', 'minimax/MiniMax-M3'],
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 1_048_576,
max_tokens: 512_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 30, // $0.30 per 1M (input <= 512k tokens)
completion_tokens: 120, // $1.20 per 1M
cached_tokens: 6, // $0.06 per 1M
},
},
// -- MiniMax M2.7 -----------------------------------------------
{
puterId: 'minimax:minimax/minimax-m2.7',
id: 'minimax-m2.7',
apiModel: 'MiniMax-M2.7',
name: 'MiniMax M2.7',
aliases: [
'minimax/minimax-m2.7',
'MiniMax-M2.7',
'minimax/MiniMax-M2.7',
],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 30, // $0.30 per 1M
completion_tokens: 120, // $1.20 per 1M
cached_tokens: 6, // $0.06 per 1M
},
},
{
puterId: 'minimax:minimax/minimax-m2.7-highspeed',
id: 'minimax-m2.7-highspeed',
apiModel: 'MiniMax-M2.7-highspeed',
name: 'MiniMax M2.7 Highspeed',
aliases: [
'minimax/minimax-m2.7-highspeed',
'MiniMax-M2.7-highspeed',
'minimax/MiniMax-M2.7-highspeed',
],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 60, // $0.60 per 1M
completion_tokens: 240, // $2.40 per 1M
cached_tokens: 6, // $0.06 per 1M
},
},
// -- MiniMax M2.5 -----------------------------------------------
{
puterId: 'minimax:minimax/minimax-m2.5',
id: 'minimax-m2.5',
apiModel: 'MiniMax-M2.5',
name: 'MiniMax M2.5',
aliases: [
'minimax/minimax-m2.5',
'MiniMax-M2.5',
'minimax/MiniMax-M2.5',
],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 30, // $0.30 per 1M
completion_tokens: 120, // $1.20 per 1M
cached_tokens: 3, // $0.03 per 1M
},
},
{
puterId: 'minimax:minimax/minimax-m2.5-highspeed',
id: 'minimax-m2.5-highspeed',
apiModel: 'MiniMax-M2.5-highspeed',
name: 'MiniMax M2.5 Highspeed',
aliases: [
'minimax/minimax-m2.5-highspeed',
'MiniMax-M2.5-highspeed',
'minimax/MiniMax-M2.5-highspeed',
],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 60, // $0.60 per 1M
completion_tokens: 240, // $2.40 per 1M
cached_tokens: 3, // $0.03 per 1M
},
},
// -- MiniMax M2.1 -----------------------------------------------
{
puterId: 'minimax:minimax/minimax-m2.1',
id: 'minimax-m2.1',
apiModel: 'MiniMax-M2.1',
name: 'MiniMax M2.1',
aliases: [
'minimax/minimax-m2.1',
'MiniMax-M2.1',
'minimax/MiniMax-M2.1',
],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 30, // $0.30 per 1M
completion_tokens: 120, // $1.20 per 1M
cached_tokens: 3, // $0.03 per 1M
},
},
{
puterId: 'minimax:minimax/minimax-m2.1-highspeed',
id: 'minimax-m2.1-highspeed',
apiModel: 'MiniMax-M2.1-highspeed',
name: 'MiniMax M2.1 Highspeed',
aliases: [
'minimax/minimax-m2.1-highspeed',
'MiniMax-M2.1-highspeed',
'minimax/MiniMax-M2.1-highspeed',
],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 60, // $0.60 per 1M
completion_tokens: 240, // $2.40 per 1M
cached_tokens: 3, // $0.03 per 1M
},
},
// -- MiniMax M2 -------------------------------------------------
{
puterId: 'minimax:minimax/minimax-m2',
id: 'minimax-m2',
apiModel: 'MiniMax-M2',
name: 'MiniMax M2',
aliases: ['minimax/minimax-m2', 'MiniMax-M2', 'minimax/MiniMax-M2'],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: 204_800,
max_tokens: 196_608,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 30, // $0.30 per 1M
completion_tokens: 120, // $1.20 per 1M
cached_tokens: 3, // $0.03 per 1M
},
},
];
@@ -655,6 +655,123 @@ describe('MistralAIProvider.complete streaming', () => {
});
});
// -- Mistral image_url coercion --
describe('MistralAIProvider image_url coercion', () => {
const baseCompletion = {
choices: [
{
message: { content: 'ok', role: 'assistant' },
finishReason: 'stop',
},
],
usage: { promptTokens: 1, completionTokens: 1 },
};
it('flattens image_url objects to plain strings before sending to Mistral', async () => {
const { provider } = makeProvider();
completeMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'mistral-small-2603',
messages: [
{
role: 'user',
content: [
{ type: 'text', text: 'describe this image' },
{
type: 'image_url',
image_url: { url: 'https://example.com/img.png' },
},
],
},
],
}),
);
const [args] = completeMock.mock.calls[0]!;
const imagePart = (args.messages[0].content as unknown[]).find(
(p: unknown) => (p as { type: string }).type === 'image_url',
) as { type: string; image_url: unknown };
// Mistral expects a plain string, not { url: string }.
expect(imagePart.image_url).toBe('https://example.com/img.png');
});
it('leaves image_url parts that are already plain strings unchanged', async () => {
const { provider } = makeProvider();
completeMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'mistral-small-2603',
messages: [
{
role: 'user',
content: [
{
type: 'image_url',
image_url: 'https://example.com/already-flat.png',
},
],
},
],
}),
);
const [args] = completeMock.mock.calls[0]!;
const imagePart = (args.messages[0].content as unknown[]).find(
(p: unknown) => (p as { type: string }).type === 'image_url',
) as { type: string; image_url: unknown };
expect(imagePart.image_url).toBe('https://example.com/already-flat.png');
});
it('leaves messages with plain string content untouched', async () => {
const { provider } = makeProvider();
completeMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'mistral-small-2603',
messages: [{ role: 'user', content: 'plain text message' }],
}),
);
const [args] = completeMock.mock.calls[0]!;
expect(args.messages[0].content).toBe('plain text message');
});
it('coerces only image_url parts and leaves other content parts intact', async () => {
const { provider } = makeProvider();
completeMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'mistral-small-2603',
messages: [
{
role: 'user',
content: [
{ type: 'text', text: 'what is in this image?' },
{
type: 'image_url',
image_url: { url: 'https://example.com/photo.jpg' },
},
],
},
],
}),
);
const [args] = completeMock.mock.calls[0]!;
const parts = args.messages[0].content as { type: string; text?: string; image_url?: unknown }[];
const textPart = parts.find((p) => p.type === 'text');
const imgPart = parts.find((p) => p.type === 'image_url');
expect(textPart?.text).toBe('what is in this image?');
expect(imgPart?.image_url).toBe('https://example.com/photo.jpg');
});
});
// ── Error mapping ───────────────────────────────────────────────────
describe('MistralAIProvider.complete error mapping', () => {
@@ -61,6 +61,38 @@ export class MistralAIProvider implements IChatProvider {
return ids;
}
/**
* Mistral's API expects `image_url` content parts to carry a plain
* string URL, not the OpenAI-style `{ url: string }` object.
* This method normalises any `{ type: 'image_url', image_url: { url } }`
* parts to `{ type: 'image_url', image_url: url }` before the request
* is sent. Messages whose `content` is a plain string are left untouched.
*/
#coerceImageUrls(
messages: { role: string; content: unknown }[],
): { role: string; content: unknown }[] {
return messages.map((message) => {
if (!Array.isArray(message.content)) return message;
const content = message.content.map(
(part: { type?: string; image_url?: unknown }) => {
if (
part.type === 'image_url' &&
part.image_url !== null &&
typeof part.image_url === 'object' &&
'url' in (part.image_url as object)
) {
return {
...part,
image_url: (part.image_url as { url: string }).url,
};
}
return part;
},
);
return { ...message, content };
});
}
async complete({
messages,
stream,
@@ -70,6 +102,7 @@ export class MistralAIProvider implements IChatProvider {
temperature,
}: ICompleteArguments): Promise<IChatCompleteResult> {
messages = await OpenAIUtil.process_input_messages(messages);
messages = this.#coerceImageUrls(messages);
for (const message of messages) {
if (message.tool_calls) {
message.toolCalls = message.tool_calls;
@@ -26,6 +26,10 @@ import type { MeteringService } from '../../../../services/metering/MeteringServ
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
import type { IChatProvider, ICompleteArguments } from '../../types.js';
import {
messagesHaveCompaction,
wantsCompaction,
} from '../../utils/compaction.js';
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import { processPuterPathUploads } from './fileUpload.js';
import { OPEN_AI_MODELS } from './models.js';
@@ -125,6 +129,20 @@ export class OpenAiChatProvider implements IChatProvider {
}
return await this.#responsesProvider.complete(params);
}
// Inline compaction is a Responses-API feature; chat.completions can't
// express `context_management` or a `compaction` content block.
// Delegate to the sibling Responses provider when the caller opted in
// OR when the messages carry a round-tripped compaction artifact.
if (wantsCompaction(params) || messagesHaveCompaction(messages)) {
if (!this.#responsesProvider) {
throw new HttpError(
400,
'compaction requires the OpenAI Responses provider, which is not configured',
{ legacyCode: 'bad_request' },
);
}
return await this.#responsesProvider.complete(params);
}
// Validate messages
if (!Array.isArray(messages)) {
throw new HttpError(400, '`messages` must be an array', {
@@ -25,6 +25,7 @@ import type { MeteringService } from '../../../../services/metering/MeteringServ
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
import type { IChatProvider, ICompleteArguments } from '../../types.js';
import { toOpenAiContextManagement } from '../../utils/compaction.js';
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import { processPuterPathUploads } from './fileUpload.js';
import { OPEN_AI_MODELS } from './models.js';
@@ -100,6 +101,8 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
parallel_tool_calls,
include,
conversation,
compaction,
context_management,
previous_response_id,
instructions,
metadata,
@@ -177,6 +180,13 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
const supportsReasoningControls =
typeof model === 'string' && model.startsWith('gpt-5');
// Translate the neutral compaction opt-in (or pass a raw
// `context_management` payload through) to OpenAI's Responses shape.
const contextManagement = toOpenAiContextManagement({
compaction,
context_management,
});
const completionParams: ResponseCreateParams = {
user: userIdentifier,
safety_identifier: userIdentifier,
@@ -188,6 +198,9 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
? { parallel_tool_calls }
: {}),
...(include !== undefined ? { include } : {}),
...(contextManagement !== undefined
? { context_management: contextManagement }
: {}),
...(conversation !== undefined ? { conversation } : {}),
...(previous_response_id !== undefined
? { previous_response_id }
@@ -363,6 +363,30 @@ export const OPEN_AI_MODELS: IChatModel[] = [
context: 128_000,
max_tokens: 128000,
},
{
puterId: 'openai:openai/gpt-5-codex',
id: 'gpt-5-codex',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-09-30',
release_date: '2025-09-15',
aliases: ['openai/gpt-5-codex'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
// models.dev openai: input $1.25 / output $10 / cache_read $0.125
// per 1M — identical to gpt-5.
tokens: 1_000_000,
prompt_tokens: 125,
cached_tokens: 13,
completion_tokens: 1000,
},
context: 128_000,
max_tokens: 128000,
responses_api_only: true,
},
{
puterId: 'openai:openai/gpt-5-mini',
id: 'gpt-5-mini-2025-08-07',
@@ -150,21 +150,7 @@ export class TogetherAIProvider implements IChatProvider {
messages,
stream,
...(tools ? { tools } : {}),
// TODO: make this better but togetherai doesn't handle max tokens properly at all
...(max_tokens
? {
max_tokens:
max_tokens -
messages.reduce((acc, curr) => {
return (
acc +
(curr.type === 'text'
? curr.text.length / 2
: 200)
);
}, 0),
}
: {}),
...(max_tokens ? { max_tokens } : {}),
...(temperature ? { temperature } : {}),
...(stream ? { stream_options: { include_usage: true } } : {}),
} as Together.Chat.Completions.CompletionCreateParamsNonStreaming);
@@ -198,6 +198,66 @@ export const XAI_MODELS: IChatModel[] = [
},
max_tokens: 30_000,
},
{
puterId: 'x-ai:x-ai/grok-4-20-reasoning',
// xAI exposes this as the dated snapshot id; `grok-4-20-reasoning`
// (and dotted forms) are accepted as aliases by callers.
id: 'grok-4.20-0309-reasoning',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-07',
release_date: '2026-03-09',
name: 'Grok 4.20 (Reasoning)',
aliases: [
'x-ai/grok-4-20-reasoning',
'grok-4-20-reasoning',
'grok-4.20-reasoning',
'x-ai/grok-4.20-reasoning',
],
context: 2_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
// models.dev xai: input $1.25 / output $2.5 / cache_read $0.2 per 1M.
prompt_tokens: 125,
completion_tokens: 250,
cached_tokens: 20,
},
max_tokens: 30_000,
},
{
puterId: 'x-ai:x-ai/grok-4-20-non-reasoning',
// xAI exposes this as the dated snapshot id; `grok-4-20-non-reasoning`
// (and dotted forms) are accepted as aliases by callers.
id: 'grok-4.20-0309-non-reasoning',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-07',
release_date: '2026-03-09',
name: 'Grok 4.20 (Non-Reasoning)',
aliases: [
'x-ai/grok-4-20-non-reasoning',
'grok-4-20-non-reasoning',
'grok-4.20-non-reasoning',
'x-ai/grok-4.20-non-reasoning',
],
context: 2_000_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
// models.dev xai: input $1.25 / output $2.5 / cache_read $0.2 per 1M.
prompt_tokens: 125,
completion_tokens: 250,
cached_tokens: 20,
},
max_tokens: 30_000,
},
{
puterId: 'x-ai:x-ai/grok-4-1-fast',
id: 'grok-4-1-fast',
@@ -69,6 +69,13 @@ const visionModel = (
// Hardcoded from https://docs.z.ai/api-reference/llm/chat-completion and
// https://docs.z.ai/guides/overview/pricing.
export const ZAI_MODELS: IChatModel[] = [
textModel(
'glm-5.2',
'GLM-5.2',
1_000 * K,
128 * K,
usdPerMToken(1.4, 4.4, 0.26),
),
textModel(
'glm-5.1',
'GLM-5.1',
+20
View File
@@ -67,6 +67,19 @@ export interface ICompleteArguments {
parallel_tool_calls?: boolean;
include?: unknown[];
conversation?: unknown;
/**
* Provider-neutral inline-compaction opt-in. `true` enables compaction with
* provider defaults; `{ trigger_tokens }` sets the token threshold at which
* the upstream summarizes earlier context. Each provider translates this to
* its own SDK shape (OpenAI `context_management:[{type:'compaction',...}]`,
* Anthropic `context_management:{edits:[{type:'compact_20260112'}]}`).
*/
compaction?: boolean | { trigger_tokens?: number };
/**
* Escape hatch: provider-native `context_management` payload, passed through
* untouched (used by `/responses` callers sending the OpenAI-native array).
*/
context_management?: unknown;
previous_response_id?: string;
instructions?: string | PuterMessage[];
metadata?: Record<string, string>;
@@ -116,6 +129,13 @@ export interface IChatMessageResult {
finally_fn?: never;
normalized?: boolean;
via_ai_chat_service?: boolean;
/**
* Inline-compaction artifact, present when the upstream compacted earlier
* context during this (non-streaming) response. Carries `type:'compaction'`
* so it's a drop-in `messages` item — the caller resends it on the next turn
* in place of the summarized history. See [[ICompleteArguments]].
*/
compaction?: { type: 'compaction'; id?: string; encrypted_content: string };
}
export type IChatCompleteResult = IChatStreamResult | IChatMessageResult;
@@ -49,6 +49,23 @@ export const normalize_single_message = (message, params = {}) => {
legacyCode: 'bad_request',
});
}
// A round-tripped inline-compaction artifact may be supplied as a bare
// top-level item (the shape the client received from the stream). Wrap it
// into an internal compaction content block so it survives normalization;
// each provider maps it back to its native input shape (OpenAI top-level
// input item, Anthropic content block).
if (!message.role && !message.content && message.type === 'compaction') {
return {
role: 'assistant',
content: [
{
type: 'compaction',
...(message.id !== undefined ? { id: message.id } : {}),
encrypted_content: message.encrypted_content,
},
],
};
}
if (!message.role) {
message.role = params.role;
}
@@ -46,6 +46,22 @@ describe('normalize_single_message', () => {
expect(result.role).toBe('system');
});
it('wraps a bare round-tripped compaction item into a compaction content block', () => {
const result = normalize_single_message({
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
});
expect(result.role).toBe('assistant');
expect(result.content).toEqual([
{
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
]);
});
it('throws 400 when message is null/undefined/array', () => {
expect(() => normalize_single_message(null)).toThrow(
expect.objectContaining({ statusCode: 400 }),
@@ -84,6 +84,39 @@ export const process_input_messages = async (messages) => {
};
export const process_input_messages_responses_api = async (messages) => {
// Pre-split round-tripped compaction blocks into standalone Responses
// compaction input items, preserving any sibling content (e.g. the
// assistant's reply text) as its own message. A compaction item represents
// prior history, so it precedes the message it was attached to. This avoids
// collapsing the whole message into a single compaction item and dropping
// the rest of its content.
const expanded = [];
for (const msg of messages) {
if (msg && Array.isArray(msg.content)) {
const compactionBlocks = msg.content.filter(
(c) => c && c.type === 'compaction',
);
if (compactionBlocks.length > 0) {
for (const block of compactionBlocks) {
expanded.push({
type: 'compaction',
...(block.id !== undefined ? { id: block.id } : {}),
encrypted_content: block.encrypted_content,
});
}
const rest = msg.content.filter(
(c) => !(c && c.type === 'compaction'),
);
if (rest.length > 0) {
expanded.push({ ...msg, content: rest });
}
continue;
}
}
expanded.push(msg);
}
messages = expanded;
for (const msg of messages) {
const content_as_string = (content) => {
if (content === undefined || content === null) return '';
@@ -253,6 +286,7 @@ export const create_chat_stream_handler =
const tool_call_blocks = [];
let last_usage = null;
let last_extra_content = null;
for await (let chunk of completion) {
chunk = deviations.chunk_but_like_actually(chunk);
const chunk_usage = deviations.index_usage_from_stream_chunk(chunk);
@@ -285,6 +319,14 @@ export const create_chat_stream_handler =
// Apps have to choose to handle extra_content themselves, it doesn't seem like theres a way we can do it in a backwards
// compatible fashion since most streaming apps will handle chat history by continuously updating content themselves
// This doesn't present us a chance to add in an extra object for gemini's chat continuing features
// Don't let a later extra_content chunk without grounding clobber an
// earlier one that carried grounding_metadata (used for metering).
if (
choice.delta.extra_content.grounding_metadata ||
!last_extra_content?.grounding_metadata
) {
last_extra_content = choice.delta.extra_content;
}
textblock.addExtraContent(choice.delta.extra_content);
}
@@ -315,7 +357,10 @@ export const create_chat_stream_handler =
}
// TODO DS: this is a bit too abstracted... this is basically just doing the metering now
const usage = usage_calculator({ usage: last_usage });
const usage = usage_calculator({
usage: last_usage,
extra_content: last_extra_content,
});
if (mode === 'text') textblock.end();
if (mode === 'tool') toolblock.end();
@@ -355,6 +400,19 @@ export const create_chat_stream_handler_responses_api =
last_usage = chunk.response.usage;
}
if (
chunk.type === 'response.output_item.done' &&
chunk.item?.type === 'compaction'
) {
// Inline compaction fired mid-response — normalize the artifact
// into the canonical internal compaction event.
chatStream.compaction({
id: chunk.item.id,
encrypted_content: chunk.item.encrypted_content,
});
continue;
}
if (
chunk.type === 'response.output_item.done' &&
chunk.item?.type === 'function_call'
@@ -488,10 +546,14 @@ export const handle_completion_output_responses_api = async ({
...(item.id ? { canonical_id: item.id } : {}),
}));
// Inline-compaction artifact, if the upstream compacted this turn.
const compactionItem = output.find((item) => item?.type === 'compaction');
const is_empty = completion.output_text.trim() === '';
if (is_empty && responseToolCalls.length < 1) {
if (is_empty && responseToolCalls.length < 1 && !compactionItem) {
// GPT refuses to generate an empty response if you ask it to,
// so this will probably only happen on an error condition.
// A compaction-only output is legitimate, so don't reject it.
throw new HttpError(400, 'an empty response was generated', {
legacyCode: 'bad_response',
});
@@ -523,6 +585,18 @@ export const handle_completion_output_responses_api = async ({
};
ret.role = output.find((item) => item?.role)?.role ?? 'assistant';
if (compactionItem) {
// Include `type` so the artifact is a drop-in `messages` item for the
// stateless round-trip — symmetric with the streaming compaction chunk.
ret.compaction = {
type: 'compaction',
...(compactionItem.id !== undefined
? { id: compactionItem.id }
: {}),
encrypted_content: compactionItem.encrypted_content,
};
}
delete ret.type;
ret.usage = usage_calculator
@@ -232,6 +232,66 @@ describe('process_input_messages_responses_api', () => {
expect(out!.output).toBe('part-apart-bpart-c');
});
it('rewrites a round-tripped compaction content block into a top-level compaction item', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
],
},
];
const [out] = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
expect(out!.type).toBe('compaction');
expect(out!.encrypted_content).toBe('ENC');
expect(out!.id).toBe('cmpct_1');
expect(out!.role).toBeUndefined();
expect(out!.content).toBeUndefined();
});
it('splits a compaction block + reply text into a compaction item plus a preserved message', async () => {
// Round-tripping an assistant turn that carries BOTH the compaction
// artifact and the reply text must keep the reply, not drop it.
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
{ type: 'text', text: 'earlier reply' },
],
},
];
const out = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
// Compaction item comes first (it represents prior history)...
expect(out).toHaveLength(2);
expect(out[0]).toEqual({
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
});
// ...followed by the assistant message with the reply preserved
// (assistant text upgraded to output_text).
expect(out[1]!.role).toBe('assistant');
expect(out[1]!.content).toEqual([
{ type: 'output_text', text: 'earlier reply' },
]);
});
it('upgrades user/system text blocks to input_text', async () => {
const messages: Array<Record<string, unknown>> = [
{
@@ -539,6 +599,39 @@ describe('create_chat_stream_handler_responses_api', () => {
});
});
it('emits a compaction event when a compaction output_item completes', async () => {
const completion = asAsyncIterable([
{
type: 'response.output_item.done',
item: {
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
},
{
type: 'response.completed',
response: { usage: { input_tokens: 1, output_tokens: 2 } },
},
]);
const init = create_chat_stream_handler_responses_api({
deviations: undefined,
completion,
usage_calculator: () => ({}),
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const compaction = harness
.events()
.find((e: { type: string }) => e.type === 'compaction');
expect(compaction).toEqual({
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
});
});
it('emits a tool_use block when a function_call output_item completes', async () => {
const completion = asAsyncIterable([
{
@@ -765,6 +858,31 @@ describe('handle_completion_output_responses_api non-stream', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('attaches a compaction artifact and allows a compaction-only output', async () => {
const completion = {
output: [
{
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
},
],
output_text: ' ',
usage: { input_tokens: 1, output_tokens: 0 },
};
const result = await handle_completion_output_responses_api({
deviations: undefined,
stream: false,
completion,
});
// Compaction-only output is not rejected as "empty".
expect(result.compaction).toEqual({
type: 'compaction',
id: 'cmpct_1',
encrypted_content: 'ENC',
});
});
it('runs moderation against output_text when a moderate fn is supplied', async () => {
const completion = {
output: [{ role: 'assistant' }],
@@ -103,6 +103,25 @@ export class AIChatStream {
this.stream.end();
}
/**
* Emit a canonical compaction event into the NDJSON stream. Both the OpenAI
* and Anthropic providers normalize their native inline-compaction artifact
* to this single shape, so downstream consumers (controllers, puter.js) see
* an identical `{ type: 'compaction', id, encrypted_content }` chunk
* regardless of which upstream served the request.
*
* @param {{ id?: string, encrypted_content: string }} compaction
*/
compaction({ id, encrypted_content }) {
this.stream.write(
`${JSON.stringify({
type: 'compaction',
...(id !== undefined ? { id } : {}),
encrypted_content,
})}\n`,
);
}
message() {
return new AIChatMessageStream(this);
}
@@ -0,0 +1,124 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Inline-compaction translation helpers.
*
* The driver-facing surface exposes a single provider-neutral opt-in
* (`compaction: boolean | { trigger_tokens }` on `ICompleteArguments`), plus a
* raw `context_management` escape hatch for callers hitting `/responses` with
* the OpenAI-native shape. These helpers map that neutral opt-in to each
* provider's SDK shape so the providers stay free of opt-in-parsing logic.
*/
/**
* @param {boolean | { trigger_tokens?: number } | undefined} compaction
* @returns {{ enabled: boolean, trigger_tokens?: number }}
*/
const readCompaction = (compaction) => {
if (compaction === true) return { enabled: true };
if (compaction && typeof compaction === 'object') {
return {
enabled: true,
...(typeof compaction.trigger_tokens === 'number'
? { trigger_tokens: compaction.trigger_tokens }
: {}),
};
}
return { enabled: false };
};
/**
* Build OpenAI Responses `context_management` from the neutral opt-in. A raw
* `context_management` passthrough (already in OpenAI shape) wins.
*
* @param {{ compaction?: boolean | { trigger_tokens?: number }, context_management?: unknown }} args
* @returns {Array<{ type: 'compaction', compact_threshold?: number }> | undefined}
*/
export const toOpenAiContextManagement = (args) => {
if (args.context_management !== undefined) {
return /** @type {any} */ (args.context_management);
}
const { enabled, trigger_tokens } = readCompaction(args.compaction);
if (!enabled) return undefined;
return [
{
type: 'compaction',
...(trigger_tokens !== undefined
? { compact_threshold: trigger_tokens }
: {}),
},
];
};
/**
* Build Anthropic `context_management` (beta `compact-2026-01-12`) from the
* neutral opt-in. A raw `context_management` passthrough wins.
*
* @param {{ compaction?: boolean | { trigger_tokens?: number }, context_management?: unknown }} args
* @returns {{ edits: Array<Record<string, unknown>> } | undefined}
*/
export const toAnthropicContextManagement = (args) => {
if (args.context_management !== undefined) {
return /** @type {any} */ (args.context_management);
}
const { enabled, trigger_tokens } = readCompaction(args.compaction);
if (!enabled) return undefined;
return {
edits: [
{
type: 'compact_20260112',
...(trigger_tokens !== undefined
? {
trigger: {
type: 'input_tokens',
value: trigger_tokens,
},
}
: {}),
},
],
};
};
/**
* Whether the request opted into inline compaction by any route.
*
* @param {{ compaction?: boolean | { trigger_tokens?: number }, context_management?: unknown }} args
*/
export const wantsCompaction = (args) =>
args.context_management !== undefined ||
readCompaction(args.compaction).enabled;
/**
* Whether the (normalized) message list carries a round-tripped compaction
* artifact. Such a request must route through a compaction-capable surface even
* if it didn't request *new* compaction — chat.completions can't represent a
* compaction content block, and Anthropic needs its compaction beta to accept
* one as input.
*
* @param {unknown} messages
*/
export const messagesHaveCompaction = (messages) =>
Array.isArray(messages) &&
messages.some(
(m) =>
Array.isArray(m?.content) &&
m.content.some((c) => c && c.type === 'compaction'),
);
@@ -0,0 +1,132 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import {
messagesHaveCompaction,
toAnthropicContextManagement,
toOpenAiContextManagement,
wantsCompaction,
} from './compaction.js';
describe('toOpenAiContextManagement', () => {
it('returns undefined when compaction is off', () => {
expect(toOpenAiContextManagement({})).toBeUndefined();
expect(
toOpenAiContextManagement({ compaction: false }),
).toBeUndefined();
});
it('builds a compaction entry from `true`', () => {
expect(toOpenAiContextManagement({ compaction: true })).toEqual([
{ type: 'compaction' },
]);
});
it('maps trigger_tokens to compact_threshold', () => {
expect(
toOpenAiContextManagement({ compaction: { trigger_tokens: 1000 } }),
).toEqual([{ type: 'compaction', compact_threshold: 1000 }]);
});
it('passes a raw context_management payload through verbatim', () => {
const raw = [{ type: 'compaction', compact_threshold: 5 }];
expect(
toOpenAiContextManagement({
compaction: true,
context_management: raw,
}),
).toBe(raw);
});
});
describe('toAnthropicContextManagement', () => {
it('returns undefined when compaction is off', () => {
expect(toAnthropicContextManagement({})).toBeUndefined();
});
it('builds a compact_20260112 edit from `true`', () => {
expect(toAnthropicContextManagement({ compaction: true })).toEqual({
edits: [{ type: 'compact_20260112' }],
});
});
it('maps trigger_tokens to an input_tokens trigger', () => {
expect(
toAnthropicContextManagement({
compaction: { trigger_tokens: 2000 },
}),
).toEqual({
edits: [
{
type: 'compact_20260112',
trigger: { type: 'input_tokens', value: 2000 },
},
],
});
});
it('passes a raw context_management payload through verbatim', () => {
const raw = { edits: [{ type: 'compact_20260112' }] };
expect(
toAnthropicContextManagement({ context_management: raw }),
).toBe(raw);
});
});
describe('wantsCompaction', () => {
it('is false without opt-in', () => {
expect(wantsCompaction({})).toBe(false);
expect(wantsCompaction({ compaction: false })).toBe(false);
});
it('is true for the neutral opt-in or a raw passthrough', () => {
expect(wantsCompaction({ compaction: true })).toBe(true);
expect(wantsCompaction({ compaction: { trigger_tokens: 1 } })).toBe(
true,
);
expect(wantsCompaction({ context_management: [] })).toBe(true);
});
});
describe('messagesHaveCompaction', () => {
it('detects a round-tripped compaction content block', () => {
expect(
messagesHaveCompaction([
{ role: 'user', content: [{ type: 'text', text: 'hi' }] },
{
role: 'assistant',
content: [
{ type: 'compaction', encrypted_content: 'ENC' },
],
},
]),
).toBe(true);
});
it('is false for ordinary messages or non-arrays', () => {
expect(
messagesHaveCompaction([
{ role: 'user', content: [{ type: 'text', text: 'hi' }] },
]),
).toBe(false);
expect(messagesHaveCompaction(undefined)).toBe(false);
expect(messagesHaveCompaction('nope')).toBe(false);
});
});
+81 -4
View File
@@ -52,6 +52,12 @@ const INDEX_URL_UNIQUENESS_EXEMPTION_CANDIDATES = [
'https://dev-center.puter.com/coming-soon',
];
// Sentinel host for builtin apps. The GUI rewrites index_urls on this
// host to `<gui origin>/builtin/<name>` (see launch_app.js), so rows
// carrying it are reserved for migration-seeded builtins — a user app
// claiming it would load its code same-origin with the desktop.
const BUILTIN_APPS_HOST = 'builtins.namespaces.puter.com';
// Canonical-uid alias namespace. When a user-created app is merged into
// an existing origin-bootstrap row, the source uid is mapped to the
// canonical (kept) uid so any client that still holds the old uid keeps
@@ -465,7 +471,7 @@ export class AppDriver extends PuterDriver {
// -- Validation ---------------------------------------------------
async #validateInput(object, { isCreate }) {
async #validateInput(object, { isCreate, existing }) {
const out = {};
if (isCreate || object.name !== undefined) {
@@ -497,6 +503,15 @@ export class AppDriver extends PuterDriver {
maxLen: 3000,
required: isCreate,
});
// Only enforce on new/changed values so rows that already
// carry a reserved host (migration-seeded builtins) can still
// have their other fields updated.
if (
out.index_url !== undefined &&
out.index_url !== existing?.index_url
) {
this.#assertIndexUrlHostAllowed(out.index_url);
}
}
if (object.icon !== undefined) {
validateString(object.icon, {
@@ -581,6 +596,54 @@ export class AppDriver extends PuterDriver {
return out;
}
/**
* App iframes run with `allow-same-origin allow-scripts`, so an
* index_url loading from the GUI host would execute third-party code
* same-origin with the desktop — a full sandbox escape. The API host
* is reserved for the same reason, and the builtin sentinel host is
* rewritten by the GUI to `<gui origin>/builtin/…` (see
* BUILTIN_APPS_HOST above). Host comparison (rather than full origin)
* deliberately also catches scheme/port variants of these hosts.
*/
#assertIndexUrlHostAllowed(indexUrl) {
let hostname;
try {
hostname = new URL(indexUrl).hostname;
} catch {
// Unparseable values are rejected by `validateUrl` upstream.
return;
}
const config = this.config ?? {};
const reserved = new Set([BUILTIN_APPS_HOST]);
// `origin`/`api_base_url` are computed at boot from `domain`; the
// domain-based fallbacks cover callers (tests, embedders) that
// construct a server without that normalization step.
const candidates = [config.origin, config.api_base_url];
if (config.domain) {
candidates.push(
`http://${config.domain}`,
`http://api.${config.domain}`,
);
}
for (const candidate of candidates) {
if (!candidate) continue;
try {
reserved.add(new URL(candidate).hostname);
} catch {
// Malformed config value — nothing to reserve from it.
}
}
if (reserved.has(hostname)) {
throw new HttpError(
400,
'`index_url` cannot point at a Puter system host',
{ legacyCode: 'bad_request' },
);
}
}
// -- Permission checks --------------------------------------------
#requireActor() {
@@ -849,7 +912,9 @@ export class AppDriver extends PuterDriver {
// `#ensure_puter_site_subdomain_is_owned`, `#ensureIndexUrlNotAlreadyInUse`,
// and the `app:canonicalUidAlias:*` kvstore pair). When a user creates
// or repoints an app at a puter-hosted subdomain (`*.puter.site`,
// `*.puter.app`, …) we want exactly one app row to back that URL:
// `*.puter.app`, …) — or at a custom host claimed by an
// `app_origin_aliases` group — we want exactly one app row to back
// that URL:
// • If a no-owner row exists (origin-bootstrap stub auto-created
// when an unknown origin first hit Puter) → claim it for the
// user, merge the new fields into it.
@@ -1156,7 +1221,8 @@ export class AppDriver extends PuterDriver {
/**
* Merge an incoming create/update into an existing app row that
* already owns the same puter-hosted index_url. Returns the joined
* already owns the same puter-hosted or alias-group index_url.
* Returns the joined
* (client-shaped) app on success, or `null` when no merge applied.
* Throws `app_index_url_already_in_use` when a conflict exists but
* cannot be merged (different owner, or same-owner non-bootstrap).
@@ -1173,7 +1239,18 @@ export class AppDriver extends PuterDriver {
excludeAppId,
} = {}) {
const indexUrl = object?.index_url;
if (!this.#isPuterHostedIndexUrl(indexUrl)) return null;
// Alias-group hosts (`app_origin_aliases`) get the same merge
// treatment as puter-hosted subdomains. Without this, a bootstrap
// stub on a custom domain could never be absorbed — and since
// `#findIndexUrlConflictRow` *does* honor alias groups, the
// uniqueness check below would hard-reject the owner's own
// create/update instead of merging into the stub.
if (
!this.#isPuterHostedIndexUrl(indexUrl) &&
!this.#findOriginAliasGroupForIndexUrl(indexUrl)
) {
return null;
}
const conflictRow = await this.#findIndexUrlConflictRow({
indexUrl,
+184
View File
@@ -143,6 +143,57 @@ describe('AppDriver.create', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
// App iframes get `allow-same-origin allow-scripts`, so an index_url
// on a Puter system host would run third-party code same-origin with
// the GUI. The test server's domain is `puter.localhost` (from
// config.default.json).
it.each([
['the GUI host', 'https://puter.localhost/evil.html'],
['the GUI host on another port/scheme', 'http://puter.localhost:4100/evil.html'],
['the API host', 'https://api.puter.localhost/evil.html'],
['the builtin sentinel host', 'https://builtins.namespaces.puter.com/emulator'],
])('rejects an index_url on %s with 400', async (_label, index_url) => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
name: uniqueName('sys-host'),
title: 't',
index_url,
},
}),
),
).rejects.toMatchObject({
statusCode: 400,
message: /system host/,
});
});
it('rejects updating an index_url to a Puter system host with 400', async () => {
const { actor } = await makeUser();
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('sys-host-upd'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
await expect(
withActor(actor, () =>
driver.update({
uid: created.uid,
object: { index_url: 'https://puter.localhost/evil.html' },
}),
),
).rejects.toMatchObject({
statusCode: 400,
message: /system host/,
});
});
it('rejects a duplicate app name with 400', async () => {
const a = await makeUser();
const b = await makeUser();
@@ -897,3 +948,136 @@ describe('AppDriver.isNameAvailable additional branches', () => {
);
});
});
// ── alias-group custom domains (`app_origin_aliases`) ──────────────
//
// Custom hosts claimed by an alias group get the same bootstrap-stub
// merge treatment as puter-hosted subdomains: creating or repointing
// an app at an aliased host absorbs the unowned origin-bootstrap row
// instead of rejecting with `app_index_url_already_in_use`.
describe('AppDriver alias-group index_url merge', () => {
const aliasHostA = `alias-a-${Math.random().toString(36).slice(2, 10)}.test`;
const aliasHostB = `alias-b-${Math.random().toString(36).slice(2, 10)}.test`;
// `config` is protected on PuterDriver; reach in to toggle the alias
// groups for this block only. `#getOriginAliasGroups` reads config at
// call time, so runtime mutation takes effect immediately.
const driverConfig = () =>
(driver as unknown as { config: Record<string, unknown> }).config;
beforeAll(() => {
driverConfig().app_origin_aliases = [[aliasHostA], [aliasHostB]];
});
afterAll(() => {
delete driverConfig().app_origin_aliases;
});
const makeBootstrapStub = async (host: string) => {
const stubUid = `app-${uuidv4()}`;
// Mirrors AuthController's get-user-app-token bootstrap path:
// origin persisted as index_url, no owner, name === uid.
await server.stores.app.createFromOrigin(stubUid, `https://${host}`);
return stubUid;
};
it('create at an aliased host absorbs the unowned bootstrap stub', async () => {
const { actor, userId } = await makeUser();
const stubUid = await makeBootstrapStub(aliasHostA);
const name = uniqueName('alias-create');
const result = await withActor(actor, () =>
driver.create({
object: {
name,
title: 'Aliased',
index_url: `https://${aliasHostA}/`,
},
}),
);
// The stub row survives as the canonical app, claimed + merged.
expect(result.uid).toBe(stubUid);
expect(result.name).toBe(name);
const stored = await server.stores.app.getByUid(stubUid);
expect(stored?.owner_user_id).toBe(userId);
});
it('rejects another user registering an app under a reserved aliased host', async () => {
const other = await makeUser();
await expect(
withActor(other.actor, () =>
driver.create({
object: {
name: uniqueName('squatter'),
title: 't',
index_url: `https://${aliasHostA}/index.html`,
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('update repointing to an aliased host merges and aliases the old uid', async () => {
const { actor, userId } = await makeUser();
const stubUid = await makeBootstrapStub(aliasHostB);
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('alias-upd'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
const updated = await withActor(actor, () =>
driver.update({
uid: created.uid,
object: { index_url: `https://${aliasHostB}/` },
}),
);
// Merged into the stub; source row deleted; old uid still resolves
// via the canonical-uid alias.
expect(updated.uid).toBe(stubUid);
expect(await server.stores.app.getByUid(created.uid as string)).toBeNull();
const stored = await server.stores.app.getByUid(stubUid);
expect(stored?.owner_user_id).toBe(userId);
const viaOldUid = await withActor(actor, () =>
driver.read({ uid: created.uid }),
);
expect(viaOldUid.uid).toBe(stubUid);
});
it('leaves unrelated custom domains untouched (no alias group, no conflict check)', async () => {
const a = await makeUser();
const b = await makeUser();
const sharedUrl = uniqueIndexUrl();
// Non-puter, non-aliased hosts keep their historical behavior:
// no uniqueness enforcement, both creates succeed.
const first = await withActor(a.actor, () =>
driver.create({
object: {
name: uniqueName('plain-a'),
title: 't',
index_url: sharedUrl,
},
}),
);
const second = await withActor(b.actor, () =>
driver.create({
object: {
name: uniqueName('plain-b'),
title: 't',
index_url: sharedUrl,
},
}),
);
expect(first.uid).not.toBe(second.uid);
});
});
+2 -2
View File
@@ -20,6 +20,6 @@
// Microcents per underlying DynamoDB capacity unit, as reported by
// SystemKVStore.KVUsage. Cost is `KV_COSTS[op] * usage.<op>`.
export const KV_COSTS = {
'kv:read': 63,
'kv:write': 125,
'kv:read': 17,
'kv:write': 90,
} as const;
+1 -26
View File
@@ -175,9 +175,7 @@ export async function loadFileInput(
// `path`/`uid`/`uuid` (e.g. AI chat `puter_path` content parts) could
// exfiltrate any user's file. Must run before the S3 read below.
await fsService.checkFSAccess(entry, actor, 'read');
// S3 object key is recorded in entry.metadata.objectKey when written by
// fsv2; older rows fall back to the entry uuid.
const objectKey = deriveObjectKey(entry);
const objectKey = entry.uuid;
const { body, contentType, contentLength } =
await stores.s3Object.getObjectStream(
{
@@ -244,29 +242,6 @@ function filenameFromMime(mime: string): string {
return `input.${ext}`;
}
// Mirrors FSService's private deriveObjectKeyFromEntry helper. fsv2-era
// rows persist an `objectKey` in metadata; older rows simply use the uuid.
function deriveObjectKey(entry: {
uuid: string;
metadata: string | null;
}): string {
if (entry.metadata) {
try {
const parsed = JSON.parse(entry.metadata);
if (
parsed &&
typeof parsed.objectKey === 'string' &&
parsed.objectKey.length > 0
) {
return parsed.objectKey;
}
} catch {
// Not JSON — fall through.
}
}
return entry.uuid;
}
export function inferFilenameFromUrlOrPath(
value: string,
fallback = 'input',
+7 -1
View File
@@ -78,6 +78,12 @@ describe('extension.import("client") optional client access', () => {
return null;
}
};
expect(probe()).toBe(fake);
// The import proxy method-binds, so the result is a binding proxy over
// `fake` rather than the raw reference (identity is intentionally not
// preserved). What the probe pattern locks is that a registered client
// surfaces a callable method.
const result = probe();
expect(result).not.toBeNull();
expect(typeof (result as { query: unknown }).query).toBe('function');
});
});
Loaded 100 of 237 files, more files were not shown because too many files have changed in this diff. Show more