mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0405de30bd | ||
|
|
40c2799b06 | ||
|
|
4bcdeefe68 | ||
|
|
b6ba6dd90e | ||
|
|
954f8b7b8d | ||
|
|
24d700673c | ||
|
|
c6d64029e0 | ||
|
|
40ff866598 | ||
|
|
36888e35d3 | ||
|
|
30432b931a | ||
|
|
e2174a14cf | ||
|
|
d3cb89d8c9 | ||
|
|
c8625b3396 | ||
|
|
b576f01d12 | ||
|
|
a87cafee3c | ||
|
|
1c51fde163 | ||
|
|
b05699db77 | ||
|
|
163e48ab27 | ||
|
|
0b41d992a4 | ||
|
|
c2ddfd6867 | ||
|
|
81b9dc1f06 | ||
|
|
5506ddb2c2 | ||
|
|
c74469b01a | ||
|
|
b706693f82 | ||
|
|
cd800148d5 | ||
|
|
900b0d7aa6 | ||
|
|
7e21c1f888 | ||
|
|
00d6f9c230 | ||
|
|
4c03080efa | ||
|
|
432a11d289 | ||
|
|
e9e233404b | ||
|
|
8101d42604 | ||
|
|
88cedd43bc | ||
|
|
d3a2934d9b | ||
|
|
13a36b6199 | ||
|
|
a155464d46 |
No files matched your search
@@ -64,9 +64,9 @@ Keep comments light. Prefer self-documenting code — clear names, small functio
|
||||
- The _why_ is non-obvious (a hidden constraint, a workaround, a subtle invariant).
|
||||
- A non-trivial usage detail would otherwise trip the next reader.
|
||||
|
||||
Don't restate what the code already says. Don't write comments that reference the current task or PR — those rot.
|
||||
Don't restate what the code already says. Don't write comments that reference the current task or PR or version of something — those rot.
|
||||
|
||||
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the *why* in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
|
||||
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the _why_ in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
|
||||
|
||||
**Use plain ASCII `-` in comment section dividers, not box-drawing characters.** Write `// -- Section name --`, not `// ── Section name ──`. The `─` (U+2500) character renders inconsistently in some editors and diff tools and adds nothing over `-`.
|
||||
|
||||
|
||||
@@ -104,7 +104,7 @@ Puterは[**puter.com**](https://puter.com)でホストサービスとして利
|
||||
|
||||
メンテナーやコミュニティと以下のチャンネルを通じてつながりましょう:
|
||||
|
||||
- バグ報告や機能リクエストがありますか? [issueを開く](https://github.com/HeyPuter/puter/issues/new/choose) してください。
|
||||
- バグ報告や機能リクエストは [新規Issue](https://github.com/HeyPuter/puter/issues/new/choose) まで
|
||||
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
|
||||
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
|
||||
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
|
||||
|
||||
@@ -199,6 +199,32 @@ Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
|
||||
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.local", ... } }
|
||||
```
|
||||
|
||||
## Running behind your own reverse proxy
|
||||
|
||||
The bundled `puter-nginx` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Caddy, Traefik, HAProxy, a cloud load balancer, another nginx). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
|
||||
|
||||
You can keep the bundled nginx as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
|
||||
|
||||
**The rules that actually matter** (getting any of these wrong is what causes the redirect loops and "Invalid Host header" failures people hit):
|
||||
|
||||
1. **Don't rewrite the `Host` header.** Puter routes entirely on Host — `api.<domain>`, `site.<domain>`, `app.<domain>` are all distinguished by the incoming host. Forward the original host through unchanged. Do **not** rewrite external hostnames to an internal name like `puter.local`; that forces you to remap every subdomain by hand and still breaks signed-URL and CORS checks. Most proxies preserve Host by default — just don't override it.
|
||||
|
||||
2. **The external domain must equal `domain` in your `config.json`.** If users reach the box at `puter.example.com`, then `domain` must be `puter.example.com` and the hosting domains must be its real subdomains (`site.puter.example.com`, `app.puter.example.com`, …) — exactly as the installer/`config.json` lays them out. Puter rejects hosts it doesn't recognize with `Invalid Host header`.
|
||||
|
||||
3. **Set `protocol` to match the public scheme.** When your proxy terminates TLS, set `"protocol": "https"` (installer: `PUTER_PROTOCOL=https`). Puter builds its origins, redirects, signed S3 URLs, and OIDC callback URLs from this — leave it `http` behind an HTTPS proxy and you get redirect loops, broken logins, and mixed-content errors.
|
||||
|
||||
4. **Forward the standard proxy headers.** Puter needs:
|
||||
- `Host` — the original request host (rule 1).
|
||||
- `X-Forwarded-Proto` — the **external** scheme (`https`), so Puter knows TLS was terminated upstream.
|
||||
- `X-Forwarded-For` — the real client IP (rate limiting + audit logs).
|
||||
- `Upgrade` / `Connection` — passed through for WebSocket / socket.io upgrades, or the realtime connection silently fails.
|
||||
|
||||
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled nginx if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
|
||||
|
||||
6. **Route the wildcard to Puter.** Your proxy must forward `*.<domain>` (covering `api.*`, `app.*`, and `s3.<domain>`) and `*.site.<domain>` (and any other hosting domains) to Puter — same wildcard DNS as Step 2. Puter does the per-subdomain routing internally; the proxy just needs to hand it the traffic with the Host intact.
|
||||
|
||||
With those in place the rest of the stack is unchanged — `docker compose up -d` as below.
|
||||
|
||||
## Step 4 — Bring it up
|
||||
|
||||
```bash
|
||||
|
||||
+1
-1
@@ -264,7 +264,7 @@ services:
|
||||
# Persistent runtime data (anything your config points at /var/puter).
|
||||
- ./puter/data/puter:/var/puter:z
|
||||
healthcheck:
|
||||
test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
|
||||
test: wget --no-verbose --tries=1 --spider http://localhost:4100/ || exit 1
|
||||
interval: 30s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import type { Request, Response } from 'express';
|
||||
import { Context } from '@heyputer/backend/src/core';
|
||||
import { HttpError } from '@heyputer/backend/src/core/http';
|
||||
import {
|
||||
@@ -6,6 +5,7 @@ import {
|
||||
driversContainers,
|
||||
} from '@heyputer/backend/src/exports';
|
||||
import { extension } from '@heyputer/backend/src/extensions';
|
||||
import type { Request, Response } from 'express';
|
||||
|
||||
const services = extension.import('service');
|
||||
const clients = extension.import('client');
|
||||
|
||||
@@ -72,6 +72,9 @@ export const handleWhoami = async (
|
||||
unconfirmed_email: user.email,
|
||||
email_confirmed: user.email_confirmed || user.username === 'admin',
|
||||
requires_email_confirmation: user.requires_email_confirmation,
|
||||
phone: user.phone,
|
||||
requires_phone_verification: user.requires_phone_verification,
|
||||
requires_card_verification: user.requires_card_verification,
|
||||
desktop_bg_url: user.desktop_bg_url,
|
||||
desktop_bg_color: user.desktop_bg_color,
|
||||
desktop_bg_fit: user.desktop_bg_fit,
|
||||
|
||||
+22
-10
@@ -16,11 +16,20 @@
|
||||
# the compose file + brings the stack up. Set PUTER_FORCE=1 to overwrite.
|
||||
#
|
||||
# Tunable env vars:
|
||||
# PUTER_DIR install directory (default: ./puter-selfhosted)
|
||||
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
|
||||
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
|
||||
# PUTER_PORT HTTP port for nginx (default: 80)
|
||||
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
|
||||
# PUTER_DIR install directory (default: ./puter-selfhosted)
|
||||
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
|
||||
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
|
||||
# PUTER_PORT HTTP port for nginx (default: 80)
|
||||
# PUTER_PROTOCOL public scheme: http | https (default: http)
|
||||
# Set https when a TLS-terminating reverse proxy (Caddy,
|
||||
# Traefik, nginx, a cloud LB) sits in front — Puter then
|
||||
# builds https:// origins, S3 URLs, and redirects. See
|
||||
# "Running behind your own reverse proxy" in doc/self-hosting.md.
|
||||
# PUTER_TRUST_PROXY number of reverse-proxy hops in front (default: 1)
|
||||
# 1 = the bundled nginx (or a single external proxy);
|
||||
# 2 = two hops (e.g. Cloudflare → your proxy → Puter).
|
||||
# PUTER_ENV prod | dev (default: prod)
|
||||
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
|
||||
|
||||
set -eu
|
||||
|
||||
@@ -28,6 +37,9 @@ PUTER_DIR="${PUTER_DIR:-puter-selfhosted}"
|
||||
PUTER_URL="${PUTER_URL:-https://raw.githubusercontent.com/HeyPuter/puter/main}"
|
||||
PUTER_DOMAIN="${PUTER_DOMAIN:-puter.localhost}"
|
||||
PUTER_PORT="${PUTER_PORT:-80}"
|
||||
PUTER_PROTOCOL="${PUTER_PROTOCOL:-http}"
|
||||
PUTER_TRUST_PROXY="${PUTER_TRUST_PROXY:-1}"
|
||||
PUTER_ENV="${PUTER_ENV:-prod}"
|
||||
PUTER_FORCE="${PUTER_FORCE:-0}"
|
||||
|
||||
log() { printf '\033[1;36m[puter-install]\033[0m %s\n' "$*"; }
|
||||
@@ -117,9 +129,9 @@ EOF
|
||||
cat > puter/config/config.json <<EOF
|
||||
{
|
||||
"domain": "$PUTER_DOMAIN",
|
||||
"protocol": "http",
|
||||
"protocol": "$PUTER_PROTOCOL",
|
||||
"pub_port": $PUTER_PORT,
|
||||
"env": "prod",
|
||||
"env": "$PUTER_ENV",
|
||||
|
||||
"static_hosting_domain": "site.$PUTER_DOMAIN",
|
||||
"static_hosting_domain_alt": "host.$PUTER_DOMAIN",
|
||||
@@ -159,7 +171,7 @@ EOF
|
||||
"s3": {
|
||||
"s3Config": {
|
||||
"endpoint": "http://s3:9000",
|
||||
"publicEndpoint": "http://s3.$PUTER_DOMAIN",
|
||||
"publicEndpoint": "$PUTER_PROTOCOL://s3.$PUTER_DOMAIN",
|
||||
"accessKeyId": "puter",
|
||||
"secretAccessKey": "$S3_SECRET_KEY",
|
||||
"region": "us-east-1",
|
||||
@@ -173,7 +185,7 @@ EOF
|
||||
"ollama": { "enabled": false }
|
||||
},
|
||||
|
||||
"trust_proxy": 1
|
||||
"trust_proxy": $PUTER_TRUST_PROXY
|
||||
}
|
||||
EOF
|
||||
fi
|
||||
@@ -187,6 +199,6 @@ log "stack starting. first boot takes ~30s while MariaDB initialises."
|
||||
log "follow puter logs:"
|
||||
log " cd $PUTER_DIR && docker compose logs -f puter"
|
||||
log ""
|
||||
log "open http://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
|
||||
log "open $PUTER_PROTOCOL://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
|
||||
log "first-boot admin password is logged once — grab it with:"
|
||||
log " cd $PUTER_DIR && docker compose logs puter | grep password"
|
||||
Generated
+3893
-3247
File diff suppressed because it is too large.
Load diff
+12
-5
@@ -44,18 +44,18 @@
|
||||
"yaml": "^2.8.1"
|
||||
},
|
||||
"scripts": {
|
||||
"test:backend": "vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:backend:postgres": "PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:backend": "npm run setupExtensions && vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:backend:postgres": "npm run setupExtensions && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
|
||||
"start=gui": "nodemon --exec \"node dev-server.js\" ",
|
||||
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
|
||||
"prestart": "npm run build:ts",
|
||||
"prestart": "npm run setupExtensions && npm run build:ts",
|
||||
"dev": "npm start",
|
||||
"build": "npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
|
||||
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
|
||||
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
|
||||
"check-translations": "node tools/check-translations.js",
|
||||
"prepare": "husky",
|
||||
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
|
||||
"postinstall": "node ./tools/extensionSetup.mjs"
|
||||
"setupExtensions": "node ./tools/extensionSetup.mjs"
|
||||
},
|
||||
"workspaces": [
|
||||
"src/*",
|
||||
@@ -77,9 +77,16 @@
|
||||
"ai": "^6.0.73",
|
||||
"dedent": "^1.5.3",
|
||||
"javascript-time-ago": "^2.5.11",
|
||||
"libphonenumber-js": "1.13.6",
|
||||
"open": "^10.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=24.0.0"
|
||||
},
|
||||
"allowScripts": {
|
||||
"[email protected]": true,
|
||||
"[email protected]": true,
|
||||
"@google/genai": false,
|
||||
"[email protected]": true
|
||||
}
|
||||
}
|
||||
@@ -86,6 +86,8 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
|
||||
[50, ['0055_username_nocase_unique.sql']],
|
||||
[51, ['0056_sessions_kind_worker.sql']],
|
||||
[52, ['0057_add_user_reputation.sql']],
|
||||
[53, ['0058_add_phone_verification.sql']],
|
||||
[54, ['0059_add_card_verification.sql']],
|
||||
];
|
||||
|
||||
export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
|
||||
@@ -24,7 +24,9 @@
|
||||
--
|
||||
|
||||
--
|
||||
-- Idempotent column-ensure helper (used by CALLs below; dropped at end of file)
|
||||
-- Idempotent column-ensure helper. Used by the CALLs below AND by later
|
||||
-- migration files, which CALL it as a one-line idempotent column add. Left
|
||||
-- resident (not dropped at end of file) so it outlives this migration.
|
||||
--
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_col;
|
||||
@@ -1333,7 +1335,10 @@ CALL _puter_add_col('user_update_audit', 'new_username', '`new_username` varchar
|
||||
CALL _puter_add_col('user_update_audit', 'reason', '`reason` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
|
||||
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_col;
|
||||
-- _puter_add_col is intentionally NOT dropped here: later migration files CALL
|
||||
-- it as a one-line idempotent column add, and migrations replay on every boot
|
||||
-- with no applied-state tracking, so the helper must outlive this file. The
|
||||
-- DROP-before-CREATE at the top keeps this migration itself replay-safe.
|
||||
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
|
||||
|
||||
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
|
||||
-- E.164 number collected during verification (indexed like `email`);
|
||||
-- `requires_phone_verification` gates account use for low-reputation signups
|
||||
-- (not indexed, mirroring `requires_email_confirmation`).
|
||||
--
|
||||
-- Idempotent: column adds use _puter_add_col (defined in mig_1, which leaves it
|
||||
-- resident for later migrations); the index add is guarded against
|
||||
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
|
||||
|
||||
CALL _puter_add_col('user', 'phone', '`phone` varchar(20) DEFAULT NULL');
|
||||
CALL _puter_add_col('user', 'requires_phone_verification', '`requires_phone_verification` tinyint(1) NOT NULL DEFAULT ''0''');
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_user_phone_index()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'user'
|
||||
AND INDEX_NAME = 'idx_user_phone'
|
||||
) THEN
|
||||
ALTER TABLE `user` ADD INDEX `idx_user_phone` (`phone`);
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_user_phone_index();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
|
||||
@@ -0,0 +1,26 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Credit-card verification column. Mirrors SQLite migration 0059.
|
||||
-- `requires_card_verification` gates account use for low-reputation signups;
|
||||
-- the card itself never touches our DB, so this is the only column (not
|
||||
-- indexed, mirroring `requires_phone_verification`).
|
||||
--
|
||||
-- Idempotent: the column add uses _puter_add_col (from mig_1) so the
|
||||
-- directory replays safely.
|
||||
|
||||
CALL _puter_add_col('user', 'requires_card_verification', '`requires_card_verification` tinyint(1) NOT NULL DEFAULT ''0''');
|
||||
@@ -0,0 +1,47 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
|
||||
-- is the Stripe card fingerprint (stable per card number) recorded when a user
|
||||
-- clears card verification — the card sibling of `phone`, indexed like it so
|
||||
-- admin tooling can find the accounts that verified with a given card. The card
|
||||
-- itself never touches our DB, only Stripe's fingerprint for it.
|
||||
--
|
||||
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
|
||||
-- leaves it resident for later migrations); the index add is guarded against
|
||||
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
|
||||
|
||||
CALL _puter_add_col('user', 'card_fingerprint', '`card_fingerprint` varchar(128) DEFAULT NULL');
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_user_card_fingerprint_index()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'user'
|
||||
AND INDEX_NAME = 'idx_user_card_fingerprint'
|
||||
) THEN
|
||||
ALTER TABLE `user` ADD INDEX `idx_user_card_fingerprint` (`card_fingerprint`);
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_user_card_fingerprint_index();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
|
||||
@@ -0,0 +1,26 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
|
||||
-- E.164 number collected during verification (indexed like `email`);
|
||||
-- `requires_phone_verification` gates account use for low-reputation signups
|
||||
-- (not indexed, mirroring `requires_email_confirmation`).
|
||||
-- Idempotent via IF NOT EXISTS.
|
||||
|
||||
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS phone varchar(20);
|
||||
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_phone_verification boolean NOT NULL DEFAULT FALSE;
|
||||
CREATE INDEX IF NOT EXISTS idx_user_phone ON "user" (phone);
|
||||
@@ -0,0 +1,24 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Credit-card verification column. Mirrors SQLite migration 0059.
|
||||
-- `requires_card_verification` gates account use for low-reputation signups;
|
||||
-- the card itself never touches our DB, so this is the only column (not
|
||||
-- indexed, mirroring `requires_phone_verification`).
|
||||
-- Idempotent via IF NOT EXISTS.
|
||||
|
||||
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_card_verification boolean NOT NULL DEFAULT FALSE;
|
||||
@@ -0,0 +1,26 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
|
||||
-- is the Stripe card fingerprint (stable per card number) recorded when a user
|
||||
-- clears card verification — the card sibling of `phone`, indexed like it so
|
||||
-- admin tooling can find the accounts that verified with a given card. The card
|
||||
-- itself never touches our DB, only Stripe's fingerprint for it.
|
||||
-- Idempotent via IF NOT EXISTS.
|
||||
|
||||
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS card_fingerprint varchar(128);
|
||||
CREATE INDEX IF NOT EXISTS idx_user_card_fingerprint ON "user" (card_fingerprint);
|
||||
@@ -0,0 +1,25 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- SMS phone verification. `phone` holds the E.164 number collected during
|
||||
-- verification (indexed like `email`). `requires_phone_verification` gates
|
||||
-- account use until verified — the abuse v2 harness sets it for low-reputation
|
||||
-- signups instead of blocking them. (Not indexed, mirroring
|
||||
-- `requires_email_confirmation`.)
|
||||
ALTER TABLE `user` ADD COLUMN `phone` varchar(20) DEFAULT NULL;
|
||||
ALTER TABLE `user` ADD COLUMN `requires_phone_verification` tinyint(1) NOT NULL DEFAULT 0;
|
||||
CREATE INDEX IF NOT EXISTS idx_user_phone ON `user` (`phone`);
|
||||
@@ -0,0 +1,23 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Credit-card verification ($0 authorization handled by a payments
|
||||
-- extension). `requires_card_verification` gates account use until verified —
|
||||
-- the abuse v2 harness sets it for low-reputation signups instead of blocking
|
||||
-- them. The card itself never touches our DB, so this is the only column.
|
||||
-- (Not indexed, mirroring `requires_phone_verification`.)
|
||||
ALTER TABLE `user` ADD COLUMN `requires_card_verification` tinyint(1) NOT NULL DEFAULT 0;
|
||||
@@ -0,0 +1,25 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Card fingerprint column. `card_fingerprint` is the Stripe card fingerprint
|
||||
-- (stable per card number) recorded when a user clears card verification — the
|
||||
-- card sibling of `phone`. Indexed like `phone` so admin tooling can find the
|
||||
-- accounts that verified with a given card. The card itself never touches our
|
||||
-- DB, only Stripe's fingerprint for it. Server-only (not on the whoami
|
||||
-- allowlist), so it never reaches clients.
|
||||
ALTER TABLE `user` ADD COLUMN `card_fingerprint` varchar(128) DEFAULT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_user_card_fingerprint ON `user` (`card_fingerprint`);
|
||||
@@ -131,6 +131,19 @@ export type EventMap = {
|
||||
data?: unknown;
|
||||
abuse?: unknown;
|
||||
trail?: Array<string>;
|
||||
/**
|
||||
* Set by the abuse harness for flagged signups — the id under which the
|
||||
* decision trail is persisted to KV (`abuse:trail:<id>`), shared back on
|
||||
* the request for log / support correlation.
|
||||
*/
|
||||
trail_id?: string;
|
||||
/** Device signals forwarded verbatim from the signup request body. */
|
||||
fingerprint?: string | null;
|
||||
dfp_telemetry_id?: string | null;
|
||||
/** Set by the abuse harness — require SMS phone verification post-signup. */
|
||||
requires_phone_verification?: boolean;
|
||||
/** Set by the abuse harness — require card verification post-signup. */
|
||||
requires_card_verification?: boolean;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
'puter.signup.success': {
|
||||
@@ -139,6 +152,9 @@ export type EventMap = {
|
||||
email: string;
|
||||
username: string;
|
||||
ip?: string | null;
|
||||
fingerprint?: string | null;
|
||||
/** True when the created account is a temp user (no email/password). */
|
||||
is_temp?: boolean;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
'email.validate': {
|
||||
@@ -158,6 +174,78 @@ export type EventMap = {
|
||||
user_uid: string;
|
||||
email: string;
|
||||
};
|
||||
// Phone-reuse cap is pure mechanism here — the abuse extension answers
|
||||
// (emitted via `emitAndWait`) by counting how many OTHER accounts have
|
||||
// already verified this number and flipping `allowed` to false when the
|
||||
// cross-account limit is hit. No extension listening → `allowed` stays
|
||||
// true and the send proceeds.
|
||||
'puter.phone-verification.check': {
|
||||
user_id: number;
|
||||
user_uid: string;
|
||||
phone: string;
|
||||
// Client-supplied device fingerprint for this request, or null. Lets
|
||||
// the abuse extension cap sends per device (across accounts) so a
|
||||
// device farm can't dodge the per-account cap with fresh signups.
|
||||
device_fingerprint: string | null;
|
||||
allowed: boolean;
|
||||
reason: string | null;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
// Fire-and-forget signal that a code was actually sent — the abuse
|
||||
// extension bumps its per-number / per-account / per-device send-velocity
|
||||
// counters off this. No-op with no extension listening.
|
||||
'puter.phone-verification.sent': {
|
||||
user_id: number;
|
||||
user_uid: string;
|
||||
phone: string;
|
||||
device_fingerprint: string | null;
|
||||
};
|
||||
'user.phone-verified': {
|
||||
user_id: number;
|
||||
user_uid: string;
|
||||
phone: string;
|
||||
};
|
||||
// Card verification is pure mechanism here — a payments extension fills
|
||||
// these in (emitted via `emitAndWait`). `enabled` stays null when no
|
||||
// extension is installed; the extension always sets it (true/false) so
|
||||
// the endpoints can distinguish "disabled" from "not installed".
|
||||
'puter.card-verification.setup': {
|
||||
user_id: number;
|
||||
user_uid: string;
|
||||
ip?: string | null;
|
||||
// Client-supplied device fingerprint for this request, or null. Lets
|
||||
// the abuse extension cap card-verification setups per device (across
|
||||
// accounts) before any Stripe SetupIntent is created.
|
||||
device_fingerprint: string | null;
|
||||
enabled: boolean | null;
|
||||
// Set false by the extension to refuse this setup (e.g. the per-device
|
||||
// setup-velocity cap); `reason` carries the opaque code. Stays true
|
||||
// when allowed or with no extension listening.
|
||||
allowed: boolean;
|
||||
reason: string | null;
|
||||
client_secret: string | null;
|
||||
publishable_key: string | null;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
'puter.card-verification.confirm': {
|
||||
user_id: number;
|
||||
user_uid: string;
|
||||
setup_intent_id: string;
|
||||
enabled: boolean | null;
|
||||
verified: boolean;
|
||||
reason: string | null;
|
||||
fingerprint: string | null;
|
||||
funding: string | null;
|
||||
country: string | null;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
'user.card-verified': {
|
||||
user_id: number;
|
||||
user_uid: string;
|
||||
fingerprint: string | null;
|
||||
funding: string | null;
|
||||
country: string | null;
|
||||
};
|
||||
'user.username-changed': {
|
||||
user_id: number;
|
||||
old_username: string;
|
||||
|
||||
@@ -24,6 +24,7 @@ import { EventClient } from './event/EventClient';
|
||||
import { DDBClient } from './dynamodb/DDBClient';
|
||||
import { RedisClient } from './redis/RedisClient';
|
||||
import { S3Client } from './s3/S3Client';
|
||||
import { PreludeClient } from './prelude/PreludeClient';
|
||||
import type { IPuterClientRegistry } from './types';
|
||||
|
||||
export const puterClients = {
|
||||
@@ -34,4 +35,5 @@ export const puterClients = {
|
||||
dynamo: DDBClient,
|
||||
redis: RedisClient,
|
||||
s3: S3Client,
|
||||
prelude: PreludeClient,
|
||||
} satisfies IPuterClientRegistry;
|
||||
@@ -0,0 +1,173 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { PreludeClient } from './PreludeClient';
|
||||
import type { IConfig } from '../../types';
|
||||
|
||||
const makeClient = (apiKey?: string) =>
|
||||
new PreludeClient({
|
||||
prelude: apiKey ? { apiKey } : undefined,
|
||||
} as unknown as IConfig);
|
||||
|
||||
const okJson = (body: unknown) =>
|
||||
({
|
||||
ok: true,
|
||||
status: 200,
|
||||
json: async () => body,
|
||||
}) as unknown as Response;
|
||||
|
||||
describe('PreludeClient', () => {
|
||||
let fetchMock: ReturnType<typeof vi.fn>;
|
||||
|
||||
beforeEach(() => {
|
||||
fetchMock = vi.fn();
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('isConfigured reflects whether an apiKey is set', () => {
|
||||
expect(makeClient('sk_test').isConfigured()).toBe(true);
|
||||
expect(makeClient().isConfigured()).toBe(false);
|
||||
});
|
||||
|
||||
describe('isCountrySupported (€0.07 cap)', () => {
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
it('allows revenue markets up to the cap (incl. the priciest)', () => {
|
||||
expect(client.isCountrySupported('US')).toBe(true); // €0.0043
|
||||
expect(client.isCountrySupported('DE')).toBe(true); // €0.0598
|
||||
expect(client.isCountrySupported('SA')).toBe(true); // €0.0638
|
||||
expect(client.isCountrySupported('us')).toBe(true); // case-insensitive
|
||||
});
|
||||
|
||||
it('rejects countries above the cap, with no SMS, or unknown', () => {
|
||||
expect(client.isCountrySupported('PK')).toBe(false); // €0.3548
|
||||
expect(client.isCountrySupported('ID')).toBe(false); // €0.2430
|
||||
expect(client.isCountrySupported('LI')).toBe(false); // null (no SMS)
|
||||
expect(client.isCountrySupported('ZZ')).toBe(false); // unknown
|
||||
expect(client.isCountrySupported(undefined)).toBe(false);
|
||||
});
|
||||
|
||||
it('honors a configured maxSmsCostEur override', () => {
|
||||
const strict = new PreludeClient({
|
||||
prelude: { apiKey: 'sk', maxSmsCostEur: 0.01 },
|
||||
} as unknown as IConfig);
|
||||
expect(strict.isCountrySupported('US')).toBe(true); // €0.0043
|
||||
expect(strict.isCountrySupported('DE')).toBe(false); // €0.0598 > 0.01
|
||||
|
||||
const loose = new PreludeClient({
|
||||
prelude: { apiKey: 'sk', maxSmsCostEur: 0.5 },
|
||||
} as unknown as IConfig);
|
||||
expect(loose.isCountrySupported('PK')).toBe(true); // €0.3548 <= 0.5
|
||||
});
|
||||
});
|
||||
|
||||
it('createVerification POSTs the phone target + ip signal with bearer auth', async () => {
|
||||
fetchMock.mockResolvedValue(
|
||||
okJson({ id: 'vrf_1', status: 'success' }),
|
||||
);
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
const res = await client.createVerification('+14155550123', {
|
||||
ip: '203.0.113.7',
|
||||
});
|
||||
|
||||
expect(res).toEqual({ id: 'vrf_1', status: 'success' });
|
||||
const [url, init] = fetchMock.mock.calls[0];
|
||||
expect(url).toBe('https://api.prelude.dev/v2/verification');
|
||||
expect(init.method).toBe('POST');
|
||||
expect(init.headers.Authorization).toBe('Bearer sk_test');
|
||||
expect(JSON.parse(init.body)).toEqual({
|
||||
target: { type: 'phone_number', value: '+14155550123' },
|
||||
// Defaults to RCS (cheaper); Prelude falls back to SMS.
|
||||
options: { code_size: 6, preferred_channel: 'rcs' },
|
||||
signals: { ip: '203.0.113.7' },
|
||||
});
|
||||
});
|
||||
|
||||
it('forwards device_id and user_agent signals when supplied', async () => {
|
||||
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
await client.createVerification('+14155550123', {
|
||||
ip: '203.0.113.7',
|
||||
device_id: 'thumb_abc123',
|
||||
user_agent: 'Mozilla/5.0',
|
||||
});
|
||||
|
||||
const [, init] = fetchMock.mock.calls[0];
|
||||
expect(JSON.parse(init.body).signals).toEqual({
|
||||
ip: '203.0.113.7',
|
||||
device_id: 'thumb_abc123',
|
||||
user_agent: 'Mozilla/5.0',
|
||||
});
|
||||
});
|
||||
|
||||
it('omits the signals object entirely when none are supplied', async () => {
|
||||
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
await client.createVerification('+14155550123');
|
||||
|
||||
expect(JSON.parse(fetchMock.mock.calls[0][1].body)).not.toHaveProperty(
|
||||
'signals',
|
||||
);
|
||||
});
|
||||
|
||||
it('includes a configured template_id + sender_id + preferred channel', async () => {
|
||||
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
|
||||
const client = new PreludeClient({
|
||||
prelude: {
|
||||
apiKey: 'sk_test',
|
||||
templateId: 'tmpl_puter',
|
||||
senderId: 'Puter',
|
||||
preferredChannel: 'sms',
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
|
||||
await client.createVerification('+14155550123');
|
||||
|
||||
const [, init] = fetchMock.mock.calls[0];
|
||||
expect(JSON.parse(init.body).options).toEqual({
|
||||
code_size: 6,
|
||||
preferred_channel: 'sms', // config override beats the rcs default
|
||||
template_id: 'tmpl_puter',
|
||||
sender_id: 'Puter',
|
||||
});
|
||||
});
|
||||
|
||||
it('checkVerification POSTs target + code and returns the status', async () => {
|
||||
fetchMock.mockResolvedValue(okJson({ status: 'success' }));
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
const res = await client.checkVerification('+14155550123', '123456');
|
||||
|
||||
expect(res).toEqual({ status: 'success' });
|
||||
const [url, init] = fetchMock.mock.calls[0];
|
||||
expect(url).toBe('https://api.prelude.dev/v2/verification/check');
|
||||
expect(JSON.parse(init.body)).toEqual({
|
||||
target: { type: 'phone_number', value: '+14155550123' },
|
||||
code: '123456',
|
||||
});
|
||||
});
|
||||
|
||||
it('throws (does not call fetch) when not configured', async () => {
|
||||
const client = makeClient();
|
||||
await expect(
|
||||
client.createVerification('+14155550123'),
|
||||
).rejects.toThrow(/not configured/i);
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws on a non-2xx Prelude response', async () => {
|
||||
fetchMock.mockResolvedValue({
|
||||
ok: false,
|
||||
status: 400,
|
||||
json: async () => ({ message: 'bad target' }),
|
||||
} as unknown as Response);
|
||||
|
||||
await expect(
|
||||
makeClient('sk_test').checkVerification('+1', 'x'),
|
||||
).rejects.toThrow(/Prelude .* failed: 400/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,222 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { IConfig } from '../../types';
|
||||
import { PuterClient } from '../types';
|
||||
import { COUNTRY_SMS_PRICES } from './countries.js';
|
||||
|
||||
const PRELUDE_API_BASE = 'https://api.prelude.dev/v2';
|
||||
const REQUEST_TIMEOUT_MS = 8000;
|
||||
/** OTP length — matches the 6-box code UI. Prelude allows 4–8. */
|
||||
const PRELUDE_CODE_SIZE = 6;
|
||||
/**
|
||||
* Channel Prelude prioritizes for delivery. RCS is far cheaper than SMS, so we
|
||||
* prefer it by default; Prelude routes to the next reachable channel (SMS) when
|
||||
* RCS isn't available on the destination. (Requires an RCS agent provisioned in
|
||||
* the Prelude account to actually use RCS — otherwise it just falls through to
|
||||
* SMS.) Override with `config.prelude.preferredChannel`.
|
||||
*/
|
||||
const DEFAULT_PREFERRED_CHANNEL: PreludeChannel = 'rcs';
|
||||
|
||||
/** Delivery channels Prelude can prioritize via `options.preferred_channel`. */
|
||||
export type PreludeChannel =
|
||||
| 'sms'
|
||||
| 'rcs'
|
||||
| 'whatsapp'
|
||||
| 'viber'
|
||||
| 'zalo'
|
||||
| 'telegram';
|
||||
/**
|
||||
* Default per-SMS cost ceiling (EUR). Countries whose Prelude SMS rate exceeds
|
||||
* this — or that have no SMS channel — are not offered phone verification. The
|
||||
* cap covers every realistic revenue market (priciest are Germany €0.0598 and
|
||||
* Saudi Arabia €0.0638) while excluding the expensive, high-fraud long tail.
|
||||
* Override per-deployment with `config.prelude.maxSmsCostEur`.
|
||||
*/
|
||||
const DEFAULT_MAX_SMS_COST_EUR = 0.07;
|
||||
|
||||
/** Status returned by Prelude when creating/retrying a verification. */
|
||||
export type PreludeCreateStatus =
|
||||
| 'success'
|
||||
| 'retry'
|
||||
| 'challenged'
|
||||
| 'blocked'
|
||||
| 'shadow_blocked';
|
||||
|
||||
/** Status returned by Prelude when checking a code. */
|
||||
export type PreludeCheckStatus =
|
||||
| 'success'
|
||||
| 'failure'
|
||||
| 'expired_or_not_found'
|
||||
| 'transaction_missing'
|
||||
| 'transaction_mismatch';
|
||||
|
||||
/**
|
||||
* Prelude Verify v2 client (https://docs.prelude.so/verify/v2). Sends and checks
|
||||
* SMS one-time codes — Prelude generates, delivers, and validates the code, so
|
||||
* we never store one ourselves. No-ops with a warning when no API key is
|
||||
* configured (so dev environments don't crash); `isConfigured()` lets callers
|
||||
* surface a clean "phone verification unavailable" error instead.
|
||||
*/
|
||||
export class PreludeClient extends PuterClient {
|
||||
private apiKey: string | null = null;
|
||||
|
||||
constructor(config: IConfig) {
|
||||
super(config);
|
||||
this.apiKey = config.prelude?.apiKey ?? null;
|
||||
}
|
||||
|
||||
override onServerStart(): void {
|
||||
if (!this.apiKey) {
|
||||
console.warn(
|
||||
'[prelude] no apiKey configured — SMS phone verification is disabled',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** True when an API key is configured and verification can be attempted. */
|
||||
isConfigured(): boolean {
|
||||
return !!this.apiKey;
|
||||
}
|
||||
|
||||
/** ISO region used to parse local-format numbers (config-driven). */
|
||||
get defaultCountry(): string | undefined {
|
||||
return this.config.prelude?.defaultCountry;
|
||||
}
|
||||
|
||||
/** Per-SMS cost ceiling in EUR (config override or the default cap). */
|
||||
get maxSmsCostEur(): number {
|
||||
return this.config.prelude?.maxSmsCostEur ?? DEFAULT_MAX_SMS_COST_EUR;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether SMS verification should be offered for a country. False when the
|
||||
* country is unknown, has no SMS channel, or its rate exceeds the cost cap.
|
||||
* @param iso ISO-3166 alpha-2 (e.g. 'US') — from the parsed phone number.
|
||||
*/
|
||||
isCountrySupported(iso: string | undefined): boolean {
|
||||
if (!iso) return false; // couldn't determine country → can't price it
|
||||
const price = COUNTRY_SMS_PRICES[iso.toUpperCase()];
|
||||
if (!price || price.sms == null) return false;
|
||||
return price.sms <= this.maxSmsCostEur;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create (or retry) a verification: Prelude sends an OTP to `target`.
|
||||
* @param target E.164 phone number, e.g. "+14155550123".
|
||||
* @param signals Optional anti-fraud signals. The more we forward, the
|
||||
* better Prelude's fraud model scores the attempt (per their docs IP +
|
||||
* device + platform are the highest-value). We pass what the backend
|
||||
* already has: the request `ip`, the client `device_id` (ThumbmarkJS
|
||||
* hash), and the `user_agent` (Prelude infers platform/model/OS from it).
|
||||
* Richer client-collected signals require Prelude's frontend SDK, which
|
||||
* produces a `dispatch_id` the backend would forward here.
|
||||
*/
|
||||
async createVerification(
|
||||
target: string,
|
||||
signals: {
|
||||
ip?: string;
|
||||
device_id?: string;
|
||||
user_agent?: string;
|
||||
} = {},
|
||||
): Promise<{ id?: string; status: PreludeCreateStatus }> {
|
||||
// Match the 6-box code UI (UIWindowPhoneVerificationRequired). Without
|
||||
// code_size Prelude uses the dashboard default (4). preferred_channel
|
||||
// prioritizes RCS (cheaper); Prelude falls back to SMS when unavailable.
|
||||
const options: Record<string, unknown> = {
|
||||
code_size: PRELUDE_CODE_SIZE,
|
||||
preferred_channel:
|
||||
this.config.prelude?.preferredChannel ??
|
||||
DEFAULT_PREFERRED_CHANNEL,
|
||||
};
|
||||
// Branding lives in the Prelude dashboard (the message text is a
|
||||
// template); these just select a Puter-branded template / sender when
|
||||
// configured. See IPreludeConfig.
|
||||
const { templateId, senderId } = this.config.prelude ?? {};
|
||||
if (templateId) options.template_id = templateId;
|
||||
if (senderId) options.sender_id = senderId;
|
||||
|
||||
const body: Record<string, unknown> = {
|
||||
target: { type: 'phone_number', value: target },
|
||||
options,
|
||||
};
|
||||
// Only attach signals we actually have — Prelude treats the object as
|
||||
// optional and an empty one adds nothing.
|
||||
const sig: Record<string, string> = {};
|
||||
if (signals.ip) sig.ip = signals.ip;
|
||||
if (signals.device_id) sig.device_id = signals.device_id;
|
||||
if (signals.user_agent) sig.user_agent = signals.user_agent;
|
||||
if (Object.keys(sig).length > 0) body.signals = sig;
|
||||
return this.#post('/verification', body) as Promise<{
|
||||
id?: string;
|
||||
status: PreludeCreateStatus;
|
||||
}>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check a code the user entered against the active verification for `target`.
|
||||
* @returns `{ status }` — `'success'` means verified.
|
||||
*/
|
||||
async checkVerification(
|
||||
target: string,
|
||||
code: string,
|
||||
): Promise<{ status: PreludeCheckStatus }> {
|
||||
return this.#post('/verification/check', {
|
||||
target: { type: 'phone_number', value: target },
|
||||
code,
|
||||
}) as Promise<{ status: PreludeCheckStatus }>;
|
||||
}
|
||||
|
||||
async #post(
|
||||
path: string,
|
||||
body: Record<string, unknown>,
|
||||
): Promise<Record<string, unknown>> {
|
||||
if (!this.apiKey) {
|
||||
throw new Error('Prelude is not configured');
|
||||
}
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
try {
|
||||
const res = await fetch(`${PRELUDE_API_BASE}${path}`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Authorization: `Bearer ${this.apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json',
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
signal: controller.signal,
|
||||
});
|
||||
const json = (await res.json().catch(() => ({}))) as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
if (!res.ok) {
|
||||
throw new Error(
|
||||
`Prelude ${path} failed: ${res.status} ${
|
||||
(json as { message?: string })?.message ?? ''
|
||||
}`.trim(),
|
||||
);
|
||||
}
|
||||
return json;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,952 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/** Per-country Prelude SMS pricing, hardcoded from the Prelude global price
|
||||
* sheet. `sms` is the average cost in EUR, or null when SMS is unavailable
|
||||
* there. Used to cap phone verification to affordable countries (see
|
||||
* PreludeClient.isCountrySupported). Regenerate from the price sheet when
|
||||
* Prelude updates rates. */
|
||||
export interface CountryPrice {
|
||||
name: string;
|
||||
/** Prelude SMS average in EUR, or null when SMS is not available. */
|
||||
sms: number | null;
|
||||
}
|
||||
|
||||
export const COUNTRY_SMS_PRICES: Record<string, CountryPrice> = {
|
||||
AD: {
|
||||
name: 'Andorra',
|
||||
sms: 0.0515,
|
||||
},
|
||||
AE: {
|
||||
name: 'United Arab Emirates',
|
||||
sms: 0.0145,
|
||||
},
|
||||
AF: {
|
||||
name: 'Afghanistan',
|
||||
sms: 0.2409,
|
||||
},
|
||||
AG: {
|
||||
name: 'Antigua and Barbuda',
|
||||
sms: 0.0755,
|
||||
},
|
||||
AI: {
|
||||
name: 'Anguilla',
|
||||
sms: 0.0765,
|
||||
},
|
||||
AL: {
|
||||
name: 'Albania',
|
||||
sms: 0.0345,
|
||||
},
|
||||
AM: {
|
||||
name: 'Armenia',
|
||||
sms: 0.118,
|
||||
},
|
||||
AN: {
|
||||
name: 'Netherlands Antilles',
|
||||
sms: null,
|
||||
},
|
||||
AO: {
|
||||
name: 'Angola',
|
||||
sms: 0.0347,
|
||||
},
|
||||
AR: {
|
||||
name: 'Argentina',
|
||||
sms: 0.0614,
|
||||
},
|
||||
AS: {
|
||||
name: 'American Samoa',
|
||||
sms: 0.077,
|
||||
},
|
||||
AT: {
|
||||
name: 'Austria',
|
||||
sms: 0.014,
|
||||
},
|
||||
AU: {
|
||||
name: 'Australia',
|
||||
sms: 0.0091,
|
||||
},
|
||||
AW: {
|
||||
name: 'Aruba',
|
||||
sms: 0.0617,
|
||||
},
|
||||
AZ: {
|
||||
name: 'Azerbaijan',
|
||||
sms: 0.0969,
|
||||
},
|
||||
BA: {
|
||||
name: 'Bosnia and Herzegovina',
|
||||
sms: 0.0801,
|
||||
},
|
||||
BB: {
|
||||
name: 'Barbados',
|
||||
sms: 0.2021,
|
||||
},
|
||||
BD: {
|
||||
name: 'Bangladesh',
|
||||
sms: 0.1844,
|
||||
},
|
||||
BE: {
|
||||
name: 'Belgium',
|
||||
sms: 0.0297,
|
||||
},
|
||||
BF: {
|
||||
name: 'Burkina Faso',
|
||||
sms: 0.063,
|
||||
},
|
||||
BG: {
|
||||
name: 'Bulgaria',
|
||||
sms: 0.0799,
|
||||
},
|
||||
BH: {
|
||||
name: 'Bahrain',
|
||||
sms: 0.014,
|
||||
},
|
||||
BI: {
|
||||
name: 'Burundi',
|
||||
sms: 0.217,
|
||||
},
|
||||
BJ: {
|
||||
name: 'Benin',
|
||||
sms: 0.13,
|
||||
},
|
||||
BM: {
|
||||
name: 'Bermuda',
|
||||
sms: 0.21,
|
||||
},
|
||||
BN: {
|
||||
name: 'Brunei',
|
||||
sms: 0.035,
|
||||
},
|
||||
BO: {
|
||||
name: 'Bolivia',
|
||||
sms: 0.0708,
|
||||
},
|
||||
BQ: {
|
||||
name: 'Caribbean Netherlands',
|
||||
sms: 0.0565,
|
||||
},
|
||||
BR: {
|
||||
name: 'Brazil',
|
||||
sms: 0.0086,
|
||||
},
|
||||
BS: {
|
||||
name: 'Bahamas',
|
||||
sms: 0.0672,
|
||||
},
|
||||
BT: {
|
||||
name: 'Bhutan',
|
||||
sms: null,
|
||||
},
|
||||
BW: {
|
||||
name: 'Botswana',
|
||||
sms: 0.0235,
|
||||
},
|
||||
BY: {
|
||||
name: 'Belarus',
|
||||
sms: 0.118,
|
||||
},
|
||||
BZ: {
|
||||
name: 'Belize',
|
||||
sms: 0.1529,
|
||||
},
|
||||
CA: {
|
||||
name: 'Canada',
|
||||
sms: 0.0052,
|
||||
},
|
||||
CD: {
|
||||
name: 'Congo RDC',
|
||||
sms: 0.1112,
|
||||
},
|
||||
CF: {
|
||||
name: 'Central African Republic',
|
||||
sms: 0.2,
|
||||
},
|
||||
CG: {
|
||||
name: 'Congo',
|
||||
sms: 0.2507,
|
||||
},
|
||||
CH: {
|
||||
name: 'Switzerland',
|
||||
sms: 0.0163,
|
||||
},
|
||||
CI: {
|
||||
name: "Cote d'Ivoire",
|
||||
sms: 0.19,
|
||||
},
|
||||
CK: {
|
||||
name: 'Cook Islands',
|
||||
sms: 0.1041,
|
||||
},
|
||||
CL: {
|
||||
name: 'Chile',
|
||||
sms: 0.0027,
|
||||
},
|
||||
CM: {
|
||||
name: 'Cameroon',
|
||||
sms: 0.1504,
|
||||
},
|
||||
CN: {
|
||||
name: 'China',
|
||||
sms: 0.0053,
|
||||
},
|
||||
CO: {
|
||||
name: 'Colombia',
|
||||
sms: 0.0008,
|
||||
},
|
||||
CR: {
|
||||
name: 'Costa Rica',
|
||||
sms: 0.0045,
|
||||
},
|
||||
CU: {
|
||||
name: 'Cuba',
|
||||
sms: 0.0512,
|
||||
},
|
||||
CV: {
|
||||
name: 'Cabo Verde',
|
||||
sms: 0.2031,
|
||||
},
|
||||
CW: {
|
||||
name: 'Curacao',
|
||||
sms: 0.0138,
|
||||
},
|
||||
CY: {
|
||||
name: 'Northern Cyprus',
|
||||
sms: 0.0065,
|
||||
},
|
||||
CZ: {
|
||||
name: 'Czech Republic',
|
||||
sms: 0.0299,
|
||||
},
|
||||
DE: {
|
||||
name: 'Germany',
|
||||
sms: 0.0598,
|
||||
},
|
||||
DJ: {
|
||||
name: 'Djibouti',
|
||||
sms: 0.0897,
|
||||
},
|
||||
DK: {
|
||||
name: 'Denmark',
|
||||
sms: 0.0301,
|
||||
},
|
||||
DM: {
|
||||
name: 'Dominica',
|
||||
sms: 0.0824,
|
||||
},
|
||||
DO: {
|
||||
name: 'Dominican Republic',
|
||||
sms: 0.0351,
|
||||
},
|
||||
DZ: {
|
||||
name: 'Algeria',
|
||||
sms: 0.196,
|
||||
},
|
||||
EC: {
|
||||
name: 'Ecuador',
|
||||
sms: 0.1041,
|
||||
},
|
||||
EE: {
|
||||
name: 'Estonia',
|
||||
sms: 0.0234,
|
||||
},
|
||||
EG: {
|
||||
name: 'Egypt',
|
||||
sms: 0.1561,
|
||||
},
|
||||
ER: {
|
||||
name: 'Eritrea',
|
||||
sms: 0.0659,
|
||||
},
|
||||
ES: {
|
||||
name: 'Spain',
|
||||
sms: 0.0193,
|
||||
},
|
||||
ET: {
|
||||
name: 'Ethiopia',
|
||||
sms: 0.2741,
|
||||
},
|
||||
FI: {
|
||||
name: 'Finland',
|
||||
sms: 0.043,
|
||||
},
|
||||
FJ: {
|
||||
name: 'Fiji',
|
||||
sms: 0.069,
|
||||
},
|
||||
FK: {
|
||||
name: 'Falkland Islands',
|
||||
sms: 0.0713,
|
||||
},
|
||||
FM: {
|
||||
name: 'Micronesia',
|
||||
sms: 0.0118,
|
||||
},
|
||||
FO: {
|
||||
name: 'Faroe Islands',
|
||||
sms: 0.0319,
|
||||
},
|
||||
FR: {
|
||||
name: 'France',
|
||||
sms: 0.03,
|
||||
},
|
||||
GA: {
|
||||
name: 'Gabon',
|
||||
sms: 0.15,
|
||||
},
|
||||
GB: {
|
||||
name: 'United Kingdom',
|
||||
sms: 0.026,
|
||||
},
|
||||
GD: {
|
||||
name: 'Grenada',
|
||||
sms: null,
|
||||
},
|
||||
GE: {
|
||||
name: 'Georgia',
|
||||
sms: 0.0788,
|
||||
},
|
||||
GF: {
|
||||
name: 'French Guiana',
|
||||
sms: 0.05,
|
||||
},
|
||||
GG: {
|
||||
name: 'Guernsey',
|
||||
sms: 0.025,
|
||||
},
|
||||
GH: {
|
||||
name: 'Ghana',
|
||||
sms: 0.15,
|
||||
},
|
||||
GI: {
|
||||
name: 'Gibraltar',
|
||||
sms: 0.0134,
|
||||
},
|
||||
GL: {
|
||||
name: 'Greenland',
|
||||
sms: 0.0048,
|
||||
},
|
||||
GM: {
|
||||
name: 'Gambia',
|
||||
sms: 0.1283,
|
||||
},
|
||||
GN: {
|
||||
name: 'Guinea',
|
||||
sms: 0.2,
|
||||
},
|
||||
GP: {
|
||||
name: 'Guadeloupe',
|
||||
sms: 0.0455,
|
||||
},
|
||||
GQ: {
|
||||
name: 'Equatorial Guinea',
|
||||
sms: 0.099,
|
||||
},
|
||||
GR: {
|
||||
name: 'Greece',
|
||||
sms: 0.0296,
|
||||
},
|
||||
GT: {
|
||||
name: 'Guatemala',
|
||||
sms: 0.1202,
|
||||
},
|
||||
GU: {
|
||||
name: 'Guam',
|
||||
sms: 0.02,
|
||||
},
|
||||
GW: {
|
||||
name: 'Guinea-Bissau',
|
||||
sms: 0.1566,
|
||||
},
|
||||
GY: {
|
||||
name: 'Guyana',
|
||||
sms: 0.2178,
|
||||
},
|
||||
HK: {
|
||||
name: 'Hong Kong',
|
||||
sms: 0.038,
|
||||
},
|
||||
HN: {
|
||||
name: 'Honduras',
|
||||
sms: 0.162,
|
||||
},
|
||||
HR: {
|
||||
name: 'Croatia',
|
||||
sms: 0.03,
|
||||
},
|
||||
HT: {
|
||||
name: 'Haiti',
|
||||
sms: 0.09,
|
||||
},
|
||||
HU: {
|
||||
name: 'Hungary',
|
||||
sms: 0.029,
|
||||
},
|
||||
ID: {
|
||||
name: 'Indonesia',
|
||||
sms: 0.243,
|
||||
},
|
||||
IE: {
|
||||
name: 'Ireland',
|
||||
sms: 0.0305,
|
||||
},
|
||||
IL: {
|
||||
name: 'Israel',
|
||||
sms: 0.01,
|
||||
},
|
||||
IM: {
|
||||
name: 'Isle of Man',
|
||||
sms: 0.0385,
|
||||
},
|
||||
IN: {
|
||||
name: 'India',
|
||||
sms: 0.0375,
|
||||
},
|
||||
IQ: {
|
||||
name: 'Iraq',
|
||||
sms: 0.151,
|
||||
},
|
||||
IR: {
|
||||
name: 'Iran',
|
||||
sms: 0.14,
|
||||
},
|
||||
IS: {
|
||||
name: 'Iceland',
|
||||
sms: 0.0493,
|
||||
},
|
||||
IT: {
|
||||
name: 'Italy',
|
||||
sms: 0.0245,
|
||||
},
|
||||
JE: {
|
||||
name: 'Jersey',
|
||||
sms: 0.025,
|
||||
},
|
||||
JM: {
|
||||
name: 'Jamaica',
|
||||
sms: 0.1525,
|
||||
},
|
||||
JO: {
|
||||
name: 'Jordan',
|
||||
sms: 0.2094,
|
||||
},
|
||||
JP: {
|
||||
name: 'Japan',
|
||||
sms: 0.017,
|
||||
},
|
||||
KE: {
|
||||
name: 'Kenya',
|
||||
sms: 0.125,
|
||||
},
|
||||
KG: {
|
||||
name: 'Kyrgyzstan',
|
||||
sms: 0.15,
|
||||
},
|
||||
KH: {
|
||||
name: 'Cambodia',
|
||||
sms: 0.1529,
|
||||
},
|
||||
KI: {
|
||||
name: 'Kiribati',
|
||||
sms: 0.025,
|
||||
},
|
||||
KM: {
|
||||
name: 'Comoros',
|
||||
sms: 0.15,
|
||||
},
|
||||
KN: {
|
||||
name: 'Saint Kitts and Nevis',
|
||||
sms: 0.1295,
|
||||
},
|
||||
KR: {
|
||||
name: 'South Korea',
|
||||
sms: 0.006,
|
||||
},
|
||||
KW: {
|
||||
name: 'Kuwait',
|
||||
sms: 0.145,
|
||||
},
|
||||
KY: {
|
||||
name: 'Cayman Islands',
|
||||
sms: 0.2172,
|
||||
},
|
||||
KZ: {
|
||||
name: 'Kazakhstan',
|
||||
sms: 0.202,
|
||||
},
|
||||
LA: {
|
||||
name: 'Laos',
|
||||
sms: 0.15,
|
||||
},
|
||||
LB: {
|
||||
name: 'Lebanon',
|
||||
sms: 0.153,
|
||||
},
|
||||
LC: {
|
||||
name: 'Saint Lucia',
|
||||
sms: 0.0892,
|
||||
},
|
||||
LI: {
|
||||
name: 'Liechtenstein',
|
||||
sms: null,
|
||||
},
|
||||
LK: {
|
||||
name: 'Sri Lanka',
|
||||
sms: 0.3593,
|
||||
},
|
||||
LR: {
|
||||
name: 'Liberia',
|
||||
sms: 0.1448,
|
||||
},
|
||||
LS: {
|
||||
name: 'Lesotho',
|
||||
sms: 0.0301,
|
||||
},
|
||||
LT: {
|
||||
name: 'Lithuania',
|
||||
sms: 0.0261,
|
||||
},
|
||||
LU: {
|
||||
name: 'Luxembourg',
|
||||
sms: 0.034,
|
||||
},
|
||||
LV: {
|
||||
name: 'Latvia',
|
||||
sms: 0.028,
|
||||
},
|
||||
LY: {
|
||||
name: 'Libya',
|
||||
sms: 0.1898,
|
||||
},
|
||||
MA: {
|
||||
name: 'Morocco',
|
||||
sms: 0.105,
|
||||
},
|
||||
MC: {
|
||||
name: 'Monaco',
|
||||
sms: 0.1219,
|
||||
},
|
||||
MD: {
|
||||
name: 'Moldova',
|
||||
sms: 0.065,
|
||||
},
|
||||
ME: {
|
||||
name: 'Montenegro',
|
||||
sms: 0.07,
|
||||
},
|
||||
MG: {
|
||||
name: 'Madagascar',
|
||||
sms: 0.24,
|
||||
},
|
||||
MH: {
|
||||
name: '',
|
||||
sms: 0.032,
|
||||
},
|
||||
MK: {
|
||||
name: 'Macedonia',
|
||||
sms: 0.0096,
|
||||
},
|
||||
ML: {
|
||||
name: 'Mali',
|
||||
sms: 0.136,
|
||||
},
|
||||
MM: {
|
||||
name: 'Myanmar',
|
||||
sms: 0.3037,
|
||||
},
|
||||
MN: {
|
||||
name: 'Mongolia',
|
||||
sms: 0.183,
|
||||
},
|
||||
MO: {
|
||||
name: 'Macao',
|
||||
sms: 0.005,
|
||||
},
|
||||
MP: {
|
||||
name: 'Northern Mariana Islands',
|
||||
sms: 0.0669,
|
||||
},
|
||||
MQ: {
|
||||
name: 'Martinique',
|
||||
sms: 0.0455,
|
||||
},
|
||||
MR: {
|
||||
name: 'Mauritania',
|
||||
sms: 0.161,
|
||||
},
|
||||
MS: {
|
||||
name: 'Montserrat',
|
||||
sms: 0.0725,
|
||||
},
|
||||
MT: {
|
||||
name: 'Malta',
|
||||
sms: 0.0343,
|
||||
},
|
||||
MU: {
|
||||
name: 'Mauritius',
|
||||
sms: 0.1431,
|
||||
},
|
||||
MV: {
|
||||
name: 'Maldives',
|
||||
sms: 0.145,
|
||||
},
|
||||
MW: {
|
||||
name: 'Malawi',
|
||||
sms: 0.19,
|
||||
},
|
||||
MX: {
|
||||
name: 'Mexico',
|
||||
sms: 0.0021,
|
||||
},
|
||||
MY: {
|
||||
name: 'Malaysia',
|
||||
sms: 0.08,
|
||||
},
|
||||
MZ: {
|
||||
name: 'Mozambique',
|
||||
sms: 0.2266,
|
||||
},
|
||||
NA: {
|
||||
name: 'Namibia',
|
||||
sms: 0.0173,
|
||||
},
|
||||
NC: {
|
||||
name: 'New Caledonia',
|
||||
sms: 0.052,
|
||||
},
|
||||
NE: {
|
||||
name: 'Niger',
|
||||
sms: 0.143,
|
||||
},
|
||||
NG: {
|
||||
name: 'Nigeria',
|
||||
sms: 0.198,
|
||||
},
|
||||
NI: {
|
||||
name: 'Nicaragua',
|
||||
sms: 0.1033,
|
||||
},
|
||||
NL: {
|
||||
name: 'Netherlands',
|
||||
sms: 0.046,
|
||||
},
|
||||
NO: {
|
||||
name: 'Norway',
|
||||
sms: 0.03,
|
||||
},
|
||||
NP: {
|
||||
name: 'Nepal',
|
||||
sms: 0.1645,
|
||||
},
|
||||
NR: {
|
||||
name: 'Nauru',
|
||||
sms: null,
|
||||
},
|
||||
NU: {
|
||||
name: 'Niue',
|
||||
sms: null,
|
||||
},
|
||||
NZ: {
|
||||
name: 'New Zealand',
|
||||
sms: 0.0372,
|
||||
},
|
||||
OM: {
|
||||
name: 'Oman',
|
||||
sms: 0.0713,
|
||||
},
|
||||
PA: {
|
||||
name: 'Panama',
|
||||
sms: 0.07,
|
||||
},
|
||||
PE: {
|
||||
name: 'Peru',
|
||||
sms: 0.13,
|
||||
},
|
||||
PF: {
|
||||
name: 'French Polynesia',
|
||||
sms: 0.0518,
|
||||
},
|
||||
PG: {
|
||||
name: 'Papua New Guinea',
|
||||
sms: 0.15,
|
||||
},
|
||||
PH: {
|
||||
name: 'Philippines',
|
||||
sms: 0.1237,
|
||||
},
|
||||
PK: {
|
||||
name: 'Pakistan',
|
||||
sms: 0.3548,
|
||||
},
|
||||
PL: {
|
||||
name: 'Poland',
|
||||
sms: 0.011,
|
||||
},
|
||||
PM: {
|
||||
name: 'Saint Pierre and Miquelon',
|
||||
sms: 0.0905,
|
||||
},
|
||||
PR: {
|
||||
name: 'Puerto Rico',
|
||||
sms: 0.02,
|
||||
},
|
||||
PS: {
|
||||
name: 'Palestine',
|
||||
sms: 0.2541,
|
||||
},
|
||||
PT: {
|
||||
name: 'Portugal',
|
||||
sms: 0.009,
|
||||
},
|
||||
PW: {
|
||||
name: 'Palau',
|
||||
sms: null,
|
||||
},
|
||||
PY: {
|
||||
name: 'Paraguay',
|
||||
sms: 0.0236,
|
||||
},
|
||||
QA: {
|
||||
name: 'Qatar',
|
||||
sms: 0.1393,
|
||||
},
|
||||
RE: {
|
||||
name: 'Reunion',
|
||||
sms: 0.034,
|
||||
},
|
||||
RO: {
|
||||
name: 'Romania',
|
||||
sms: 0.0255,
|
||||
},
|
||||
RS: {
|
||||
name: 'Serbia',
|
||||
sms: 0.1935,
|
||||
},
|
||||
RU: {
|
||||
name: 'Russia',
|
||||
sms: 0.2028,
|
||||
},
|
||||
RW: {
|
||||
name: 'Rwanda',
|
||||
sms: 0.1177,
|
||||
},
|
||||
SA: {
|
||||
name: 'Saudi Arabia',
|
||||
sms: 0.0638,
|
||||
},
|
||||
SB: {
|
||||
name: 'Solomon Islands',
|
||||
sms: 0.0357,
|
||||
},
|
||||
SC: {
|
||||
name: 'Seychelles',
|
||||
sms: 0.0404,
|
||||
},
|
||||
SD: {
|
||||
name: 'Sudan',
|
||||
sms: 0.224,
|
||||
},
|
||||
SE: {
|
||||
name: 'Sweden',
|
||||
sms: 0.026,
|
||||
},
|
||||
SG: {
|
||||
name: 'Singapore',
|
||||
sms: 0.0256,
|
||||
},
|
||||
SI: {
|
||||
name: 'Slovenia',
|
||||
sms: 0.1,
|
||||
},
|
||||
SK: {
|
||||
name: 'Slovakia',
|
||||
sms: 0.0245,
|
||||
},
|
||||
SL: {
|
||||
name: 'Sierra Leone',
|
||||
sms: 0.2322,
|
||||
},
|
||||
SM: {
|
||||
name: 'San Marino',
|
||||
sms: null,
|
||||
},
|
||||
SN: {
|
||||
name: 'Senegal',
|
||||
sms: 0.1151,
|
||||
},
|
||||
SO: {
|
||||
name: 'Somalia',
|
||||
sms: 0.06,
|
||||
},
|
||||
SR: {
|
||||
name: 'Suriname',
|
||||
sms: 0.119,
|
||||
},
|
||||
SS: {
|
||||
name: 'South Sudan',
|
||||
sms: 0.15,
|
||||
},
|
||||
ST: {
|
||||
name: 'Sao Tome and Principe',
|
||||
sms: 0.0133,
|
||||
},
|
||||
SV: {
|
||||
name: 'El Salvador',
|
||||
sms: 0.06,
|
||||
},
|
||||
SX: {
|
||||
name: 'Sint Maarten',
|
||||
sms: 0.0623,
|
||||
},
|
||||
SY: {
|
||||
name: 'Syria',
|
||||
sms: 0.221,
|
||||
},
|
||||
SZ: {
|
||||
name: 'Eswatini',
|
||||
sms: 0.12,
|
||||
},
|
||||
TC: {
|
||||
name: 'Turks and Caicos Islands',
|
||||
sms: null,
|
||||
},
|
||||
TD: {
|
||||
name: 'Chad',
|
||||
sms: 0.1711,
|
||||
},
|
||||
TG: {
|
||||
name: 'Togo',
|
||||
sms: 0.1806,
|
||||
},
|
||||
TH: {
|
||||
name: 'Thailand',
|
||||
sms: 0.003,
|
||||
},
|
||||
TJ: {
|
||||
name: 'Tajikistan',
|
||||
sms: 0.2626,
|
||||
},
|
||||
TL: {
|
||||
name: 'Timor-Leste',
|
||||
sms: 0.0675,
|
||||
},
|
||||
TM: {
|
||||
name: 'Turkmenistan',
|
||||
sms: 0.1677,
|
||||
},
|
||||
TN: {
|
||||
name: 'Tunisia',
|
||||
sms: 0.225,
|
||||
},
|
||||
TO: {
|
||||
name: 'Tonga',
|
||||
sms: 0.0902,
|
||||
},
|
||||
TR: {
|
||||
name: 'Turkey',
|
||||
sms: 0.0008,
|
||||
},
|
||||
TT: {
|
||||
name: 'Trinidad and Tobago',
|
||||
sms: 0.1556,
|
||||
},
|
||||
TW: {
|
||||
name: 'Taiwan',
|
||||
sms: 0.0165,
|
||||
},
|
||||
TZ: {
|
||||
name: 'Tanzania',
|
||||
sms: 0.2753,
|
||||
},
|
||||
UA: {
|
||||
name: 'Ukraine',
|
||||
sms: 0.094,
|
||||
},
|
||||
UG: {
|
||||
name: 'Uganda',
|
||||
sms: 0.1707,
|
||||
},
|
||||
US: {
|
||||
name: 'United States',
|
||||
sms: 0.0043,
|
||||
},
|
||||
UY: {
|
||||
name: 'Uruguay',
|
||||
sms: 0.0174,
|
||||
},
|
||||
UZ: {
|
||||
name: 'Uzbekistan',
|
||||
sms: 0.292,
|
||||
},
|
||||
VC: {
|
||||
name: 'Saint Vincent and The Grenadines',
|
||||
sms: 0.1216,
|
||||
},
|
||||
VE: {
|
||||
name: 'Venezuela',
|
||||
sms: 0.0427,
|
||||
},
|
||||
VG: {
|
||||
name: 'British Virgin Islands',
|
||||
sms: 0.083,
|
||||
},
|
||||
VI: {
|
||||
name: 'Virgin Island',
|
||||
sms: 0.0017,
|
||||
},
|
||||
VN: {
|
||||
name: 'Vietnam',
|
||||
sms: 0.0885,
|
||||
},
|
||||
VU: {
|
||||
name: 'Vanuatu',
|
||||
sms: 0.1347,
|
||||
},
|
||||
WF: {
|
||||
name: 'Wallis and Futuna',
|
||||
sms: 0.09,
|
||||
},
|
||||
WS: {
|
||||
name: 'Samoa',
|
||||
sms: 0.2,
|
||||
},
|
||||
XK: {
|
||||
name: 'Kosovo',
|
||||
sms: 0.1394,
|
||||
},
|
||||
YE: {
|
||||
name: 'Yemen',
|
||||
sms: 0.1486,
|
||||
},
|
||||
YT: {
|
||||
name: 'Mayotte',
|
||||
sms: 0.06,
|
||||
},
|
||||
ZA: {
|
||||
name: 'South Africa',
|
||||
sms: 0.039,
|
||||
},
|
||||
ZM: {
|
||||
name: 'Zambia',
|
||||
sms: 0.21,
|
||||
},
|
||||
ZW: {
|
||||
name: 'Zimbabwe',
|
||||
sms: 0.13,
|
||||
},
|
||||
};
|
||||
File diff suppressed because it is too large.
Load diff
@@ -47,6 +47,7 @@ import {
|
||||
} from '../../services/auth/OTPUtil.js';
|
||||
import { sessionCookieFlags } from '../../util/cookieFlags.js';
|
||||
import { cleanEmail, isBlockedEmail } from '../../util/email.js';
|
||||
import { parsePhone } from '../../util/phone.js';
|
||||
import { generate_identifier } from '../../util/identifier.js';
|
||||
import { getTaskbarItems } from '../../util/taskbarItems.js';
|
||||
import {
|
||||
@@ -57,6 +58,8 @@ import { PuterController } from '../types.js';
|
||||
|
||||
const USERNAME_REGEX = /^\w{1,}$/;
|
||||
const USERNAME_MAX_LENGTH = 45;
|
||||
const FINGERPRINT_MAX_LENGTH = 128;
|
||||
const DFP_TELEMETRY_ID_MAX_LENGTH = 64;
|
||||
const RESERVED_USERNAMES = new Set([
|
||||
'admin',
|
||||
'administrator',
|
||||
@@ -375,6 +378,43 @@ export class AuthController extends PuterController {
|
||||
return;
|
||||
}
|
||||
|
||||
// Optional device signals (browser fingerprint hash, DFP telemetry
|
||||
// id). Core only enforces shape and forwards the values verbatim —
|
||||
// signup-abuse policy built on them lives in extensions. Checked
|
||||
// before the reauth short-circuit so a malformed value is rejected
|
||||
// on every /signup path.
|
||||
if (body.fingerprint !== undefined && body.fingerprint !== null) {
|
||||
if (typeof body.fingerprint !== 'string')
|
||||
throw new HttpError(400, 'fingerprint must be a string.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
if (body.fingerprint.length > FINGERPRINT_MAX_LENGTH)
|
||||
throw new HttpError(
|
||||
400,
|
||||
`fingerprint cannot be longer than ${FINGERPRINT_MAX_LENGTH} characters.`,
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
if (
|
||||
body.dfp_telemetry_id !== undefined &&
|
||||
body.dfp_telemetry_id !== null
|
||||
) {
|
||||
if (typeof body.dfp_telemetry_id !== 'string')
|
||||
throw new HttpError(400, 'dfp_telemetry_id must be a string.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
if (body.dfp_telemetry_id.length > DFP_TELEMETRY_ID_MAX_LENGTH)
|
||||
throw new HttpError(
|
||||
400,
|
||||
`dfp_telemetry_id cannot be longer than ${DFP_TELEMETRY_ID_MAX_LENGTH} characters.`,
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
// Empty strings are treated as absent — a signal that wasn't
|
||||
// collected, not a malformed request.
|
||||
const fingerprint: string | null = body.fingerprint || null;
|
||||
const dfp_telemetry_id: string | null = body.dfp_telemetry_id || null;
|
||||
|
||||
// Temp-user reauth short-circuit: when an existing temp user is
|
||||
// forced through the reauth flow, the GUI re-submits /signup with
|
||||
// is_temp=true plus the server-signed reauth_token from the 401.
|
||||
@@ -554,12 +594,26 @@ export class AuthController extends PuterController {
|
||||
allow: true,
|
||||
no_temp_user: false,
|
||||
requires_email_confirmation: false,
|
||||
// Set by the abuse harness for low-reputation signups: the account is
|
||||
// created + logged in but gated behind SMS phone verification (in
|
||||
// addition to email confirmation) instead of being blocked.
|
||||
requires_phone_verification: false,
|
||||
// Same idea, one rung up the ladder: gate the account behind
|
||||
// credit-card verification (a $0 auth handled by an extension).
|
||||
requires_card_verification: false,
|
||||
message: null,
|
||||
code: null,
|
||||
user_agent: req?.headers?.['user-agent'] ?? null,
|
||||
fingerprint,
|
||||
dfp_telemetry_id,
|
||||
// Populated by the abuse extension's v2 harness; persisted to the
|
||||
// user row below so the signup-time reputation is referable later.
|
||||
reputation: null as number | null,
|
||||
// Stamped by the abuse harness for flagged signups — the id keying
|
||||
// the `abuse:trail:<id>` decision trail (carrying both the live and
|
||||
// shadow trails). Surfaced to a blocked user as the Request Code so
|
||||
// the code they quote support leads straight to their trail.
|
||||
trail_id: undefined as string | undefined,
|
||||
};
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
@@ -571,9 +625,14 @@ export class AuthController extends PuterController {
|
||||
console.warn('[signup] validate hook failed:', e);
|
||||
}
|
||||
if (!validateEvent.allow) {
|
||||
// Pass the trail id back to a blocked user as the Request Code (when
|
||||
// the harness stamped one), embedded in the message so the existing
|
||||
// signup-block UI surfaces it without a GUI change.
|
||||
const requestCode = validateEvent.trail_id;
|
||||
throw new HttpError(
|
||||
403,
|
||||
validateEvent.message ?? 'Signup blocked',
|
||||
(validateEvent.message ?? 'Signup blocked') +
|
||||
(requestCode ? ` Request Code: ${requestCode}` : ''),
|
||||
{
|
||||
...(validateEvent.code
|
||||
? { legacyCode: validateEvent.code as never }
|
||||
@@ -596,6 +655,17 @@ export class AuthController extends PuterController {
|
||||
const force_email_confirmation = Boolean(
|
||||
validateEvent.requires_email_confirmation,
|
||||
);
|
||||
const force_phone_verification =
|
||||
Boolean(validateEvent.requires_phone_verification) ||
|
||||
// Test/QA switch: force the SMS gate on every signup regardless of
|
||||
// reputation (see config.always_require_phone_verification).
|
||||
Boolean(this.config.always_require_phone_verification);
|
||||
const force_card_verification = Boolean(
|
||||
validateEvent.requires_card_verification ||
|
||||
// Test/QA switch: force the card gate on every signup regardless of
|
||||
// reputation (see config.always_require_card_verification).
|
||||
this.config.always_require_card_verification,
|
||||
);
|
||||
|
||||
// Prepare shared fields
|
||||
const user_uuid = uuidv4();
|
||||
@@ -620,16 +690,13 @@ export class AuthController extends PuterController {
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
email_confirmed: 0,
|
||||
// Pseudo claims always require email confirmation — the
|
||||
// validate hook can only tighten, not loosen, so `1`
|
||||
// stays hardcoded here.
|
||||
requires_email_confirmation: 1,
|
||||
last_activity_ts: signupSqlTs,
|
||||
// Record the v2 reputation from this claim (skip if unset so we
|
||||
// don't clobber an existing score with a placeholder).
|
||||
...(validateEvent.reputation != null
|
||||
? { reputation: validateEvent.reputation }
|
||||
: {}),
|
||||
requires_phone_verification: force_phone_verification ? 1 : 0,
|
||||
requires_card_verification: force_card_verification ? 1 : 0,
|
||||
});
|
||||
|
||||
// Move from temp group to regular user group
|
||||
@@ -678,6 +745,8 @@ export class AuthController extends PuterController {
|
||||
ip_fwd: proxyIpChain,
|
||||
user_agent: req.headers?.['user-agent'],
|
||||
origin: req.headers?.origin,
|
||||
fingerprint,
|
||||
dfp_telemetry_id,
|
||||
},
|
||||
signup_ip: clientIp,
|
||||
signup_ip_forwarded: proxyIpChain,
|
||||
@@ -687,6 +756,10 @@ export class AuthController extends PuterController {
|
||||
referrer: req.body.referrer ?? null,
|
||||
last_activity_ts: signupSqlTs,
|
||||
reputation: validateEvent.reputation,
|
||||
// Phone collected later in the verification dialog (null now).
|
||||
phone: null,
|
||||
requires_phone_verification: force_phone_verification,
|
||||
requires_card_verification: force_card_verification,
|
||||
} as never);
|
||||
|
||||
// Add to default group
|
||||
@@ -757,6 +830,11 @@ export class AuthController extends PuterController {
|
||||
user_uuid: user!.uuid,
|
||||
email: user!.email,
|
||||
username: user!.username,
|
||||
fingerprint,
|
||||
// Reflects the row that was actually created/claimed —
|
||||
// a pseudo-user claim ends up with credentials, so it
|
||||
// reports false here. Same signal completeLogin uses.
|
||||
is_temp: user!.password === null && user!.email === null,
|
||||
ip:
|
||||
(req?.headers?.['x-forwarded-for'] as
|
||||
| string
|
||||
@@ -957,6 +1035,507 @@ export class AuthController extends PuterController {
|
||||
res.json({ email_confirmed: true, original_client_socket_id });
|
||||
}
|
||||
|
||||
// -- Phone verification (SMS via Prelude) ------------------------
|
||||
|
||||
@Post('/send-confirm-phone', {
|
||||
subdomain: ['api', ''],
|
||||
requireUserActor: true,
|
||||
allowUnconfirmed: true,
|
||||
rateLimit: {
|
||||
scope: 'send-confirm-phone',
|
||||
limit: 10,
|
||||
window: 60 * 60_000,
|
||||
key: 'user',
|
||||
},
|
||||
})
|
||||
async handleSendConfirmPhone(req: Request, res: Response): Promise<void> {
|
||||
const user = await this.stores.user.getById(req.actor!.user.id!, {
|
||||
force: true,
|
||||
});
|
||||
if (!user)
|
||||
throw new HttpError(404, 'User not found.', {
|
||||
legacyCode: 'user_not_found' as never,
|
||||
});
|
||||
if (user.suspended)
|
||||
throw new HttpError(403, 'Account suspended.', {
|
||||
legacyCode: 'account_suspended',
|
||||
});
|
||||
if (!this.clients.prelude?.isConfigured())
|
||||
throw new HttpError(503, 'Phone verification is unavailable.', {
|
||||
legacyCode: 'service_unavailable' as never,
|
||||
});
|
||||
|
||||
// Parse to E.164 (Prelude's required form + the stored form) and the
|
||||
// country, so we can apply the per-country cost cap.
|
||||
const parsed = parsePhone(
|
||||
req.body?.phone,
|
||||
this.clients.prelude.defaultCountry,
|
||||
);
|
||||
if (!parsed)
|
||||
throw new HttpError(400, 'Invalid phone number.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
// Cost cap: skip countries with no SMS channel or rates above the cap
|
||||
// (see PreludeClient / countries.ts). Avoids paying exorbitant per-SMS
|
||||
// rates in low-revenue, high-fraud geographies.
|
||||
if (!this.clients.prelude.isCountrySupported(parsed.country))
|
||||
throw new HttpError(
|
||||
400,
|
||||
'Phone verification is not available for this country.',
|
||||
{ legacyCode: 'phone_country_not_supported' as never },
|
||||
);
|
||||
|
||||
// Abuse caps live ENTIRELY in a listening abuse extension, consulted
|
||||
// via `puter.phone-verification.check`. The backend ships no thresholds
|
||||
// or detection of its own (so none of it is readable in the open-source
|
||||
// repo): it forwards the user / number / ip, and the extension decides
|
||||
// `allowed` plus an opaque `reason` (per-account + per-number send
|
||||
// velocity, cross-account reuse, …). With no extension listening
|
||||
// `allowed` stays true. Fail-open on a hook error — this is abuse/cost
|
||||
// control, not a security boundary (the route rate limit and the country
|
||||
// cost cap remain), so a flaky hook must not lock signups out.
|
||||
const abuseCheck = {
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
phone: parsed.e164,
|
||||
device_fingerprint: req.deviceFingerprint ?? null,
|
||||
allowed: true,
|
||||
reason: null as string | null,
|
||||
};
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
'puter.phone-verification.check',
|
||||
abuseCheck,
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[send-confirm-phone] abuse-check hook failed:', e);
|
||||
}
|
||||
// Forward the verdict verbatim: a generic 429 plus the opaque reason for
|
||||
// the client to message on. The backend never interprets the reason —
|
||||
// its meaning lives in the extension (which sets it) and the GUI (which
|
||||
// displays it), so no abuse semantics leak into the OSS repo.
|
||||
if (abuseCheck.allowed === false)
|
||||
throw new HttpError(
|
||||
429,
|
||||
'Phone verification is unavailable for this number right now.',
|
||||
{
|
||||
legacyCode: 'phone_verification_unavailable' as never,
|
||||
fields: abuseCheck.reason
|
||||
? { reason: abuseCheck.reason }
|
||||
: {},
|
||||
},
|
||||
);
|
||||
|
||||
// Stage the parsed number as pending in KV (NOT on the user row) so a
|
||||
// never-confirmed number is never written to the indexed `phone`
|
||||
// column. /confirm-phone reads it back and persists it to the row only
|
||||
// once Prelude confirms the code. ~1h TTL covers the code's lifetime.
|
||||
// Stored before the send so we never dispatch an SMS we couldn't later
|
||||
// confirm against.
|
||||
const pendingPhoneKey = `phone-verify-pending:${user.id}`;
|
||||
try {
|
||||
await this.stores.kv.set({
|
||||
key: pendingPhoneKey,
|
||||
value: parsed.e164,
|
||||
expireAt: Math.floor(Date.now() / 1000) + 60 * 60,
|
||||
});
|
||||
} catch (e) {
|
||||
console.warn('[send-confirm-phone] pending-store failed:', e);
|
||||
throw new HttpError(503, 'Could not start phone verification.', {
|
||||
legacyCode: 'service_unavailable' as never,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = req.ip || req.socket?.remoteAddress || undefined;
|
||||
const userAgent =
|
||||
typeof req.headers['user-agent'] === 'string'
|
||||
? req.headers['user-agent']
|
||||
: undefined;
|
||||
try {
|
||||
const result = await this.clients.prelude.createVerification(
|
||||
parsed.e164,
|
||||
{
|
||||
ip,
|
||||
device_id: req.deviceFingerprint ?? undefined,
|
||||
user_agent: userAgent,
|
||||
},
|
||||
);
|
||||
// Prelude rejected the attempt as abusive — surface as rate-limit.
|
||||
if (
|
||||
result.status === 'blocked' ||
|
||||
result.status === 'shadow_blocked'
|
||||
) {
|
||||
throw new HttpError(
|
||||
429,
|
||||
'Phone verification is temporarily unavailable for this number.',
|
||||
{ legacyCode: 'too_many_requests' as never },
|
||||
);
|
||||
}
|
||||
} catch (e) {
|
||||
if (e instanceof HttpError) throw e;
|
||||
console.warn('[send-confirm-phone] createVerification failed:', e);
|
||||
throw new HttpError(502, 'Could not send verification code.', {
|
||||
legacyCode: 'upstream_error' as never,
|
||||
});
|
||||
}
|
||||
|
||||
// Tell the abuse extension a code was actually sent, so it can bump its
|
||||
// send-velocity counters (per number + per account). Fire-and-forget;
|
||||
// the backend keeps no send counts of its own. Only reached after a
|
||||
// successful send, so an upstream error never burns quota.
|
||||
try {
|
||||
this.clients.event?.emit(
|
||||
'puter.phone-verification.sent' as never,
|
||||
{
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
phone: parsed.e164,
|
||||
device_fingerprint: req.deviceFingerprint ?? null,
|
||||
} as never,
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
// ignore — best-effort velocity signal
|
||||
}
|
||||
res.json({});
|
||||
}
|
||||
|
||||
@Post('/confirm-phone', {
|
||||
subdomain: ['api', ''],
|
||||
requireUserActor: true,
|
||||
allowUnconfirmed: true,
|
||||
rateLimit: {
|
||||
scope: 'confirm-phone',
|
||||
limit: 10,
|
||||
window: 10 * 60_000,
|
||||
key: 'user',
|
||||
},
|
||||
})
|
||||
async handleConfirmPhone(req: Request, res: Response): Promise<void> {
|
||||
const { code, original_client_socket_id } = req.body ?? {};
|
||||
if (!code)
|
||||
throw new HttpError(400, 'Missing `code`.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const user = await this.stores.user.getById(req.actor!.user.id!, {
|
||||
force: true,
|
||||
});
|
||||
if (!user)
|
||||
throw new HttpError(404, 'User not found.', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
if (!user.requires_phone_verification) {
|
||||
res.json({ phone_verified: true, original_client_socket_id });
|
||||
return;
|
||||
}
|
||||
// The number being verified is the one staged at send time (KV
|
||||
// pending), not the user row — we don't persist an unverified number.
|
||||
// Fall back to a row value for accounts that already have one on file
|
||||
// (legacy / a number persisted by a prior verified flow).
|
||||
const pendingPhoneKey = `phone-verify-pending:${user.id}`;
|
||||
let pendingPhone: string | null = null;
|
||||
try {
|
||||
const { res: staged } = await this.stores.kv.get({
|
||||
key: pendingPhoneKey,
|
||||
});
|
||||
if (typeof staged === 'string' && staged) pendingPhone = staged;
|
||||
} catch (e) {
|
||||
console.warn('[confirm-phone] pending read failed:', e);
|
||||
}
|
||||
if (!pendingPhone) pendingPhone = user.phone ?? null;
|
||||
if (!pendingPhone)
|
||||
throw new HttpError(
|
||||
400,
|
||||
'No phone number on file. Request a code first.',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
if (!this.clients.prelude?.isConfigured())
|
||||
throw new HttpError(503, 'Phone verification is unavailable.', {
|
||||
legacyCode: 'service_unavailable' as never,
|
||||
});
|
||||
|
||||
let status;
|
||||
try {
|
||||
({ status } = await this.clients.prelude.checkVerification(
|
||||
pendingPhone,
|
||||
String(code),
|
||||
));
|
||||
} catch (e) {
|
||||
console.warn('[confirm-phone] checkVerification failed:', e);
|
||||
throw new HttpError(502, 'Could not verify code.', {
|
||||
legacyCode: 'upstream_error' as never,
|
||||
});
|
||||
}
|
||||
|
||||
if (status !== 'success') {
|
||||
res.json({ phone_verified: false, original_client_socket_id });
|
||||
return;
|
||||
}
|
||||
|
||||
// Verified — persist the number now (and only now) and clear the gate.
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_phone_verification: 0,
|
||||
phone: pendingPhone,
|
||||
});
|
||||
|
||||
try {
|
||||
this.clients.event?.emit(
|
||||
'user.phone-verified' as never,
|
||||
{
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
phone: pendingPhone,
|
||||
} as never,
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
// ignore — event is a side-channel signal, not load-bearing
|
||||
}
|
||||
// Notify other tabs/devices for this user so they refresh + drop the gate.
|
||||
try {
|
||||
await this.services.socket?.send(
|
||||
{ room: user.id },
|
||||
'user.phone_verified',
|
||||
{ original_client_socket_id },
|
||||
);
|
||||
} catch {
|
||||
// ignore — best-effort
|
||||
}
|
||||
|
||||
res.json({ phone_verified: true, original_client_socket_id });
|
||||
}
|
||||
|
||||
// -- Card verification ($0 auth via a payments extension) --------
|
||||
|
||||
/**
|
||||
* Start card verification for the calling user. Pure mechanism: the
|
||||
* endpoint emits `puter.card-verification.setup` and a payments
|
||||
* extension fills in the client credentials — the OSS backend holds
|
||||
* no provider knowledge or config. Phone verification (when required)
|
||||
* must be completed first; the ordering is enforced here so a client
|
||||
* can't skip the cheaper gate.
|
||||
*/
|
||||
@Post('/card-verification/setup', {
|
||||
subdomain: ['api', ''],
|
||||
requireUserActor: true,
|
||||
allowUnconfirmed: true,
|
||||
rateLimit: {
|
||||
scope: 'card-verification-setup',
|
||||
limit: 5,
|
||||
window: 60 * 60_000,
|
||||
key: 'user',
|
||||
},
|
||||
})
|
||||
async handleCardVerificationSetup(
|
||||
req: Request,
|
||||
res: Response,
|
||||
): Promise<void> {
|
||||
const user = await this.stores.user.getById(req.actor!.user.id!, {
|
||||
force: true,
|
||||
});
|
||||
if (!user)
|
||||
throw new HttpError(404, 'User not found.', {
|
||||
legacyCode: 'user_not_found' as never,
|
||||
});
|
||||
if (user.suspended)
|
||||
throw new HttpError(403, 'Account suspended.', {
|
||||
legacyCode: 'account_suspended',
|
||||
});
|
||||
if (!user.requires_card_verification) {
|
||||
res.json({ card_verified: true });
|
||||
return;
|
||||
}
|
||||
if (user.requires_phone_verification)
|
||||
throw new HttpError(
|
||||
409,
|
||||
'Phone verification must be completed first.',
|
||||
{ legacyCode: 'conflict' },
|
||||
);
|
||||
|
||||
// `enabled` stays null when no extension is listening; an installed
|
||||
// extension always sets it (true/false) before doing any work.
|
||||
const setupEvent = {
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
ip: (req.ip || req.socket?.remoteAddress || null) as string | null,
|
||||
device_fingerprint: req.deviceFingerprint ?? null,
|
||||
enabled: null as boolean | null,
|
||||
allowed: true,
|
||||
reason: null as string | null,
|
||||
client_secret: null as string | null,
|
||||
publishable_key: null as string | null,
|
||||
};
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
'puter.card-verification.setup',
|
||||
setupEvent,
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[card-verification/setup] setup hook failed:', e);
|
||||
}
|
||||
|
||||
// Abuse veto (e.g. per-device setup-velocity cap): the extension refused
|
||||
// before any Stripe work. Forward the opaque reason verbatim as a 429,
|
||||
// same as the phone gate — the backend never interprets it.
|
||||
if (setupEvent.allowed === false)
|
||||
throw new HttpError(
|
||||
429,
|
||||
'Card verification is unavailable right now.',
|
||||
{
|
||||
legacyCode: 'too_many_requests' as never,
|
||||
fields: setupEvent.reason
|
||||
? { reason: setupEvent.reason }
|
||||
: {},
|
||||
},
|
||||
);
|
||||
|
||||
// Kill switch: the extension reports the feature disabled — unstick
|
||||
// any user still carrying the flag instead of dead-ending them.
|
||||
if (setupEvent.enabled === false) {
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_card_verification: 0,
|
||||
});
|
||||
res.json({ card_verified: true, disabled: true });
|
||||
return;
|
||||
}
|
||||
if (!setupEvent.client_secret || !setupEvent.publishable_key)
|
||||
throw new HttpError(503, 'Card verification is not available.', {
|
||||
legacyCode: 'service_unavailable' as never,
|
||||
});
|
||||
|
||||
res.json({
|
||||
client_secret: setupEvent.client_secret,
|
||||
publishable_key: setupEvent.publishable_key,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete card verification. The client confirms the setup intent with
|
||||
* the payment provider directly, then posts the resulting id here; the
|
||||
* payments extension checks it (and applies its own abuse limits) via
|
||||
* `puter.card-verification.confirm`. On success the gate clears exactly
|
||||
* like `/confirm-phone` clears the phone gate.
|
||||
*/
|
||||
@Post('/card-verification/confirm', {
|
||||
subdomain: ['api', ''],
|
||||
requireUserActor: true,
|
||||
allowUnconfirmed: true,
|
||||
rateLimit: {
|
||||
scope: 'card-verification-confirm',
|
||||
limit: 10,
|
||||
window: 10 * 60_000,
|
||||
key: 'user',
|
||||
},
|
||||
})
|
||||
async handleCardVerificationConfirm(
|
||||
req: Request,
|
||||
res: Response,
|
||||
): Promise<void> {
|
||||
const { setup_intent_id, original_client_socket_id } = req.body ?? {};
|
||||
if (
|
||||
typeof setup_intent_id !== 'string' ||
|
||||
setup_intent_id.length === 0 ||
|
||||
setup_intent_id.length > 255
|
||||
)
|
||||
throw new HttpError(400, 'Invalid `setup_intent_id`.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const user = await this.stores.user.getById(req.actor!.user.id!, {
|
||||
force: true,
|
||||
});
|
||||
if (!user)
|
||||
throw new HttpError(404, 'User not found.', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
if (!user.requires_card_verification) {
|
||||
res.json({ card_verified: true });
|
||||
return;
|
||||
}
|
||||
if (user.requires_phone_verification)
|
||||
throw new HttpError(
|
||||
409,
|
||||
'Phone verification must be completed first.',
|
||||
{ legacyCode: 'conflict' },
|
||||
);
|
||||
|
||||
const confirmEvent = {
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
setup_intent_id,
|
||||
enabled: null as boolean | null,
|
||||
verified: false,
|
||||
reason: null as string | null,
|
||||
fingerprint: null as string | null,
|
||||
funding: null as string | null,
|
||||
country: null as string | null,
|
||||
};
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
'puter.card-verification.confirm',
|
||||
confirmEvent,
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[card-verification/confirm] confirm hook failed:', e);
|
||||
}
|
||||
|
||||
// Kill switch — same semantics as /card-verification/setup.
|
||||
if (confirmEvent.enabled === false) {
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_card_verification: 0,
|
||||
});
|
||||
res.json({ card_verified: true, disabled: true });
|
||||
return;
|
||||
}
|
||||
// No extension listening — nothing could have verified anything.
|
||||
if (confirmEvent.enabled === null)
|
||||
throw new HttpError(503, 'Card verification is not available.', {
|
||||
legacyCode: 'service_unavailable' as never,
|
||||
});
|
||||
|
||||
if (confirmEvent.verified !== true) {
|
||||
res.json({ card_verified: false, reason: confirmEvent.reason });
|
||||
return;
|
||||
}
|
||||
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_card_verification: 0,
|
||||
});
|
||||
|
||||
try {
|
||||
this.clients.event?.emit(
|
||||
'user.card-verified' as never,
|
||||
{
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
fingerprint: confirmEvent.fingerprint,
|
||||
funding: confirmEvent.funding,
|
||||
country: confirmEvent.country,
|
||||
} as never,
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
// ignore — event is a side-channel signal, not load-bearing
|
||||
}
|
||||
// Notify other tabs/devices for this user so they refresh + drop the gate.
|
||||
try {
|
||||
await this.services.socket?.send(
|
||||
{ room: user.id },
|
||||
'user.card_verified',
|
||||
{ original_client_socket_id },
|
||||
);
|
||||
} catch {
|
||||
// ignore — best-effort
|
||||
}
|
||||
|
||||
res.json({ card_verified: true });
|
||||
}
|
||||
|
||||
// -- Password recovery -------------------------------------------
|
||||
|
||||
@Post('/send-pass-recovery-email', {
|
||||
@@ -3085,6 +3664,9 @@ export class AuthController extends PuterController {
|
||||
password?: string | null;
|
||||
email_confirmed?: number | boolean;
|
||||
requires_email_confirmation?: number | boolean;
|
||||
phone?: string | null;
|
||||
requires_phone_verification?: number | boolean;
|
||||
requires_card_verification?: number | boolean;
|
||||
},
|
||||
): Promise<void> {
|
||||
const meta = {
|
||||
@@ -3133,6 +3715,9 @@ export class AuthController extends PuterController {
|
||||
email: user.email,
|
||||
email_confirmed: user.email_confirmed,
|
||||
requires_email_confirmation: user.requires_email_confirmation,
|
||||
phone: user.phone,
|
||||
requires_phone_verification: user.requires_phone_verification,
|
||||
requires_card_verification: user.requires_card_verification,
|
||||
is_temp: user.password === null && user.email === null,
|
||||
taskbar_items,
|
||||
},
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import type { Readable } from 'node:stream';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
@@ -124,6 +125,14 @@ const makeRes = () => {
|
||||
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
|
||||
runWithContext({ actor }, fn);
|
||||
|
||||
const streamToString = async (stream: Readable): Promise<string> => {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of stream) {
|
||||
chunks.push(Buffer.from(chunk as Buffer));
|
||||
}
|
||||
return Buffer.concat(chunks).toString('utf8');
|
||||
};
|
||||
|
||||
// ── /startBatchWrite ────────────────────────────────────────────────
|
||||
|
||||
describe('FSController.startBatchWrites', () => {
|
||||
@@ -549,9 +558,7 @@ describe('FSController.completeBatchWrites', () => {
|
||||
.body as SignedWriteResponse[];
|
||||
|
||||
const emitSpy = vi.spyOn(server.clients.event, 'emit');
|
||||
let updatedCall:
|
||||
| (typeof emitSpy.mock.calls)[number]
|
||||
| undefined;
|
||||
let updatedCall: (typeof emitSpy.mock.calls)[number] | undefined;
|
||||
try {
|
||||
await withActor(actor, () =>
|
||||
controller.completeBatchWrites(
|
||||
@@ -726,9 +733,9 @@ describe('FSController.statEntry', () => {
|
||||
...makeReq({ body: { path: '/x' }, actor }),
|
||||
actor: undefined,
|
||||
} as unknown as Request;
|
||||
await expect(
|
||||
controller.statEntry(req, res),
|
||||
).rejects.toMatchObject({ statusCode: 401 });
|
||||
await expect(controller.statEntry(req, res)).rejects.toMatchObject({
|
||||
statusCode: 401,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -898,9 +905,7 @@ describe('FSController.searchEntries', () => {
|
||||
).toBe(true);
|
||||
}
|
||||
expect(
|
||||
results.some(
|
||||
(r) => r.path === `/${username}/Documents/${needle}`,
|
||||
),
|
||||
results.some((r) => r.path === `/${username}/Documents/${needle}`),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
@@ -1318,10 +1323,6 @@ describe('FSController.copyEntry', () => {
|
||||
// ── /read (readEntry, full read) ────────────────────────────────────
|
||||
|
||||
describe('FSController.readEntry (file streaming)', () => {
|
||||
// makeRes here adds a real Writable surface so that
|
||||
// `pipeline(download.body, res)` inside the controller can pipe
|
||||
// the in-memory S3 stream into the test's response and capture
|
||||
// bytes for assertions.
|
||||
const makeStreamingRes = () => {
|
||||
const captured = {
|
||||
statusCode: 200,
|
||||
@@ -1329,7 +1330,8 @@ describe('FSController.readEntry (file streaming)', () => {
|
||||
bodyChunks: [] as Buffer[],
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
const { Writable } = require('node:stream') as typeof import('node:stream');
|
||||
const { Writable } =
|
||||
require('node:stream') as typeof import('node:stream');
|
||||
const writable = new Writable({
|
||||
write(chunk: Buffer, _enc, cb) {
|
||||
captured.bodyChunks.push(chunk);
|
||||
@@ -1389,7 +1391,9 @@ describe('FSController.readEntry (file streaming)', () => {
|
||||
// Pipeline awaits the stream-end on success.
|
||||
expect(captured.statusCode).toBe(200);
|
||||
expect(captured.headers['Content-Type']).toMatch(/text\/plain/);
|
||||
expect(captured.headers['Content-Length']).toBe(String(body.byteLength));
|
||||
expect(captured.headers['Content-Length']).toBe(
|
||||
String(body.byteLength),
|
||||
);
|
||||
expect(captured.headers['Content-Disposition']).toMatch(
|
||||
/inline; filename=/,
|
||||
);
|
||||
@@ -1833,3 +1837,158 @@ describe('FSController.mkshortcutEntry', () => {
|
||||
expect(body.isShortcut).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('FSController metadata.objectKey injection', () => {
|
||||
const writeFile = async (
|
||||
actor: Actor,
|
||||
path: string,
|
||||
content: string,
|
||||
metadata?: Record<string, unknown>,
|
||||
) => {
|
||||
await withActor(actor, () =>
|
||||
controller.write(
|
||||
makeReq({
|
||||
body: {
|
||||
fileMetadata: {
|
||||
path,
|
||||
size: Buffer.byteLength(content),
|
||||
contentType: 'text/plain',
|
||||
overwrite: true,
|
||||
...(metadata ? { metadata } : {}),
|
||||
},
|
||||
fileContent: content,
|
||||
encoding: 'utf8',
|
||||
},
|
||||
actor,
|
||||
}) as unknown as Request<
|
||||
Record<string, never>,
|
||||
null,
|
||||
import('./requestTypes.js').WriteRequest
|
||||
>,
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
const entry = await server.stores.fsEntry.getEntryByPath(path, {
|
||||
skipCache: true,
|
||||
});
|
||||
if (!entry) throw new Error(`entry not found after write: ${path}`);
|
||||
return entry;
|
||||
};
|
||||
|
||||
it("does not stream another user's file when a client injects metadata.objectKey on write", async () => {
|
||||
const victim = await makeUser();
|
||||
const attacker = await makeUser();
|
||||
const victimSecret = 'VICTIM-TOP-SECRET-PAYLOAD';
|
||||
const attackerDecoy = 'attacker-own-decoy-bytes';
|
||||
|
||||
const victimEntry = await writeFile(
|
||||
victim.actor,
|
||||
`/${victim.actor.user!.username}/Documents/secret.txt`,
|
||||
victimSecret,
|
||||
);
|
||||
|
||||
const victimRead = await server.services.fs.readContent(victimEntry);
|
||||
expect(await streamToString(victimRead.body)).toBe(victimSecret);
|
||||
|
||||
const attackerEntry = await writeFile(
|
||||
attacker.actor,
|
||||
`/${attacker.actor.user!.username}/Documents/loot.txt`,
|
||||
attackerDecoy,
|
||||
{ objectKey: victimEntry.uuid },
|
||||
);
|
||||
|
||||
const persisted = attackerEntry.metadata
|
||||
? (JSON.parse(attackerEntry.metadata) as Record<string, unknown>)
|
||||
: {};
|
||||
expect(persisted.objectKey).toBeUndefined();
|
||||
|
||||
const attackerRead =
|
||||
await server.services.fs.readContent(attackerEntry);
|
||||
const got = await streamToString(attackerRead.body);
|
||||
expect(got).toBe(attackerDecoy);
|
||||
expect(got).not.toBe(victimSecret);
|
||||
});
|
||||
|
||||
it('read path ignores a divergent metadata.objectKey on an already-poisoned row', async () => {
|
||||
const victim = await makeUser();
|
||||
const attacker = await makeUser();
|
||||
const victimSecret = 'VICTIM-SECRET-FOR-POISON-TEST';
|
||||
const attackerDecoy = 'attacker-decoy-for-poison-test';
|
||||
|
||||
const victimEntry = await writeFile(
|
||||
victim.actor,
|
||||
`/${victim.actor.user!.username}/Documents/secret2.txt`,
|
||||
victimSecret,
|
||||
);
|
||||
const attackerEntry = await writeFile(
|
||||
attacker.actor,
|
||||
`/${attacker.actor.user!.username}/Documents/loot2.txt`,
|
||||
attackerDecoy,
|
||||
);
|
||||
|
||||
await server.stores.fsEntry.updateEntry(attackerEntry.uuid, {
|
||||
metadata: JSON.stringify({ objectKey: victimEntry.uuid }),
|
||||
});
|
||||
const poisoned = await server.stores.fsEntry.getEntryByPath(
|
||||
attackerEntry.path,
|
||||
{ skipCache: true },
|
||||
);
|
||||
if (!poisoned) throw new Error('poisoned entry not found');
|
||||
expect(
|
||||
(JSON.parse(poisoned.metadata!) as { objectKey: string }).objectKey,
|
||||
).toBe(victimEntry.uuid);
|
||||
|
||||
const read = await server.services.fs.readContent(poisoned);
|
||||
expect(await streamToString(read.body)).toBe(attackerDecoy);
|
||||
});
|
||||
|
||||
it('scrubs objectKey from move newMetadata while preserving legit trash metadata', async () => {
|
||||
const victim = await makeUser();
|
||||
const attacker = await makeUser();
|
||||
const victimSecret = 'VICTIM-SECRET-FOR-MOVE-TEST';
|
||||
const attackerDecoy = 'attacker-decoy-for-move-test';
|
||||
const username = attacker.actor.user!.username!;
|
||||
|
||||
const victimEntry = await writeFile(
|
||||
victim.actor,
|
||||
`/${victim.actor.user!.username}/Documents/secret3.txt`,
|
||||
victimSecret,
|
||||
);
|
||||
const attackerEntry = await writeFile(
|
||||
attacker.actor,
|
||||
`/${username}/Documents/loot3.txt`,
|
||||
attackerDecoy,
|
||||
);
|
||||
const documents = await server.stores.fsEntry.getEntryByPath(
|
||||
`/${username}/Documents`,
|
||||
{ skipCache: true },
|
||||
);
|
||||
if (!documents) throw new Error('Documents dir not found');
|
||||
|
||||
const moved = await withActor(attacker.actor, () =>
|
||||
server.services.fs.move(attacker.userId, {
|
||||
source: attackerEntry,
|
||||
destinationParent: documents,
|
||||
newName: 'loot3-moved.txt',
|
||||
newMetadata: {
|
||||
original_path: `/${username}/Documents/loot3.txt`,
|
||||
trashed_ts: 1700000000,
|
||||
objectKey: victimEntry.uuid,
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const persisted = JSON.parse(moved.metadata!) as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
expect(persisted.objectKey).toBeUndefined();
|
||||
expect(persisted.original_path).toBe(
|
||||
`/${username}/Documents/loot3.txt`,
|
||||
);
|
||||
expect(persisted.trashed_ts).toBe(1700000000);
|
||||
|
||||
const read = await server.services.fs.readContent(moved);
|
||||
expect(await streamToString(read.body)).toBe(attackerDecoy);
|
||||
});
|
||||
});
|
||||
@@ -46,14 +46,6 @@ import {
|
||||
} from './legacyFsHelpers.js';
|
||||
import { RouteOptions } from '../../core/http/index.js';
|
||||
|
||||
/**
|
||||
* Legacy FS routes, implemented as thin shims over `FSService`.
|
||||
*
|
||||
* Each shim parses the request shape (FSNodeParam-style `{ path, uid, id }`
|
||||
* or `{ parent, name }`), invokes the service method, and returns the
|
||||
* snake_case response clients expect.
|
||||
*/
|
||||
|
||||
type RouterCache = Map<string, RequestHandler | null>;
|
||||
|
||||
const additionalRoutePaths: Record<string, string> = {};
|
||||
@@ -1631,17 +1623,6 @@ export class LegacyFSController extends PuterController {
|
||||
download.body.pipe(res);
|
||||
};
|
||||
|
||||
// Helpers for writeFile
|
||||
// -- GUI event emission -------------------------------------------
|
||||
//
|
||||
// Fire-and-forget `outer.gui.item.*` events so SocketService,
|
||||
// BroadcastService, WorkerDriver (hot-reload), and cache-invalidation
|
||||
// listeners pick up mutations made through the legacy (bare-path) routes.
|
||||
// FSController (v2-native /fs/* routes) emits these from its own handlers;
|
||||
// LegacyFSController delegates to the same FSService but needs its
|
||||
// own emissions because the service layer deliberately doesn't emit GUI
|
||||
// events (that's a controller concern).
|
||||
|
||||
async #emitGuiEvent(
|
||||
eventName:
|
||||
| 'outer.gui.item.added'
|
||||
|
||||
@@ -1506,6 +1506,212 @@ describe('PuterAIController.anthropicMessages streaming + helpers', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── Inline compaction ───────────────────────────────────────────────
|
||||
|
||||
describe('PuterAIController inline compaction', () => {
|
||||
// Extract the single `event: compaction\ndata: {...}` SSE frame.
|
||||
const compactionFrame = (out: string): string | null => {
|
||||
const m = out.match(/event: compaction\ndata: [^\n]*\n\n/);
|
||||
return m ? m[0] : null;
|
||||
};
|
||||
|
||||
it('forwards the compaction opt-in to the driver (/responses)', async () => {
|
||||
const spy = stubChatComplete({
|
||||
message: { role: 'assistant', content: 'ok' },
|
||||
finish_reason: 'stop',
|
||||
});
|
||||
await controller.openaiResponses(
|
||||
makeReq({
|
||||
body: { model: 'gpt-test', input: 'hi', compaction: true },
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
makeRes().res,
|
||||
);
|
||||
expect(spy.mock.calls[0]![0].compaction).toBe(true);
|
||||
});
|
||||
|
||||
it('round-trips a compaction `input` item into a messages compaction item', async () => {
|
||||
const spy = stubChatComplete({
|
||||
message: { role: 'assistant', content: 'ack' },
|
||||
finish_reason: 'stop',
|
||||
});
|
||||
await controller.openaiResponses(
|
||||
makeReq({
|
||||
body: {
|
||||
model: 'gpt-test',
|
||||
input: [
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
],
|
||||
},
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
makeRes().res,
|
||||
);
|
||||
expect(spy.mock.calls[0]![0].messages).toContainEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
});
|
||||
});
|
||||
|
||||
it('emits a canonical compaction SSE event and output item (/responses stream)', async () => {
|
||||
stubChatComplete({
|
||||
dataType: 'stream',
|
||||
content_type: 'application/x-ndjson',
|
||||
stream: ndjsonStreamFrom([
|
||||
{ type: 'text', text: 'hi' },
|
||||
{ type: 'compaction', id: 'cmpct_9', encrypted_content: 'ENC9' },
|
||||
{ type: 'usage', usage: { prompt_tokens: 1, completion_tokens: 1 } },
|
||||
]),
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await controller.openaiResponses(
|
||||
makeReq({
|
||||
body: {
|
||||
model: 'gpt-test',
|
||||
input: 'hi',
|
||||
stream: true,
|
||||
compaction: true,
|
||||
},
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
res,
|
||||
);
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
|
||||
const out = captured.written.join('');
|
||||
expect(compactionFrame(out)).toBe(
|
||||
'event: compaction\ndata: {"type":"compaction","id":"cmpct_9","encrypted_content":"ENC9"}\n\n',
|
||||
);
|
||||
// Native shape also lands in the final response.completed output[].
|
||||
const completed = out
|
||||
.split('event: response.completed\n')[1]
|
||||
?.split('\n\n')[0];
|
||||
expect(completed).toContain('"type":"compaction"');
|
||||
expect(completed).toContain('"encrypted_content":"ENC9"');
|
||||
});
|
||||
|
||||
it('emits the compaction item in non-streaming /responses output', async () => {
|
||||
stubChatComplete({
|
||||
message: { role: 'assistant', content: 'done' },
|
||||
finish_reason: 'stop',
|
||||
compaction: { id: 'cmpct_n', encrypted_content: 'ENCN' },
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await controller.openaiResponses(
|
||||
makeReq({
|
||||
body: { model: 'gpt-test', input: 'hi', compaction: true },
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
res,
|
||||
);
|
||||
const body = captured.body as { output: Array<Record<string, unknown>> };
|
||||
expect(body.output).toContainEqual(
|
||||
expect.objectContaining({
|
||||
type: 'compaction',
|
||||
encrypted_content: 'ENCN',
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('emits an identical canonical compaction SSE event on the Anthropic surface', async () => {
|
||||
// /responses frame
|
||||
stubChatComplete({
|
||||
dataType: 'stream',
|
||||
content_type: 'application/x-ndjson',
|
||||
stream: ndjsonStreamFrom([
|
||||
{ type: 'compaction', id: 'cmpct_x', encrypted_content: 'ENCX' },
|
||||
]),
|
||||
});
|
||||
const r1 = makeRes();
|
||||
await controller.openaiResponses(
|
||||
makeReq({
|
||||
body: { model: 'gpt-test', input: 'hi', stream: true },
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
r1.res,
|
||||
);
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
|
||||
// /anthropic/v1/messages frame
|
||||
stubChatComplete({
|
||||
dataType: 'stream',
|
||||
content_type: 'application/x-ndjson',
|
||||
stream: ndjsonStreamFrom([
|
||||
{ type: 'compaction', id: 'cmpct_x', encrypted_content: 'ENCX' },
|
||||
]),
|
||||
});
|
||||
const r2 = makeRes();
|
||||
await controller.anthropicMessages(
|
||||
makeReq({
|
||||
body: {
|
||||
model: 'claude-test',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
stream: true,
|
||||
},
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
r2.res,
|
||||
);
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
|
||||
const a = compactionFrame(r1.captured.written.join(''));
|
||||
const b = compactionFrame(r2.captured.written.join(''));
|
||||
expect(a).not.toBeNull();
|
||||
expect(a).toBe(b); // byte-identical streaming shape across providers
|
||||
});
|
||||
|
||||
it('renders a native compaction content block in non-streaming /messages', async () => {
|
||||
stubChatComplete({
|
||||
message: { role: 'assistant', content: 'done' },
|
||||
finish_reason: 'stop',
|
||||
compaction: { id: 'cmpct_m', encrypted_content: 'ENCM' },
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await controller.anthropicMessages(
|
||||
makeReq({
|
||||
body: {
|
||||
model: 'claude-test',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
},
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
res,
|
||||
);
|
||||
const body = captured.body as { content: Array<Record<string, unknown>> };
|
||||
expect(body.content).toContainEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_m',
|
||||
encrypted_content: 'ENCM',
|
||||
});
|
||||
});
|
||||
|
||||
it('does not emit compaction frames for a normal stream (regression)', async () => {
|
||||
stubChatComplete({
|
||||
dataType: 'stream',
|
||||
content_type: 'application/x-ndjson',
|
||||
stream: ndjsonStreamFrom([
|
||||
{ type: 'text', text: 'hello' },
|
||||
{ type: 'usage', usage: { prompt_tokens: 1, completion_tokens: 1 } },
|
||||
]),
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await controller.openaiResponses(
|
||||
makeReq({
|
||||
body: { model: 'gpt-test', input: 'hi', stream: true },
|
||||
actor: makeUserActor(),
|
||||
}),
|
||||
res,
|
||||
);
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
expect(captured.written.join('')).not.toContain('event: compaction');
|
||||
});
|
||||
});
|
||||
|
||||
// ── Model details listing ───────────────────────────────────────────
|
||||
|
||||
describe('PuterAIController model details', () => {
|
||||
|
||||
@@ -587,6 +587,15 @@ export class PuterAIController extends PuterController {
|
||||
? { metadata: body.metadata as Record<string, string> }
|
||||
: {}),
|
||||
...(body.conversation ? { conversation: body.conversation } : {}),
|
||||
...(body.context_management !== undefined
|
||||
? { context_management: body.context_management }
|
||||
: {}),
|
||||
...(body.compaction !== undefined
|
||||
? {
|
||||
compaction:
|
||||
body.compaction as ICompleteArguments['compaction'],
|
||||
}
|
||||
: {}),
|
||||
...(body.previous_response_id
|
||||
? { previous_response_id: String(body.previous_response_id) }
|
||||
: {}),
|
||||
@@ -750,6 +759,30 @@ export class PuterAIController extends PuterController {
|
||||
output_index: outputIndex,
|
||||
item,
|
||||
});
|
||||
} else if (ev.type === 'compaction') {
|
||||
// Native shape in the final `output[]`, plus the
|
||||
// canonical SSE event shared with the Anthropic surface.
|
||||
const item = {
|
||||
type: 'compaction',
|
||||
...(ev.id !== undefined ? { id: ev.id } : {}),
|
||||
encrypted_content: ev.encrypted_content,
|
||||
};
|
||||
const outputIndex = output.length;
|
||||
output.push(item);
|
||||
sendEvent({
|
||||
type: 'response.output_item.added',
|
||||
output_index: outputIndex,
|
||||
item,
|
||||
});
|
||||
sendEvent({
|
||||
type: 'response.output_item.done',
|
||||
output_index: outputIndex,
|
||||
item,
|
||||
});
|
||||
writeCompactionEvent(res, {
|
||||
id: ev.id,
|
||||
encrypted_content: ev.encrypted_content,
|
||||
});
|
||||
} else if (ev.type === 'usage') {
|
||||
usage = buildResponsesUsage(
|
||||
ev.usage as Record<string, unknown>,
|
||||
@@ -869,6 +902,15 @@ export class PuterAIController extends PuterController {
|
||||
...(body.max_tokens !== undefined
|
||||
? { max_tokens: Number(body.max_tokens) }
|
||||
: {}),
|
||||
...(body.context_management !== undefined
|
||||
? { context_management: body.context_management }
|
||||
: {}),
|
||||
...(body.compaction !== undefined
|
||||
? {
|
||||
compaction:
|
||||
body.compaction as ICompleteArguments['compaction'],
|
||||
}
|
||||
: {}),
|
||||
...(body.provider
|
||||
? { provider: toStringOrEmpty(body.provider) }
|
||||
: { provider: DEFAULTS.anthropic }),
|
||||
@@ -968,6 +1010,14 @@ export class PuterAIController extends PuterController {
|
||||
},
|
||||
});
|
||||
closeBlock();
|
||||
} else if (ev.type === 'compaction') {
|
||||
// Close any open content block, then emit the canonical
|
||||
// compaction SSE event — byte-identical to /responses.
|
||||
closeBlock();
|
||||
writeCompactionEvent(res, {
|
||||
id: ev.id,
|
||||
encrypted_content: ev.encrypted_content,
|
||||
});
|
||||
} else if (ev.type === 'usage') {
|
||||
usage = ev.usage as Record<string, unknown>;
|
||||
}
|
||||
@@ -1026,6 +1076,18 @@ export class PuterAIController extends PuterController {
|
||||
if (textContent)
|
||||
contentBlocks.push({ type: 'text', text: textContent });
|
||||
contentBlocks.push(...toolUseBlocks);
|
||||
// Native Anthropic-shaped compaction block (non-streaming bodies stay
|
||||
// provider-native, unlike the unified streaming event).
|
||||
const compaction = (
|
||||
messageResult as { compaction?: Record<string, unknown> }
|
||||
).compaction;
|
||||
if (compaction) {
|
||||
contentBlocks.push({
|
||||
type: 'compaction',
|
||||
...(compaction.id !== undefined ? { id: compaction.id } : {}),
|
||||
encrypted_content: compaction.encrypted_content,
|
||||
});
|
||||
}
|
||||
if (contentBlocks.length === 0)
|
||||
contentBlocks.push({ type: 'text', text: '' });
|
||||
|
||||
@@ -1082,6 +1144,25 @@ const setSseHeaders = (res: Response): void => {
|
||||
res.setHeader('Connection', 'keep-alive');
|
||||
};
|
||||
|
||||
/**
|
||||
* Inline-compaction is emitted in a single canonical SSE shape that is
|
||||
* byte-identical across the `/responses` and `/anthropic/v1/messages` streaming
|
||||
* surfaces, so a streaming client parses compaction the same way regardless of
|
||||
* which upstream served the request. (Non-streaming bodies stay provider-native.)
|
||||
*/
|
||||
const writeCompactionEvent = (
|
||||
res: Response,
|
||||
compaction: { id?: unknown; encrypted_content?: unknown },
|
||||
): void => {
|
||||
const payload = {
|
||||
type: 'compaction',
|
||||
...(compaction.id !== undefined ? { id: compaction.id } : {}),
|
||||
encrypted_content: compaction.encrypted_content,
|
||||
};
|
||||
res.write(`event: compaction\n`);
|
||||
res.write(`data: ${JSON.stringify(payload)}\n\n`);
|
||||
};
|
||||
|
||||
/**
|
||||
* The chat driver returns either a stream-result envelope or a plain
|
||||
* message result. Proxy routes invoked with `stream: true` expect the
|
||||
@@ -1325,6 +1406,17 @@ const responseInputToMessages = (input: unknown): unknown[] => {
|
||||
if (!item || typeof item !== 'object') continue;
|
||||
const it = item as Record<string, unknown>;
|
||||
|
||||
if (it.type === 'compaction') {
|
||||
// Round-tripped compaction artifact: preserve it as a bare item so
|
||||
// `normalize_single_message` wraps it into an internal compaction
|
||||
// content block (providers map it back to their native input shape).
|
||||
messages.push({
|
||||
type: 'compaction',
|
||||
...(it.id !== undefined ? { id: it.id } : {}),
|
||||
encrypted_content: it.encrypted_content,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
if (it.type === 'function_call_output') {
|
||||
messages.push({
|
||||
role: 'tool',
|
||||
@@ -1416,6 +1508,16 @@ const responseOutputFromResult = (
|
||||
});
|
||||
}
|
||||
|
||||
const compaction = (result as { compaction?: Record<string, unknown> })
|
||||
.compaction;
|
||||
if (compaction) {
|
||||
output.push({
|
||||
id: (compaction.id as string | undefined) || generateId('cmpct'),
|
||||
type: 'compaction',
|
||||
encrypted_content: compaction.encrypted_content,
|
||||
});
|
||||
}
|
||||
|
||||
return output;
|
||||
};
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ import type { Request, Response } from 'express';
|
||||
import { Readable } from 'node:stream';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { hash as bcryptHash } from 'bcrypt';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
@@ -228,6 +229,112 @@ describe('WebDAVController', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('pending-verification gate', () => {
|
||||
// WebDAV must enforce the same gate every other authenticated route
|
||||
// gets from requireVerifiedAccount — it dispatches off a single use()
|
||||
// with no route options, so the middleware is never wired in and it
|
||||
// has to call assertVerifiedAccount itself. Without it, an account
|
||||
// still pending email/phone/card verification could read/write its
|
||||
// whole filesystem over the `dav` subdomain.
|
||||
const gatedActor = (flags: Record<string, unknown>) => ({
|
||||
user: {
|
||||
id: 1,
|
||||
uuid: 'gated-uuid',
|
||||
username: 'gated',
|
||||
...flags,
|
||||
},
|
||||
});
|
||||
|
||||
it('rejects a session actor pending phone verification with 403', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'PROPFIND',
|
||||
actor: gatedActor({ requires_phone_verification: true }),
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
expect(captured.statusCode).toBe(403);
|
||||
});
|
||||
|
||||
it('rejects a session actor pending card verification with 403', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'GET',
|
||||
actor: gatedActor({ requires_card_verification: true }),
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
expect(captured.statusCode).toBe(403);
|
||||
});
|
||||
|
||||
it('rejects a session actor with an unconfirmed email with 403', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'PROPFIND',
|
||||
actor: gatedActor({
|
||||
requires_email_confirmation: true,
|
||||
email_confirmed: false,
|
||||
}),
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
expect(captured.statusCode).toBe(403);
|
||||
});
|
||||
|
||||
it('enforces the gate on the Basic-auth path (flags carried onto the built actor)', async () => {
|
||||
const username = `webdav-gated-${Math.random()
|
||||
.toString(36)
|
||||
.slice(2, 10)}`;
|
||||
const created = await server.stores.user.create({
|
||||
username,
|
||||
uuid: uuidv4(),
|
||||
password: await bcryptHash('correct-horse', 4),
|
||||
email: `${username}@test.local`,
|
||||
free_storage: 100 * 1024 * 1024,
|
||||
requires_email_confirmation: false,
|
||||
});
|
||||
await server.stores.user.update(created.id, {
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'PROPFIND',
|
||||
headers: {
|
||||
authorization: basicAuth(username, 'correct-horse'),
|
||||
},
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
expect(captured.statusCode).toBe(403);
|
||||
});
|
||||
|
||||
it('lets a fully-verified session actor through the gate', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'OPTIONS',
|
||||
actor: gatedActor({
|
||||
requires_phone_verification: false,
|
||||
requires_card_verification: false,
|
||||
requires_email_confirmation: false,
|
||||
}),
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
expect(captured.statusCode).toBe(200);
|
||||
});
|
||||
});
|
||||
|
||||
describe('unsupported methods', () => {
|
||||
it('returns 405 for unknown HTTP methods', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
|
||||
@@ -23,6 +23,7 @@ import { posix as pathPosix } from 'node:path';
|
||||
import { EventMap } from '../../clients/event/types.js';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { assertVerifiedAccount } from '../../core/http/middleware/gates.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
|
||||
import { expandTildePath } from '../../services/fs/resolveNode.js';
|
||||
@@ -89,6 +90,16 @@ export class WebDAVController extends PuterController {
|
||||
const actor = await this.#resolveActor(req, res);
|
||||
if (!actor) return; // 401 already sent
|
||||
|
||||
// Apply the same pending-verification gate every other authenticated
|
||||
// route gets from `requireVerifiedAccount`. WebDAV dispatches every
|
||||
// method off a single `router.use` with no route options, so that
|
||||
// middleware is never inserted into its chain — without this call an
|
||||
// account still pending email / phone / card verification could read,
|
||||
// write, and delete its entire filesystem over the `dav` subdomain,
|
||||
// bypassing the gate. Throws a 403 HttpError, surfaced by the catch in
|
||||
// registerRoutes.
|
||||
assertVerifiedAccount(actor.user);
|
||||
|
||||
// Expand `~`/`~/...` against the authenticated actor's username.
|
||||
// WebDAV doesn't standardize `~`, but some clients do — and the
|
||||
// pre-existing behaviour silently expanded it via the FS store.
|
||||
@@ -233,6 +244,15 @@ export class WebDAVController extends PuterController {
|
||||
email_confirmed: user.email_confirmed ?? false,
|
||||
requires_email_confirmation:
|
||||
user.requires_email_confirmation ?? false,
|
||||
// Carry the signup-time verification flags so the
|
||||
// assertVerifiedAccount() gate in #dispatch can see them on
|
||||
// the Basic-auth path too (the cookie / `-token` paths get
|
||||
// them from AuthService#actorUserFromRow). Omitting these is
|
||||
// what let phone/card-gated accounts through WebDAV.
|
||||
requires_phone_verification:
|
||||
user.requires_phone_verification ?? false,
|
||||
requires_card_verification:
|
||||
user.requires_card_verification ?? false,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -66,6 +66,21 @@ declare global {
|
||||
/** True when the request's Host is a custom domain (not one of the configured Puter domains). */
|
||||
is_custom_domain?: boolean;
|
||||
|
||||
/**
|
||||
* Coarse server-derived request fingerprint (IP + UA + accept
|
||||
* headers), populated by the global fingerprint middleware. Always
|
||||
* present; the same value the rate limiter keys on.
|
||||
*/
|
||||
networkFingerprint?: string;
|
||||
|
||||
/**
|
||||
* Client-supplied device fingerprint (ThumbmarkJS hash) from the
|
||||
* body or `x-puter-device-fingerprint` header, populated by the
|
||||
* global fingerprint middleware. Present only when the client sent a
|
||||
* well-shaped value; spoofable but stable per device across IPs.
|
||||
*/
|
||||
deviceFingerprint?: string;
|
||||
|
||||
/** Parsed cookies, populated by the global `cookie-parser` middleware. */
|
||||
cookies?: Record<string, string>;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { createFingerprintMiddleware } from './fingerprint';
|
||||
|
||||
const VALID_FP = 'dev-fingerprint-abc123';
|
||||
|
||||
const run = (reqPartial: Partial<Request> & Record<string, unknown>) => {
|
||||
const middleware = createFingerprintMiddleware();
|
||||
const req = {
|
||||
headers: {},
|
||||
...reqPartial,
|
||||
} as unknown as Request;
|
||||
const next = vi.fn();
|
||||
middleware(req, {} as Response, next);
|
||||
return { req, next };
|
||||
};
|
||||
|
||||
describe('createFingerprintMiddleware', () => {
|
||||
it('always sets a well-formed networkFingerprint and calls next', () => {
|
||||
const { req, next } = run({ ip: '203.0.113.1' });
|
||||
|
||||
expect(req.networkFingerprint).toMatch(/^[A-Za-z0-9_-]{16}$/);
|
||||
expect(next).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('networkFingerprint is stable for identical inputs and varies by UA', () => {
|
||||
const headers = { 'user-agent': 'UA/1.0' };
|
||||
const a = run({ ip: '203.0.113.1', headers }).req.networkFingerprint;
|
||||
const b = run({ ip: '203.0.113.1', headers }).req.networkFingerprint;
|
||||
const c = run({
|
||||
ip: '203.0.113.1',
|
||||
headers: { 'user-agent': 'UA/2.0' },
|
||||
}).req.networkFingerprint;
|
||||
|
||||
expect(a).toBe(b);
|
||||
expect(a).not.toBe(c);
|
||||
});
|
||||
|
||||
it('reads a well-shaped device fingerprint from the body', () => {
|
||||
const { req } = run({ body: { fingerprint: VALID_FP } });
|
||||
expect(req.deviceFingerprint).toBe(VALID_FP);
|
||||
});
|
||||
|
||||
it('falls back to the x-puter-device-fingerprint header', () => {
|
||||
const { req } = run({
|
||||
headers: { 'x-puter-device-fingerprint': VALID_FP },
|
||||
});
|
||||
expect(req.deviceFingerprint).toBe(VALID_FP);
|
||||
});
|
||||
|
||||
it('prefers the body fingerprint over the header', () => {
|
||||
const { req } = run({
|
||||
body: { fingerprint: VALID_FP },
|
||||
headers: { 'x-puter-device-fingerprint': 'other-fingerprint-xyz' },
|
||||
});
|
||||
expect(req.deviceFingerprint).toBe(VALID_FP);
|
||||
});
|
||||
|
||||
it('drops a malformed fingerprint to undefined', () => {
|
||||
expect(run({ body: { fingerprint: 'bad fp!' } }).req.deviceFingerprint)
|
||||
.toBeUndefined();
|
||||
expect(run({ body: { fingerprint: 'short' } }).req.deviceFingerprint)
|
||||
.toBeUndefined();
|
||||
expect(
|
||||
run({ body: { fingerprint: 123 as unknown as string } }).req
|
||||
.deviceFingerprint,
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('leaves deviceFingerprint undefined when nothing was supplied', () => {
|
||||
const { req } = run({ ip: '203.0.113.1' });
|
||||
expect(req.deviceFingerprint).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,76 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { RequestHandler } from 'express';
|
||||
import { computeNetworkFingerprint } from './rateLimit.js';
|
||||
import '../expressAugmentation';
|
||||
|
||||
/**
|
||||
* Conservative charset/length for the client-supplied device fingerprint. It
|
||||
* becomes part of KV keys downstream (abuse extension), so anything outside
|
||||
* this shape is dropped to `undefined` rather than trusted. Kept in sync with
|
||||
* the abuse extension's `DEVICE_FINGERPRINT_SHAPE`; the core only enforces the
|
||||
* shape, never any abuse policy built on the value.
|
||||
*/
|
||||
const DEVICE_FINGERPRINT_SHAPE = /^[A-Za-z0-9._-]{8,128}$/;
|
||||
|
||||
/** Header the GUI may send the device fingerprint on for non-signup requests. */
|
||||
const DEVICE_FINGERPRINT_HEADER = 'x-puter-device-fingerprint';
|
||||
|
||||
function readDeviceFingerprint(req: {
|
||||
body?: unknown;
|
||||
headers?: Record<string, unknown>;
|
||||
}): string | undefined {
|
||||
// Body first (what /signup already sends), then a header fallback so
|
||||
// authenticated, bodyless-or-different-shape requests can still carry it.
|
||||
const body = req.body as { fingerprint?: unknown } | undefined;
|
||||
const candidate =
|
||||
(typeof body?.fingerprint === 'string'
|
||||
? body.fingerprint
|
||||
: undefined) ??
|
||||
(typeof req.headers?.[DEVICE_FINGERPRINT_HEADER] === 'string'
|
||||
? (req.headers[DEVICE_FINGERPRINT_HEADER] as string)
|
||||
: undefined);
|
||||
if (candidate && DEVICE_FINGERPRINT_SHAPE.test(candidate)) return candidate;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stamp request-scoped fingerprints onto `req` so any downstream gate, handler,
|
||||
* or service (via the ALS `Context.get('req')`) can read them without recomputing:
|
||||
*
|
||||
* - `req.networkFingerprint` — always set; a coarse IP+headers hash (the same
|
||||
* value the rate limiter keys on). Server-derived, so it can't be forged
|
||||
* away, but it's coarse (shared behind NAT/VPN, rotates with UA).
|
||||
* - `req.deviceFingerprint` — set only when the client supplied a well-shaped
|
||||
* device fingerprint (ThumbmarkJS hash) in the body or the
|
||||
* `x-puter-device-fingerprint` header; `undefined` otherwise. Client-supplied
|
||||
* and spoofable, but stable per real device across IP rotation.
|
||||
*
|
||||
* Install AFTER the body parsers (so the body fingerprint is readable) and
|
||||
* before `requestContext` (so the snapshot into ALS already carries them).
|
||||
* Never rejects — a missing/invalid device fingerprint is simply absent.
|
||||
*/
|
||||
export const createFingerprintMiddleware = (): RequestHandler => {
|
||||
return (req, _res, next) => {
|
||||
req.networkFingerprint = computeNetworkFingerprint(req);
|
||||
req.deviceFingerprint = readDeviceFingerprint(req);
|
||||
next();
|
||||
};
|
||||
};
|
||||
@@ -25,7 +25,7 @@ import {
|
||||
adminOnlyGate,
|
||||
allowedAppIdsGate,
|
||||
requireAuthGate,
|
||||
requireEmailConfirmedGate,
|
||||
requireVerifiedAccount,
|
||||
requireNonAccessTokenGate,
|
||||
requireUserActorGate,
|
||||
requireVerifiedGate,
|
||||
@@ -440,11 +440,11 @@ describe('requireVerifiedGate', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── requireEmailConfirmedGate ───────────────────────────────────────
|
||||
// ── requireVerifiedAccount ──────────────────────────────────────────
|
||||
|
||||
describe('requireEmailConfirmedGate', () => {
|
||||
describe('requireVerifiedAccount', () => {
|
||||
it('passes through users that do not require confirmation (e.g. legacy/temp)', () => {
|
||||
const got = runGate(requireEmailConfirmedGate(), {
|
||||
const got = runGate(requireVerifiedAccount(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
@@ -457,7 +457,7 @@ describe('requireEmailConfirmedGate', () => {
|
||||
});
|
||||
|
||||
it('passes through confirmed users even when confirmation is required', () => {
|
||||
const got = runGate(requireEmailConfirmedGate(), {
|
||||
const got = runGate(requireVerifiedAccount(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
@@ -470,7 +470,7 @@ describe('requireEmailConfirmedGate', () => {
|
||||
});
|
||||
|
||||
it('returns 403 email_confirmation_required for pending-confirmation users', () => {
|
||||
const got = runGate(requireEmailConfirmedGate(), {
|
||||
const got = runGate(requireVerifiedAccount(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
@@ -482,8 +482,51 @@ describe('requireEmailConfirmedGate', () => {
|
||||
expectHttpError(got, 403, 'email_confirmation_required');
|
||||
});
|
||||
|
||||
it('returns 403 phone_verification_required while the phone gate is set', () => {
|
||||
const got = runGate(requireVerifiedAccount(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
requires_email_confirmation: false,
|
||||
email_confirmed: true,
|
||||
requires_phone_verification: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
expectHttpError(got, 403, 'phone_verification_required');
|
||||
});
|
||||
|
||||
it('returns 403 card_verification_required while the card gate is set', () => {
|
||||
const got = runGate(requireVerifiedAccount(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
requires_email_confirmation: false,
|
||||
email_confirmed: true,
|
||||
requires_card_verification: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
expectHttpError(got, 403, 'card_verification_required');
|
||||
});
|
||||
|
||||
it('passes through once every gate is cleared', () => {
|
||||
const got = runGate(requireVerifiedAccount(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
requires_email_confirmation: true,
|
||||
email_confirmed: true,
|
||||
requires_phone_verification: false,
|
||||
requires_card_verification: false,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(got).toBeUndefined();
|
||||
});
|
||||
|
||||
it("passes through when there's no actor (auth gate handled it)", () => {
|
||||
const got = runGate(requireEmailConfirmedGate(), {});
|
||||
const got = runGate(requireVerifiedAccount(), {});
|
||||
expect(got).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -224,29 +224,84 @@ export const requireVerifiedGate = (strictFlag: boolean): RequestHandler => {
|
||||
};
|
||||
|
||||
/**
|
||||
* Reject authenticated users whose account is pending email confirmation
|
||||
* (`requires_email_confirmation && !email_confirmed`). Runs on every
|
||||
* authenticated route by default; routes that set `allowUnconfirmed: true`
|
||||
* skip this gate.
|
||||
* Reject authenticated users whose account is still pending any signup-time
|
||||
* verification — email confirmation, SMS phone verification, or credit-card
|
||||
* verification. The abuse harness sets the phone/card flags on low-reputation
|
||||
* signups (in place of a hard block), and this gate is what actually keeps
|
||||
* those accounts out of the product until the flag clears: the flags live on
|
||||
* `req.actor.user`, so every authenticated route enforces them, not just the
|
||||
* GUI modal.
|
||||
*
|
||||
* Returns 403 with `email_confirmation_required` so clients can show the
|
||||
* confirmation prompt instead of a generic error.
|
||||
* Runs on every authenticated route by default; routes that set
|
||||
* `allowUnconfirmed: true` opt out (the verification endpoints themselves,
|
||||
* plus essential flows like whoami / logout / save-account so a pending
|
||||
* account can still reach the screens that clear the gate).
|
||||
*
|
||||
* Returns 403 with a per-gate legacy code (`email_confirmation_required` /
|
||||
* `phone_verification_required` / `card_verification_required`) so clients can
|
||||
* show the right prompt instead of a generic error. There is no state where a
|
||||
* user should be allowed in with one verification pending, so any pending gate
|
||||
* rejects.
|
||||
*/
|
||||
export const requireEmailConfirmedGate = (): RequestHandler => {
|
||||
export const requireVerifiedAccount = (): RequestHandler => {
|
||||
return (req, _res, next) => {
|
||||
const user = req.actor?.user;
|
||||
if (user?.requires_email_confirmation && !user?.email_confirmed) {
|
||||
next(
|
||||
new HttpError(403, 'Please confirm your email to continue', {
|
||||
legacyCode: 'email_confirmation_required',
|
||||
}),
|
||||
);
|
||||
try {
|
||||
assertVerifiedAccount(req.actor?.user);
|
||||
} catch (err) {
|
||||
next(err);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* The pending-verification check, factored out of {@link requireVerifiedAccount}
|
||||
* so auth paths that build their own actor outside the route-option machinery
|
||||
* can enforce the exact same gate. The WebDAV controller is the motivating
|
||||
* case: it dispatches every method off a single `router.use`, so
|
||||
* `requireVerifiedAccount` is never wired into its chain — it has to call this
|
||||
* directly. Keeping one implementation is the point: a verification gate added
|
||||
* here is picked up by every caller, so the paths can't drift (which is how
|
||||
* WebDAV came to bypass the phone/card gate to begin with).
|
||||
*
|
||||
* Throws 403 with a per-gate legacy code (`email_confirmation_required` /
|
||||
* `phone_verification_required` / `card_verification_required`) so clients can
|
||||
* show the right prompt instead of a generic error. There is no state where a
|
||||
* user should be let in with any verification pending, so the first pending
|
||||
* gate rejects.
|
||||
*/
|
||||
export const assertVerifiedAccount = (
|
||||
user:
|
||||
| {
|
||||
requires_email_confirmation?: unknown;
|
||||
email_confirmed?: unknown;
|
||||
requires_phone_verification?: unknown;
|
||||
requires_card_verification?: unknown;
|
||||
}
|
||||
| undefined,
|
||||
): void => {
|
||||
if (user?.requires_email_confirmation && !user?.email_confirmed) {
|
||||
throw new HttpError(403, 'Please confirm your email to continue', {
|
||||
legacyCode: 'email_confirmation_required',
|
||||
});
|
||||
}
|
||||
if (user?.requires_phone_verification) {
|
||||
throw new HttpError(
|
||||
403,
|
||||
'Please verify your phone number to continue',
|
||||
{
|
||||
legacyCode: 'phone_verification_required' as never,
|
||||
},
|
||||
);
|
||||
}
|
||||
if (user?.requires_card_verification) {
|
||||
throw new HttpError(403, 'Please verify your card to continue', {
|
||||
legacyCode: 'card_verification_required' as never,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Reject unless the actor is acting through one of the named apps.
|
||||
* App-under-user actors are permitted iff `actor.app.uid` is in the allowList;
|
||||
|
||||
@@ -264,6 +264,79 @@ describe('createUserSubdomainRedirect', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
const selfHosted = {
|
||||
domain: 'puter.localhost',
|
||||
static_hosting_domain: 'site.puter.localhost',
|
||||
static_hosting_domain_alt: 'host.puter.localhost',
|
||||
private_app_hosting_domain: 'app.puter.localhost',
|
||||
private_app_hosting_domain_alt: 'dev.puter.localhost',
|
||||
} as IConfig;
|
||||
|
||||
it('still redirects a bare subdomain on the main domain to the hosting domain (self-hosted)', () => {
|
||||
const { out, next } = run(
|
||||
createUserSubdomainRedirect(selfHosted),
|
||||
makeReq({
|
||||
subdomains: ['localhost', 'puter', 'foo'],
|
||||
host: 'foo.puter.localhost',
|
||||
originalUrl: '/bar?x=1',
|
||||
protocol: 'http',
|
||||
}),
|
||||
);
|
||||
expect(out.redirectArgs).toEqual([
|
||||
302,
|
||||
'http://foo.site.puter.localhost/bar?x=1',
|
||||
]);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('passes through hosts already on the static hosting domain (no redirect loop)', () => {
|
||||
const { out, next } = run(
|
||||
createUserSubdomainRedirect(selfHosted),
|
||||
makeReq({
|
||||
subdomains: ['localhost', 'puter', 'site', 'foo'],
|
||||
host: 'foo.site.puter.localhost',
|
||||
originalUrl: '/',
|
||||
}),
|
||||
);
|
||||
expect(out.redirectArgs).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('passes through hosts on the alt / private-app hosting domains too', () => {
|
||||
for (const host of [
|
||||
'foo.host.puter.localhost',
|
||||
'foo.app.puter.localhost',
|
||||
'foo.dev.puter.localhost',
|
||||
]) {
|
||||
const { out, next } = run(
|
||||
createUserSubdomainRedirect(selfHosted),
|
||||
makeReq({
|
||||
subdomains: [
|
||||
'localhost',
|
||||
'puter',
|
||||
host.split('.')[1],
|
||||
'foo',
|
||||
],
|
||||
host,
|
||||
}),
|
||||
);
|
||||
expect(out.redirectArgs).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
}
|
||||
});
|
||||
|
||||
it('passes through the hosting-domain root itself (exact match, no loop)', () => {
|
||||
const { out, next } = run(
|
||||
createUserSubdomainRedirect(selfHosted),
|
||||
makeReq({
|
||||
subdomains: ['localhost', 'puter', 'site'],
|
||||
host: 'site.puter.localhost',
|
||||
}),
|
||||
);
|
||||
expect(out.redirectArgs).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("passes through when the request host has a port the configured domain doesn't", () => {
|
||||
// Edge case worth pinning: the suffix check is exact-`endsWith`,
|
||||
// so a port mismatch silently bypasses the redirect. Documenting
|
||||
|
||||
@@ -88,6 +88,15 @@ export const createUserSubdomainRedirect = (
|
||||
if (!domain || !target) {
|
||||
return (_req, _res, next) => next();
|
||||
}
|
||||
|
||||
const hostingDomains = [
|
||||
config.static_hosting_domain,
|
||||
config.static_hosting_domain_alt,
|
||||
config.private_app_hosting_domain,
|
||||
config.private_app_hosting_domain_alt,
|
||||
]
|
||||
.map((d) => (d ?? '').toLowerCase().split(':')[0])
|
||||
.filter((d) => d.length > 0);
|
||||
return (req, res, next) => {
|
||||
const active = (
|
||||
req.subdomains?.[req.subdomains.length - 1] ?? ''
|
||||
@@ -95,6 +104,14 @@ export const createUserSubdomainRedirect = (
|
||||
if (active === '' || RESERVED_SUBDOMAINS.has(active)) return next();
|
||||
|
||||
const host = (req.headers.host ?? '').toLowerCase();
|
||||
const hostName = host.split(':')[0];
|
||||
if (
|
||||
hostingDomains.some(
|
||||
(d) => hostName === d || hostName.endsWith(`.${d}`),
|
||||
)
|
||||
) {
|
||||
return next();
|
||||
}
|
||||
if (!host.endsWith(domain)) return next();
|
||||
|
||||
// host ends in domain — swap the domain suffix for the hosting one,
|
||||
|
||||
@@ -534,6 +534,51 @@ describe('createPuterSiteMiddleware — file serving', () => {
|
||||
expect(piped!.equals(body)).toBe(true);
|
||||
});
|
||||
|
||||
it('emits site.htmlServed with the original URL target including query string', async () => {
|
||||
const owner = await makeUserWithHome();
|
||||
const homePath = `/${owner.username}`;
|
||||
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
|
||||
const sub = `serveq-${Math.random().toString(36).slice(2, 8)}`;
|
||||
await server.stores.subdomain.create({
|
||||
userId: owner.id,
|
||||
subdomain: sub,
|
||||
rootDirId: homeEntry!.id,
|
||||
});
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/index.html`,
|
||||
Buffer.from('<html>hi</html>'),
|
||||
'text/html',
|
||||
);
|
||||
|
||||
const emitSpy = vi.spyOn(server.clients.event, 'emit');
|
||||
try {
|
||||
const mw = buildMiddleware();
|
||||
const { res } = makeRes();
|
||||
await mw(
|
||||
makeReq({
|
||||
hostname: `${sub}.site.puter.localhost`,
|
||||
path: '/index.html',
|
||||
originalUrl: '/index.html?brand=paypal',
|
||||
}),
|
||||
res,
|
||||
vi.fn(),
|
||||
);
|
||||
|
||||
const htmlServedCall = emitSpy.mock.calls.find(
|
||||
(call) => call[0] === 'site.htmlServed',
|
||||
);
|
||||
expect(htmlServedCall?.[1]).toMatchObject({
|
||||
subdomain: sub,
|
||||
requestPath: '/index.html',
|
||||
requestUrl: '/index.html?brand=paypal',
|
||||
mime: 'text/html',
|
||||
});
|
||||
} finally {
|
||||
emitSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('returns 206 status when a Range header is supplied', async () => {
|
||||
const owner = await makeUserWithHome();
|
||||
const homePath = `/${owner.username}`;
|
||||
|
||||
@@ -555,8 +555,12 @@ export const createPuterSiteMiddleware = (
|
||||
res.setHeader('Content-Type', mime);
|
||||
|
||||
// Fire-and-forget signal for downstream extensions
|
||||
if (mime === 'text/html' || mime === 'application/xhtml+xml') {
|
||||
const mimeBase = mime.split(';', 1)[0].trim().toLowerCase();
|
||||
if (mimeBase === 'text/html' || mimeBase === 'application/xhtml+xml') {
|
||||
try {
|
||||
const requestUrl = (req.originalUrl || '/').startsWith('/')
|
||||
? req.originalUrl || '/'
|
||||
: `/${req.originalUrl}`;
|
||||
layers.clients.event.emit(
|
||||
'site.htmlServed',
|
||||
{
|
||||
@@ -564,7 +568,8 @@ export const createPuterSiteMiddleware = (
|
||||
entry,
|
||||
host,
|
||||
requestPath: req.path,
|
||||
mime,
|
||||
requestUrl,
|
||||
mime: mimeBase,
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
@@ -392,7 +392,14 @@ function ip(req) {
|
||||
return req.ip || req.socket?.remoteAddress || 'unknown';
|
||||
}
|
||||
|
||||
function fingerprint(req) {
|
||||
/**
|
||||
* A coarse network fingerprint for a request: a short hash of the (proxy-aware)
|
||||
* IP plus the headers a client can't trivially vary per-request without also
|
||||
* changing how the request looks. Used as the default rate-limit key here, and
|
||||
* exported so the global fingerprint middleware can stamp the identical value
|
||||
* on `req.networkFingerprint` (one key space shared by both).
|
||||
*/
|
||||
export function computeNetworkFingerprint(req) {
|
||||
const parts = [
|
||||
ip(req),
|
||||
req.headers?.['user-agent'] || '',
|
||||
@@ -406,6 +413,10 @@ function fingerprint(req) {
|
||||
.slice(0, 16);
|
||||
}
|
||||
|
||||
function fingerprint(req) {
|
||||
return computeNetworkFingerprint(req);
|
||||
}
|
||||
|
||||
// -- Route middleware ------------------------------------------------
|
||||
|
||||
/**
|
||||
|
||||
@@ -26,6 +26,10 @@ import type { MeteringService } from '../../../../services/metering/MeteringServ
|
||||
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
|
||||
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
|
||||
import type { IChatProvider, ICompleteArguments } from '../../types.js';
|
||||
import {
|
||||
messagesHaveCompaction,
|
||||
wantsCompaction,
|
||||
} from '../../utils/compaction.js';
|
||||
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import { processPuterPathUploads } from '../openai/fileUpload.js';
|
||||
import { AZURE_MODELS } from './models.js';
|
||||
@@ -141,6 +145,20 @@ export class AzureChatProvider implements IChatProvider {
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Inline compaction is a Responses-API feature; chat.completions can't
|
||||
// express `context_management` or a `compaction` content block.
|
||||
// Delegate to the Responses provider when the caller opted in OR when
|
||||
// the messages carry a round-tripped compaction artifact.
|
||||
if (wantsCompaction(params) || messagesHaveCompaction(messages)) {
|
||||
if (!this.#responsesProvider) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'compaction requires the OpenAI Responses provider, which is not configured',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new HttpError(400, '`messages` must be an array', {
|
||||
|
||||
@@ -25,6 +25,7 @@ import type { MeteringService } from '../../../../services/metering/MeteringServ
|
||||
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
|
||||
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
|
||||
import type { IChatProvider, ICompleteArguments } from '../../types.js';
|
||||
import { toOpenAiContextManagement } from '../../utils/compaction.js';
|
||||
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import { processPuterPathUploads } from '../openai/fileUpload.js';
|
||||
import { AZURE_MODELS } from './models.js';
|
||||
@@ -109,6 +110,8 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
@@ -182,6 +185,13 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
const supportsReasoningControls =
|
||||
typeof model === 'string' && model.startsWith('gpt-5');
|
||||
|
||||
// Translate the neutral compaction opt-in (or pass a raw
|
||||
// `context_management` payload through) to OpenAI's Responses shape.
|
||||
const contextManagement = toOpenAiContextManagement({
|
||||
compaction,
|
||||
context_management,
|
||||
});
|
||||
|
||||
const completionParams: ResponseCreateParams = {
|
||||
user: userIdentifier,
|
||||
safety_identifier: userIdentifier,
|
||||
@@ -193,6 +203,9 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
? { parallel_tool_calls }
|
||||
: {}),
|
||||
...(include !== undefined ? { include } : {}),
|
||||
...(contextManagement !== undefined
|
||||
? { context_management: contextManagement }
|
||||
: {}),
|
||||
...(conversation !== undefined ? { conversation } : {}),
|
||||
...(previous_response_id !== undefined
|
||||
? { previous_response_id }
|
||||
|
||||
@@ -486,6 +486,52 @@ describe('ClaudeProvider.complete non-stream output', () => {
|
||||
10 * Number(haiku.costs.cache_read_input_tokens),
|
||||
);
|
||||
});
|
||||
|
||||
it('bills the compaction pass by summing usage.iterations', async () => {
|
||||
const { provider } = makeProvider();
|
||||
// Per Anthropic: top-level input/output reflect only the message pass;
|
||||
// the compaction pass lives in `iterations` and must be summed to bill.
|
||||
const msg = {
|
||||
content: [{ type: 'text', text: 'done' }],
|
||||
usage: {
|
||||
input_tokens: 23000, // message pass only (NOT the total)
|
||||
output_tokens: 1000,
|
||||
iterations: [
|
||||
{
|
||||
type: 'compaction',
|
||||
input_tokens: 180000,
|
||||
output_tokens: 3500,
|
||||
},
|
||||
{ type: 'message', input_tokens: 23000, output_tokens: 1000 },
|
||||
],
|
||||
},
|
||||
};
|
||||
messagesCreateMock.mockResolvedValueOnce(msg);
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
compaction: true,
|
||||
}),
|
||||
)) as { usage: Record<string, number> };
|
||||
|
||||
// Totals are the SUM across iterations, not the top-level fields.
|
||||
expect(result.usage.input_tokens).toBe(203000); // 180000 + 23000
|
||||
expect(result.usage.output_tokens).toBe(4500); // 3500 + 1000
|
||||
|
||||
const haiku = CLAUDE_MODELS.find(
|
||||
(m) => m.id === 'claude-haiku-4-5-20251001',
|
||||
)!;
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.input_tokens).toBe(203000);
|
||||
expect(overrides.input_tokens).toBe(
|
||||
203000 * Number(haiku.costs.input_tokens),
|
||||
);
|
||||
expect(overrides.output_tokens).toBe(
|
||||
4500 * Number(haiku.costs.output_tokens),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Streaming deltas ────────────────────────────────────────────────
|
||||
@@ -618,6 +664,220 @@ describe('ClaudeProvider.complete streaming', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── Inline compaction ───────────────────────────────────────────────
|
||||
|
||||
describe('ClaudeProvider.complete compaction', () => {
|
||||
it('translates the neutral opt-in to context_management + the compaction beta', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'hi' }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
compaction: { trigger_tokens: 50000 },
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = messagesCreateMock.mock.calls[0]!;
|
||||
expect(args.context_management).toEqual({
|
||||
edits: [
|
||||
{
|
||||
type: 'compact_20260112',
|
||||
trigger: { type: 'input_tokens', value: 50000 },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(args.betas).toContain('compact-2026-01-12');
|
||||
});
|
||||
|
||||
it('emits a canonical compaction event from a streamed compaction block', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesStreamMock.mockReturnValueOnce(
|
||||
makeStreamLike(
|
||||
[
|
||||
{ type: 'message_start' },
|
||||
{
|
||||
type: 'content_block_start',
|
||||
content_block: {
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
content: 'ENC', // Anthropic carries the summary here
|
||||
},
|
||||
},
|
||||
{ type: 'content_block_stop' },
|
||||
{
|
||||
type: 'message_delta',
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
},
|
||||
{ type: 'message_stop' },
|
||||
],
|
||||
{ input_tokens: 1, output_tokens: 1 },
|
||||
),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
stream: true,
|
||||
compaction: true,
|
||||
}),
|
||||
);
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const compaction = harness
|
||||
.events()
|
||||
.find((e) => e.type === 'compaction');
|
||||
expect(compaction).toEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
});
|
||||
});
|
||||
|
||||
it('enables the compaction beta when a round-tripped compaction block is resent (no opt-in)', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [
|
||||
{ role: 'user', content: 'continue' },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
// note: no `compaction`/`context_management` opt-in
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = messagesCreateMock.mock.calls[0]!;
|
||||
expect(args.betas).toContain('compact-2026-01-12');
|
||||
// No new compaction was requested, so no context_management is sent.
|
||||
expect(args.context_management).toBeUndefined();
|
||||
});
|
||||
|
||||
it('surfaces a compaction block from a non-streaming response as result.compaction', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [
|
||||
{ type: 'text', text: 'done' },
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_2',
|
||||
content: 'ENC2', // Anthropic carries the summary here
|
||||
},
|
||||
],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
});
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
compaction: true,
|
||||
}),
|
||||
)) as { compaction?: { id?: string; encrypted_content: string } };
|
||||
|
||||
// Anthropic's `content` is surfaced under the unified `encrypted_content`.
|
||||
expect(result.compaction).toEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_2',
|
||||
encrypted_content: 'ENC2',
|
||||
});
|
||||
});
|
||||
|
||||
it('maps a round-tripped compaction block back to Anthropic `content` on input', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [
|
||||
{ role: 'user', content: 'continue' },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'compaction', encrypted_content: 'SUMMARY' },
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = messagesCreateMock.mock.calls[0]!;
|
||||
const compactionBlock = args.messages
|
||||
.flatMap((m: { content?: unknown[] }) =>
|
||||
Array.isArray(m.content) ? m.content : [],
|
||||
)
|
||||
.find((c: { type?: string }) => c?.type === 'compaction');
|
||||
// Internal `encrypted_content` carrier → Anthropic native `content`.
|
||||
expect(compactionBlock).toEqual({
|
||||
type: 'compaction',
|
||||
content: 'SUMMARY',
|
||||
});
|
||||
});
|
||||
|
||||
it('preserves the reply text when an assistant turn carries compaction + text', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
messages: [
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'compaction', encrypted_content: 'SUMMARY' },
|
||||
{ type: 'text', text: 'earlier reply' },
|
||||
],
|
||||
},
|
||||
{ role: 'user', content: 'continue' },
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = messagesCreateMock.mock.calls[0]!;
|
||||
const blocks = args.messages.flatMap((m: { content?: unknown[] }) =>
|
||||
Array.isArray(m.content) ? m.content : [],
|
||||
);
|
||||
// Compaction mapped to Anthropic `content`, AND the reply text kept.
|
||||
expect(blocks).toContainEqual({
|
||||
type: 'compaction',
|
||||
content: 'SUMMARY',
|
||||
});
|
||||
expect(blocks).toContainEqual({ type: 'text', text: 'earlier reply' });
|
||||
});
|
||||
});
|
||||
|
||||
// ── Moderation ──────────────────────────────────────────────────────
|
||||
|
||||
describe('ClaudeProvider.checkModeration', () => {
|
||||
|
||||
@@ -34,6 +34,10 @@ import type {
|
||||
ICompleteArguments,
|
||||
IChatCompleteResult,
|
||||
} from '../../types.js';
|
||||
import {
|
||||
messagesHaveCompaction,
|
||||
toAnthropicContextManagement,
|
||||
} from '../../utils/compaction.js';
|
||||
import { make_claude_tools } from '../../utils/FunctionCalling.js';
|
||||
import { extract_and_remove_system_messages } from '../../utils/Messages.js';
|
||||
import type {
|
||||
@@ -44,6 +48,11 @@ import type {
|
||||
import { FILES_API_BETA, processPuterPathUploads } from './fileUpload.js';
|
||||
import { CLAUDE_MODELS } from './models.js';
|
||||
|
||||
// Anthropic inline-compaction beta. The vendored SDK (0.68.0) doesn't type the
|
||||
// `compact_20260112` edit or the `compaction` content block, so the request
|
||||
// params and streamed/returned blocks are handled with `as any` casts.
|
||||
const COMPACTION_BETA = 'compact-2026-01-12';
|
||||
|
||||
export class ClaudeProvider implements IChatProvider {
|
||||
anthropic: Anthropic;
|
||||
|
||||
@@ -97,9 +106,18 @@ export class ClaudeProvider implements IChatProvider {
|
||||
temperature,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
compaction,
|
||||
context_management,
|
||||
}: ICompleteArguments): Promise<IChatCompleteResult> {
|
||||
tools = make_claude_tools(tools);
|
||||
|
||||
// Translate the neutral compaction opt-in (or pass a raw
|
||||
// `context_management` payload through) to Anthropic's beta shape.
|
||||
const contextManagement = toAnthropicContextManagement({
|
||||
compaction,
|
||||
context_management,
|
||||
});
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
let system_prompts: string | any[];
|
||||
[system_prompts, messages] =
|
||||
@@ -218,6 +236,23 @@ export class ClaudeProvider implements IChatProvider {
|
||||
return message;
|
||||
});
|
||||
|
||||
// Map round-tripped compaction blocks back to Anthropic's native shape.
|
||||
// The internal/unified carrier field is `encrypted_content`; Anthropic's
|
||||
// compaction block uses `content` (a plaintext summary).
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
messages = messages.map((message: any) => {
|
||||
if (!Array.isArray(message.content)) return message;
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
message.content = message.content.map((part: any) => {
|
||||
if (part?.type !== 'compaction') return part;
|
||||
return {
|
||||
type: 'compaction',
|
||||
content: part.content ?? part.encrypted_content ?? '',
|
||||
};
|
||||
});
|
||||
return message;
|
||||
});
|
||||
|
||||
const modelUsed =
|
||||
this.models().find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
@@ -262,6 +297,18 @@ export class ClaudeProvider implements IChatProvider {
|
||||
actor,
|
||||
);
|
||||
const usesBetaFiles = uploadedFileIds.length > 0;
|
||||
// The compaction beta is needed both to *request* compaction
|
||||
// (contextManagement) and to *accept a round-tripped* compaction block
|
||||
// back as input (messagesHaveCompaction).
|
||||
const usesCompaction =
|
||||
!!contextManagement || messagesHaveCompaction(messages);
|
||||
// Compaction and Files API both require the beta endpoint; combine their
|
||||
// beta headers and route through `beta.messages.*` if either is active.
|
||||
const betas = [
|
||||
...(usesBetaFiles ? [FILES_API_BETA] : []),
|
||||
...(usesCompaction ? [COMPACTION_BETA] : []),
|
||||
];
|
||||
const usesBeta = betas.length > 0;
|
||||
|
||||
const sdkParams: MessageCreateParams & {
|
||||
betas?: string[];
|
||||
@@ -291,7 +338,11 @@ export class ClaudeProvider implements IChatProvider {
|
||||
...(supportsEffort && requestedReasoningEffort
|
||||
? { output_config: { effort: requestedReasoningEffort } }
|
||||
: {}),
|
||||
...(usesBetaFiles ? { betas: [FILES_API_BETA] } : {}),
|
||||
// Cast: `context_management` compaction edits aren't typed in SDK 0.68.0.
|
||||
...(contextManagement
|
||||
? { context_management: contextManagement as any }
|
||||
: {}),
|
||||
...(usesBeta ? { betas } : {}),
|
||||
} as MessageCreateParams & { betas?: string[] };
|
||||
|
||||
const cleanupUploads = async () => {
|
||||
@@ -315,13 +366,23 @@ export class ClaudeProvider implements IChatProvider {
|
||||
}: {
|
||||
chatStream: AIChatStream;
|
||||
}) => {
|
||||
const completion = usesBetaFiles
|
||||
const completion = usesBeta
|
||||
? this.anthropic.beta.messages.stream(sdkParams)
|
||||
: this.anthropic.messages.stream(sdkParams);
|
||||
const usageSum: Record<string, number> = {};
|
||||
|
||||
let message, contentBlock;
|
||||
let currentContentBlockType: string | null = null;
|
||||
// Inline-compaction block is an untyped beta block; capture its
|
||||
// artifact across start/delta and emit on stop. Anthropic carries
|
||||
// the summary in `content` (plaintext) — unlike OpenAI's
|
||||
// `encrypted_content` — so read `content` first.
|
||||
let compactionData: {
|
||||
id?: string;
|
||||
payload: string;
|
||||
buffer: string;
|
||||
} | null = null;
|
||||
let emittedCompaction = false;
|
||||
for await (const event of completion) {
|
||||
if (event.type === 'message_delta') {
|
||||
const meteredData = this.#usageFormatterUtil(
|
||||
@@ -345,6 +406,24 @@ export class ClaudeProvider implements IChatProvider {
|
||||
}
|
||||
if (event.type === 'content_block_start') {
|
||||
currentContentBlockType = event.content_block.type;
|
||||
if (
|
||||
(event.content_block.type as string) ===
|
||||
'compaction'
|
||||
) {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const block = event.content_block as any;
|
||||
compactionData = {
|
||||
id: block.id,
|
||||
// Anthropic uses `content`; fall back to
|
||||
// `encrypted_content` in case the field varies.
|
||||
payload:
|
||||
block.content ??
|
||||
block.encrypted_content ??
|
||||
'',
|
||||
buffer: '',
|
||||
};
|
||||
continue;
|
||||
}
|
||||
if (event.content_block.type === 'tool_use') {
|
||||
contentBlock = message!.contentBlock({
|
||||
type: event.content_block.type,
|
||||
@@ -363,12 +442,42 @@ export class ClaudeProvider implements IChatProvider {
|
||||
continue;
|
||||
}
|
||||
if (event.type === 'content_block_stop') {
|
||||
if (currentContentBlockType === 'compaction') {
|
||||
const encrypted_content =
|
||||
compactionData?.payload ||
|
||||
compactionData?.buffer ||
|
||||
'';
|
||||
// Only emit (and mark done) if we actually captured
|
||||
// the summary; otherwise let the finalMessage
|
||||
// fallback recover it from the complete block.
|
||||
if (encrypted_content) {
|
||||
chatStream.compaction({
|
||||
id: compactionData?.id,
|
||||
encrypted_content,
|
||||
});
|
||||
emittedCompaction = true;
|
||||
}
|
||||
compactionData = null;
|
||||
currentContentBlockType = null;
|
||||
continue;
|
||||
}
|
||||
contentBlock!.end();
|
||||
contentBlock = null;
|
||||
currentContentBlockType = null;
|
||||
continue;
|
||||
}
|
||||
if (event.type === 'content_block_delta') {
|
||||
if (currentContentBlockType === 'compaction') {
|
||||
// Capture any streamed payload for the artifact.
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const d = event.delta as any;
|
||||
const chunk =
|
||||
d.partial_json ?? d.text ?? d.data ?? '';
|
||||
if (typeof chunk === 'string' && compactionData) {
|
||||
compactionData.buffer += chunk;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (event.delta.type === 'input_json_delta') {
|
||||
(contentBlock as AIChatToolUseStream)!.addPartialJSON(
|
||||
event.delta.partial_json,
|
||||
@@ -391,18 +500,37 @@ export class ClaudeProvider implements IChatProvider {
|
||||
// signature_delta — ignored
|
||||
}
|
||||
}
|
||||
const finalUsage = await completion
|
||||
const finalMessage = await completion
|
||||
.finalMessage()
|
||||
.then((msg) =>
|
||||
this.#usageFormatterUtil(
|
||||
msg.usage as Usage | BetaUsage,
|
||||
),
|
||||
)
|
||||
.catch(() => null);
|
||||
if (finalUsage) {
|
||||
if (finalMessage) {
|
||||
const finalUsage = this.#usageFormatterUtil(
|
||||
finalMessage.usage as Usage | BetaUsage,
|
||||
);
|
||||
for (const [key, value] of Object.entries(finalUsage)) {
|
||||
usageSum[key] = value;
|
||||
}
|
||||
// Fallback: some SDK versions surface the compaction block
|
||||
// only in the final message, not as a streamed block.
|
||||
if (!emittedCompaction) {
|
||||
const block = (
|
||||
(finalMessage.content as unknown[]) ?? []
|
||||
).find(
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
(c: any) => c?.type === 'compaction',
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
) as any;
|
||||
if (block) {
|
||||
chatStream.compaction({
|
||||
id: block.id,
|
||||
encrypted_content:
|
||||
block.content ??
|
||||
block.encrypted_content ??
|
||||
'',
|
||||
});
|
||||
emittedCompaction = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
chatStream.end(usageSum);
|
||||
const costsOverrideFromModel =
|
||||
@@ -423,7 +551,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
try {
|
||||
const msg = await (usesBetaFiles
|
||||
const msg = await (usesBeta
|
||||
? this.anthropic.beta.messages.create(sdkParams)
|
||||
: this.anthropic.messages.create(sdkParams));
|
||||
const usage = this.#usageFormatterUtil(
|
||||
@@ -440,7 +568,37 @@ export class ClaudeProvider implements IChatProvider {
|
||||
costsOverrideFromModel,
|
||||
);
|
||||
|
||||
return { message: msg, usage, finish_reason: 'stop' };
|
||||
// Surface any inline-compaction artifact for stateless round-trip.
|
||||
const compactionBlock = (
|
||||
((msg as Message).content as unknown[]) ?? []
|
||||
).find(
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
(c: any) => c?.type === 'compaction',
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
) as any;
|
||||
|
||||
return {
|
||||
message: msg,
|
||||
usage,
|
||||
finish_reason: 'stop',
|
||||
...(compactionBlock
|
||||
? {
|
||||
compaction: {
|
||||
// `type` makes the artifact a drop-in `messages`
|
||||
// item for the round-trip (symmetric with the
|
||||
// streaming compaction chunk).
|
||||
type: 'compaction' as const,
|
||||
...(compactionBlock.id !== undefined
|
||||
? { id: compactionBlock.id }
|
||||
: {}),
|
||||
encrypted_content:
|
||||
compactionBlock.content ??
|
||||
compactionBlock.encrypted_content ??
|
||||
'',
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
} finally {
|
||||
await cleanupUploads();
|
||||
}
|
||||
@@ -448,8 +606,31 @@ export class ClaudeProvider implements IChatProvider {
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
#usageFormatterUtil(usage: any) {
|
||||
// Compaction responses report per-pass usage in `usage.iterations` (the
|
||||
// compaction pass + the message pass). Per Anthropic's docs the
|
||||
// top-level `input_tokens`/`output_tokens` reflect ONLY the
|
||||
// non-compaction passes, so to bill the (often large) compaction pass we
|
||||
// must sum across all iterations. Compaction is billed at normal token
|
||||
// rates, and iterations don't break out cache fields — those stay
|
||||
// top-level (the message pass's), which bills the compaction input at
|
||||
// the full input rate.
|
||||
const iterations = Array.isArray(usage?.iterations)
|
||||
? usage.iterations
|
||||
: null;
|
||||
const inputTokens = iterations
|
||||
? iterations.reduce(
|
||||
(sum: number, it: any) => sum + (it?.input_tokens || 0),
|
||||
0,
|
||||
)
|
||||
: usage?.input_tokens || 0;
|
||||
const outputTokens = iterations
|
||||
? iterations.reduce(
|
||||
(sum: number, it: any) => sum + (it?.output_tokens || 0),
|
||||
0,
|
||||
)
|
||||
: usage?.output_tokens || 0;
|
||||
return {
|
||||
input_tokens: usage?.input_tokens || 0,
|
||||
input_tokens: inputTokens,
|
||||
ephemeral_5m_input_tokens:
|
||||
usage?.cache_creation?.ephemeral_5m_input_tokens ||
|
||||
usage?.cache_creation_input_tokens ||
|
||||
@@ -457,7 +638,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
ephemeral_1h_input_tokens:
|
||||
usage?.cache_creation?.ephemeral_1h_input_tokens || 0,
|
||||
cache_read_input_tokens: usage?.cache_read_input_tokens || 0,
|
||||
output_tokens: usage?.output_tokens || 0,
|
||||
output_tokens: outputTokens,
|
||||
thinking_tokens:
|
||||
usage?.thinking_tokens ||
|
||||
usage?.output_tokens_details?.thinking_tokens ||
|
||||
|
||||
@@ -355,6 +355,8 @@ describe('GeminiChatProvider.complete non-stream output', () => {
|
||||
prompt_tokens: 90,
|
||||
completion_tokens: 50,
|
||||
cached_tokens: 10,
|
||||
thinking_tokens: 0,
|
||||
grounding_requests: 0,
|
||||
});
|
||||
|
||||
// gemini-2.5-flash costs: prompt=30, completion=250, cached=3.
|
||||
@@ -367,11 +369,15 @@ describe('GeminiChatProvider.complete non-stream output', () => {
|
||||
prompt_tokens: 90,
|
||||
completion_tokens: 50,
|
||||
cached_tokens: 10,
|
||||
thinking_tokens: 0,
|
||||
grounding_requests: 0,
|
||||
});
|
||||
expect(overrides).toEqual({
|
||||
prompt_tokens: 90 * Number(flash.costs.prompt_tokens),
|
||||
completion_tokens: 50 * Number(flash.costs.completion_tokens),
|
||||
cached_tokens: 10 * Number(flash.costs.cached_tokens ?? 0),
|
||||
thinking_tokens: 0,
|
||||
grounding_requests: 0,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -446,6 +452,8 @@ describe('GeminiChatProvider.complete streaming', () => {
|
||||
prompt_tokens: 3, // 4 - 1 cached
|
||||
completion_tokens: 2,
|
||||
cached_tokens: 1,
|
||||
thinking_tokens: 0,
|
||||
grounding_requests: 0,
|
||||
});
|
||||
|
||||
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
|
||||
@@ -456,10 +464,192 @@ describe('GeminiChatProvider.complete streaming', () => {
|
||||
prompt_tokens: 3 * Number(flash.costs.prompt_tokens),
|
||||
completion_tokens: 2 * Number(flash.costs.completion_tokens),
|
||||
cached_tokens: 1 * Number(flash.costs.cached_tokens ?? 0),
|
||||
thinking_tokens: 0,
|
||||
grounding_requests: 0,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ── Thinking-token metering ─────────────────────────────────────────
|
||||
|
||||
describe('GeminiChatProvider.complete thinking token metering', () => {
|
||||
it('splits thinking tokens out of completion_tokens and bills each at its own rate', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'result', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 60,
|
||||
completion_tokens_details: { reasoning_tokens: 20 },
|
||||
prompt_tokens_details: { cached_tokens: 10 },
|
||||
},
|
||||
});
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.5-flash',
|
||||
messages: [{ role: 'user', content: 'think hard' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect((result as { usage: unknown }).usage).toEqual({
|
||||
prompt_tokens: 90, // 100 - 10 cached
|
||||
completion_tokens: 40, // 60 - 20 thinking
|
||||
cached_tokens: 10,
|
||||
thinking_tokens: 20,
|
||||
grounding_requests: 0,
|
||||
});
|
||||
|
||||
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.thinking_tokens).toBe(20);
|
||||
expect(usage.completion_tokens).toBe(40);
|
||||
expect(overrides).toMatchObject({
|
||||
thinking_tokens: 20 * Number(flash.costs.thinking_tokens),
|
||||
completion_tokens: 40 * Number(flash.costs.completion_tokens),
|
||||
});
|
||||
});
|
||||
|
||||
it('sets thinking_tokens to 0 when completion_tokens_details is absent', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 5, completion_tokens: 3 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.5-flash',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const [usage] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.thinking_tokens).toBe(0);
|
||||
expect(usage.completion_tokens).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Grounding-request metering ───────────────────────────────────────
|
||||
|
||||
describe('GeminiChatProvider.complete grounding request metering', () => {
|
||||
it('charges one grounding request when non-stream response contains grounding_metadata', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
content: 'result',
|
||||
role: 'assistant',
|
||||
extra_content: {
|
||||
grounding_metadata: { web_search_queries: ['foo'] },
|
||||
},
|
||||
},
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 5 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.5-flash',
|
||||
messages: [{ role: 'user', content: 'search for foo' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.grounding_requests).toBe(1);
|
||||
expect(overrides!.grounding_requests).toBe(
|
||||
1 * Number(flash.costs.grounding_requests),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not charge a grounding request when no grounding_metadata is present', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 5, completion_tokens: 3 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.5-flash',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const [usage] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.grounding_requests).toBe(0);
|
||||
});
|
||||
|
||||
it('charges one grounding request when streaming response contains grounding extra_content', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{ choices: [{ delta: { content: 'ans' } }] },
|
||||
{
|
||||
choices: [
|
||||
{
|
||||
delta: {
|
||||
extra_content: {
|
||||
grounding_metadata: {
|
||||
web_search_queries: ['bar'],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
choices: [{ delta: {} }],
|
||||
usage: { prompt_tokens: 8, completion_tokens: 4 },
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.5-flash',
|
||||
messages: [{ role: 'user', content: 'search bar' }],
|
||||
stream: true,
|
||||
}),
|
||||
);
|
||||
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const flash = GEMINI_MODELS.find((m) => m.id === 'gemini-2.5-flash')!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.grounding_requests).toBe(1);
|
||||
expect(overrides!.grounding_requests).toBe(
|
||||
1 * Number(flash.costs.grounding_requests),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Error mapping ───────────────────────────────────────────────────
|
||||
|
||||
describe('GeminiChatProvider.complete error mapping', () => {
|
||||
|
||||
@@ -102,24 +102,52 @@ export class GeminiChatProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
return handle_completion_output({
|
||||
usage_calculator: ({ usage }) => {
|
||||
usage_calculator: (args) => {
|
||||
// Cast to access Gemini-specific extras passed alongside usage:
|
||||
// - choices: non-stream grounding metadata lives in choices[0].message.extra_content
|
||||
// - extra_content: streaming grounding metadata accumulated by the stream handler
|
||||
const { usage, choices, extra_content } = args as {
|
||||
usage: typeof args.usage;
|
||||
choices?: Array<{
|
||||
message?: {
|
||||
extra_content?: { grounding_metadata?: unknown };
|
||||
};
|
||||
}>;
|
||||
extra_content?: { grounding_metadata?: unknown };
|
||||
};
|
||||
|
||||
const cached_tokens =
|
||||
usage?.prompt_tokens_details?.cached_tokens ?? 0;
|
||||
|
||||
// Thinking tokens are a subset of completion_tokens billed at a different rate
|
||||
const thinking_tokens =
|
||||
usage?.completion_tokens_details?.reasoning_tokens ?? 0;
|
||||
|
||||
const trackedUsage = {
|
||||
prompt_tokens:
|
||||
(usage.prompt_tokens ?? 0) -
|
||||
(usage.prompt_tokens_details?.cached_tokens ?? 0),
|
||||
completion_tokens: usage.completion_tokens ?? 0,
|
||||
cached_tokens:
|
||||
usage.prompt_tokens_details?.cached_tokens ?? 0,
|
||||
prompt_tokens: (usage?.prompt_tokens ?? 0) - cached_tokens,
|
||||
completion_tokens: Math.max(
|
||||
0,
|
||||
(usage?.completion_tokens ?? 0) - thinking_tokens,
|
||||
),
|
||||
cached_tokens,
|
||||
thinking_tokens,
|
||||
// Grounding search is a per-request fee not reflected in token counts
|
||||
grounding_requests:
|
||||
(choices?.[0]?.message?.extra_content
|
||||
?.grounding_metadata ??
|
||||
extra_content?.grounding_metadata)
|
||||
? 1
|
||||
: 0,
|
||||
};
|
||||
|
||||
const costsOverrideFromModel = Object.fromEntries(
|
||||
Object.entries(trackedUsage).map(([k, v]) => {
|
||||
return [k, v * modelUsed.costs[k]];
|
||||
return [k, v * (modelUsed.costs[k] ?? 0)];
|
||||
}),
|
||||
);
|
||||
this.meteringService.utilRecordUsageObject(
|
||||
trackedUsage,
|
||||
actor,
|
||||
actor!,
|
||||
`gemini:${modelUsed?.id}`,
|
||||
costsOverrideFromModel,
|
||||
);
|
||||
|
||||
@@ -43,7 +43,10 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 150,
|
||||
completion_tokens: 900,
|
||||
thinking_tokens: 900,
|
||||
cached_tokens: 15,
|
||||
// Gemini 3.x grounding is $14 / 1,000 requests
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -68,6 +71,8 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
prompt_tokens: 10,
|
||||
completion_tokens: 40,
|
||||
cached_tokens: 3,
|
||||
// Gemini 2.x grounding is $35 / 1,000 requests
|
||||
grounding_requests: 3_500_000,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
@@ -115,8 +120,13 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
// Output is $2.50/M; thinking tokens bill at the same output rate
|
||||
completion_tokens: 250,
|
||||
thinking_tokens: 250,
|
||||
// Cache read is $0.03/M (10% of input)
|
||||
cached_tokens: 3,
|
||||
// Gemini 2.x grounding is $35 / 1,000 requests
|
||||
grounding_requests: 3_500_000,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
@@ -141,7 +151,10 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
completion_tokens: 40,
|
||||
thinking_tokens: 40,
|
||||
cached_tokens: 1,
|
||||
// Gemini 2.x grounding is $35 / 1,000 requests
|
||||
grounding_requests: 3_500_000,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
@@ -166,7 +179,10 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
completion_tokens: 1000,
|
||||
cached_tokens: 13,
|
||||
thinking_tokens: 1000,
|
||||
cached_tokens: 31,
|
||||
// Gemini 2.x grounding is $35 / 1,000 requests
|
||||
grounding_requests: 3_500_000,
|
||||
},
|
||||
max_tokens: 200_000,
|
||||
},
|
||||
@@ -191,7 +207,9 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
completion_tokens: 1200,
|
||||
thinking_tokens: 1200,
|
||||
cached_tokens: 20,
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
@@ -216,7 +234,9 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
completion_tokens: 300,
|
||||
thinking_tokens: 300,
|
||||
cached_tokens: 5,
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
@@ -245,7 +265,9 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
completion_tokens: 150,
|
||||
thinking_tokens: 150,
|
||||
cached_tokens: 2.5,
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
|
||||
@@ -143,6 +143,11 @@ describe('MiniMaxProvider model catalog', () => {
|
||||
|
||||
it('uses MiniMax completion-token limits instead of the context window', () => {
|
||||
for (const model of MINIMAX_MODELS) {
|
||||
if (model.id === 'minimax-m3') {
|
||||
expect(model.context).toBe(1_048_576);
|
||||
expect(model.max_tokens).toBe(512_000);
|
||||
continue;
|
||||
}
|
||||
expect(model.context).toBe(204_800);
|
||||
expect(model.max_tokens).toBe(196_608);
|
||||
}
|
||||
|
||||
@@ -18,60 +18,215 @@
|
||||
*/
|
||||
|
||||
import type { IChatModel } from '../../types.js';
|
||||
import { usdPerMToken } from '../../utils/pricing.js';
|
||||
|
||||
const CONTEXT_WINDOW = 204_800;
|
||||
const MAX_OUTPUT_TOKENS = 196_608;
|
||||
|
||||
type MiniMaxChatModel = IChatModel & {
|
||||
apiModel: string;
|
||||
};
|
||||
|
||||
const minimaxModel = (
|
||||
apiModel: string,
|
||||
name: string,
|
||||
costs: IChatModel['costs'],
|
||||
): MiniMaxChatModel => {
|
||||
const id = apiModel.toLowerCase();
|
||||
return {
|
||||
puterId: `minimax:minimax/${id}`,
|
||||
id,
|
||||
apiModel,
|
||||
name,
|
||||
aliases: [`minimax/${id}`, apiModel, `minimax/${apiModel}`],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: CONTEXT_WINDOW,
|
||||
max_tokens: MAX_OUTPUT_TOKENS,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs,
|
||||
};
|
||||
};
|
||||
|
||||
// Hardcoded from MiniMax OpenAI-compatible API docs and pay-as-you-go pricing:
|
||||
// https://platform.minimax.io/docs/api-reference/text-openai-api
|
||||
// https://platform.minimax.io/docs/guides/pricing-paygo
|
||||
export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
minimaxModel('MiniMax-M2.7', 'MiniMax M2.7', usdPerMToken(0.3, 1.2, 0.06)),
|
||||
minimaxModel(
|
||||
'MiniMax-M2.7-highspeed',
|
||||
'MiniMax M2.7 Highspeed',
|
||||
usdPerMToken(0.6, 2.4, 0.06),
|
||||
),
|
||||
minimaxModel('MiniMax-M2.5', 'MiniMax M2.5', usdPerMToken(0.3, 1.2, 0.03)),
|
||||
minimaxModel(
|
||||
'MiniMax-M2.5-highspeed',
|
||||
'MiniMax M2.5 Highspeed',
|
||||
usdPerMToken(0.6, 2.4, 0.03),
|
||||
),
|
||||
minimaxModel('MiniMax-M2.1', 'MiniMax M2.1', usdPerMToken(0.3, 1.2, 0.03)),
|
||||
minimaxModel(
|
||||
'MiniMax-M2.1-highspeed',
|
||||
'MiniMax M2.1 Highspeed',
|
||||
usdPerMToken(0.6, 2.4, 0.03),
|
||||
),
|
||||
minimaxModel('MiniMax-M2', 'MiniMax M2', usdPerMToken(0.3, 1.2, 0.03)),
|
||||
// -- MiniMax M3 (Flagship) --------------------------------------
|
||||
// MiniMax Sparse Attention, 1M context, native multimodal input.
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m3',
|
||||
id: 'minimax-m3',
|
||||
apiModel: 'MiniMax-M3',
|
||||
name: 'MiniMax M3',
|
||||
aliases: ['minimax/minimax-m3', 'MiniMax-M3', 'minimax/MiniMax-M3'],
|
||||
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 1_048_576,
|
||||
max_tokens: 512_000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M (input <= 512k tokens)
|
||||
completion_tokens: 120, // $1.20 per 1M
|
||||
cached_tokens: 6, // $0.06 per 1M
|
||||
},
|
||||
},
|
||||
|
||||
// -- MiniMax M2.7 -----------------------------------------------
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2.7',
|
||||
id: 'minimax-m2.7',
|
||||
apiModel: 'MiniMax-M2.7',
|
||||
name: 'MiniMax M2.7',
|
||||
aliases: [
|
||||
'minimax/minimax-m2.7',
|
||||
'MiniMax-M2.7',
|
||||
'minimax/MiniMax-M2.7',
|
||||
],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
completion_tokens: 120, // $1.20 per 1M
|
||||
cached_tokens: 6, // $0.06 per 1M
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2.7-highspeed',
|
||||
id: 'minimax-m2.7-highspeed',
|
||||
apiModel: 'MiniMax-M2.7-highspeed',
|
||||
name: 'MiniMax M2.7 Highspeed',
|
||||
aliases: [
|
||||
'minimax/minimax-m2.7-highspeed',
|
||||
'MiniMax-M2.7-highspeed',
|
||||
'minimax/MiniMax-M2.7-highspeed',
|
||||
],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60, // $0.60 per 1M
|
||||
completion_tokens: 240, // $2.40 per 1M
|
||||
cached_tokens: 6, // $0.06 per 1M
|
||||
},
|
||||
},
|
||||
|
||||
// -- MiniMax M2.5 -----------------------------------------------
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2.5',
|
||||
id: 'minimax-m2.5',
|
||||
apiModel: 'MiniMax-M2.5',
|
||||
name: 'MiniMax M2.5',
|
||||
aliases: [
|
||||
'minimax/minimax-m2.5',
|
||||
'MiniMax-M2.5',
|
||||
'minimax/MiniMax-M2.5',
|
||||
],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
completion_tokens: 120, // $1.20 per 1M
|
||||
cached_tokens: 3, // $0.03 per 1M
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2.5-highspeed',
|
||||
id: 'minimax-m2.5-highspeed',
|
||||
apiModel: 'MiniMax-M2.5-highspeed',
|
||||
name: 'MiniMax M2.5 Highspeed',
|
||||
aliases: [
|
||||
'minimax/minimax-m2.5-highspeed',
|
||||
'MiniMax-M2.5-highspeed',
|
||||
'minimax/MiniMax-M2.5-highspeed',
|
||||
],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60, // $0.60 per 1M
|
||||
completion_tokens: 240, // $2.40 per 1M
|
||||
cached_tokens: 3, // $0.03 per 1M
|
||||
},
|
||||
},
|
||||
|
||||
// -- MiniMax M2.1 -----------------------------------------------
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2.1',
|
||||
id: 'minimax-m2.1',
|
||||
apiModel: 'MiniMax-M2.1',
|
||||
name: 'MiniMax M2.1',
|
||||
aliases: [
|
||||
'minimax/minimax-m2.1',
|
||||
'MiniMax-M2.1',
|
||||
'minimax/MiniMax-M2.1',
|
||||
],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
completion_tokens: 120, // $1.20 per 1M
|
||||
cached_tokens: 3, // $0.03 per 1M
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2.1-highspeed',
|
||||
id: 'minimax-m2.1-highspeed',
|
||||
apiModel: 'MiniMax-M2.1-highspeed',
|
||||
name: 'MiniMax M2.1 Highspeed',
|
||||
aliases: [
|
||||
'minimax/minimax-m2.1-highspeed',
|
||||
'MiniMax-M2.1-highspeed',
|
||||
'minimax/MiniMax-M2.1-highspeed',
|
||||
],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60, // $0.60 per 1M
|
||||
completion_tokens: 240, // $2.40 per 1M
|
||||
cached_tokens: 3, // $0.03 per 1M
|
||||
},
|
||||
},
|
||||
|
||||
// -- MiniMax M2 -------------------------------------------------
|
||||
{
|
||||
puterId: 'minimax:minimax/minimax-m2',
|
||||
id: 'minimax-m2',
|
||||
apiModel: 'MiniMax-M2',
|
||||
name: 'MiniMax M2',
|
||||
aliases: ['minimax/minimax-m2', 'MiniMax-M2', 'minimax/MiniMax-M2'],
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
context: 204_800,
|
||||
max_tokens: 196_608,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
completion_tokens: 120, // $1.20 per 1M
|
||||
cached_tokens: 3, // $0.03 per 1M
|
||||
},
|
||||
},
|
||||
];
|
||||
@@ -26,6 +26,10 @@ import type { MeteringService } from '../../../../services/metering/MeteringServ
|
||||
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
|
||||
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
|
||||
import type { IChatProvider, ICompleteArguments } from '../../types.js';
|
||||
import {
|
||||
messagesHaveCompaction,
|
||||
wantsCompaction,
|
||||
} from '../../utils/compaction.js';
|
||||
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import { processPuterPathUploads } from './fileUpload.js';
|
||||
import { OPEN_AI_MODELS } from './models.js';
|
||||
@@ -125,6 +129,20 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Inline compaction is a Responses-API feature; chat.completions can't
|
||||
// express `context_management` or a `compaction` content block.
|
||||
// Delegate to the sibling Responses provider when the caller opted in
|
||||
// OR when the messages carry a round-tripped compaction artifact.
|
||||
if (wantsCompaction(params) || messagesHaveCompaction(messages)) {
|
||||
if (!this.#responsesProvider) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'compaction requires the OpenAI Responses provider, which is not configured',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new HttpError(400, '`messages` must be an array', {
|
||||
|
||||
@@ -25,6 +25,7 @@ import type { MeteringService } from '../../../../services/metering/MeteringServ
|
||||
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
|
||||
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
|
||||
import type { IChatProvider, ICompleteArguments } from '../../types.js';
|
||||
import { toOpenAiContextManagement } from '../../utils/compaction.js';
|
||||
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import { processPuterPathUploads } from './fileUpload.js';
|
||||
import { OPEN_AI_MODELS } from './models.js';
|
||||
@@ -100,6 +101,8 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
@@ -177,6 +180,13 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
const supportsReasoningControls =
|
||||
typeof model === 'string' && model.startsWith('gpt-5');
|
||||
|
||||
// Translate the neutral compaction opt-in (or pass a raw
|
||||
// `context_management` payload through) to OpenAI's Responses shape.
|
||||
const contextManagement = toOpenAiContextManagement({
|
||||
compaction,
|
||||
context_management,
|
||||
});
|
||||
|
||||
const completionParams: ResponseCreateParams = {
|
||||
user: userIdentifier,
|
||||
safety_identifier: userIdentifier,
|
||||
@@ -188,6 +198,9 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
? { parallel_tool_calls }
|
||||
: {}),
|
||||
...(include !== undefined ? { include } : {}),
|
||||
...(contextManagement !== undefined
|
||||
? { context_management: contextManagement }
|
||||
: {}),
|
||||
...(conversation !== undefined ? { conversation } : {}),
|
||||
...(previous_response_id !== undefined
|
||||
? { previous_response_id }
|
||||
|
||||
@@ -150,21 +150,7 @@ export class TogetherAIProvider implements IChatProvider {
|
||||
messages,
|
||||
stream,
|
||||
...(tools ? { tools } : {}),
|
||||
// TODO: make this better but togetherai doesn't handle max tokens properly at all
|
||||
...(max_tokens
|
||||
? {
|
||||
max_tokens:
|
||||
max_tokens -
|
||||
messages.reduce((acc, curr) => {
|
||||
return (
|
||||
acc +
|
||||
(curr.type === 'text'
|
||||
? curr.text.length / 2
|
||||
: 200)
|
||||
);
|
||||
}, 0),
|
||||
}
|
||||
: {}),
|
||||
...(max_tokens ? { max_tokens } : {}),
|
||||
...(temperature ? { temperature } : {}),
|
||||
...(stream ? { stream_options: { include_usage: true } } : {}),
|
||||
} as Together.Chat.Completions.CompletionCreateParamsNonStreaming);
|
||||
|
||||
@@ -69,6 +69,13 @@ const visionModel = (
|
||||
// Hardcoded from https://docs.z.ai/api-reference/llm/chat-completion and
|
||||
// https://docs.z.ai/guides/overview/pricing.
|
||||
export const ZAI_MODELS: IChatModel[] = [
|
||||
textModel(
|
||||
'glm-5.2',
|
||||
'GLM-5.2',
|
||||
1_000 * K,
|
||||
128 * K,
|
||||
usdPerMToken(1.4, 4.4, 0.26),
|
||||
),
|
||||
textModel(
|
||||
'glm-5.1',
|
||||
'GLM-5.1',
|
||||
|
||||
@@ -67,6 +67,19 @@ export interface ICompleteArguments {
|
||||
parallel_tool_calls?: boolean;
|
||||
include?: unknown[];
|
||||
conversation?: unknown;
|
||||
/**
|
||||
* Provider-neutral inline-compaction opt-in. `true` enables compaction with
|
||||
* provider defaults; `{ trigger_tokens }` sets the token threshold at which
|
||||
* the upstream summarizes earlier context. Each provider translates this to
|
||||
* its own SDK shape (OpenAI `context_management:[{type:'compaction',...}]`,
|
||||
* Anthropic `context_management:{edits:[{type:'compact_20260112'}]}`).
|
||||
*/
|
||||
compaction?: boolean | { trigger_tokens?: number };
|
||||
/**
|
||||
* Escape hatch: provider-native `context_management` payload, passed through
|
||||
* untouched (used by `/responses` callers sending the OpenAI-native array).
|
||||
*/
|
||||
context_management?: unknown;
|
||||
previous_response_id?: string;
|
||||
instructions?: string | PuterMessage[];
|
||||
metadata?: Record<string, string>;
|
||||
@@ -116,6 +129,13 @@ export interface IChatMessageResult {
|
||||
finally_fn?: never;
|
||||
normalized?: boolean;
|
||||
via_ai_chat_service?: boolean;
|
||||
/**
|
||||
* Inline-compaction artifact, present when the upstream compacted earlier
|
||||
* context during this (non-streaming) response. Carries `type:'compaction'`
|
||||
* so it's a drop-in `messages` item — the caller resends it on the next turn
|
||||
* in place of the summarized history. See [[ICompleteArguments]].
|
||||
*/
|
||||
compaction?: { type: 'compaction'; id?: string; encrypted_content: string };
|
||||
}
|
||||
|
||||
export type IChatCompleteResult = IChatStreamResult | IChatMessageResult;
|
||||
|
||||
@@ -49,6 +49,23 @@ export const normalize_single_message = (message, params = {}) => {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
// A round-tripped inline-compaction artifact may be supplied as a bare
|
||||
// top-level item (the shape the client received from the stream). Wrap it
|
||||
// into an internal compaction content block so it survives normalization;
|
||||
// each provider maps it back to its native input shape (OpenAI top-level
|
||||
// input item, Anthropic content block).
|
||||
if (!message.role && !message.content && message.type === 'compaction') {
|
||||
return {
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{
|
||||
type: 'compaction',
|
||||
...(message.id !== undefined ? { id: message.id } : {}),
|
||||
encrypted_content: message.encrypted_content,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
if (!message.role) {
|
||||
message.role = params.role;
|
||||
}
|
||||
|
||||
@@ -46,6 +46,22 @@ describe('normalize_single_message', () => {
|
||||
expect(result.role).toBe('system');
|
||||
});
|
||||
|
||||
it('wraps a bare round-tripped compaction item into a compaction content block', () => {
|
||||
const result = normalize_single_message({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
});
|
||||
expect(result.role).toBe('assistant');
|
||||
expect(result.content).toEqual([
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('throws 400 when message is null/undefined/array', () => {
|
||||
expect(() => normalize_single_message(null)).toThrow(
|
||||
expect.objectContaining({ statusCode: 400 }),
|
||||
|
||||
@@ -84,6 +84,39 @@ export const process_input_messages = async (messages) => {
|
||||
};
|
||||
|
||||
export const process_input_messages_responses_api = async (messages) => {
|
||||
// Pre-split round-tripped compaction blocks into standalone Responses
|
||||
// compaction input items, preserving any sibling content (e.g. the
|
||||
// assistant's reply text) as its own message. A compaction item represents
|
||||
// prior history, so it precedes the message it was attached to. This avoids
|
||||
// collapsing the whole message into a single compaction item and dropping
|
||||
// the rest of its content.
|
||||
const expanded = [];
|
||||
for (const msg of messages) {
|
||||
if (msg && Array.isArray(msg.content)) {
|
||||
const compactionBlocks = msg.content.filter(
|
||||
(c) => c && c.type === 'compaction',
|
||||
);
|
||||
if (compactionBlocks.length > 0) {
|
||||
for (const block of compactionBlocks) {
|
||||
expanded.push({
|
||||
type: 'compaction',
|
||||
...(block.id !== undefined ? { id: block.id } : {}),
|
||||
encrypted_content: block.encrypted_content,
|
||||
});
|
||||
}
|
||||
const rest = msg.content.filter(
|
||||
(c) => !(c && c.type === 'compaction'),
|
||||
);
|
||||
if (rest.length > 0) {
|
||||
expanded.push({ ...msg, content: rest });
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
expanded.push(msg);
|
||||
}
|
||||
messages = expanded;
|
||||
|
||||
for (const msg of messages) {
|
||||
const content_as_string = (content) => {
|
||||
if (content === undefined || content === null) return '';
|
||||
@@ -253,6 +286,7 @@ export const create_chat_stream_handler =
|
||||
const tool_call_blocks = [];
|
||||
|
||||
let last_usage = null;
|
||||
let last_extra_content = null;
|
||||
for await (let chunk of completion) {
|
||||
chunk = deviations.chunk_but_like_actually(chunk);
|
||||
const chunk_usage = deviations.index_usage_from_stream_chunk(chunk);
|
||||
@@ -285,6 +319,14 @@ export const create_chat_stream_handler =
|
||||
// Apps have to choose to handle extra_content themselves, it doesn't seem like theres a way we can do it in a backwards
|
||||
// compatible fashion since most streaming apps will handle chat history by continuously updating content themselves
|
||||
// This doesn't present us a chance to add in an extra object for gemini's chat continuing features
|
||||
// Don't let a later extra_content chunk without grounding clobber an
|
||||
// earlier one that carried grounding_metadata (used for metering).
|
||||
if (
|
||||
choice.delta.extra_content.grounding_metadata ||
|
||||
!last_extra_content?.grounding_metadata
|
||||
) {
|
||||
last_extra_content = choice.delta.extra_content;
|
||||
}
|
||||
textblock.addExtraContent(choice.delta.extra_content);
|
||||
}
|
||||
|
||||
@@ -315,7 +357,10 @@ export const create_chat_stream_handler =
|
||||
}
|
||||
|
||||
// TODO DS: this is a bit too abstracted... this is basically just doing the metering now
|
||||
const usage = usage_calculator({ usage: last_usage });
|
||||
const usage = usage_calculator({
|
||||
usage: last_usage,
|
||||
extra_content: last_extra_content,
|
||||
});
|
||||
|
||||
if (mode === 'text') textblock.end();
|
||||
if (mode === 'tool') toolblock.end();
|
||||
@@ -355,6 +400,19 @@ export const create_chat_stream_handler_responses_api =
|
||||
last_usage = chunk.response.usage;
|
||||
}
|
||||
|
||||
if (
|
||||
chunk.type === 'response.output_item.done' &&
|
||||
chunk.item?.type === 'compaction'
|
||||
) {
|
||||
// Inline compaction fired mid-response — normalize the artifact
|
||||
// into the canonical internal compaction event.
|
||||
chatStream.compaction({
|
||||
id: chunk.item.id,
|
||||
encrypted_content: chunk.item.encrypted_content,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
if (
|
||||
chunk.type === 'response.output_item.done' &&
|
||||
chunk.item?.type === 'function_call'
|
||||
@@ -488,10 +546,14 @@ export const handle_completion_output_responses_api = async ({
|
||||
...(item.id ? { canonical_id: item.id } : {}),
|
||||
}));
|
||||
|
||||
// Inline-compaction artifact, if the upstream compacted this turn.
|
||||
const compactionItem = output.find((item) => item?.type === 'compaction');
|
||||
|
||||
const is_empty = completion.output_text.trim() === '';
|
||||
if (is_empty && responseToolCalls.length < 1) {
|
||||
if (is_empty && responseToolCalls.length < 1 && !compactionItem) {
|
||||
// GPT refuses to generate an empty response if you ask it to,
|
||||
// so this will probably only happen on an error condition.
|
||||
// A compaction-only output is legitimate, so don't reject it.
|
||||
throw new HttpError(400, 'an empty response was generated', {
|
||||
legacyCode: 'bad_response',
|
||||
});
|
||||
@@ -523,6 +585,18 @@ export const handle_completion_output_responses_api = async ({
|
||||
};
|
||||
ret.role = output.find((item) => item?.role)?.role ?? 'assistant';
|
||||
|
||||
if (compactionItem) {
|
||||
// Include `type` so the artifact is a drop-in `messages` item for the
|
||||
// stateless round-trip — symmetric with the streaming compaction chunk.
|
||||
ret.compaction = {
|
||||
type: 'compaction',
|
||||
...(compactionItem.id !== undefined
|
||||
? { id: compactionItem.id }
|
||||
: {}),
|
||||
encrypted_content: compactionItem.encrypted_content,
|
||||
};
|
||||
}
|
||||
|
||||
delete ret.type;
|
||||
|
||||
ret.usage = usage_calculator
|
||||
|
||||
@@ -232,6 +232,66 @@ describe('process_input_messages_responses_api', () => {
|
||||
expect(out!.output).toBe('part-apart-bpart-c');
|
||||
});
|
||||
|
||||
it('rewrites a round-tripped compaction content block into a top-level compaction item', async () => {
|
||||
const messages: Array<Record<string, unknown>> = [
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
const [out] = (await process_input_messages_responses_api(
|
||||
messages,
|
||||
)) as Array<Record<string, unknown>>;
|
||||
expect(out!.type).toBe('compaction');
|
||||
expect(out!.encrypted_content).toBe('ENC');
|
||||
expect(out!.id).toBe('cmpct_1');
|
||||
expect(out!.role).toBeUndefined();
|
||||
expect(out!.content).toBeUndefined();
|
||||
});
|
||||
|
||||
it('splits a compaction block + reply text into a compaction item plus a preserved message', async () => {
|
||||
// Round-tripping an assistant turn that carries BOTH the compaction
|
||||
// artifact and the reply text must keep the reply, not drop it.
|
||||
const messages: Array<Record<string, unknown>> = [
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
{ type: 'text', text: 'earlier reply' },
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
const out = (await process_input_messages_responses_api(
|
||||
messages,
|
||||
)) as Array<Record<string, unknown>>;
|
||||
|
||||
// Compaction item comes first (it represents prior history)...
|
||||
expect(out).toHaveLength(2);
|
||||
expect(out[0]).toEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
});
|
||||
// ...followed by the assistant message with the reply preserved
|
||||
// (assistant text upgraded to output_text).
|
||||
expect(out[1]!.role).toBe('assistant');
|
||||
expect(out[1]!.content).toEqual([
|
||||
{ type: 'output_text', text: 'earlier reply' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('upgrades user/system text blocks to input_text', async () => {
|
||||
const messages: Array<Record<string, unknown>> = [
|
||||
{
|
||||
@@ -539,6 +599,39 @@ describe('create_chat_stream_handler_responses_api', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('emits a compaction event when a compaction output_item completes', async () => {
|
||||
const completion = asAsyncIterable([
|
||||
{
|
||||
type: 'response.output_item.done',
|
||||
item: {
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'response.completed',
|
||||
response: { usage: { input_tokens: 1, output_tokens: 2 } },
|
||||
},
|
||||
]);
|
||||
const init = create_chat_stream_handler_responses_api({
|
||||
deviations: undefined,
|
||||
completion,
|
||||
usage_calculator: () => ({}),
|
||||
});
|
||||
const harness = makeCapturingChatStream();
|
||||
await init({ chatStream: harness.chatStream });
|
||||
|
||||
const compaction = harness
|
||||
.events()
|
||||
.find((e: { type: string }) => e.type === 'compaction');
|
||||
expect(compaction).toEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
});
|
||||
});
|
||||
|
||||
it('emits a tool_use block when a function_call output_item completes', async () => {
|
||||
const completion = asAsyncIterable([
|
||||
{
|
||||
@@ -765,6 +858,31 @@ describe('handle_completion_output_responses_api non-stream', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('attaches a compaction artifact and allows a compaction-only output', async () => {
|
||||
const completion = {
|
||||
output: [
|
||||
{
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
},
|
||||
],
|
||||
output_text: ' ',
|
||||
usage: { input_tokens: 1, output_tokens: 0 },
|
||||
};
|
||||
const result = await handle_completion_output_responses_api({
|
||||
deviations: undefined,
|
||||
stream: false,
|
||||
completion,
|
||||
});
|
||||
// Compaction-only output is not rejected as "empty".
|
||||
expect(result.compaction).toEqual({
|
||||
type: 'compaction',
|
||||
id: 'cmpct_1',
|
||||
encrypted_content: 'ENC',
|
||||
});
|
||||
});
|
||||
|
||||
it('runs moderation against output_text when a moderate fn is supplied', async () => {
|
||||
const completion = {
|
||||
output: [{ role: 'assistant' }],
|
||||
|
||||
@@ -103,6 +103,25 @@ export class AIChatStream {
|
||||
this.stream.end();
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit a canonical compaction event into the NDJSON stream. Both the OpenAI
|
||||
* and Anthropic providers normalize their native inline-compaction artifact
|
||||
* to this single shape, so downstream consumers (controllers, puter.js) see
|
||||
* an identical `{ type: 'compaction', id, encrypted_content }` chunk
|
||||
* regardless of which upstream served the request.
|
||||
*
|
||||
* @param {{ id?: string, encrypted_content: string }} compaction
|
||||
*/
|
||||
compaction({ id, encrypted_content }) {
|
||||
this.stream.write(
|
||||
`${JSON.stringify({
|
||||
type: 'compaction',
|
||||
...(id !== undefined ? { id } : {}),
|
||||
encrypted_content,
|
||||
})}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
message() {
|
||||
return new AIChatMessageStream(this);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Inline-compaction translation helpers.
|
||||
*
|
||||
* The driver-facing surface exposes a single provider-neutral opt-in
|
||||
* (`compaction: boolean | { trigger_tokens }` on `ICompleteArguments`), plus a
|
||||
* raw `context_management` escape hatch for callers hitting `/responses` with
|
||||
* the OpenAI-native shape. These helpers map that neutral opt-in to each
|
||||
* provider's SDK shape so the providers stay free of opt-in-parsing logic.
|
||||
*/
|
||||
|
||||
/**
|
||||
* @param {boolean | { trigger_tokens?: number } | undefined} compaction
|
||||
* @returns {{ enabled: boolean, trigger_tokens?: number }}
|
||||
*/
|
||||
const readCompaction = (compaction) => {
|
||||
if (compaction === true) return { enabled: true };
|
||||
if (compaction && typeof compaction === 'object') {
|
||||
return {
|
||||
enabled: true,
|
||||
...(typeof compaction.trigger_tokens === 'number'
|
||||
? { trigger_tokens: compaction.trigger_tokens }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
return { enabled: false };
|
||||
};
|
||||
|
||||
/**
|
||||
* Build OpenAI Responses `context_management` from the neutral opt-in. A raw
|
||||
* `context_management` passthrough (already in OpenAI shape) wins.
|
||||
*
|
||||
* @param {{ compaction?: boolean | { trigger_tokens?: number }, context_management?: unknown }} args
|
||||
* @returns {Array<{ type: 'compaction', compact_threshold?: number }> | undefined}
|
||||
*/
|
||||
export const toOpenAiContextManagement = (args) => {
|
||||
if (args.context_management !== undefined) {
|
||||
return /** @type {any} */ (args.context_management);
|
||||
}
|
||||
const { enabled, trigger_tokens } = readCompaction(args.compaction);
|
||||
if (!enabled) return undefined;
|
||||
return [
|
||||
{
|
||||
type: 'compaction',
|
||||
...(trigger_tokens !== undefined
|
||||
? { compact_threshold: trigger_tokens }
|
||||
: {}),
|
||||
},
|
||||
];
|
||||
};
|
||||
|
||||
/**
|
||||
* Build Anthropic `context_management` (beta `compact-2026-01-12`) from the
|
||||
* neutral opt-in. A raw `context_management` passthrough wins.
|
||||
*
|
||||
* @param {{ compaction?: boolean | { trigger_tokens?: number }, context_management?: unknown }} args
|
||||
* @returns {{ edits: Array<Record<string, unknown>> } | undefined}
|
||||
*/
|
||||
export const toAnthropicContextManagement = (args) => {
|
||||
if (args.context_management !== undefined) {
|
||||
return /** @type {any} */ (args.context_management);
|
||||
}
|
||||
const { enabled, trigger_tokens } = readCompaction(args.compaction);
|
||||
if (!enabled) return undefined;
|
||||
return {
|
||||
edits: [
|
||||
{
|
||||
type: 'compact_20260112',
|
||||
...(trigger_tokens !== undefined
|
||||
? {
|
||||
trigger: {
|
||||
type: 'input_tokens',
|
||||
value: trigger_tokens,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
],
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether the request opted into inline compaction by any route.
|
||||
*
|
||||
* @param {{ compaction?: boolean | { trigger_tokens?: number }, context_management?: unknown }} args
|
||||
*/
|
||||
export const wantsCompaction = (args) =>
|
||||
args.context_management !== undefined ||
|
||||
readCompaction(args.compaction).enabled;
|
||||
|
||||
/**
|
||||
* Whether the (normalized) message list carries a round-tripped compaction
|
||||
* artifact. Such a request must route through a compaction-capable surface even
|
||||
* if it didn't request *new* compaction — chat.completions can't represent a
|
||||
* compaction content block, and Anthropic needs its compaction beta to accept
|
||||
* one as input.
|
||||
*
|
||||
* @param {unknown} messages
|
||||
*/
|
||||
export const messagesHaveCompaction = (messages) =>
|
||||
Array.isArray(messages) &&
|
||||
messages.some(
|
||||
(m) =>
|
||||
Array.isArray(m?.content) &&
|
||||
m.content.some((c) => c && c.type === 'compaction'),
|
||||
);
|
||||
@@ -0,0 +1,132 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
messagesHaveCompaction,
|
||||
toAnthropicContextManagement,
|
||||
toOpenAiContextManagement,
|
||||
wantsCompaction,
|
||||
} from './compaction.js';
|
||||
|
||||
describe('toOpenAiContextManagement', () => {
|
||||
it('returns undefined when compaction is off', () => {
|
||||
expect(toOpenAiContextManagement({})).toBeUndefined();
|
||||
expect(
|
||||
toOpenAiContextManagement({ compaction: false }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('builds a compaction entry from `true`', () => {
|
||||
expect(toOpenAiContextManagement({ compaction: true })).toEqual([
|
||||
{ type: 'compaction' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('maps trigger_tokens to compact_threshold', () => {
|
||||
expect(
|
||||
toOpenAiContextManagement({ compaction: { trigger_tokens: 1000 } }),
|
||||
).toEqual([{ type: 'compaction', compact_threshold: 1000 }]);
|
||||
});
|
||||
|
||||
it('passes a raw context_management payload through verbatim', () => {
|
||||
const raw = [{ type: 'compaction', compact_threshold: 5 }];
|
||||
expect(
|
||||
toOpenAiContextManagement({
|
||||
compaction: true,
|
||||
context_management: raw,
|
||||
}),
|
||||
).toBe(raw);
|
||||
});
|
||||
});
|
||||
|
||||
describe('toAnthropicContextManagement', () => {
|
||||
it('returns undefined when compaction is off', () => {
|
||||
expect(toAnthropicContextManagement({})).toBeUndefined();
|
||||
});
|
||||
|
||||
it('builds a compact_20260112 edit from `true`', () => {
|
||||
expect(toAnthropicContextManagement({ compaction: true })).toEqual({
|
||||
edits: [{ type: 'compact_20260112' }],
|
||||
});
|
||||
});
|
||||
|
||||
it('maps trigger_tokens to an input_tokens trigger', () => {
|
||||
expect(
|
||||
toAnthropicContextManagement({
|
||||
compaction: { trigger_tokens: 2000 },
|
||||
}),
|
||||
).toEqual({
|
||||
edits: [
|
||||
{
|
||||
type: 'compact_20260112',
|
||||
trigger: { type: 'input_tokens', value: 2000 },
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it('passes a raw context_management payload through verbatim', () => {
|
||||
const raw = { edits: [{ type: 'compact_20260112' }] };
|
||||
expect(
|
||||
toAnthropicContextManagement({ context_management: raw }),
|
||||
).toBe(raw);
|
||||
});
|
||||
});
|
||||
|
||||
describe('wantsCompaction', () => {
|
||||
it('is false without opt-in', () => {
|
||||
expect(wantsCompaction({})).toBe(false);
|
||||
expect(wantsCompaction({ compaction: false })).toBe(false);
|
||||
});
|
||||
|
||||
it('is true for the neutral opt-in or a raw passthrough', () => {
|
||||
expect(wantsCompaction({ compaction: true })).toBe(true);
|
||||
expect(wantsCompaction({ compaction: { trigger_tokens: 1 } })).toBe(
|
||||
true,
|
||||
);
|
||||
expect(wantsCompaction({ context_management: [] })).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('messagesHaveCompaction', () => {
|
||||
it('detects a round-tripped compaction content block', () => {
|
||||
expect(
|
||||
messagesHaveCompaction([
|
||||
{ role: 'user', content: [{ type: 'text', text: 'hi' }] },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'compaction', encrypted_content: 'ENC' },
|
||||
],
|
||||
},
|
||||
]),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('is false for ordinary messages or non-arrays', () => {
|
||||
expect(
|
||||
messagesHaveCompaction([
|
||||
{ role: 'user', content: [{ type: 'text', text: 'hi' }] },
|
||||
]),
|
||||
).toBe(false);
|
||||
expect(messagesHaveCompaction(undefined)).toBe(false);
|
||||
expect(messagesHaveCompaction('nope')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -175,9 +175,7 @@ export async function loadFileInput(
|
||||
// `path`/`uid`/`uuid` (e.g. AI chat `puter_path` content parts) could
|
||||
// exfiltrate any user's file. Must run before the S3 read below.
|
||||
await fsService.checkFSAccess(entry, actor, 'read');
|
||||
// S3 object key is recorded in entry.metadata.objectKey when written by
|
||||
// fsv2; older rows fall back to the entry uuid.
|
||||
const objectKey = deriveObjectKey(entry);
|
||||
const objectKey = entry.uuid;
|
||||
const { body, contentType, contentLength } =
|
||||
await stores.s3Object.getObjectStream(
|
||||
{
|
||||
@@ -244,29 +242,6 @@ function filenameFromMime(mime: string): string {
|
||||
return `input.${ext}`;
|
||||
}
|
||||
|
||||
// Mirrors FSService's private deriveObjectKeyFromEntry helper. fsv2-era
|
||||
// rows persist an `objectKey` in metadata; older rows simply use the uuid.
|
||||
function deriveObjectKey(entry: {
|
||||
uuid: string;
|
||||
metadata: string | null;
|
||||
}): string {
|
||||
if (entry.metadata) {
|
||||
try {
|
||||
const parsed = JSON.parse(entry.metadata);
|
||||
if (
|
||||
parsed &&
|
||||
typeof parsed.objectKey === 'string' &&
|
||||
parsed.objectKey.length > 0
|
||||
) {
|
||||
return parsed.objectKey;
|
||||
}
|
||||
} catch {
|
||||
// Not JSON — fall through.
|
||||
}
|
||||
}
|
||||
return entry.uuid;
|
||||
}
|
||||
|
||||
export function inferFilenameFromUrlOrPath(
|
||||
value: string,
|
||||
fallback = 'input',
|
||||
|
||||
@@ -78,6 +78,12 @@ describe('extension.import("client") optional client access', () => {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
expect(probe()).toBe(fake);
|
||||
// The import proxy method-binds, so the result is a binding proxy over
|
||||
// `fake` rather than the raw reference (identity is intentionally not
|
||||
// preserved). What the probe pattern locks is that a registered client
|
||||
// surfaces a callable method.
|
||||
const result = probe();
|
||||
expect(result).not.toBeNull();
|
||||
expect(typeof (result as { query: unknown }).query).toBe('function');
|
||||
});
|
||||
});
|
||||
@@ -201,6 +201,35 @@ const makeUseFn = (): ExtensionUseFn => {
|
||||
* `'store:baz'` / `'controller:qux'` / `'driver:fred'` — returns a lazy
|
||||
* proxy to the registered instance (thrown on use-before-init).
|
||||
*/
|
||||
/**
|
||||
* Wrap a resolved layer instance so that pulling a method off the import
|
||||
* comes out *bound* to the instance. Extensions routinely grab a method as a
|
||||
* bare reference — `const { write } = extension.import('service').fs` or
|
||||
* `const w = svc.fs.write` — then call it detached; without binding, `this`
|
||||
* is `undefined` and the method's private-field access throws on the first
|
||||
* line. Getters keep the real instance as their receiver, so private-field
|
||||
* reads inside accessors still resolve. Only `get` is trapped; writes, `in`,
|
||||
* and descriptor reads fall through to the instance unchanged.
|
||||
*
|
||||
* Trade-off: each method access returns a fresh bound function, so reference
|
||||
* identity is not stable (`svc.fs.write !== svc.fs.write`). That's acceptable
|
||||
* for the import surface, where instances are grabbed once and methods called.
|
||||
*/
|
||||
const bindLayerMethods = <T>(instance: T): T => {
|
||||
if (instance === null || typeof instance !== 'object') {
|
||||
return instance;
|
||||
}
|
||||
return new Proxy(instance as object, {
|
||||
get(target, prop) {
|
||||
const value = Reflect.get(target, prop, target);
|
||||
return typeof value === 'function'
|
||||
? // eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
(value as (...a: any[]) => unknown).bind(target)
|
||||
: value;
|
||||
},
|
||||
}) as T;
|
||||
};
|
||||
|
||||
export const extension = {
|
||||
// -- Config access -----------------------------------------------
|
||||
//
|
||||
@@ -324,7 +353,7 @@ export const extension = {
|
||||
};
|
||||
return new Proxy({}, proxyProxyHandler) as object;
|
||||
}
|
||||
return proxiedObj;
|
||||
return bindLayerMethods(proxiedObj);
|
||||
},
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -352,7 +381,7 @@ export const extension = {
|
||||
};
|
||||
return new Proxy({}, proxyProxyHandler) as object;
|
||||
}
|
||||
return proxiedObj;
|
||||
return bindLayerMethods(proxiedObj);
|
||||
},
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -380,7 +409,7 @@ export const extension = {
|
||||
};
|
||||
return new Proxy({}, proxyProxyHandler) as object;
|
||||
}
|
||||
return proxiedObj;
|
||||
return bindLayerMethods(proxiedObj);
|
||||
},
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -408,7 +437,7 @@ export const extension = {
|
||||
};
|
||||
return new Proxy({}, proxyProxyHandler) as object;
|
||||
}
|
||||
return proxiedObj;
|
||||
return bindLayerMethods(proxiedObj);
|
||||
},
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -436,7 +465,7 @@ export const extension = {
|
||||
};
|
||||
return new Proxy({}, proxyProxyHandler) as object;
|
||||
}
|
||||
return proxiedObj;
|
||||
return bindLayerMethods(proxiedObj);
|
||||
},
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
|
||||
+10
-10
@@ -9,7 +9,7 @@
|
||||
"bench": "vitest bench --config=vitest.bench.config.ts --run"
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "^0.68.0",
|
||||
"@anthropic-ai/sdk": "^0.105.0",
|
||||
"@aws-sdk/client-dynamodb": "^3.490.0",
|
||||
"@aws-sdk/client-polly": "^3.1028.0",
|
||||
"@aws-sdk/client-s3": "^3.1028.0",
|
||||
@@ -22,13 +22,13 @@
|
||||
"@heyputer/putility": "^1.0.0",
|
||||
"@mistralai/mistralai": "^1.15.1",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
"@opentelemetry/auto-instrumentations-node": "^0.52.0",
|
||||
"@opentelemetry/exporter-metrics-otlp-grpc": "^0.55.0",
|
||||
"@opentelemetry/exporter-trace-otlp-grpc": "^0.55.0",
|
||||
"@opentelemetry/resources": "^1.28.0",
|
||||
"@opentelemetry/sdk-metrics": "^1.28.0",
|
||||
"@opentelemetry/sdk-node": "^0.55.0",
|
||||
"@opentelemetry/sdk-trace-base": "^1.28.0",
|
||||
"@opentelemetry/auto-instrumentations-node": "^0.77.0",
|
||||
"@opentelemetry/exporter-metrics-otlp-grpc": "^0.219.0",
|
||||
"@opentelemetry/exporter-trace-otlp-grpc": "^0.219.0",
|
||||
"@opentelemetry/resources": "^2.8.0",
|
||||
"@opentelemetry/sdk-metrics": "^2.8.0",
|
||||
"@opentelemetry/sdk-node": "^0.219.0",
|
||||
"@opentelemetry/sdk-trace-base": "^2.8.0",
|
||||
"@opentelemetry/semantic-conventions": "^1.28.0",
|
||||
"@pagerduty/pdjs": "^2.2.4",
|
||||
"@smithy/node-http-handler": "^2.5.0",
|
||||
@@ -57,7 +57,7 @@
|
||||
"mime-types": "^2.1.35",
|
||||
"murmurhash": "^2.0.1",
|
||||
"mysql2": "^3.22.4",
|
||||
"nodemailer": "^8.0.7",
|
||||
"nodemailer": "^9.0.1",
|
||||
"openai": "^6.34.0",
|
||||
"otpauth": "^9.2.4",
|
||||
"parse-domain": "^8.2.2",
|
||||
@@ -69,8 +69,8 @@
|
||||
"svg-captcha": "^1.4.0",
|
||||
"together-ai": "^0.33.0",
|
||||
"ua-parser-js": "^1.0.41",
|
||||
"undici": "^7.25.0",
|
||||
"uglify-js": "^3.17.4",
|
||||
"undici": "^7.25.0",
|
||||
"uuid": "^14.0.0",
|
||||
"validator": "^13.15.35"
|
||||
},
|
||||
|
||||
+18
-7
@@ -31,13 +31,14 @@ import { puterClients } from './clients';
|
||||
import { puterControllers } from './controllers';
|
||||
import { createAuthProbe } from './core/http/middleware/authProbe';
|
||||
import { createRequestContextMiddleware } from './core/http/middleware/requestContext';
|
||||
import { createFingerprintMiddleware } from './core/http/middleware/fingerprint';
|
||||
import { createErrorHandler } from './core/http/middleware/errorHandler';
|
||||
import { isHttpError } from './core/http/HttpError';
|
||||
import {
|
||||
adminOnlyGate,
|
||||
allowedAppIdsGate,
|
||||
requireAuthGate,
|
||||
requireEmailConfirmedGate,
|
||||
requireVerifiedAccount,
|
||||
requireNonAccessTokenGate,
|
||||
requireUserActorGate,
|
||||
requireVerifiedGate,
|
||||
@@ -485,6 +486,13 @@ export class PuterServer {
|
||||
);
|
||||
}
|
||||
|
||||
// -- Request fingerprints ------------------------------------
|
||||
// Stamp `req.networkFingerprint` (server-derived) and
|
||||
// `req.deviceFingerprint` (client-supplied, from body/header). Runs
|
||||
// AFTER body parsing so the body fingerprint is readable, and before
|
||||
// the ALS context so the snapshot carries them.
|
||||
this.#app.use(createFingerprintMiddleware());
|
||||
|
||||
// -- Per-request ALS context ---------------------------------
|
||||
// Runs AFTER auth probe so `req.actor` is already populated when
|
||||
// we snapshot it into the context.
|
||||
@@ -830,13 +838,16 @@ export class PuterServer {
|
||||
mwChain.push(requireAuthGate());
|
||||
}
|
||||
|
||||
// Default-on email confirmation gate. Every authenticated route
|
||||
// rejects users pending confirmation unless `allowUnconfirmed`
|
||||
// opts out. This prevents unconfirmed accounts from accessing
|
||||
// AI, FS, driver, etc. endpoints while still allowing essential
|
||||
// flows (logout, confirm-email, whoami, save-account, …).
|
||||
// Default-on account-verification gate. Every authenticated route
|
||||
// rejects accounts still pending any signup-time verification —
|
||||
// email confirmation, SMS phone verification, or card verification —
|
||||
// unless `allowUnconfirmed` opts out. This is what keeps low-reputation
|
||||
// signups (which the abuse harness flags instead of hard-blocking) out
|
||||
// of AI, FS, driver, etc. endpoints server-side, not just behind the
|
||||
// GUI modal, while still allowing essential flows (logout,
|
||||
// confirm-email / -phone, card verification, whoami, save-account, …).
|
||||
if (needsAuth && !opts.allowUnconfirmed) {
|
||||
mwChain.push(requireEmailConfirmedGate());
|
||||
mwChain.push(requireVerifiedAccount());
|
||||
}
|
||||
|
||||
// block access tokens by default
|
||||
|
||||
@@ -26,7 +26,9 @@ import { PuterService } from '../types.js';
|
||||
*/
|
||||
const RECOMMENDED_APP_NAMES = [
|
||||
'app-center',
|
||||
'builder',
|
||||
'dev-center',
|
||||
'butler',
|
||||
'editor',
|
||||
'code',
|
||||
'camera',
|
||||
|
||||
@@ -1919,6 +1919,11 @@ export class AuthService extends PuterService {
|
||||
email_confirmed: user.email_confirmed ?? false,
|
||||
requires_email_confirmation:
|
||||
user.requires_email_confirmation ?? false,
|
||||
phone: user.phone ?? null,
|
||||
requires_phone_verification:
|
||||
user.requires_phone_verification ?? false,
|
||||
requires_card_verification:
|
||||
user.requires_card_verification ?? false,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import crypto from 'node:crypto';
|
||||
import jwt from 'jsonwebtoken';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import type { OIDCService } from './OIDCService.js';
|
||||
|
||||
const TEST_ORIGIN = 'http://test.local';
|
||||
const MS_CLIENT_ID = 'ms-client';
|
||||
// Home tenant of personal Microsoft accounts — mirrors the constant in
|
||||
// OIDCService.
|
||||
const MSA_TENANT = '9188040d-6c67-4c5b-b112-36a304b66dad';
|
||||
const ENTRA_TENANT = '3a8757eb-bf01-4b5d-83b2-90e0eaf21d10';
|
||||
const KID = 'ms-key-1';
|
||||
|
||||
const MS_DISCOVERY_URL =
|
||||
'https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration';
|
||||
const MS_JWKS_URI =
|
||||
'https://login.microsoftonline.com/common/discovery/v2.0/keys';
|
||||
const MS_USERINFO = 'https://graph.microsoft.com/oidc/userinfo';
|
||||
|
||||
let server: PuterServer;
|
||||
let privateKey: crypto.KeyObject;
|
||||
let jwk: Record<string, unknown>;
|
||||
const fetchedUrls: string[] = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer({
|
||||
origin: TEST_ORIGIN,
|
||||
oidc: {
|
||||
providers: {
|
||||
microsoft: {
|
||||
client_id: MS_CLIENT_ID,
|
||||
client_secret: 'ms-secret',
|
||||
},
|
||||
},
|
||||
},
|
||||
} as never);
|
||||
|
||||
const pair = crypto.generateKeyPairSync('rsa', { modulusLength: 2048 });
|
||||
privateKey = pair.privateKey;
|
||||
jwk = {
|
||||
...(pair.publicKey.export({ format: 'jwk' }) as Record<
|
||||
string,
|
||||
unknown
|
||||
>),
|
||||
kid: KID,
|
||||
};
|
||||
|
||||
// Serve Microsoft discovery + JWKS over a fake fetch. The Graph
|
||||
// userinfo endpoint is deliberately NOT handled — Microsoft claims
|
||||
// must come from the verified id_token, never from userinfo.
|
||||
vi.stubGlobal('fetch', (async (input: unknown) => {
|
||||
const url = String(input);
|
||||
fetchedUrls.push(url);
|
||||
if (url === MS_DISCOVERY_URL) {
|
||||
return {
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
issuer: 'https://login.microsoftonline.com/{tenantid}/v2.0',
|
||||
authorization_endpoint:
|
||||
'https://login.microsoftonline.com/common/oauth2/v2.0/authorize',
|
||||
token_endpoint:
|
||||
'https://login.microsoftonline.com/common/oauth2/v2.0/token',
|
||||
userinfo_endpoint: MS_USERINFO,
|
||||
jwks_uri: MS_JWKS_URI,
|
||||
}),
|
||||
} as Response;
|
||||
}
|
||||
if (url === MS_JWKS_URI) {
|
||||
return {
|
||||
ok: true,
|
||||
json: async () => ({ keys: [jwk] }),
|
||||
} as Response;
|
||||
}
|
||||
throw new Error(`unexpected fetch in test: ${url}`);
|
||||
}) as typeof fetch);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
vi.unstubAllGlobals();
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
const oidc = (): OIDCService =>
|
||||
server.services.oidc as unknown as OIDCService;
|
||||
|
||||
const signMsIdToken = (
|
||||
tid: string,
|
||||
payload: Record<string, unknown> = {},
|
||||
key: crypto.KeyObject = privateKey,
|
||||
): string =>
|
||||
jwt.sign({ tid, ...payload }, key, {
|
||||
algorithm: 'RS256',
|
||||
keyid: KID,
|
||||
subject: 'ms-sub-1',
|
||||
audience: MS_CLIENT_ID,
|
||||
issuer: `https://login.microsoftonline.com/${tid}/v2.0`,
|
||||
expiresIn: '5m',
|
||||
});
|
||||
|
||||
describe('OIDCService.getUserInfo (microsoft)', () => {
|
||||
it('reads claims from the verified id_token, never Graph userinfo', async () => {
|
||||
const info = await oidc().getUserInfo(
|
||||
'microsoft',
|
||||
'access-token',
|
||||
signMsIdToken(MSA_TENANT, { email: '[email protected]' }),
|
||||
);
|
||||
expect(info).toEqual({
|
||||
sub: 'ms-sub-1',
|
||||
email: '[email protected]',
|
||||
email_verified: true,
|
||||
});
|
||||
expect(fetchedUrls).not.toContain(MS_USERINFO);
|
||||
});
|
||||
|
||||
it('marks Entra emails verified only when xms_edov attests them', async () => {
|
||||
const withEdov = await oidc().getUserInfo(
|
||||
'microsoft',
|
||||
'access-token',
|
||||
signMsIdToken(ENTRA_TENANT, {
|
||||
email: '[email protected]',
|
||||
xms_edov: true,
|
||||
}),
|
||||
);
|
||||
expect(withEdov?.email_verified).toBe(true);
|
||||
|
||||
const withoutEdov = await oidc().getUserInfo(
|
||||
'microsoft',
|
||||
'access-token',
|
||||
signMsIdToken(ENTRA_TENANT, { email: '[email protected]' }),
|
||||
);
|
||||
expect(withoutEdov?.email_verified).toBe(false);
|
||||
});
|
||||
|
||||
it('omits email (rather than inventing one) when the token has none', async () => {
|
||||
const info = await oidc().getUserInfo(
|
||||
'microsoft',
|
||||
'access-token',
|
||||
signMsIdToken(ENTRA_TENANT),
|
||||
);
|
||||
expect(info?.sub).toBe('ms-sub-1');
|
||||
expect(info?.email).toBeUndefined();
|
||||
});
|
||||
|
||||
it('returns null when no id_token is supplied', async () => {
|
||||
const info = await oidc().getUserInfo('microsoft', 'access-token');
|
||||
expect(info).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null for an id_token signed by an unknown key', async () => {
|
||||
const otherKey = crypto.generateKeyPairSync('rsa', {
|
||||
modulusLength: 2048,
|
||||
}).privateKey;
|
||||
const forged = signMsIdToken(
|
||||
MSA_TENANT,
|
||||
{ email: '[email protected]' },
|
||||
otherKey,
|
||||
);
|
||||
const info = await oidc().getUserInfo(
|
||||
'microsoft',
|
||||
'access-token',
|
||||
forged,
|
||||
);
|
||||
expect(info).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('OIDCService.createUserFromOIDC', () => {
|
||||
const req = { headers: {}, ip: '127.0.0.1', socket: {} } as never;
|
||||
|
||||
it('refuses to create an account when the provider returned no email', async () => {
|
||||
const result = await runWithContext({ req }, () =>
|
||||
oidc().createUserFromOIDC('microsoft', {
|
||||
sub: 'no-email-sub',
|
||||
email_verified: true,
|
||||
}),
|
||||
);
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.error).toMatch(/email/i);
|
||||
});
|
||||
|
||||
it('refuses when the provider explicitly reports the email unverified', async () => {
|
||||
const result = await runWithContext({ req }, () =>
|
||||
oidc().createUserFromOIDC('microsoft', {
|
||||
sub: 'unverified-sub',
|
||||
email: '[email protected]',
|
||||
email_verified: false,
|
||||
}),
|
||||
);
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.error).toMatch(/verify/i);
|
||||
});
|
||||
});
|
||||
@@ -37,6 +37,10 @@ const MICROSOFT_DISCOVERY_URL_TEMPLATE =
|
||||
const GOOGLE_SCOPES = 'openid email profile';
|
||||
const APPLE_SCOPES = 'openid email';
|
||||
const MICROSOFT_SCOPES = 'openid email profile';
|
||||
// Home tenant of personal Microsoft accounts (outlook.com, hotmail, …).
|
||||
// Microsoft verifies those emails itself; work/school (Entra) emails are
|
||||
// admin-editable and only attested via the opt-in `xms_edov` claim.
|
||||
const MICROSOFT_CONSUMER_TENANT = '9188040d-6c67-4c5b-b112-36a304b66dad';
|
||||
const STATE_EXPIRY_SEC = 600; // 10 minutes
|
||||
const VALID_OIDC_FLOWS = ['login', 'signup', 'revalidate'] as const;
|
||||
const REVALIDATION_EXPIRY_SEC = 300; // 5 minutes
|
||||
@@ -300,8 +304,29 @@ export class OIDCService extends PuterService {
|
||||
idToken?: string,
|
||||
): Promise<OIDCUserInfo | null> {
|
||||
const config = await this.getProviderConfig(providerId);
|
||||
if (!config) return null;
|
||||
|
||||
if (config?.userinfo_endpoint) {
|
||||
// Microsoft: Graph's userinfo endpoint omits `email` for many Entra
|
||||
// accounts and never returns `email_verified`, so read claims from
|
||||
// the verified id_token instead. The email only counts as verified
|
||||
// for personal accounts (Microsoft verifies those itself) or when
|
||||
// the `xms_edov` claim attests the email's domain belongs to the
|
||||
// issuing tenant — an Entra admin can put any address in `mail`
|
||||
// (nOAuth), so everything else is unverified.
|
||||
if (providerId === 'microsoft') {
|
||||
if (!idToken) return null;
|
||||
const claims = await this.#verifyIdToken(idToken, config);
|
||||
if (!claims) return null;
|
||||
return {
|
||||
sub: claims.sub,
|
||||
email: claims.email,
|
||||
email_verified:
|
||||
claims.tid === MICROSOFT_CONSUMER_TENANT ||
|
||||
claims.xms_edov === true,
|
||||
};
|
||||
}
|
||||
|
||||
if (config.userinfo_endpoint) {
|
||||
const res = await fetch(config.userinfo_endpoint, {
|
||||
headers: { Authorization: `Bearer ${accessToken}` },
|
||||
});
|
||||
@@ -311,7 +336,7 @@ export class OIDCService extends PuterService {
|
||||
|
||||
// No userinfo endpoint — verify the id_token signature against the
|
||||
// provider's JWKS and read claims from it (e.g. Apple).
|
||||
if (idToken && config) {
|
||||
if (idToken) {
|
||||
return this.#verifyIdToken(idToken, config);
|
||||
}
|
||||
|
||||
@@ -407,6 +432,17 @@ export class OIDCService extends PuterService {
|
||||
};
|
||||
}
|
||||
|
||||
// No email, no account. Providers can legitimately omit the claim
|
||||
// (e.g. Entra accounts with an empty `mail` attribute); refuse
|
||||
// rather than mint an account we can never contact or recover.
|
||||
const email = claims.email;
|
||||
if (!email) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'Provider did not supply an email address.',
|
||||
};
|
||||
}
|
||||
|
||||
// Generate a unique username
|
||||
let username: string;
|
||||
let attempts = 0;
|
||||
@@ -434,7 +470,7 @@ export class OIDCService extends PuterService {
|
||||
// IdP already authenticated the user, so captcha listeners
|
||||
// (e.g. Turnstile) should skip — abuse/IP/email checks still run.
|
||||
source: 'oidc' as const,
|
||||
data: { username, email: claims.email ?? '' },
|
||||
data: { username, email },
|
||||
ip:
|
||||
(req?.headers?.['x-forwarded-for'] as string | undefined) ||
|
||||
req?.connection?.remoteAddress ||
|
||||
@@ -442,10 +478,13 @@ export class OIDCService extends PuterService {
|
||||
req?.socket?.remoteAddress ||
|
||||
null,
|
||||
user_agent: req?.headers?.['user-agent'] ?? null,
|
||||
email: claims.email ?? '',
|
||||
email,
|
||||
allow: true,
|
||||
no_temp_user: false,
|
||||
requires_email_confirmation: false,
|
||||
requires_phone_verification: false,
|
||||
requires_card_verification: false,
|
||||
reputation: null as number | null,
|
||||
message: null as string | null,
|
||||
code: null as string | null,
|
||||
};
|
||||
@@ -466,48 +505,62 @@ export class OIDCService extends PuterService {
|
||||
};
|
||||
}
|
||||
|
||||
// Email validation — mirrors AuthController#validateEmail. Skipped
|
||||
// when the IdP didn't return an email (createUserFromOIDC is
|
||||
// reachable with `email` undefined).
|
||||
if (claims.email) {
|
||||
if (isBlockedEmail(claims.email, this.config.blockedEmailDomains)) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'This email is not allowed.',
|
||||
};
|
||||
}
|
||||
const emailEvent = {
|
||||
email: cleanEmail(claims.email),
|
||||
allow: true,
|
||||
message: null as string | null,
|
||||
// Email validation — mirrors AuthController#validateEmail.
|
||||
if (isBlockedEmail(email, this.config.blockedEmailDomains)) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'This email is not allowed.',
|
||||
};
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
'email.validate',
|
||||
emailEvent,
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[oidc] email validate hook failed:', e);
|
||||
}
|
||||
if (!emailEvent.allow) {
|
||||
return {
|
||||
success: false,
|
||||
error:
|
||||
emailEvent.message ??
|
||||
'This email cannot be used. Please try a different email address.',
|
||||
};
|
||||
}
|
||||
}
|
||||
const emailEvent = {
|
||||
email: cleanEmail(email),
|
||||
allow: true,
|
||||
message: null as string | null,
|
||||
};
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
'email.validate',
|
||||
emailEvent,
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[oidc] email validate hook failed:', e);
|
||||
}
|
||||
if (!emailEvent.allow) {
|
||||
return {
|
||||
success: false,
|
||||
error:
|
||||
emailEvent.message ??
|
||||
'This email cannot be used. Please try a different email address.',
|
||||
};
|
||||
}
|
||||
|
||||
const cfg = this.config as {
|
||||
always_require_phone_verification?: boolean;
|
||||
always_require_card_verification?: boolean;
|
||||
};
|
||||
const force_phone_verification =
|
||||
Boolean(validateEvent.requires_phone_verification) ||
|
||||
Boolean(cfg.always_require_phone_verification);
|
||||
const force_card_verification =
|
||||
Boolean(validateEvent.requires_card_verification) ||
|
||||
Boolean(cfg.always_require_card_verification);
|
||||
|
||||
const created = await this.stores.user.create({
|
||||
username,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email: claims.email ?? null,
|
||||
clean_email: claims.email ? cleanEmail(claims.email) : null,
|
||||
email,
|
||||
clean_email: cleanEmail(email),
|
||||
free_storage: this.config.storage_capacity ?? null,
|
||||
// Email is provider-verified, so the email step is always skipped;
|
||||
// the phone/card gates still apply when the harness flagged them.
|
||||
requires_email_confirmation: false,
|
||||
requires_phone_verification: force_phone_verification,
|
||||
requires_card_verification: force_card_verification,
|
||||
...(validateEvent.reputation != null
|
||||
? { reputation: validateEvent.reputation }
|
||||
: {}),
|
||||
audit_metadata: {
|
||||
ip: clientIp,
|
||||
ip_fwd: proxyIpChain,
|
||||
@@ -680,6 +733,8 @@ export class OIDCService extends PuterService {
|
||||
sub: claims.sub,
|
||||
email: claims.email,
|
||||
email_verified: claims.email_verified,
|
||||
tid: claims.tid,
|
||||
xms_edov: claims.xms_edov,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -54,9 +54,10 @@ const signToken = (
|
||||
audience?: string;
|
||||
issuer?: string;
|
||||
key?: crypto.KeyObject;
|
||||
payload?: Record<string, unknown>;
|
||||
} = {},
|
||||
): string =>
|
||||
jwt.sign({ email: '[email protected]', email_verified: true }, overrides.key ?? privateKey, {
|
||||
jwt.sign({ email: '[email protected]', email_verified: true, ...overrides.payload }, overrides.key ?? privateKey, {
|
||||
algorithm: 'RS256',
|
||||
keyid: overrides.kid ?? KID,
|
||||
subject: 'user-123',
|
||||
@@ -210,3 +211,87 @@ describe('verifyOidcIdToken', () => {
|
||||
expect(calls()).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// Multi-tenant Microsoft discovery returns the issuer as a template with a
|
||||
// literal '{tenantid}' placeholder; the verifier substitutes the token's own
|
||||
// `tid` claim so that `iss` and `tid` must agree.
|
||||
describe('verifyOidcIdToken with a {tenantid} issuer template', () => {
|
||||
const TEMPLATE_ISSUER = 'https://login.microsoftonline.com/{tenantid}/v2.0';
|
||||
const TID = '3a8757eb-bf01-4b5d-83b2-90e0eaf21d10';
|
||||
const tenantIssuer = `https://login.microsoftonline.com/${TID}/v2.0`;
|
||||
const templateOpts = {
|
||||
jwksUri: JWKS_URI,
|
||||
issuer: TEMPLATE_ISSUER,
|
||||
audience: AUDIENCE,
|
||||
};
|
||||
|
||||
it('accepts a token whose iss matches its own tid, and passes tid/xms_edov through', async () => {
|
||||
const { fetchImpl } = makeFetch([jwk]);
|
||||
const claims = await verifyOidcIdToken(
|
||||
signToken({
|
||||
issuer: tenantIssuer,
|
||||
payload: { tid: TID, xms_edov: true },
|
||||
}),
|
||||
templateOpts,
|
||||
deps(fetchImpl),
|
||||
);
|
||||
expect(claims).toEqual({
|
||||
sub: 'user-123',
|
||||
email: '[email protected]',
|
||||
email_verified: true,
|
||||
tid: TID,
|
||||
xms_edov: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects a token whose iss names a different tenant than its tid', async () => {
|
||||
const { fetchImpl } = makeFetch([jwk]);
|
||||
const claims = await verifyOidcIdToken(
|
||||
signToken({
|
||||
issuer:
|
||||
'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/v2.0',
|
||||
payload: { tid: TID },
|
||||
}),
|
||||
templateOpts,
|
||||
deps(fetchImpl),
|
||||
);
|
||||
expect(claims).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects a token whose tid is not a UUID (no substitution into the issuer)', async () => {
|
||||
const { fetchImpl, calls } = makeFetch([jwk]);
|
||||
const claims = await verifyOidcIdToken(
|
||||
signToken({
|
||||
issuer: 'https://login.microsoftonline.com/evil/v2.0',
|
||||
payload: { tid: 'evil' },
|
||||
}),
|
||||
templateOpts,
|
||||
deps(fetchImpl),
|
||||
);
|
||||
expect(claims).toBeNull();
|
||||
expect(calls()).toBe(0);
|
||||
});
|
||||
|
||||
it('rejects a token with no tid claim at all', async () => {
|
||||
const { fetchImpl } = makeFetch([jwk]);
|
||||
const claims = await verifyOidcIdToken(
|
||||
signToken({ issuer: tenantIssuer }),
|
||||
templateOpts,
|
||||
deps(fetchImpl),
|
||||
);
|
||||
expect(claims).toBeNull();
|
||||
});
|
||||
|
||||
it('normalizes a string-encoded xms_edov to boolean', async () => {
|
||||
const { fetchImpl } = makeFetch([jwk]);
|
||||
const claims = await verifyOidcIdToken(
|
||||
signToken({
|
||||
issuer: tenantIssuer,
|
||||
payload: { tid: TID, xms_edov: 'true' },
|
||||
}),
|
||||
templateOpts,
|
||||
deps(fetchImpl),
|
||||
);
|
||||
expect(claims?.xms_edov).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -37,6 +37,14 @@ export interface IdTokenClaims {
|
||||
sub: string;
|
||||
email?: string;
|
||||
email_verified?: boolean;
|
||||
/** Microsoft: tenant id of the account's home tenant. */
|
||||
tid?: string;
|
||||
/**
|
||||
* Microsoft: "email domain owner verified" — true when the email's
|
||||
* domain is a verified domain of the issuing tenant. Opt-in claim,
|
||||
* configured on the Azure app registration.
|
||||
*/
|
||||
xms_edov?: boolean;
|
||||
}
|
||||
|
||||
export interface VerifyIdTokenOptions {
|
||||
@@ -149,6 +157,27 @@ export const verifyOidcIdToken = async (
|
||||
: null;
|
||||
if (!kid) return null;
|
||||
|
||||
// Multi-tenant Microsoft discovery returns the issuer as a template
|
||||
// ('https://login.microsoftonline.com/{tenantid}/v2.0'). Substitute the
|
||||
// token's own `tid` before verification — `jwt.verify` then enforces
|
||||
// that `iss` agrees with `tid`, and the signature check pins both to
|
||||
// the IdP.
|
||||
let issuer = opts.issuer;
|
||||
if (issuer?.includes('{tenantid}')) {
|
||||
const unverified =
|
||||
decoded && typeof decoded.payload === 'object'
|
||||
? (decoded.payload as Record<string, unknown>)
|
||||
: null;
|
||||
const tid = unverified?.tid;
|
||||
if (
|
||||
typeof tid !== 'string' ||
|
||||
!/^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$/i.test(tid)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
issuer = issuer.replace('{tenantid}', tid);
|
||||
}
|
||||
|
||||
const pem = await getSigningKey(opts.jwksUri, kid, deps);
|
||||
if (!pem) return null;
|
||||
|
||||
@@ -156,12 +185,22 @@ export const verifyOidcIdToken = async (
|
||||
const payload = jwt.verify(idToken, pem, {
|
||||
algorithms: ['RS256', 'ES256'],
|
||||
audience: opts.audience,
|
||||
issuer: opts.issuer,
|
||||
issuer,
|
||||
}) as Record<string, unknown>;
|
||||
return {
|
||||
sub: payload.sub as string,
|
||||
email: payload.email as string | undefined,
|
||||
email_verified: payload.email_verified as boolean | undefined,
|
||||
tid: payload.tid as string | undefined,
|
||||
// Documented as boolean; normalize string encodings defensively
|
||||
// so a representation change can't silently flip accounts to
|
||||
// unverified.
|
||||
xms_edov:
|
||||
payload.xms_edov === undefined
|
||||
? undefined
|
||||
: payload.xms_edov === true ||
|
||||
payload.xms_edov === 'true' ||
|
||||
payload.xms_edov === '1',
|
||||
};
|
||||
} catch (e) {
|
||||
console.warn('[oidc] id_token verification failed', e);
|
||||
|
||||
@@ -70,8 +70,8 @@ import { AclMode } from '../acl/ACLService.js';
|
||||
const DEFAULT_CONTENT_TYPE = 'application/octet-stream';
|
||||
const DEFAULT_SIGNED_UPLOAD_EXPIRY_SECONDS = 60 * 15;
|
||||
|
||||
// AWS SDK v3 surfaces missing-key errors with both `name` and `Code` set to
|
||||
// "NoSuchKey". Check both — `Code` is the wire field, `name` is the JS class.
|
||||
const RESERVED_METADATA_KEYS: readonly string[] = ['objectKey'];
|
||||
|
||||
const isNoSuchKeyError = (err: unknown): boolean => {
|
||||
if (!err || typeof err !== 'object') return false;
|
||||
const e = err as { name?: unknown; Code?: unknown };
|
||||
@@ -377,7 +377,7 @@ export class FSService extends PuterService {
|
||||
size,
|
||||
contentType: metadata.contentType ?? DEFAULT_CONTENT_TYPE,
|
||||
checksumSha256: metadata.checksumSha256,
|
||||
metadata: metadata.metadata,
|
||||
metadata: this.#sanitizeClientMetadata(metadata.metadata),
|
||||
thumbnail: metadata.thumbnail,
|
||||
associatedAppId: metadata.associatedAppId,
|
||||
overwrite: Boolean(metadata.overwrite),
|
||||
@@ -391,6 +391,50 @@ export class FSService extends PuterService {
|
||||
};
|
||||
}
|
||||
|
||||
#sanitizeClientMetadata(
|
||||
metadata: FSEntryWriteInput['metadata'],
|
||||
): FSEntryWriteInput['metadata'] {
|
||||
if (metadata === null || metadata === undefined) {
|
||||
return metadata;
|
||||
}
|
||||
if (typeof metadata === 'string') {
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(metadata);
|
||||
} catch {
|
||||
return metadata;
|
||||
}
|
||||
if (
|
||||
!parsed ||
|
||||
typeof parsed !== 'object' ||
|
||||
Array.isArray(parsed)
|
||||
) {
|
||||
return metadata;
|
||||
}
|
||||
return JSON.stringify(
|
||||
this.#stripReservedMetadataKeys(
|
||||
parsed as Record<string, unknown>,
|
||||
),
|
||||
);
|
||||
}
|
||||
if (typeof metadata === 'object' && !Array.isArray(metadata)) {
|
||||
return this.#stripReservedMetadataKeys(
|
||||
metadata as Record<string, unknown>,
|
||||
);
|
||||
}
|
||||
return metadata;
|
||||
}
|
||||
|
||||
#stripReservedMetadataKeys(
|
||||
record: Record<string, unknown>,
|
||||
): Record<string, unknown> {
|
||||
const cleaned: Record<string, unknown> = { ...record };
|
||||
for (const key of RESERVED_METADATA_KEYS) {
|
||||
delete cleaned[key];
|
||||
}
|
||||
return cleaned;
|
||||
}
|
||||
|
||||
async #findDedupedPath(
|
||||
targetPath: string,
|
||||
reservedPaths: Set<string>,
|
||||
@@ -2762,10 +2806,7 @@ export class FSService extends PuterService {
|
||||
{ legacyCode: 'shortcut_target_not_found' },
|
||||
);
|
||||
}
|
||||
// Derive the S3 object key from entry metadata if present, else fall
|
||||
// back to the uuid convention used elsewhere (objectKey defaults to
|
||||
// uuid during write when no metadata override is set).
|
||||
const objectKey = this.#deriveObjectKeyFromEntry(entry);
|
||||
const objectKey = entry.uuid;
|
||||
try {
|
||||
return await this.stores.s3Object.getObjectStream(
|
||||
{
|
||||
@@ -2815,27 +2856,6 @@ export class FSService extends PuterService {
|
||||
}
|
||||
}
|
||||
|
||||
// Objects written by fsv2 use the pending-session's objectKey, which is
|
||||
// persisted in FSEntry.metadata JSON under `objectKey`. Falls back to the
|
||||
// entry uuid for entries that didn't record it (older data).
|
||||
#deriveObjectKeyFromEntry(entry: FSEntry): string {
|
||||
if (entry.metadata) {
|
||||
try {
|
||||
const parsed = JSON.parse(entry.metadata);
|
||||
if (
|
||||
parsed &&
|
||||
typeof parsed.objectKey === 'string' &&
|
||||
parsed.objectKey.length > 0
|
||||
) {
|
||||
return parsed.objectKey;
|
||||
}
|
||||
} catch {
|
||||
// Not JSON — fall through.
|
||||
}
|
||||
}
|
||||
return entry.uuid;
|
||||
}
|
||||
|
||||
// -- Mutation: mkdir / touch / rename / mkshortcut ---------
|
||||
|
||||
/**
|
||||
@@ -3186,7 +3206,7 @@ export class FSService extends PuterService {
|
||||
try {
|
||||
await this.stores.s3Object.deleteObject(
|
||||
entry.bucket,
|
||||
this.#deriveObjectKeyFromEntry(entry),
|
||||
entry.uuid,
|
||||
entry.bucketRegion,
|
||||
);
|
||||
} catch {
|
||||
@@ -3280,7 +3300,7 @@ export class FSService extends PuterService {
|
||||
region: child.bucketRegion,
|
||||
keys: [],
|
||||
};
|
||||
group.keys.push(this.#deriveObjectKeyFromEntry(child));
|
||||
group.keys.push(child.uuid);
|
||||
grouped.set(groupKey, group);
|
||||
// Fire individual removal events so thumbnail extension can clean up.
|
||||
this.#emitRemoveEvent(child);
|
||||
@@ -3420,13 +3440,12 @@ export class FSService extends PuterService {
|
||||
? `/${name}`
|
||||
: `${destinationParent.path}/${name}`;
|
||||
|
||||
// `metadata` column is a TEXT field; serialize when the caller sends
|
||||
// an object, pass-through a bare string, and `null` clears it.
|
||||
// `undefined` leaves the column untouched.
|
||||
let metadataPatch: string | null | undefined;
|
||||
if (input.newMetadata === null) metadataPatch = null;
|
||||
else if (typeof input.newMetadata === 'object')
|
||||
metadataPatch = JSON.stringify(input.newMetadata);
|
||||
else if (input.newMetadata && typeof input.newMetadata === 'object')
|
||||
metadataPatch = JSON.stringify(
|
||||
this.#stripReservedMetadataKeys(input.newMetadata),
|
||||
);
|
||||
|
||||
const updated = await this.stores.fsEntry.updateEntry(source.uuid, {
|
||||
name,
|
||||
@@ -3627,10 +3646,8 @@ export class FSService extends PuterService {
|
||||
});
|
||||
}
|
||||
|
||||
// Regular file: duplicate the S3 object under a new key (the new
|
||||
// entry's uuid), then insert the DB row pointing at it.
|
||||
const newUuid = uuidv4();
|
||||
const sourceObjectKey = this.#deriveObjectKeyFromEntry(source);
|
||||
const sourceObjectKey = source.uuid;
|
||||
const resolvedBucket = this.stores.s3Object.resolveBucket(
|
||||
source.bucket,
|
||||
);
|
||||
@@ -3644,14 +3661,8 @@ export class FSService extends PuterService {
|
||||
this.stores.s3Object.resolveRegion(source.bucketRegion),
|
||||
);
|
||||
|
||||
// Re-serialize metadata, swapping in the new objectKey.
|
||||
const nextMetadata = this.#metadataWithObjectKey(
|
||||
source.metadata,
|
||||
newUuid,
|
||||
);
|
||||
const nextMetadata = this.#sanitizeClientMetadata(source.metadata);
|
||||
|
||||
// Insert as a file row. We reuse the files INSERT path (batchCreateEntries)
|
||||
// since it handles bucket/metadata correctly. A single-row call is fine.
|
||||
const [created] = await this.stores.fsEntry.batchCreateEntries(
|
||||
[
|
||||
{
|
||||
@@ -3697,27 +3708,6 @@ export class FSService extends PuterService {
|
||||
return created;
|
||||
}
|
||||
|
||||
// Preserves existing metadata JSON fields, overriding only objectKey.
|
||||
#metadataWithObjectKey(metadata: string | null, objectKey: string): string {
|
||||
let parsed: Record<string, unknown> = {};
|
||||
if (metadata) {
|
||||
try {
|
||||
const tentative = JSON.parse(metadata);
|
||||
if (
|
||||
tentative &&
|
||||
typeof tentative === 'object' &&
|
||||
!Array.isArray(tentative)
|
||||
) {
|
||||
parsed = tentative as Record<string, unknown>;
|
||||
}
|
||||
} catch {
|
||||
// Non-JSON legacy metadata — drop and replace.
|
||||
}
|
||||
}
|
||||
parsed.objectKey = objectKey;
|
||||
return JSON.stringify(parsed);
|
||||
}
|
||||
|
||||
/**
|
||||
* This method checks if the specified actor has permission to access the entry provided. It will throw an error if the actor is not permitted
|
||||
*/
|
||||
|
||||
@@ -24,6 +24,8 @@ import { PuterService } from '../types.js';
|
||||
import type { UserRow } from '../../stores/user/UserStore.js';
|
||||
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import type { AppIconService } from '../appIcon/AppIconService.js';
|
||||
import { LOCAL_UNLIMITED_USER } from '../../data/subPolicies/localUnlimitedUserPolicy.js';
|
||||
import { UNLIMITED_SUBSCRIPTION } from '../metering/consts.js';
|
||||
|
||||
const USERNAME = 'admin';
|
||||
const ADMIN_GROUP_UID = 'ca342a5e-b13d-4dee-9048-58b11a57cc55';
|
||||
@@ -43,6 +45,20 @@ const ADMIN_STORAGE_BYTES = 10 * 1024 * 1024 * 1024;
|
||||
*/
|
||||
export class DefaultUserService extends PuterService {
|
||||
override async onServerStart(): Promise<void> {
|
||||
// Dev convenience: grant the bootstrap `admin` user unlimited metering.
|
||||
// Gated to env === 'dev' so prod deployments never get a free-usage
|
||||
// actor. Registering the policy makes its `'unlimited'` id resolvable
|
||||
// (extraPolicies is always in the available set); the resolver returns
|
||||
// null for everyone else, so all other users keep their normal tier.
|
||||
if (this.config.env === 'dev') {
|
||||
this.services.metering.registerPolicy(LOCAL_UNLIMITED_USER);
|
||||
this.services.metering.registerSubscriptionResolver((actor) =>
|
||||
actor.user?.username === USERNAME
|
||||
? UNLIMITED_SUBSCRIPTION
|
||||
: null,
|
||||
);
|
||||
}
|
||||
|
||||
if (this.config.no_default_user) return;
|
||||
let user = await this.stores.user.getByUsername(USERNAME);
|
||||
let tmpPassword: string;
|
||||
|
||||
@@ -45,6 +45,12 @@ export interface UserRow {
|
||||
metadata?: Record<string, unknown>;
|
||||
/** Abuse v2 reputation score recorded at signup (DB default 100). */
|
||||
reputation?: number;
|
||||
/** E.164 phone number collected during SMS verification. */
|
||||
phone?: string | null;
|
||||
/** True while the account must complete SMS phone verification before use. */
|
||||
requires_phone_verification?: boolean;
|
||||
/** True while the account must complete credit-card verification before use. */
|
||||
requires_card_verification?: boolean;
|
||||
password?: string;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
@@ -86,10 +92,13 @@ const LATIN1_USER_COLUMNS: ReadonlySet<string> = new Set([
|
||||
'email',
|
||||
'username',
|
||||
'clean_email',
|
||||
'phone',
|
||||
]);
|
||||
const USER_BOOLEAN_COLUMNS: ReadonlySet<string> = new Set([
|
||||
'requires_email_confirmation',
|
||||
'email_confirmed',
|
||||
'requires_phone_verification',
|
||||
'requires_card_verification',
|
||||
'dev_approved_for_incentive_program',
|
||||
'dev_joined_incentive_program',
|
||||
'suspended',
|
||||
@@ -327,6 +336,9 @@ export class UserStore extends PuterStore {
|
||||
referrer?: string | null;
|
||||
last_activity_ts?: string | null;
|
||||
reputation?: number | null;
|
||||
phone?: string | null;
|
||||
requires_phone_verification?: boolean;
|
||||
requires_card_verification?: boolean;
|
||||
}): Promise<UserRow> {
|
||||
assertLatin1Writable(fields as Record<string, unknown>);
|
||||
const result = await this.clients.db.write(
|
||||
@@ -348,8 +360,11 @@ export class UserStore extends PuterStore {
|
||||
signup_server,
|
||||
referrer,
|
||||
last_activity_ts,
|
||||
reputation)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)${this.clients.db.returningIdClause()}`,
|
||||
reputation,
|
||||
phone,
|
||||
requires_phone_verification,
|
||||
requires_card_verification)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)${this.clients.db.returningIdClause()}`,
|
||||
[
|
||||
fields.username,
|
||||
fields.email,
|
||||
@@ -374,6 +389,13 @@ export class UserStore extends PuterStore {
|
||||
fields.last_activity_ts ?? null,
|
||||
// Default matches the DB column default + v2's STARTING_REPUTATION.
|
||||
fields.reputation ?? 100,
|
||||
fields.phone ?? null,
|
||||
this.clients.db.booleanValue(
|
||||
Boolean(fields.requires_phone_verification),
|
||||
),
|
||||
this.clients.db.booleanValue(
|
||||
Boolean(fields.requires_card_verification),
|
||||
),
|
||||
],
|
||||
);
|
||||
|
||||
@@ -583,6 +605,11 @@ export class UserStore extends PuterStore {
|
||||
requires_email_confirmation: asBool(
|
||||
rest.requires_email_confirmation,
|
||||
),
|
||||
requires_phone_verification: asBool(
|
||||
rest.requires_phone_verification,
|
||||
),
|
||||
requires_card_verification: asBool(rest.requires_card_verification),
|
||||
phone: rest.phone == null ? null : String(rest.phone),
|
||||
reputation:
|
||||
rest.reputation == null ? undefined : Number(rest.reputation),
|
||||
metadata,
|
||||
|
||||
@@ -112,6 +112,43 @@ export interface IEmailConfig {
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
/** Prelude (https://prelude.so) Verify v2 — SMS phone verification provider. */
|
||||
export interface IPreludeConfig {
|
||||
/** Prelude v2 API key (sent as `Authorization: Bearer <apiKey>`). */
|
||||
apiKey?: string;
|
||||
/** Default region for parsing local-format phone numbers (e.g. 'US'). */
|
||||
defaultCountry?: string;
|
||||
/**
|
||||
* Per-SMS cost ceiling in EUR.
|
||||
*/
|
||||
maxSmsCostEur?: number;
|
||||
/**
|
||||
* Verification template id (from the Prelude dashboard) that controls the
|
||||
* SMS wording — e.g. a "Your Puter verification code is {{code}}" template.
|
||||
* The message text itself is authored in Prelude, not here; this just
|
||||
* selects it. Omit to use the dashboard default.
|
||||
*/
|
||||
templateId?: string;
|
||||
/**
|
||||
* Alphanumeric Sender ID to brand who the SMS is "from" (e.g. "Puter").
|
||||
* Must be pre-enabled by Prelude and isn't supported by all carriers/regions
|
||||
* (notably US long/short codes). Omit to use Prelude's default sender.
|
||||
*/
|
||||
senderId?: string;
|
||||
/**
|
||||
* Channel Prelude prioritizes for delivery. Defaults to 'rcs' (much cheaper
|
||||
* than SMS); Prelude falls back to SMS when RCS isn't reachable. Requires an
|
||||
* RCS agent provisioned in the Prelude account to actually use RCS.
|
||||
*/
|
||||
preferredChannel?:
|
||||
| 'sms'
|
||||
| 'rcs'
|
||||
| 'whatsapp'
|
||||
| 'viber'
|
||||
| 'zalo'
|
||||
| 'telegram';
|
||||
}
|
||||
|
||||
/**
|
||||
* S3-compatible bucket the thumbnails extension uses for storing generated
|
||||
* thumbnails. When unset, the extension falls back to the main `S3Client`
|
||||
@@ -526,6 +563,20 @@ interface IConfigOptional {
|
||||
allow_system_login: boolean;
|
||||
/** Reject auth-gated routes unless the user has confirmed their email. */
|
||||
strict_email_verification_required: boolean;
|
||||
/**
|
||||
* Force SMS phone verification on every new signup, regardless of abuse
|
||||
* reputation. Off by default; mainly a test/QA switch so the phone gate can
|
||||
* be exercised on demand (it otherwise only triggers for low-reputation
|
||||
* signups). Requires `prelude.apiKey` to actually deliver codes.
|
||||
*/
|
||||
always_require_phone_verification: boolean;
|
||||
/**
|
||||
* Force credit-card verification on every new signup, regardless of abuse
|
||||
* reputation. Off by default; mainly a test/QA switch so the card gate can
|
||||
* be exercised on demand (it otherwise only triggers for low-reputation
|
||||
* signups). Requires a payments extension to actually run the $0 auth.
|
||||
*/
|
||||
always_require_card_verification: boolean;
|
||||
/** Captcha configuration. */
|
||||
captcha: { enabled: boolean; difficulty?: 'easy' | 'medium' | 'hard' };
|
||||
/** OIDC / OAuth2 providers (google + custom). */
|
||||
@@ -569,6 +620,8 @@ interface IConfigOptional {
|
||||
redis: IRedisConfig;
|
||||
pager: IPagerConfig;
|
||||
email: IEmailConfig;
|
||||
/** Optional — only set when SMS phone verification (Prelude) is wired in. */
|
||||
prelude: IPreludeConfig;
|
||||
/** Optional — only set when a ClickHouse analytics client is wired in. */
|
||||
clickhouse?: IClickhouseConfig;
|
||||
cf_file_cache: ICfFileCacheConfig;
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
// Importing npm `undici` has a process-wide side effect: if the global
|
||||
// dispatcher slot — `Symbol.for('undici.globalDispatcher.1')`, shared with
|
||||
// Node's built-in fetch — is empty, undici's module initializer installs its
|
||||
// own Agent there. From that moment EVERY plain `fetch()` in the process
|
||||
// (the AI provider SDKs, everything) is silently served by the npm copy of
|
||||
// undici instead of the copy bundled with — and security-patched by — Node.
|
||||
//
|
||||
// The hijack is irreversible once it happens: loading npm undici also makes
|
||||
// Node's bundled fetch machinery initialize, and the bundled copy only
|
||||
// claims the slot at that initialization — evicting the npm Agent afterwards
|
||||
// just leaves plain fetch() throwing `assert(dispatcher)`. So the one
|
||||
// winning move is to claim the slot for Node FIRST: a data: fetch never
|
||||
// touches the network but synchronously installs the bundled dispatcher.
|
||||
//
|
||||
// This module must therefore be imported before `undici` — it intentionally
|
||||
// imports nothing itself, and secureHttp.ts (the backend's only undici
|
||||
// importer) imports it before importing undici.
|
||||
const DISPATCHER_SLOT = Symbol.for('undici.globalDispatcher.1');
|
||||
const globalSlots = globalThis as unknown as Record<symbol, unknown>;
|
||||
|
||||
void fetch('data:text/plain,dispatcher-guard').catch(() => {
|
||||
// data: fetch cannot fail meaningfully; if it ever does, the slot is
|
||||
// simply claimed by whichever copy of undici initializes first.
|
||||
});
|
||||
|
||||
// AI generation endpoints (Gemini image/video, etc.) can hold a response
|
||||
// back for minutes while generating; undici kills any request whose response
|
||||
// headers haven't arrived within 5 minutes (UND_ERR_HEADERS_TIMEOUT), and the
|
||||
// provider SDKs go through plain fetch with no way to extend that. Wrap the
|
||||
// claimed dispatcher to give every plain fetch a 10-minute budget instead.
|
||||
// secureFetch is unaffected: it passes its own explicit dispatchers, keeping
|
||||
// undici's tighter defaults for user-supplied URLs.
|
||||
const OUTBOUND_FETCH_TIMEOUT_MS = 10 * 60 * 1000;
|
||||
// Symbol.for so a re-imported copy of this module (fresh module registry,
|
||||
// same process) recognizes an already-wrapped dispatcher and doesn't re-wrap.
|
||||
const TIMEOUTS_APPLIED = Symbol.for('puter.outboundFetchTimeouts');
|
||||
|
||||
type DispatchFn = (opts: Record<string, unknown>, handler: unknown) => boolean;
|
||||
interface ComposableDispatcher {
|
||||
compose: (
|
||||
interceptor: (dispatch: DispatchFn) => DispatchFn,
|
||||
) => ComposableDispatcher;
|
||||
}
|
||||
|
||||
const claimed = globalSlots[DISPATCHER_SLOT] as
|
||||
| (ComposableDispatcher & Record<symbol, unknown>)
|
||||
| undefined;
|
||||
if (
|
||||
claimed &&
|
||||
typeof claimed.compose === 'function' &&
|
||||
!(TIMEOUTS_APPLIED in claimed)
|
||||
) {
|
||||
const wrapped = claimed.compose(
|
||||
(dispatch) => (opts, handler) =>
|
||||
// Defaults first so a per-request timeout, if one is ever passed,
|
||||
// still wins.
|
||||
dispatch(
|
||||
{
|
||||
headersTimeout: OUTBOUND_FETCH_TIMEOUT_MS,
|
||||
bodyTimeout: OUTBOUND_FETCH_TIMEOUT_MS,
|
||||
...opts,
|
||||
},
|
||||
handler,
|
||||
),
|
||||
) as ComposableDispatcher & Record<symbol, unknown>;
|
||||
wrapped[TIMEOUTS_APPLIED] = true;
|
||||
globalSlots[DISPATCHER_SLOT] = wrapped;
|
||||
}
|
||||
|
||||
export {};
|
||||
@@ -0,0 +1,60 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { parsePhone, sanitizePhone } from './phone';
|
||||
|
||||
describe('sanitizePhone', () => {
|
||||
it('normalizes a valid E.164 number unchanged', () => {
|
||||
expect(sanitizePhone('+14155550123')).toBe('+14155550123');
|
||||
});
|
||||
|
||||
it('normalizes messy but valid input to E.164', () => {
|
||||
expect(sanitizePhone('+1 (415) 555-0123')).toBe('+14155550123');
|
||||
expect(sanitizePhone(' +44 20 7946 0958 ')).toBe('+442079460958');
|
||||
});
|
||||
|
||||
it('parses local format using the default country', () => {
|
||||
expect(sanitizePhone('(415) 555-0123', 'US')).toBe('+14155550123');
|
||||
});
|
||||
|
||||
it('rejects local format with no default country (ambiguous)', () => {
|
||||
expect(sanitizePhone('4155550123')).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects invalid / too-short / non-numeric input', () => {
|
||||
expect(sanitizePhone('+1 555')).toBeNull();
|
||||
expect(sanitizePhone('not a phone')).toBeNull();
|
||||
expect(sanitizePhone('')).toBeNull();
|
||||
expect(sanitizePhone(' ')).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects non-string input', () => {
|
||||
expect(sanitizePhone(undefined)).toBeNull();
|
||||
expect(sanitizePhone(null)).toBeNull();
|
||||
expect(sanitizePhone(14155550123)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('parsePhone', () => {
|
||||
it('returns E.164 + ISO country for a valid number', () => {
|
||||
expect(parsePhone('+14155550123')).toEqual({
|
||||
e164: '+14155550123',
|
||||
country: 'US',
|
||||
});
|
||||
expect(parsePhone('+442079460958')).toEqual({
|
||||
e164: '+442079460958',
|
||||
country: 'GB',
|
||||
});
|
||||
});
|
||||
|
||||
it('infers country from the default region for local format', () => {
|
||||
expect(parsePhone('(415) 555-0123', 'US')).toEqual({
|
||||
e164: '+14155550123',
|
||||
country: 'US',
|
||||
});
|
||||
});
|
||||
|
||||
it('returns null for invalid input', () => {
|
||||
expect(parsePhone('not a phone')).toBeNull();
|
||||
expect(parsePhone('')).toBeNull();
|
||||
expect(parsePhone(undefined)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import {
|
||||
parsePhoneNumberFromString,
|
||||
type CountryCode,
|
||||
} from 'libphonenumber-js';
|
||||
|
||||
export interface ParsedPhone {
|
||||
/** E.164 form, e.g. "+14155550123" — the format Prelude requires. */
|
||||
e164: string;
|
||||
/** ISO 3166-1 alpha-2 region (e.g. 'US'), or undefined if undetermined. */
|
||||
country?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse + validate raw user-entered phone input, returning its E.164 form and
|
||||
* country. Returns `null` when the input isn't a valid number — callers should
|
||||
* treat that as a 400 (bad phone). `defaultCountry` lets a local-format number
|
||||
* (no "+") be interpreted (e.g. "(415) 555-0123" with 'US').
|
||||
*
|
||||
* @param input Raw phone string from the client.
|
||||
* @param defaultCountry ISO 3166-1 alpha-2 region for local-format numbers.
|
||||
*/
|
||||
export function parsePhone(
|
||||
input: unknown,
|
||||
defaultCountry?: string,
|
||||
): ParsedPhone | null {
|
||||
if (typeof input !== 'string') return null;
|
||||
const trimmed = input.trim();
|
||||
if (!trimmed) return null;
|
||||
|
||||
const parsed = parsePhoneNumberFromString(
|
||||
trimmed,
|
||||
defaultCountry as CountryCode | undefined,
|
||||
);
|
||||
if (!parsed || !parsed.isValid()) return null;
|
||||
|
||||
return { e164: parsed.number, country: parsed.country };
|
||||
}
|
||||
|
||||
/**
|
||||
* Sanitize raw user-entered phone input to a validated E.164 string, or `null`
|
||||
* if invalid. Thin wrapper over {@link parsePhone} for callers that only need
|
||||
* the number.
|
||||
*/
|
||||
export function sanitizePhone(
|
||||
input: unknown,
|
||||
defaultCountry?: string,
|
||||
): string | null {
|
||||
return parsePhone(input, defaultCountry)?.e164 ?? null;
|
||||
}
|
||||
@@ -17,6 +17,7 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { Agent as UndiciAgent } from 'undici';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { configContainer } from '../exports.js';
|
||||
import {
|
||||
@@ -52,6 +53,94 @@ vi.mock('node:dns', async importOriginal => {
|
||||
};
|
||||
});
|
||||
|
||||
describe('global fetch dispatcher isolation', () => {
|
||||
// Importing secureHttp pulls in npm undici, whose module initializer
|
||||
// installs its own Agent into the dispatcher slot shared with Node's
|
||||
// built-in fetch if nothing claimed it first — silently rerouting every
|
||||
// plain fetch() in the process through the npm copy of undici, and
|
||||
// irreversibly so (once npm undici loads, the bundled copy can no longer
|
||||
// claim the slot). nodeFetchDispatcherGuard claims the slot for Node's
|
||||
// bundled dispatcher before the undici import runs. That race is won at
|
||||
// backend boot, but not necessarily under test runners (this very file
|
||||
// imports undici for assertions), so the universal invariants to assert
|
||||
// are: the slot is never left empty and plain fetch() always works.
|
||||
it('leaves global fetch in a working state after the guard runs', async () => {
|
||||
const slot = (globalThis as unknown as Record<symbol, unknown>)[
|
||||
Symbol.for('undici.globalDispatcher.1')
|
||||
];
|
||||
expect(slot).toBeDefined();
|
||||
const response = await fetch('data:text/plain,ok');
|
||||
expect(await response.text()).toBe('ok');
|
||||
});
|
||||
|
||||
it('wraps an already-claimed slot exactly once and keeps fetch working', async () => {
|
||||
// Re-run the guard with an Agent already in the slot (as when the
|
||||
// guard loses the import race): it must not reclaim the slot, only
|
||||
// wrap it with the outbound timeouts — and re-running again must not
|
||||
// wrap a second time.
|
||||
const globalSlots = globalThis as unknown as Record<symbol, unknown>;
|
||||
const SLOT = Symbol.for('undici.globalDispatcher.1');
|
||||
const before = globalSlots[SLOT];
|
||||
const planted = new UndiciAgent();
|
||||
globalSlots[SLOT] = planted;
|
||||
try {
|
||||
vi.resetModules();
|
||||
await import('./nodeFetchDispatcherGuard.js');
|
||||
const wrapped = globalSlots[SLOT];
|
||||
expect(wrapped).not.toBe(planted);
|
||||
expect(wrapped).toBeDefined();
|
||||
|
||||
vi.resetModules();
|
||||
await import('./nodeFetchDispatcherGuard.js');
|
||||
expect(globalSlots[SLOT]).toBe(wrapped);
|
||||
|
||||
const response = await fetch('data:text/plain,ok');
|
||||
expect(await response.text()).toBe('ok');
|
||||
} finally {
|
||||
globalSlots[SLOT] = before;
|
||||
}
|
||||
});
|
||||
|
||||
it('injects 10-minute header/body timeouts into dispatched requests', async () => {
|
||||
// Plant a fake composable dispatcher recording what reaches its
|
||||
// dispatch, re-run the guard, and dispatch through the wrapper.
|
||||
const globalSlots = globalThis as unknown as Record<symbol, unknown>;
|
||||
const SLOT = Symbol.for('undici.globalDispatcher.1');
|
||||
const before = globalSlots[SLOT];
|
||||
const seen: Record<string, unknown>[] = [];
|
||||
type DispatchFn = (
|
||||
opts: Record<string, unknown>,
|
||||
handler: unknown,
|
||||
) => boolean;
|
||||
globalSlots[SLOT] = {
|
||||
compose(interceptor: (dispatch: DispatchFn) => DispatchFn) {
|
||||
const record: DispatchFn = (opts) => {
|
||||
seen.push(opts);
|
||||
return true;
|
||||
};
|
||||
return { dispatch: interceptor(record) };
|
||||
},
|
||||
};
|
||||
try {
|
||||
vi.resetModules();
|
||||
await import('./nodeFetchDispatcherGuard.js');
|
||||
const wrapped = globalSlots[SLOT] as { dispatch: DispatchFn };
|
||||
wrapped.dispatch({ origin: 'https://example.com' }, {});
|
||||
expect(seen).toHaveLength(1);
|
||||
expect(seen[0].headersTimeout).toBe(10 * 60 * 1000);
|
||||
expect(seen[0].bodyTimeout).toBe(10 * 60 * 1000);
|
||||
// A per-request value must win over the injected default.
|
||||
wrapped.dispatch(
|
||||
{ origin: 'https://example.com', headersTimeout: 1234 },
|
||||
{},
|
||||
);
|
||||
expect(seen[1].headersTimeout).toBe(1234);
|
||||
} finally {
|
||||
globalSlots[SLOT] = before;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('secureHttp URL validation', () => {
|
||||
it('rejects raw IP and localhost URLs before fetching', () => {
|
||||
expect(() => validateUrlNoIP('http://127.0.0.1/')).toThrow();
|
||||
|
||||
@@ -17,6 +17,9 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
// Must be imported before 'undici' — see the module comment in the guard.
|
||||
import './nodeFetchDispatcherGuard.js';
|
||||
|
||||
import { lookup as dnsLookup, type LookupAddress } from 'node:dns';
|
||||
import net, { BlockList } from 'node:net';
|
||||
import { Agent as UndiciAgent, fetch as undiciFetch } from 'undici';
|
||||
@@ -154,6 +157,12 @@ const ssrfGuardDispatcher = new UndiciAgent({
|
||||
connect: { lookup: guardedLookup },
|
||||
});
|
||||
|
||||
// Proxied requests skip the SSRF lookup guard (the only locally-resolved
|
||||
// host is the admin-configured proxy itself) but still get an explicit
|
||||
// dispatcher: undici's fetch would otherwise fall back to the global
|
||||
// dispatcher slot it shares with Node's built-in fetch.
|
||||
const proxyDispatcher = new UndiciAgent();
|
||||
|
||||
function proxyConfig(): ISecureCorsProxyConfig | undefined {
|
||||
const cfg = configContainer.secureCorsProxy;
|
||||
if (cfg?.url && cfg?.secret) return cfg;
|
||||
@@ -219,7 +228,7 @@ export async function secureFetch(
|
||||
...rest,
|
||||
headers,
|
||||
redirect: 'manual',
|
||||
...(proxied ? {} : { dispatcher: ssrfGuardDispatcher }),
|
||||
dispatcher: proxied ? proxyDispatcher : ssrfGuardDispatcher,
|
||||
} as unknown as Parameters<typeof undiciFetch>[1])) as unknown as Response;
|
||||
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
|
||||
@@ -54,6 +54,7 @@ interface TaskbarDeps {
|
||||
|
||||
const DEFAULT_TASKBAR_ITEMS: TaskbarEntry[] = [
|
||||
{ name: 'app-center', type: 'app' },
|
||||
{ name: 'builder', type: 'app' },
|
||||
{ name: 'dev-center', type: 'app' },
|
||||
{ name: 'editor', type: 'app' },
|
||||
{ name: 'code', type: 'app' },
|
||||
|
||||
@@ -253,12 +253,12 @@ function generate_website_card (website) {
|
||||
return `
|
||||
<tr class="website-card" data-name="${html_encode(website.subdomain)}">
|
||||
<td style="width:30px; vertical-align: middle; line-height: 1;">
|
||||
<input type="checkbox" class="website-checkbox" data-website-name="${website.subdomain}">
|
||||
<input type="checkbox" class="website-checkbox" data-website-name="${html_encode(website.subdomain)}">
|
||||
</td>
|
||||
<td style="font-family: monospace; font-size: 14px; vertical-align: middle;"><a href="https://${website.subdomain}.puter.site" target="_blank">${website.subdomain}.puter.site</a></td>
|
||||
<td style="font-size: 14px; vertical-align: middle;"> <span class="root-dir-name" data-root-dir-path="${website.root_dir ? html_encode(website.root_dir.path) : ''}">${website.root_dir ? website.root_dir.name : ''}</span></td>
|
||||
<td style="font-size: 14px; vertical-align: middle;">${website.created_at}</td>
|
||||
<td style="vertical-align: middle;"><img class="options-icon options-icon-website" data-website-name="${website.subdomain}" src="./img/options.svg"></td>
|
||||
<td style="font-family: monospace; font-size: 14px; vertical-align: middle;"><a href="https://${html_encode(website.subdomain)}.puter.site" target="_blank">${html_encode(website.subdomain)}.puter.site</a></td>
|
||||
<td style="font-size: 14px; vertical-align: middle;"> <span class="root-dir-name" data-root-dir-path="${website.root_dir ? html_encode(website.root_dir.path) : ''}">${website.root_dir ? html_encode(website.root_dir.name) : ''}</span></td>
|
||||
<td style="font-size: 14px; vertical-align: middle;">${html_encode(website.created_at)}</td>
|
||||
<td style="vertical-align: middle;"><img class="options-icon options-icon-website" data-website-name="${html_encode(website.subdomain)}" src="./img/options.svg"></td>
|
||||
</tr>
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -271,12 +271,12 @@ function generate_worker_card (worker) {
|
||||
return `
|
||||
<tr class="worker-card" data-name="${html_encode(worker.name)}">
|
||||
<td style="width:50px; vertical-align: middle; line-height: 1;">
|
||||
<input type="checkbox" class="worker-checkbox" data-worker-name="${worker.name}">
|
||||
<input type="checkbox" class="worker-checkbox" data-worker-name="${html_encode(worker.name)}">
|
||||
</td>
|
||||
<td style="font-family: monospace; font-size: 14px; vertical-align: middle;">${worker.name}</td>
|
||||
<td style="font-family: monospace; font-size: 14px; vertical-align: middle;"><span class="worker-file-path" data-worker-file-path="${html_encode(worker.file_path)}">${worker.file_path ? worker.file_path : ''}</span></td>
|
||||
<td style="font-size: 14px; vertical-align: middle;">${worker.created_at}</td>
|
||||
<td style="vertical-align: middle;"><img class="options-icon options-icon-worker" data-worker-name="${worker.name}" src="./img/options.svg"></td>
|
||||
<td style="font-family: monospace; font-size: 14px; vertical-align: middle;">${html_encode(worker.name)}</td>
|
||||
<td style="font-family: monospace; font-size: 14px; vertical-align: middle;"><span class="worker-file-path" data-worker-file-path="${html_encode(worker.file_path)}">${worker.file_path ? html_encode(worker.file_path) : ''}</span></td>
|
||||
<td style="font-size: 14px; vertical-align: middle;">${html_encode(worker.created_at)}</td>
|
||||
<td style="vertical-align: middle;"><img class="options-icon options-icon-worker" data-worker-name="${html_encode(worker.name)}" src="./img/options.svg"></td>
|
||||
</tr>
|
||||
`;
|
||||
}
|
||||
|
||||
+114
-1
@@ -33,7 +33,8 @@ An object containing the following properties:
|
||||
- `temperature` (Number) - A number between 0 and 2 indicating the randomness of the completion. Lower values make the output more focused and deterministic, while higher values make it more random. By default, the specific model's temperature is used.
|
||||
- `tools` (Array) (Optional) - Function definitions the AI can call. See [Function Calling](#function-calling) for details.
|
||||
- `reasoning_effort` / `reasoning.effort` (String) (Optional) - Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Lower values give faster responses with less reasoning. OpenAI models only.
|
||||
- `text` / `text_verbosity` (String) (Optional) - Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only.
|
||||
- `verbosity` / `text.verbosity` (String) (Optional) - Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only.
|
||||
- `compaction` (Boolean | Object) (Optional) - Opt into inline context compaction for long conversations. Pass `true` to enable it with provider defaults, or `{ trigger_tokens: number }` to set the token threshold at which earlier context is summarized. When the model compacts, you receive a `compaction` chunk while streaming (or a `compaction` field on the result when not streaming) containing an opaque `encrypted_content` summary. Resend that item in `messages` on the next turn in place of the summarized history. The compaction chunk shape is identical across providers, so the same code works whether `model` is an OpenAI or Anthropic model. See [Compaction](#compaction).
|
||||
|
||||
#### `testMode` (Boolean) (Optional)
|
||||
|
||||
@@ -183,6 +184,118 @@ Pass in the `cache_control` parameter inside the object in the `messages` array.
|
||||
|
||||
You can find the implementation in our [prompt caching example](/playground/ai-claude-cache-control/). Find more details about cache control in [Anthropic documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
|
||||
|
||||
## Compaction
|
||||
|
||||
For long, multi-turn conversations that you keep on the client, enable
|
||||
`compaction` so the model can summarize earlier context before it overflows the
|
||||
context window. When it fires, the model summarizes the older turns into a single
|
||||
**compaction artifact** and answers from that summary instead of the full
|
||||
history — so the request stays small.
|
||||
|
||||
You get the artifact back as a `compaction` item (a stream chunk when streaming,
|
||||
or `result.compaction` when not). On the next turn you carry it forward **as the
|
||||
first block of the assistant turn that produced it**, together with that turn's
|
||||
reply text — the artifact stands in for the older turns it summarized, and the
|
||||
recent exchange is preserved. The item shape is the same across providers, so the
|
||||
same code works for OpenAI and Anthropic models.
|
||||
|
||||
#### Enabling it
|
||||
|
||||
Pass `compaction` in the options object:
|
||||
|
||||
- `compaction: true` — enable with provider defaults.
|
||||
- `compaction: { trigger_tokens: 60000 }` — set the token threshold at which the
|
||||
model compacts.
|
||||
|
||||
The artifact is `{ type: 'compaction', id, encrypted_content }`. `encrypted_content`
|
||||
is an opaque payload; treat it as a black box and just carry it forward.
|
||||
|
||||
#### Streaming
|
||||
|
||||
```js
|
||||
const resp = await puter.ai.chat(messages, {
|
||||
model: 'gpt-5.4', // or 'claude-opus-4-8' — same code
|
||||
stream: true,
|
||||
compaction: { trigger_tokens: 60000 },
|
||||
});
|
||||
|
||||
let text = '';
|
||||
let compaction = null;
|
||||
for await ( const part of resp ) {
|
||||
if ( part.type === 'text' ) text += part.text;
|
||||
else if ( part.type === 'compaction' ) compaction = part; // { type, id, encrypted_content }
|
||||
else if ( part.type === 'error' ) console.error('stream error:', part.message);
|
||||
}
|
||||
|
||||
// Next turn: rebuild the assistant turn from the compaction artifact + the reply
|
||||
// text it came with (artifact first), then add the new user message. The artifact
|
||||
// replaces the older compacted turns; the recent exchange is kept. Keep
|
||||
// compaction enabled so it can compact again later.
|
||||
if ( compaction ) {
|
||||
const next = await puter.ai.chat(
|
||||
[
|
||||
{ role: 'system', content: 'You are a helpful assistant.' },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [ compaction, { type: 'text', text } ],
|
||||
},
|
||||
{ role: 'user', content: 'now compare the two approaches' },
|
||||
],
|
||||
{ model: 'gpt-5.4', stream: true, compaction: true }
|
||||
);
|
||||
for await ( const part of next ) {
|
||||
if ( part.type === 'text' ) document.write(part.text);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Non-streaming
|
||||
|
||||
```js
|
||||
const result = await puter.ai.chat(messages, {
|
||||
model: 'gpt-5.4',
|
||||
compaction: { trigger_tokens: 60000 },
|
||||
});
|
||||
console.log(result.message.content);
|
||||
|
||||
if ( result.compaction ) {
|
||||
// result.compaction is { type: 'compaction', id, encrypted_content } — the
|
||||
// same item you get from the stream. Rebuild the assistant turn from it plus
|
||||
// the reply text (artifact first), then add the new user message:
|
||||
const next = await puter.ai.chat(
|
||||
[
|
||||
{ role: 'system', content: 'You are a helpful assistant.' },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
result.compaction,
|
||||
{ type: 'text', text: result.message.content },
|
||||
],
|
||||
},
|
||||
{ role: 'user', content: 'now compare the two approaches' },
|
||||
],
|
||||
{ model: 'gpt-5.4', compaction: true }
|
||||
);
|
||||
console.log(next.message.content);
|
||||
}
|
||||
```
|
||||
|
||||
#### Notes
|
||||
|
||||
- **Keep `compaction` enabled on every turn** of the conversation so it can
|
||||
compact again as the conversation keeps growing.
|
||||
- **Carry the artifact as the first block of its assistant turn**, alongside that
|
||||
turn's reply text, then continue with new turns. The artifact replaces the older
|
||||
compacted turns; don't drop the recent exchange.
|
||||
- **It only fires once the context is large enough.** Anthropic models require a
|
||||
minimum threshold of **50,000 tokens**, and the conversation must actually
|
||||
exceed your `trigger_tokens`. OpenAI models don't enforce that floor, so they
|
||||
can compact smaller conversations. If nothing compacts, your input was below
|
||||
the threshold.
|
||||
- **Handle the `error` chunk** when streaming — provider errors (e.g. a
|
||||
`trigger_tokens` below a provider's minimum) arrive as an `error` chunk, not a
|
||||
thrown exception.
|
||||
|
||||
## Image Generation (Gemini Image Models)
|
||||
|
||||
Certain Gemini models can generate and edit images as part of a chat conversation. These models accept text and image inputs, and return text and images in the response.
|
||||
|
||||
@@ -18,7 +18,7 @@ puter.ai.img2txt({ source: image, ...options })
|
||||
|
||||
#### `image` / `source` (String|File|Blob) (required)
|
||||
|
||||
A string containing the URL or Puter path, or a `File`/`Blob` object containing the source image or file. When calling with an options object, pass it as `{ source: ... }`.
|
||||
A string containing the URL or Puter path, or a `File`/`Blob` object containing the source image or file. When calling with an options object, pass it as `{ source: ... }`. Maximum input size at 10MB.
|
||||
|
||||
#### `testMode` (Boolean) (Optional)
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ Fine-tune how transcription runs.
|
||||
- `chunking_strategy` (String): Required for `gpt-4o-transcribe-diarize` inputs longer than 30 seconds (recommend `"auto"`).
|
||||
- `known_speaker_names` / `known_speaker_references` (Array): Optional diarization references encoded as data URLs.
|
||||
- `extra_body` (Object): Forwarded verbatim to the OpenAI API for experimental flags.
|
||||
- `stream` (Boolean): Reserved for future streaming support. Currently rejected when `true`.
|
||||
- `stream` (Boolean): Reserved for future streaming support. Streaming is not currently supported.
|
||||
- `test_mode` (Boolean): When `true`, returns a sample response without using credits. Defaults to `false`.
|
||||
|
||||
**xAI-specific options** (when `provider: 'xai'`):
|
||||
@@ -65,8 +65,8 @@ When `true`, skips the live API call and returns a static sample transcript so y
|
||||
|
||||
Returns a `Promise` that resolves to either:
|
||||
|
||||
- A string (when `response_format: "text"` or you pass a shorthand `source` with no options), or
|
||||
- An object of [`Speech2TxtResult`](/Objects/speech2txtresult) containing the transcription payload (including diarization segments, timestamps, etc., depending on the selected model and format).
|
||||
- A string (when `response_format: "text"`), or
|
||||
- An object of [`Speech2TxtResult`](/Objects/speech2txtresult) containing the transcription payload (including diarization segments, timestamps, etc., depending on the selected model and format). This is the default, including when you pass a bare `source` with no options.
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -79,7 +79,7 @@ Returns a `Promise` that resolves to either:
|
||||
<script>
|
||||
(async () => {
|
||||
const transcript = await puter.ai.speech2txt('https://assets.puter.site/example.mp3');
|
||||
puter.print('Transcript:', transcript.text ?? transcript);
|
||||
puter.print('Transcript:', transcript.text);
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
|
||||
@@ -32,14 +32,7 @@ Common aliases are also accepted (e.g. `'eleven'`, `'google'`, `'grok'`).
|
||||
|
||||
## Return value
|
||||
|
||||
A `Promise` that resolves to an array of engine objects. Each object contains:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `String` | Engine/model identifier |
|
||||
| `name` | `String` | Human-readable engine name |
|
||||
| `provider` | `String` | Provider this engine belongs to |
|
||||
| `pricing_per_million_chars` | `Number` | Cost per million characters (may be absent) |
|
||||
A `Promise` that resolves to an array of [`TTSEngine`](/Objects/ttsengine) objects.
|
||||
|
||||
Example response:
|
||||
|
||||
|
||||
@@ -26,19 +26,7 @@ When `options` is a plain string it is treated as an `engine` filter for the def
|
||||
|
||||
## Return value
|
||||
|
||||
A `Promise` that resolves to an array of voice objects. Each object contains:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | `String` | Voice identifier to pass to `txt2speech()` |
|
||||
| `name` | `String` | Human-readable voice name |
|
||||
| `provider` | `String` | Provider this voice belongs to |
|
||||
| `language` | `Object` | `{ name, code }` language info (may be absent) |
|
||||
| `description` | `String` | Short description of the voice (may be absent) |
|
||||
| `category` | `String` | Voice category, e.g. `'premade'` (may be absent) |
|
||||
| `labels` | `Object` | Provider-specific labels (may be absent) |
|
||||
| `supported_models` | `Array` | Model IDs this voice works with (may be absent) |
|
||||
| `supported_engines` | `Array` | Engine types this voice supports (may be absent) |
|
||||
A `Promise` that resolves to an array of [`TTSVoice`](/Objects/ttsvoice) objects.
|
||||
|
||||
Example response:
|
||||
|
||||
|
||||
@@ -96,7 +96,7 @@ Available when `provider: 'xai'`:
|
||||
| `language` | `String` | BCP-47 language code. Defaults to `'en'`. Supports `'auto'` for auto-detection and 20+ languages |
|
||||
| `output_format` | `String` | Output codec. Available: `'mp3'` (default), `'wav'`, `'pcm'`, `'mulaw'`, `'alaw'` |
|
||||
|
||||
Text supports inline speech tags like `[pause]`, `[laugh]` and wrapping tags like `<whisper>text</whisper>` for expressive delivery. Maximum 15,000 characters per request.
|
||||
Text supports inline speech tags like `[pause]`, `[laugh]` and wrapping tags like `<whisper>text</whisper>` for expressive delivery.
|
||||
|
||||
For more details, see the [xAI TTS documentation](https://x.ai/news/grok-stt-and-tts-apis).
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ The name of the app to delete.
|
||||
|
||||
## Return value
|
||||
|
||||
A `Promise` that will resolve to an object `{ success: true }` indicating whether the deletion was successful.
|
||||
A `Promise` that will resolve to an object `{ success: true, uid: <app uid> }` indicating whether the deletion was successful, along with the `uid` of the deleted app.
|
||||
|
||||
## Examples
|
||||
|
||||
|
||||
@@ -33,6 +33,16 @@ puter.auth.signIn(options)
|
||||
|
||||
A `Promise` that will resolve to a [`SignInResult`](/Objects/signinresult/) object when the user has signed in.
|
||||
|
||||
## Rejection
|
||||
|
||||
The promise will reject with an object containing an `error` code and a human-readable `msg` in the following cases:
|
||||
|
||||
- `popup_blocked`: The sign-in popup was blocked by the browser. This usually happens when `signIn()` is not called from a user action (such as a click event).
|
||||
|
||||
- `auth_window_closed`: The user closed the sign-in window (or cancelled the consent dialog) without completing the sign-in process.
|
||||
|
||||
The promise may also reject with the failure response returned by the authentication window itself.
|
||||
|
||||
## Example
|
||||
|
||||
```html;auth-sign-in
|
||||
|
||||
@@ -19,9 +19,9 @@ puter.fs.getReadURL(path, expiresIn)
|
||||
|
||||
The path to the file to read.
|
||||
|
||||
#### `expiresIn` (Number) (Optional)
|
||||
#### `expiresIn` (String | Number) (Optional)
|
||||
|
||||
The number of milliseconds until the URL expires. If not provided, the URL will expire in 24 hours.
|
||||
How long the URL stays valid, in [jsonwebtoken](https://github.com/auth0/node-jsonwebtoken#usage) duration format: a string like `'24h'`, `'30d'`, or `'1h'` (units: `s`, `m`, `h`, `d`, `w`, `y`), or a number of seconds. If not provided, defaults to `'24h'`.
|
||||
|
||||
## Return value
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ The path of the directory to upload the items to. If not set, the items will be
|
||||
A set of key/value pairs that configure the upload process. The following options are supported:
|
||||
|
||||
- `overwrite` (Boolean) - Whether to overwrite the destination file if it already exists. Defaults to `false`.
|
||||
- `dedupeName` (Boolean) - Whether to deduplicate the file name if it already exists. Defaults to `false`.
|
||||
- `dedupeName` (Boolean) - Whether to deduplicate the file name if it already exists. Defaults to `true`. Ignored when `overwrite` is `true`.
|
||||
- `createMissingParents` (Boolean) - Whether to create missing parent directories. Defaults to `false`.
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -63,9 +63,9 @@ The API provides comprehensive hosting management features including creating, r
|
||||
let site_3 = puter.randName();
|
||||
|
||||
// (2) Create 3 empty websites with the subdomains we generated
|
||||
await puter.hosting.create(site_1);
|
||||
await puter.hosting.create(site_2);
|
||||
await puter.hosting.create(site_3);
|
||||
await puter.hosting.create(site_1, '.');
|
||||
await puter.hosting.create(site_2, '.');
|
||||
await puter.hosting.create(site_3, '.');
|
||||
|
||||
// (3) Get all subdomains
|
||||
let sites = await puter.hosting.list();
|
||||
@@ -97,11 +97,11 @@ The API provides comprehensive hosting management features including creating, r
|
||||
(async () => {
|
||||
// (1) Create a random website
|
||||
let subdomain = puter.randName();
|
||||
const site = await puter.hosting.create(subdomain)
|
||||
const site = await puter.hosting.create(subdomain, '.')
|
||||
puter.print(`Website hosted at: ${site.subdomain}.puter.site (This is an empty website with no files)<br>`);
|
||||
|
||||
// (2) Delete the website using delete()
|
||||
const site2 = await puter.hosting.delete(site.subdomain);
|
||||
await puter.hosting.delete(site.subdomain);
|
||||
puter.print('Website deleted<br>');
|
||||
|
||||
// (3) Try to retrieve the website (should fail)
|
||||
@@ -131,7 +131,7 @@ The API provides comprehensive hosting management features including creating, r
|
||||
(async () => {
|
||||
// (1) Create a random website
|
||||
let subdomain = puter.randName();
|
||||
const site = await puter.hosting.create(subdomain)
|
||||
const site = await puter.hosting.create(subdomain, '.')
|
||||
puter.print(`Website hosted at: ${site.subdomain}.puter.site<br>`);
|
||||
|
||||
// (2) Create a random directory
|
||||
@@ -144,7 +144,7 @@ The API provides comprehensive hosting management features including creating, r
|
||||
puter.print(`Changed subdomain's root directory to "${dir.path}"<br>`);
|
||||
|
||||
// (4) Delete the app (cleanup)
|
||||
await puter.hosting.delete(updatedSite.subdomain)
|
||||
await puter.hosting.delete(subdomain)
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
@@ -165,7 +165,7 @@ The API provides comprehensive hosting management features including creating, r
|
||||
(async () => {
|
||||
// (1) Create a random website
|
||||
let subdomain = puter.randName();
|
||||
const site = await puter.hosting.create(subdomain)
|
||||
const site = await puter.hosting.create(subdomain, '.')
|
||||
puter.print(`Website hosted at: ${site.subdomain}.puter.site (This is an empty website with no files)<br>`);
|
||||
|
||||
// (2) Retrieve the website using get()
|
||||
|
||||
@@ -4,13 +4,12 @@ description: Create and host a website from a directory on Puter.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Will create a new subdomain that will be served by the hosting service. Optionally, you can specify a path to a directory that will be served by the subdomain.
|
||||
Will create a new subdomain that will be served by the hosting service. You must specify a path to a directory that will be served by the subdomain.
|
||||
|
||||
## Syntax
|
||||
|
||||
```js
|
||||
puter.hosting.create(subdomain, dirPath)
|
||||
puter.hosting.create(subdomain)
|
||||
puter.hosting.create(options)
|
||||
```
|
||||
|
||||
@@ -20,16 +19,16 @@ puter.hosting.create(options)
|
||||
|
||||
A string containing the name of the subdomain you want to create.
|
||||
|
||||
#### `dirPath` (String) (optional)
|
||||
#### `dirPath` (String) (required)
|
||||
|
||||
A string containing the path to the directory you want to serve. If not specified, the subdomain will be created without a directory.
|
||||
A string containing the path to the directory you want to serve.
|
||||
|
||||
#### `options` (Object) (optional)
|
||||
|
||||
Alternative way to create hosting via options.
|
||||
|
||||
- `subdomain` (String) - Name of the subdomain you want to create.
|
||||
- `root_dir` (String) (optional) - Path to the directory you want to serve, similar to `dirPath`.
|
||||
- `root_dir` (String) (required) - Absolute path to the directory you want to serve. Unlike `dirPath`, this value is not resolved against the app's root directory, so it must be an absolute path.
|
||||
|
||||
## Return value
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ A string containing the name of the subdomain you want to delete.
|
||||
|
||||
## Return value
|
||||
|
||||
A `Promise` that will resolve to `true` when the subdomain has been deleted. If a subdomain with the given name does not exist, the promise will be rejected with an error.
|
||||
A `Promise` that will resolve to an object of the form `{ success: true, uid: <subdomain-uid> }` when the subdomain has been deleted. If a subdomain with the given name does not exist, the promise will be rejected with an error.
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -34,11 +34,11 @@ A `Promise` that will resolve to `true` when the subdomain has been deleted. If
|
||||
(async () => {
|
||||
// (1) Create a random website
|
||||
let subdomain = puter.randName();
|
||||
const site = await puter.hosting.create(subdomain)
|
||||
const site = await puter.hosting.create(subdomain, '.')
|
||||
puter.print(`Website hosted at: ${site.subdomain}.puter.site (This is an empty website with no files)<br>`);
|
||||
|
||||
// (2) Delete the website using delete()
|
||||
const site2 = await puter.hosting.delete(site.subdomain);
|
||||
await puter.hosting.delete(site.subdomain);
|
||||
puter.print('Website deleted<br>');
|
||||
|
||||
// (3) Try to retrieve the website (should fail)
|
||||
|
||||
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user