mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
82
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5918e3f4e2 | ||
|
|
2c9e818e5f | ||
|
|
8826a9fc51 | ||
|
|
8307e8c9f0 | ||
|
|
732786a029 | ||
|
|
f0cd251626 | ||
|
|
943adae622 | ||
|
|
65572dcf23 | ||
|
|
b3923f86da | ||
|
|
729114bd09 | ||
|
|
da1c85d3c5 | ||
|
|
66410d7f1f | ||
|
|
76bd05f228 | ||
|
|
fef1337b2b | ||
|
|
58794f4f76 | ||
|
|
9809e428b5 | ||
|
|
eaddd1d44e | ||
|
|
53b792c52c | ||
|
|
2dd7073034 | ||
|
|
a552ce0b87 | ||
|
|
c04827fd3f | ||
|
|
9debdffc5d | ||
|
|
57b4cbab66 | ||
|
|
5e1645d40e | ||
|
|
b795b219a8 | ||
|
|
401b6066b3 | ||
|
|
ec3c33fea6 | ||
|
|
95d797b047 | ||
|
|
bd06e88185 | ||
|
|
cabedc30b0 | ||
|
|
f81b76c1d1 | ||
|
|
9604cc3f23 | ||
|
|
2c852bf6b3 | ||
|
|
fc861da89f | ||
|
|
facb844747 | ||
|
|
3b791bb334 | ||
|
|
a64a444a17 | ||
|
|
d3c9828207 | ||
|
|
87180b1081 | ||
|
|
d22b656df8 | ||
|
|
19cd5f4476 | ||
|
|
fc56075115 | ||
|
|
e1998e39ab | ||
|
|
fb7968a1c7 | ||
|
|
c88eeb3868 | ||
|
|
9e25ce1401 | ||
|
|
f15d835eeb | ||
|
|
27c4506e02 | ||
|
|
6a7c09f54e | ||
|
|
6254714484 | ||
|
|
efa2cb4441 | ||
|
|
4bc969b99e | ||
|
|
65f6d38432 | ||
|
|
d641f6f7a1 | ||
|
|
00828b9998 | ||
|
|
126ec09556 | ||
|
|
25a5799088 | ||
|
|
ea2f9967a6 | ||
|
|
ba9b885a5a | ||
|
|
6cc241ac31 | ||
|
|
7f8e24f06d | ||
|
|
9b6627b1ea | ||
|
|
7576e0b03a | ||
|
|
d980a9f590 | ||
|
|
d18ea1adb5 | ||
|
|
08f075d774 | ||
|
|
664de0016b | ||
|
|
e273431f14 | ||
|
|
8ed8feed4b | ||
|
|
22f5bf5429 | ||
|
|
198184f986 | ||
|
|
026ae46530 | ||
|
|
0615c3aa3c | ||
|
|
748461aabb | ||
|
|
78cfe0c0f6 | ||
|
|
81f9f00fc0 | ||
|
|
556380cf16 | ||
|
|
908ec23eda | ||
|
|
d5ae5a0049 | ||
|
|
419d0aaa89 | ||
|
|
c3a0c28705 | ||
|
|
7ceb2090b7 |
No files matched your search
@@ -52,6 +52,7 @@ When in doubt, return less. Auth-, permission-, or data-export-related changes d
|
||||
- **Test new behavior.** Every new function, endpoint, driver method, or logic branch gets a test; every bug fix gets a regression test that fails before the fix. If something is genuinely hard to test, skip it but say so in the PR.
|
||||
- **Boy Scout Rule, proportional to the change.** Fix the obvious typo or dead import in files you're already touching; don't ride a refactor along with a bug fix.
|
||||
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't defend in review is not.
|
||||
- **A limit change is not done until the docs change with it.** Every rate limit, concurrency cap, quota, and allowance in the code is published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) — an undisclosed limit is one developers discover as a service failure. If a PR moves any of these numbers, the same PR updates that page. The numbers live in `src/backend/controllers/fs/limits.ts` (filesystem), `src/backend/drivers/util/aiLimits.ts` (all AI drivers), per-driver `rateLimit`/`concurrent` configs, and `src/backend/data/subPolicies/` (free-tier allowances).
|
||||
|
||||
---
|
||||
|
||||
|
||||
+39
-35
@@ -6,33 +6,33 @@ We at **Puter** are committed to maintaining a secure experience for our users a
|
||||
|
||||
The following are in scope for this program:
|
||||
|
||||
* **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
|
||||
* **`puter.com`**
|
||||
* **`api.puter.com`**
|
||||
- **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
|
||||
- **`puter.com`**
|
||||
- **`api.puter.com`**
|
||||
|
||||
Out-of-scope:
|
||||
|
||||
* Third-party services, applications, or libraries not maintained by Puter.
|
||||
* Social engineering attacks (e.g., phishing against staff).
|
||||
* Denial of Service (DoS), spam, or volumetric attacks.
|
||||
* Physical security issues.
|
||||
- Third-party services, applications, or libraries not maintained by Puter.
|
||||
- Social engineering attacks (e.g., phishing against staff).
|
||||
- Denial of Service (DoS), spam, or volumetric attacks.
|
||||
- Physical security issues.
|
||||
|
||||
## Known Non-Issues (Please Check Before Submitting)
|
||||
|
||||
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
|
||||
|
||||
* **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we *do* evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
|
||||
* **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
|
||||
* **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
|
||||
* **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
|
||||
* **App metadata or app user-count "leaks".** This information is currently public by design.
|
||||
* **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
|
||||
* **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
|
||||
* **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
|
||||
* **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
|
||||
* **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
|
||||
* **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
|
||||
* **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
|
||||
- **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we _do_ evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
|
||||
- **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
|
||||
- **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
|
||||
- **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
|
||||
- **App metadata or app user-count "leaks".** This information is currently public by design.
|
||||
- **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
|
||||
- **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
|
||||
- **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
|
||||
- **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
|
||||
- **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
|
||||
- **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
|
||||
- **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
|
||||
|
||||
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
|
||||
|
||||
@@ -40,7 +40,7 @@ If you believe you have a **genuinely new** exploit chain that defeats one of th
|
||||
|
||||
To participate, you must:
|
||||
|
||||
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see *Proof of Reproduction* below).
|
||||
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see _Proof of Reproduction_ below).
|
||||
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
|
||||
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
|
||||
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
|
||||
@@ -51,26 +51,30 @@ Reports that do not meet these guidelines may not be eligible for a reward.
|
||||
|
||||
## Proof of Reproduction
|
||||
|
||||
Reports must demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
|
||||
Reports **MUST** demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
|
||||
|
||||
* Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
|
||||
* The **observed** result versus the **expected** result.
|
||||
* For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
|
||||
* For server-side bugs: a runnable proof-of-concept.
|
||||
- Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
|
||||
- The **observed** result versus the **expected** result.
|
||||
- For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
|
||||
- For server-side bugs: a runnable proof-of-concept and screenshot or video showing the attack process.
|
||||
|
||||
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and are generally not eligible**. If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
|
||||
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and not eligible for bounty**.
|
||||
If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
|
||||
|
||||
Please submit **one issue per report**. Bundled "audit packs" of many speculative findings will be declined; send each confirmed issue separately.
|
||||
Please submit **one issue per report**.
|
||||
Bundled "audit packs" of many speculative findings will not be elligible for bounty; send each confirmed issue separately.
|
||||
Additionally please **DO NOT** submit too many issues at once; these will be treated as a bundled report, and will not be elligible for bounty.
|
||||
|
||||
## Reporting Process
|
||||
|
||||
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
|
||||
Include:
|
||||
|
||||
* A description of the vulnerability
|
||||
* Steps to reproduce
|
||||
* Potential impact
|
||||
* Suggested remediation (if available)
|
||||
- A description of the vulnerability
|
||||
- Steps to reproduce
|
||||
- Potential impact
|
||||
- Suggested remediation (if available)
|
||||
- Paypal id where we can send the bounty (if eligible)
|
||||
|
||||
We aim to acknowledge receipt within **72 hours** and provide a resolution timeline.
|
||||
|
||||
@@ -78,10 +82,10 @@ We aim to acknowledge receipt within **72 hours** and provide a resolution timel
|
||||
|
||||
We offer monetary rewards based on the severity of the vulnerability, as determined by our internal assessment (using CVSS as a guide).
|
||||
|
||||
* **Critical: \$1,000 – \$2,000**
|
||||
* **High: \$500 – \$1,000**
|
||||
* **Medium: \$200 – \$500**
|
||||
* **Low: \$50 – \$100**
|
||||
- **Critical: \$1,000 – \$2,000**
|
||||
- **High: \$500 – \$1,000**
|
||||
- **Medium: \$200 – \$500**
|
||||
- **Low: \$50 – \$100**
|
||||
|
||||
Non-security issues, suggestions, and best practices feedback are always welcome, but may not qualify for a reward.
|
||||
If multiple researchers report the same issue, the bounty will be awarded to the first eligible report we receive.
|
||||
|
||||
@@ -32,6 +32,8 @@ Don't commit code you couldn't have written, debugged, or defended yourself. Rea
|
||||
|
||||
If you add or change a public API (an endpoint, driver method, or puter-js method), follow [doc/contributing-apis.md](doc/contributing-apis.md) — backward compatibility, [developer docs](src/docs/), types, and tests all move in the same PR.
|
||||
|
||||
The same rule applies to limits: rate limits, concurrency caps, quotas, and allowances are published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md), and a PR that changes one of those numbers updates that page in the same PR. A limit nobody published is a limit developers only discover as a service failure.
|
||||
|
||||
## 6. Boy Scout Rule — leave it 1% better
|
||||
|
||||

|
||||
|
||||
@@ -12,10 +12,6 @@
|
||||
·
|
||||
<a href="https://developer.puter.com" target="_blank">Developers</a>
|
||||
·
|
||||
<a href="https://discord.com/invite/PQcx7Teh8u">Discord</a>
|
||||
·
|
||||
<a href="https://reddit.com/r/puter">Reddit</a>
|
||||
·
|
||||
<a href="https://twitter.com/HeyPuter">X</a>
|
||||
</p>
|
||||
|
||||
@@ -82,10 +78,7 @@ Puter is available as a hosted service at [**puter.com**](https://puter.com).
|
||||
Connect with the maintainers and community through these channels:
|
||||
|
||||
- Bug report or feature request? Please [open an issue](https://github.com/HeyPuter/puter/issues/new/choose).
|
||||
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
|
||||
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
|
||||
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
|
||||
- Mastodon: [mastodon.social/@puter](https://mastodon.social/@puter)
|
||||
- Security issues or abuse reports? [[email protected]](mailto:[email protected])
|
||||
- Email maintainers at [[email protected]](mailto:[email protected])
|
||||
|
||||
|
||||
+41
-2
@@ -251,6 +251,28 @@
|
||||
}
|
||||
},
|
||||
|
||||
// ── Sharing ─────────────────────────────────────────────────────────
|
||||
// Email a recipient who already has an account about a new share. On
|
||||
// unless set to false; they opt out via the unsubscribe link, or by
|
||||
// blocking a sender. An invite to an address with no account always goes.
|
||||
// "share_email_notifications": false,
|
||||
//
|
||||
// New shares one account may create per UTC day. Default 200.
|
||||
// "share_daily_limit": 200,
|
||||
//
|
||||
// How often a share may interrupt its recipient — the notification pushed
|
||||
// to their screen and the email with it. Over budget the share still
|
||||
// succeeds and their notification is still brought up to date; only the
|
||||
// interruption is dropped. The recipient bounds are what stop many senders
|
||||
// burying one person between them. A non-positive value removes a bound.
|
||||
// "share_notify_limits": {
|
||||
// "pairWindowSeconds": 900,
|
||||
// "pairDaily": 20,
|
||||
// "recipientHourly": 10,
|
||||
// "recipientDaily": 50,
|
||||
// "emailBatchSeconds": 90
|
||||
// },
|
||||
|
||||
// ── Alarms / alerting ───────────────────────────────────────────────
|
||||
// Where system alarms go. Severity is the routing decision — each
|
||||
// transport takes everything at or above its own `minSeverity`:
|
||||
@@ -324,6 +346,10 @@
|
||||
"apiURL": ""
|
||||
},
|
||||
"gemini": { "apiKey": "" },
|
||||
// Meta Model API (Muse Spark). `muse-spark-1.2-contributor`
|
||||
// trades a ~12x discount for Meta training on its prompts and
|
||||
// completions, so it is only served to callers that name it.
|
||||
"meta": { "apiKey": "" },
|
||||
"groq": { "apiKey": "" },
|
||||
"deepseek": { "apiKey": "" },
|
||||
"mistral": { "apiKey": "" },
|
||||
@@ -341,6 +367,16 @@
|
||||
"apiKey": "",
|
||||
"apiBaseUrl": "https://llm.onerouter.pro/v1"
|
||||
},
|
||||
// BytePlus ModelArk. One key powers chat (Seed/GLM/DeepSeek), image
|
||||
// generation (Seedream) and video generation (Seedance); `apiBaseUrl`
|
||||
// selects the region; see
|
||||
// https://docs.byteplus.com/en/docs/ModelArk/1330310 for options.
|
||||
"byteplus": {
|
||||
"apiKey": "",
|
||||
"apiBaseUrl": "https://ark.ap-southeast.bytepluses.com/api/v3"
|
||||
},
|
||||
"zai": { "apiKey": "" },
|
||||
"alibaba": { "apiKey": "" },
|
||||
"together-ai": { "apiKey": "" },
|
||||
// Local Ollama. `enabled: false` skips the auto-probe at startup
|
||||
// (otherwise Puter logs ECONNREFUSED on every boot when no Ollama
|
||||
@@ -448,8 +484,11 @@
|
||||
// operations that spend it — file transfers, KV calls. Recording is
|
||||
// unaffected either way. `workers` extends the same refusal to
|
||||
// worker-driven calls, which are exempt by default because a deployed
|
||||
// worker has nowhere to surface a payment prompt.
|
||||
// "meteringEnforcement": { "enabled": true, "workers": false },
|
||||
// worker has nowhere to surface a payment prompt. `subscriptions` covers
|
||||
// the separate question of surfaces reserved for paid plans (the
|
||||
// vendor-compatible AI endpoints): turn it off on a deployment with no
|
||||
// paid plans, or every account is refused them.
|
||||
// "meteringEnforcement": { "enabled": true, "workers": false, "subscriptions": true },
|
||||
|
||||
// Fleet-wide spend rate, in micro-cents per minute, past which metering
|
||||
// raises the `metering:excessiveGlobalUsageRate` alarm. Omit it (the
|
||||
|
||||
+9
-1
@@ -44,7 +44,15 @@ pass one explicitly unless you really mean "page someone".
|
||||
event; the 429 is the whole signal, and alarming on it only produces noise
|
||||
proportional to traffic. An *upstream provider* rate-limiting us is the
|
||||
opposite case and still alarms (`upstream_rate_limited`, `info`) — that one
|
||||
is not something we chose.
|
||||
is not something we chose. The exception is a free model: nothing is billed,
|
||||
nothing is actionable, and the throttling is the price of the model, so the
|
||||
AI chat driver marks those `noAlarm` and only paid models still record.
|
||||
|
||||
`noAlarm` is the code-level mute: an `HttpError` carrying it skips the
|
||||
terminal gate entirely, no matter its status or code. It is for failures the
|
||||
call site *already knows* are expected and traffic-proportional — reach for it
|
||||
there, not as a way to quiet an alarm you haven't diagnosed
|
||||
(`severityOverrides` below is the knob for that).
|
||||
|
||||
An extension whose signals are all one tier can default its own local
|
||||
`raiseAlarm` helper to that tier instead of repeating it at every call site —
|
||||
|
||||
@@ -53,7 +53,9 @@ Both are supported ways to define an API. **Prefer a controller when you need fi
|
||||
#### Middleware and gates
|
||||
|
||||
- **Controller routes** (and extension routes — same options) take [`RouteOptions`](../src/backend/core/http/types.ts): auth gates (`requireAuth`, `requireUserActor`, `noUserSession`, `adminOnly`, `allowedAppIds`, and the access-token controls), `subdomain` routing, per-route `rateLimit`, body parsers, and arbitrary extra `middleware`. The auth flavors are subtle and default-deny — read the JSDoc on each field before picking.
|
||||
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), and `noUserSession`. The `/drivers/call` surface enforces them.
|
||||
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), `requireSubscription`, and `noUserSession`. The `/drivers/call` surface enforces them.
|
||||
- **A surface only paying accounts should reach** declares `requireSubscription` — the route option on a controller endpoint, the per-method `@Driver({ requireSubscription })` block on a driver (`/drivers/call` is one shared route, so a driver's requirement can't live in route options). `true` accepts any plan that isn't free, so a plan registered by an extension counts without core naming it; an array of policy ids (`['business', 'pro']`) accepts only those; `false` is "no requirement", the same as leaving it out. An empty array is a boot error rather than a silent no-op — it reads as subscribers-only while admitting everyone. Off unless asked for, and answered from the metering service's cached per-actor subscription, so it costs a map lookup. Distinct from `requireCredits`: this asks which plan the account is on, not whether it has budget left. Deployments with no paid plans (self-hosted installs) turn the whole thing off with `meteringEnforcement.subscriptions: false`.
|
||||
- **An account must have verified a specific factor** — `requireVerified` (email, and only under `strict_email_verification_required`), `requirePhoneVerified`, `requireCardVerified`. These are opt-in and require the factor to have actually been verified, unlike the default-on gate that turns away accounts still carrying a pending verification the abuse harness asked for.
|
||||
- **An endpoint that spends metered resources** on the caller's behalf — moving file content, making object-store requests, anything else the account is billed for — also declares `requireCredits: true`, which turns an account with nothing left of its budget away with a 402 before the handler runs. Endpoints that only describe or delete things deliberately don't: an account that has run out still has to be able to see what it has, clear it, and reach its billing pages. Drivers have no route options to declare this on, so they call `assertActorHasCredits` themselves ([src/backend/services/metering/enforcement.ts](../src/backend/services/metering/enforcement.ts)) — see `KVStoreDriver`, which does it once for every method.
|
||||
|
||||
### 3. puter.js
|
||||
|
||||
@@ -122,6 +122,10 @@ cat > puter/config/config.json <<EOF
|
||||
"ollama": { "enabled": false }
|
||||
},
|
||||
|
||||
"meteringEnforcement": {
|
||||
"subscriptions": false
|
||||
},
|
||||
|
||||
"trust_proxy": 1
|
||||
}
|
||||
EOF
|
||||
@@ -136,6 +140,7 @@ Why these knobs:
|
||||
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
|
||||
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
|
||||
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
|
||||
- `meteringEnforcement.subscriptions: false` — a few surfaces (the OpenAI/Anthropic-compatible AI endpoints) are declared paid-plan-only. A self-hosted install has no paid plans, so every account sits on the free policy and those routes would answer `402 subscription_required` for everyone. Turning the plan gates off opens them; nothing else about metering changes.
|
||||
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
|
||||
- `s3.s3Config.forcePathStyle: true` — RustFS / MinIO / fauxqs need path-style URLs (`<endpoint>/<bucket>`). Real AWS S3 wants virtual-hosted (`<bucket>.<endpoint>`) — drop this flag (or set `false`) when you swap to real S3.
|
||||
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. Caddy routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
|
||||
@@ -295,6 +300,18 @@ Used for password resets, email confirmation, and notifications. Without it thos
|
||||
|
||||
To require email confirmation before login, also set `"strict_email_verification_required": true`.
|
||||
|
||||
To read the mail yourself instead of sending it, point the transport at a local SMTP sink — [MailHog](https://github.com/mailhog/MailHog) catches everything and serves it at `http://localhost:8025`. Publish both ports; a container that only exposes them is unreachable from the host:
|
||||
|
||||
```sh
|
||||
docker run -d -p 1025:1025 -p 8025:8025 mailhog/mailhog
|
||||
```
|
||||
|
||||
```json
|
||||
"email": { "from": "\"Puter\" <[email protected]>", "host": "127.0.0.1", "port": 1025, "secure": false, "ignoreTLS": true }
|
||||
```
|
||||
|
||||
Share email is on once a transport exists: a recipient who already has an account is emailed as well as notified in-app, and they opt out with the unsubscribe link the mail carries. Set `"share_email_notifications": false` to keep those in the app only — an address with no account is still emailed, since there is nothing else to reach them with.
|
||||
|
||||
### Sign in with Google (or another OIDC provider)
|
||||
|
||||
```json
|
||||
|
||||
+107
-3
@@ -6,6 +6,10 @@ import {
|
||||
servicesContainers,
|
||||
} from '@heyputer/backend/src/exports';
|
||||
import { extension } from '@heyputer/backend/src/extensions';
|
||||
import {
|
||||
creditMultiplierFrom,
|
||||
toCredits,
|
||||
} from '@heyputer/backend/src/services/metering/utils';
|
||||
import type { Request, Response } from 'express';
|
||||
|
||||
const services = extension.import('service');
|
||||
@@ -50,6 +54,42 @@ function collectAllCosts(): Record<string, unknown>[] {
|
||||
return all;
|
||||
}
|
||||
|
||||
// -- Credits scaling ---------------------------------------------------
|
||||
//
|
||||
// When the deployment configures a credit multiplier, every monetary amount
|
||||
// is scaled to display credits BEFORE it leaves the server — raw metered
|
||||
// amounts are never reported. Counts, units, and byte figures pass through
|
||||
// untouched. A response that was scaled says so (`unit: 'credits'`); without
|
||||
// a configured multiplier amounts pass through raw and clients render them
|
||||
// as dollars.
|
||||
|
||||
/** The per-usage-type records of a UsageByType map, costs scaled. */
|
||||
const scaleUsageByType = (
|
||||
usage: Record<string, unknown>,
|
||||
multiplier: number,
|
||||
): Record<string, unknown> => {
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(usage)) {
|
||||
// `allowanceUsed` is the month's allowance-charged spend — monetary,
|
||||
// scalar, scaled like the total.
|
||||
if (key === 'total' || key === 'allowanceUsed') {
|
||||
out[key] = toCredits(Number(value) || 0, multiplier);
|
||||
} else if (
|
||||
value &&
|
||||
typeof value === 'object' &&
|
||||
typeof (value as { cost?: unknown }).cost === 'number'
|
||||
) {
|
||||
out[key] = {
|
||||
...(value as Record<string, unknown>),
|
||||
cost: toCredits((value as { cost: number }).cost, multiplier),
|
||||
};
|
||||
} else {
|
||||
out[key] = value;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
export const handleMeteringUsage = async (
|
||||
_req: Request,
|
||||
res: Response,
|
||||
@@ -61,7 +101,44 @@ export const handleMeteringUsage = async (
|
||||
services.metering.getActorCurrentMonthUsageDetails(actor),
|
||||
services.metering.getAllowedUsage(actor),
|
||||
]);
|
||||
res.json({ ...actorUsage, allowanceInfo });
|
||||
const multiplier = creditMultiplierFrom(extension.config);
|
||||
if (!multiplier) {
|
||||
res.json({ ...actorUsage, allowanceInfo });
|
||||
return;
|
||||
}
|
||||
res.json({
|
||||
usage: scaleUsageByType(
|
||||
actorUsage.usage as unknown as Record<string, unknown>,
|
||||
multiplier,
|
||||
),
|
||||
appTotals: Object.fromEntries(
|
||||
Object.entries(actorUsage.appTotals).map(([appId, totals]) => [
|
||||
appId,
|
||||
{ ...totals, total: toCredits(totals.total, multiplier) },
|
||||
]),
|
||||
),
|
||||
allowanceInfo: {
|
||||
...allowanceInfo,
|
||||
remaining: toCredits(allowanceInfo.remaining, multiplier),
|
||||
monthUsageAllowance: toCredits(
|
||||
allowanceInfo.monthUsageAllowance,
|
||||
multiplier,
|
||||
),
|
||||
// Monetary addon fields scale; absent ones stay absent rather
|
||||
// than materializing as NaN/null. purchasedStorage is bytes.
|
||||
addons: Object.fromEntries(
|
||||
Object.entries(allowanceInfo.addons ?? {}).map(([k, v]) => [
|
||||
k,
|
||||
(k === 'purchasedCredits' ||
|
||||
k === 'consumedPurchaseCredits') &&
|
||||
typeof v === 'number'
|
||||
? toCredits(v, multiplier)
|
||||
: v,
|
||||
]),
|
||||
),
|
||||
unit: 'credits',
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
export const handleMeteringUsageForApp = async (
|
||||
@@ -92,7 +169,18 @@ export const handleMeteringUsageForApp = async (
|
||||
actor,
|
||||
appId,
|
||||
);
|
||||
res.json(appUsage);
|
||||
const multiplier = creditMultiplierFrom(extension.config);
|
||||
if (!multiplier) {
|
||||
res.json(appUsage);
|
||||
return;
|
||||
}
|
||||
res.json({
|
||||
...scaleUsageByType(
|
||||
appUsage as unknown as Record<string, unknown>,
|
||||
multiplier,
|
||||
),
|
||||
unit: 'credits',
|
||||
});
|
||||
};
|
||||
|
||||
export const handleMeteringGlobalUsage = async (
|
||||
@@ -109,7 +197,23 @@ export const handleMeteringAllCosts = async (
|
||||
res: Response,
|
||||
): Promise<void> => {
|
||||
if (!cachedAllCosts) {
|
||||
cachedAllCosts = collectAllCosts();
|
||||
const raw = collectAllCosts();
|
||||
const multiplier = creditMultiplierFrom(extension.config);
|
||||
// Price the catalogue in the same unit the usage endpoints report:
|
||||
// per-unit credits when a multiplier is configured, raw otherwise.
|
||||
cachedAllCosts = multiplier
|
||||
? raw.map(({ ucentsPerUnit, costs_currency: _cc, ...rest }) =>
|
||||
typeof ucentsPerUnit === 'number'
|
||||
? {
|
||||
...rest,
|
||||
creditsPerUnit: toCredits(
|
||||
ucentsPerUnit,
|
||||
multiplier,
|
||||
),
|
||||
}
|
||||
: rest,
|
||||
)
|
||||
: raw;
|
||||
}
|
||||
res.json({ costs: cachedAllCosts });
|
||||
};
|
||||
|
||||
@@ -206,6 +206,84 @@ describe('whoami extension — handleWhoami', () => {
|
||||
expect(JSON.stringify(body)).not.toContain('bootstrap-secret');
|
||||
});
|
||||
|
||||
it('never exposes phone, card fingerprint or signup identity', async () => {
|
||||
const user = await seedUser();
|
||||
await server.stores.user.update(user.id as number, {
|
||||
phone: '+15551234567',
|
||||
card_fingerprint: 'fp_ABC123',
|
||||
requires_phone_verification: false,
|
||||
requires_card_verification: false,
|
||||
});
|
||||
// Guard against a vacuous assertion below: the columns really do hold
|
||||
// the values we then expect never to see on the wire.
|
||||
const stored = await server.stores.user.getById(user.id as number, {
|
||||
cached: false,
|
||||
force: true,
|
||||
});
|
||||
expect(stored?.phone).toBe('+15551234567');
|
||||
expect(stored?.card_fingerprint).toBe('fp_ABC123');
|
||||
|
||||
for (const actor of [
|
||||
{ user: { uuid: user.uuid, id: user.id as number } },
|
||||
{
|
||||
user: { uuid: user.uuid, id: user.id as number },
|
||||
app: { uid: 'app-test-actor' },
|
||||
},
|
||||
]) {
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext({ actor }, () =>
|
||||
handleWhoami(makeReq(), res),
|
||||
);
|
||||
|
||||
const body = captured.body as Record<string, unknown>;
|
||||
expect(body.phone).toBeUndefined();
|
||||
expect(body.card_fingerprint).toBeUndefined();
|
||||
expect(body.password).toBeUndefined();
|
||||
expect(body.otp_secret).toBeUndefined();
|
||||
expect(body.signup_ip).toBeUndefined();
|
||||
// Not just absent as a key — the values must not appear anywhere
|
||||
// in the payload (nested under metadata, taskbar items, …).
|
||||
const serialized = JSON.stringify(body);
|
||||
expect(serialized).not.toContain('5551234567');
|
||||
expect(serialized).not.toContain('fp_ABC123');
|
||||
// The verification flags, which the GUI acts on, still ship.
|
||||
expect(body).toHaveProperty('requires_phone_verification');
|
||||
expect(body).toHaveProperty('requires_card_verification');
|
||||
}
|
||||
});
|
||||
|
||||
it('scrubs sensitive keys added to metadata, and leaves the cached row intact', async () => {
|
||||
const user = await seedUser();
|
||||
await server.stores.user.updateMetadata(user.id as number, {
|
||||
tmp_password: 'bootstrap-secret',
|
||||
billing: { card_fingerprint: 'fp_NESTED', tier: 'pro' },
|
||||
});
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext(
|
||||
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
|
||||
() => handleWhoami(makeReq(), res),
|
||||
);
|
||||
|
||||
const body = captured.body as Record<string, unknown>;
|
||||
const metadata = body.metadata as Record<string, unknown>;
|
||||
expect(metadata.tmp_password).toBeUndefined();
|
||||
// Nested sensitive keys are removed too; siblings survive.
|
||||
expect(metadata.billing).toEqual({ tier: 'pro' });
|
||||
expect(JSON.stringify(body)).not.toContain('fp_NESTED');
|
||||
|
||||
// The scrub works on a copy — server-side state is untouched.
|
||||
const fresh = await server.stores.user.getById(user.id as number, {
|
||||
cached: false,
|
||||
force: true,
|
||||
});
|
||||
expect(fresh?.metadata?.tmp_password).toBe('bootstrap-secret');
|
||||
expect(
|
||||
(fresh?.metadata?.billing as Record<string, unknown>)
|
||||
?.card_fingerprint,
|
||||
).toBe('fp_NESTED');
|
||||
});
|
||||
|
||||
it('marks the user as oidc_only when password is null', async () => {
|
||||
const slug = Math.random().toString(36).slice(2, 8);
|
||||
const oidcUser = await server.stores.user.create({
|
||||
|
||||
+80
-3
@@ -33,6 +33,68 @@ const CLIENT_VISIBLE_FEATURE_FLAGS: ReadonlySet<string> = new Set([
|
||||
'prompt_user_when_navigation_away_from_puter',
|
||||
]);
|
||||
|
||||
// Keys that must never leave the server, whoever put them on the response.
|
||||
// `details` is an explicit pick, but the `whoami.details` event hands
|
||||
// listeners the full UserRow next to the object they may write to, and
|
||||
// `metadata` is a free-form blob — so any of these can arrive on the
|
||||
// response without an edit to the pick above. The scrub runs last, over the
|
||||
// whole payload, and is the one place that decides what "sensitive" means.
|
||||
//
|
||||
// Credentials and single-use tokens, the payment/phone identifiers used for
|
||||
// verification (`card_fingerprint` is the Stripe fingerprint, stable per
|
||||
// card number), the network identity recorded at signup, and internal
|
||||
// anti-abuse bookkeeping. `requires_phone_verification` /
|
||||
// `requires_card_verification` stay: they are the flags the GUI acts on, and
|
||||
// they carry no identifier.
|
||||
const SENSITIVE_KEYS: ReadonlySet<string> = new Set([
|
||||
'password',
|
||||
'tmp_password',
|
||||
'pass_recovery_token',
|
||||
'email_confirm_code',
|
||||
'email_confirm_token',
|
||||
'change_email_confirm_token',
|
||||
'otp_secret',
|
||||
'otp_recovery_codes',
|
||||
'card_fingerprint',
|
||||
'phone',
|
||||
'clean_email',
|
||||
'signup_ip',
|
||||
'signup_ip_forwarded',
|
||||
'signup_user_agent',
|
||||
'signup_origin',
|
||||
'signup_server',
|
||||
'audit_metadata',
|
||||
]);
|
||||
|
||||
// Depth-limited, cycle-safe walk deleting every SENSITIVE_KEYS entry it finds
|
||||
// at any level (`metadata` and `taskbar_items` are both nested structures).
|
||||
const scrubSensitive = (
|
||||
value: unknown,
|
||||
seen: Set<object> = new Set(),
|
||||
depth = 0,
|
||||
): void => {
|
||||
if (depth > 8 || value === null || typeof value !== 'object') return;
|
||||
if (seen.has(value as object)) return;
|
||||
seen.add(value as object);
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
for (const entry of value) scrubSensitive(entry, seen, depth + 1);
|
||||
return;
|
||||
}
|
||||
|
||||
for (const key of Object.keys(value as Record<string, unknown>)) {
|
||||
if (SENSITIVE_KEYS.has(key)) {
|
||||
delete (value as Record<string, unknown>)[key];
|
||||
continue;
|
||||
}
|
||||
scrubSensitive(
|
||||
(value as Record<string, unknown>)[key],
|
||||
seen,
|
||||
depth + 1,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
export const handleWhoami = async (
|
||||
req: Request,
|
||||
res: Response,
|
||||
@@ -74,8 +136,12 @@ export const handleWhoami = async (
|
||||
}
|
||||
}
|
||||
|
||||
const metadata = user.metadata ? { ...user.metadata } : user.metadata;
|
||||
if (metadata) delete metadata.tmp_password;
|
||||
// Deep-copied (it is decoded JSON) so the scrub below edits the response
|
||||
// and not the cached UserRow. Sensitive keys inside it — tmp_password and
|
||||
// anything else on the denylist — are removed by scrubSensitive.
|
||||
const metadata = user.metadata
|
||||
? structuredClone(user.metadata)
|
||||
: user.metadata;
|
||||
|
||||
const details: Record<string, unknown> = {
|
||||
username: user.username,
|
||||
@@ -84,7 +150,9 @@ export const handleWhoami = async (
|
||||
unconfirmed_email: user.email,
|
||||
email_confirmed: user.email_confirmed || user.username === 'admin',
|
||||
requires_email_confirmation: user.requires_email_confirmation,
|
||||
phone: user.phone,
|
||||
// The phone number itself is deliberately absent: nothing on the
|
||||
// client reads it, and it is PII that would otherwise be handed to
|
||||
// every app actor. Only the verification flag ships.
|
||||
requires_phone_verification: user.requires_phone_verification,
|
||||
requires_card_verification: user.requires_card_verification,
|
||||
desktop_bg_url: user.desktop_bg_url,
|
||||
@@ -92,6 +160,10 @@ export const handleWhoami = async (
|
||||
desktop_bg_fit: user.desktop_bg_fit,
|
||||
is_temp: user.password === null && user.email === null,
|
||||
is_user_token: true,
|
||||
// Present only once the account has actually asked for a code (see the
|
||||
// referral extension) — null until then, and never minted from here:
|
||||
// this endpoint is polled, and a mint is a write.
|
||||
referral_code: user.referral_code,
|
||||
oidc_only: oidcOnly,
|
||||
taskbar_items: isUser
|
||||
? await getTaskbarItems(
|
||||
@@ -172,6 +244,8 @@ export const handleWhoami = async (
|
||||
delete details.created_ts;
|
||||
delete details.is_user_token;
|
||||
delete details.metadata;
|
||||
// An app has no business reading the code its user earns credit with.
|
||||
delete details.referral_code;
|
||||
}
|
||||
|
||||
if (actor.app) {
|
||||
@@ -196,6 +270,9 @@ export const handleWhoami = async (
|
||||
delete subscription.offering.price_id;
|
||||
}
|
||||
|
||||
// Last word on what ships, after every listener has had its say.
|
||||
scrubSensitive(details);
|
||||
|
||||
res.json(details);
|
||||
};
|
||||
|
||||
|
||||
@@ -131,6 +131,11 @@ S3_SECRET_KEY=$S3_SECRET_KEY
|
||||
S3_BUCKET=puter-local
|
||||
EOF
|
||||
|
||||
# `meteringEnforcement.subscriptions` is off here because a self-hosted
|
||||
# install has no paid plans to be on: every account resolves to the free
|
||||
# policy, so leaving the plan gates armed would 402 the surfaces that
|
||||
# declare one (the OpenAI/Anthropic-compatible AI endpoints) for everybody.
|
||||
# The code default is on — this is a property of the install, not of Puter.
|
||||
log "writing puter/config/config.json"
|
||||
cat > puter/config/config.json <<EOF
|
||||
{
|
||||
@@ -191,6 +196,10 @@ EOF
|
||||
"ollama": { "enabled": false }
|
||||
},
|
||||
|
||||
"meteringEnforcement": {
|
||||
"subscriptions": false
|
||||
},
|
||||
|
||||
"trust_proxy": $PUTER_TRUST_PROXY
|
||||
}
|
||||
EOF
|
||||
|
||||
Generated
+2102
-3710
File diff suppressed because it is too large.
Load diff
+14
-25
@@ -13,14 +13,9 @@
|
||||
"@babel/core": "^7.29.7",
|
||||
"@eslint/js": "^9.35.0",
|
||||
"@playwright/test": "^1.56.1",
|
||||
"@stylistic/eslint-plugin": "^5.3.1",
|
||||
"@types/better-sqlite3": "^7.6.13",
|
||||
"@types/express": "^5.0.0",
|
||||
"@types/mime-types": "^3.0.1",
|
||||
"@typescript-eslint/eslint-plugin": "^8.46.1",
|
||||
"@typescript-eslint/parser": "^8.46.1",
|
||||
"@vitest/coverage-v8": "^4.0.14",
|
||||
"@vitest/ui": "^4.0.14",
|
||||
"babel-loader": "^10.1.1",
|
||||
"babel-plugin-istanbul": "^8.0.0",
|
||||
"chalk": "^4.1.0",
|
||||
@@ -30,25 +25,19 @@
|
||||
"eslint": "^9.35.0",
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
"eslint-plugin-prettier": "^5.5.5",
|
||||
"eslint-rule-composer": "^0.3.0",
|
||||
"globals": "^15.15.0",
|
||||
"html-webpack-plugin": "^5.6.0",
|
||||
"husky": "^9.1.7",
|
||||
"istanbul-lib-coverage": "^3.2.2",
|
||||
"istanbul-lib-report": "^3.0.1",
|
||||
"istanbul-reports": "^3.2.0",
|
||||
"license-check-and-add": "^4.0.5",
|
||||
"nodemon": "^3.1.0",
|
||||
"prettier": "^3.8.3",
|
||||
"prettier-plugin-jsdoc": "^1.8.1",
|
||||
"simple-git": "^3.32.3",
|
||||
"typescript": "^5.4.5",
|
||||
"uglify-js": "^3.17.4",
|
||||
"vite-plugin-static-copy": "^3.3.0",
|
||||
"vitest": "^4.1.5",
|
||||
"webpack": "^5.88.2",
|
||||
"webpack-cli": "^5.1.1",
|
||||
"yaml": "^2.8.1"
|
||||
"webpack-cli": "^5.1.1"
|
||||
},
|
||||
"scripts": {
|
||||
"test:backend": "npm run setupExtensions && npm run build:workerLib && vitest run --config src/backend/vitest.config.ts ",
|
||||
@@ -73,9 +62,7 @@
|
||||
"setupExtensions": "node ./tools/extensionSetup.mjs"
|
||||
},
|
||||
"workspaces": [
|
||||
"src/*",
|
||||
"tools/*",
|
||||
"experiments/js-parse-and-output"
|
||||
"src/*"
|
||||
],
|
||||
"nodemonConfig": {
|
||||
"ext": "js, json, mjs, jsx, svg, css",
|
||||
@@ -85,11 +72,8 @@
|
||||
]
|
||||
},
|
||||
"dependencies": {
|
||||
"@ai-sdk/openai": "^3.0.25",
|
||||
"@aws-sdk/client-s3": "^3.1020.0",
|
||||
"@aws-sdk/s3-request-presigner": "^3.1028.0",
|
||||
"@heyputer/putility": "^1.0.2",
|
||||
"ai": "^6.0.73",
|
||||
"dedent": "^1.5.3",
|
||||
"javascript-time-ago": "^2.5.11",
|
||||
"libphonenumber-js": "1.13.6",
|
||||
@@ -101,12 +85,17 @@
|
||||
},
|
||||
"allowScripts": {
|
||||
"@google/genai": false,
|
||||
"bcrypt@5.1.1": true,
|
||||
"better-sqlite3@12.11.1": true,
|
||||
"browser-tabs-lock@1.3.0": true,
|
||||
"classic-level@3.0.0": true,
|
||||
"[email protected]": true,
|
||||
"[email protected]": true,
|
||||
"[email protected]": true
|
||||
"bcrypt": true,
|
||||
"better-sqlite3": true,
|
||||
"browser-tabs-lock": true,
|
||||
"classic-level": true,
|
||||
"esbuild": true,
|
||||
"fsevents": true,
|
||||
"protobufjs": true,
|
||||
"sharp": true,
|
||||
"workerd": true
|
||||
},
|
||||
"overrides": {
|
||||
"undici": "^7.29.0"
|
||||
}
|
||||
}
|
||||
@@ -18,7 +18,7 @@
|
||||
*/
|
||||
|
||||
import Database from 'better-sqlite3';
|
||||
import { existsSync, mkdtempSync, rmSync } from 'node:fs';
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
@@ -27,7 +27,15 @@ import { DatabaseClientFactory } from './index.js';
|
||||
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
|
||||
|
||||
/** Highest schema version the migration table can reach. */
|
||||
const CURRENT_SCHEMA_VERSION = 61;
|
||||
const CURRENT_SCHEMA_VERSION = 66;
|
||||
|
||||
/**
|
||||
* These suites migrate real files on disk. Idle they finish in well under a
|
||||
* second, but the whole migration chain runs against the filesystem — enough
|
||||
* that a loaded runner blows the 5s default and reports a timeout where there
|
||||
* is no fault.
|
||||
*/
|
||||
const DISK_MIGRATION_TIMEOUT_MS = 30_000;
|
||||
const SYSTEM_USER_UUID = '5d4adce0-a381-4982-9c02-6e2540026238';
|
||||
|
||||
const sqliteConfig = (
|
||||
@@ -52,7 +60,7 @@ const userVersionOf = async (client: SqliteDatabaseClient): Promise<number> => {
|
||||
return row.user_version as number;
|
||||
};
|
||||
|
||||
describe('SqliteDatabaseClient — boot and migrations', () => {
|
||||
describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATION_TIMEOUT_MS }, () => {
|
||||
let client: SqliteDatabaseClient;
|
||||
|
||||
beforeEach(async () => {
|
||||
@@ -83,6 +91,105 @@ describe('SqliteDatabaseClient — boot and migrations', () => {
|
||||
expect(rows).toEqual([{ username: 'system' }]);
|
||||
});
|
||||
|
||||
// 0070 sweeps three by-hand groups no code reads. Both tables it can reach
|
||||
// cascade on delete, so the guard is the point: a group that still carries
|
||||
// permissions or members has to survive, or the sweep silently revokes
|
||||
// them. Re-running the statement is safe — it is a plain conditional DELETE.
|
||||
describe('0070 drop-orphaned-default-groups', () => {
|
||||
const MIGRATION = readFileSync(
|
||||
new URL(
|
||||
'./migrations/sqlite/0070_drop-orphaned-default-groups.sql',
|
||||
import.meta.url,
|
||||
),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
const seedGroup = async (name: string): Promise<number> => {
|
||||
const [system] = await client.read(
|
||||
'SELECT `id` FROM `user` WHERE `uuid` = ?',
|
||||
[SYSTEM_USER_UUID],
|
||||
);
|
||||
const uid = `grp-${name}-${Math.random().toString(36).slice(2, 10)}`;
|
||||
await client.write(
|
||||
'INSERT INTO `group` (`uid`, `owner_user_id`, `extra`, `metadata`) ' +
|
||||
'VALUES (?, ?, ?, ?)',
|
||||
[uid, system.id, JSON.stringify({ name }), '{}'],
|
||||
);
|
||||
const [row] = await client.read(
|
||||
'SELECT `id` FROM `group` WHERE `uid` = ?',
|
||||
[uid],
|
||||
);
|
||||
return Number(row.id);
|
||||
};
|
||||
|
||||
const stillThere = async (id: number): Promise<boolean> => {
|
||||
const rows = await client.read(
|
||||
'SELECT `id` FROM `group` WHERE `id` = ?',
|
||||
[id],
|
||||
);
|
||||
return rows.length > 0;
|
||||
};
|
||||
|
||||
it('drops an orphaned freeai/experimental/dangerous group', async () => {
|
||||
const ids = await Promise.all(
|
||||
['freeai', 'experimental', 'dangerous'].map(seedGroup),
|
||||
);
|
||||
await client.write(MIGRATION);
|
||||
for (const id of ids) expect(await stillThere(id)).toBe(false);
|
||||
});
|
||||
|
||||
it('spares one that still carries a permission row', async () => {
|
||||
const id = await seedGroup('freeai');
|
||||
const [system] = await client.read(
|
||||
'SELECT `id` FROM `user` WHERE `uuid` = ?',
|
||||
[SYSTEM_USER_UUID],
|
||||
);
|
||||
await client.write(
|
||||
'INSERT INTO `user_to_group_permissions` ' +
|
||||
'(`user_id`, `group_id`, `permission`, `extra`) VALUES (?, ?, ?, ?)',
|
||||
[system.id, id, 'service:some-paid-thing', '{}'],
|
||||
);
|
||||
|
||||
await client.write(MIGRATION);
|
||||
|
||||
expect(await stillThere(id)).toBe(true);
|
||||
// And the grant it carries is still intact, not cascaded away.
|
||||
const perms = await client.read(
|
||||
'SELECT `permission` FROM `user_to_group_permissions` WHERE `group_id` = ?',
|
||||
[id],
|
||||
);
|
||||
expect(perms).toEqual([{ permission: 'service:some-paid-thing' }]);
|
||||
});
|
||||
|
||||
it('spares one that still has members', async () => {
|
||||
const id = await seedGroup('experimental');
|
||||
const [system] = await client.read(
|
||||
'SELECT `id` FROM `user` WHERE `uuid` = ?',
|
||||
[SYSTEM_USER_UUID],
|
||||
);
|
||||
await client.write(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[system.id, id],
|
||||
);
|
||||
|
||||
await client.write(MIGRATION);
|
||||
|
||||
expect(await stillThere(id)).toBe(true);
|
||||
});
|
||||
|
||||
it('leaves the groups the platform actually depends on alone', async () => {
|
||||
// system, admin, user and temp are named by config or code.
|
||||
await client.write(MIGRATION);
|
||||
const rows = await client.read(
|
||||
"SELECT json_extract(`extra`, '$.name') AS name FROM `group`",
|
||||
);
|
||||
const names = rows.map((r) => String(r.name));
|
||||
for (const name of ['system', 'admin', 'user']) {
|
||||
expect(names).toContain(name);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
it('leaves an already-migrated database untouched on a second boot', async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
|
||||
const path = join(dir, 'nested', 'puter.sqlite');
|
||||
@@ -137,7 +244,10 @@ describe('SqliteDatabaseClient — boot and migrations', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('SqliteDatabaseClient — legacy version inference', () => {
|
||||
describe(
|
||||
'SqliteDatabaseClient — legacy version inference',
|
||||
{ timeout: DISK_MIGRATION_TIMEOUT_MS },
|
||||
() => {
|
||||
let dir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
|
||||
@@ -95,6 +95,11 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
|
||||
[58, ['0063_add_suspended_reason.sql']],
|
||||
[59, ['0064_abuse-moderation-events.sql']],
|
||||
[60, ['0065_app-feedback.sql']],
|
||||
[61, ['0066_owned-email-unique.sql']],
|
||||
[62, ['0067_share_entries.sql']],
|
||||
[63, ['0068_referral-code-unique.sql']],
|
||||
[64, ['0069_user-block.sql']],
|
||||
[65, ['0070_drop-orphaned-default-groups.sql']],
|
||||
];
|
||||
|
||||
export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Enforce "at most one account owns an email address". Mirrors SQLite
|
||||
-- migration 0066.
|
||||
--
|
||||
-- `user.email` is deliberately not UNIQUE: several rows may legitimately hold
|
||||
-- the same address while unconfirmed (admin-provisioned placeholders, signups
|
||||
-- that were never confirmed, temp accounts on their way to becoming real). What
|
||||
-- must never happen is two rows both *owning* an address — owning meaning the
|
||||
-- row is confirmed, or holds a password and so can drive password recovery for
|
||||
-- that inbox.
|
||||
--
|
||||
-- Signup, save-account, change-email, OIDC and admin provisioning each check for
|
||||
-- an owner before writing, but a check and an insert are not one operation: two
|
||||
-- requests can both read "free" and both write. This index is what actually
|
||||
-- holds the invariant; the application checks just produce a nicer error most of
|
||||
-- the time.
|
||||
--
|
||||
-- SQLite expresses that with a partial index. MySQL has none, so the predicate
|
||||
-- lives in a generated column that evaluates to NULL for every row that does not
|
||||
-- own its address — and NULLs do not collide in an InnoDB unique index, which is
|
||||
-- exactly the "unlimited unconfirmed placeholders" behaviour we need.
|
||||
--
|
||||
-- The column is VIRTUAL, not STORED, on purpose: adding a stored generated
|
||||
-- column rebuilds the table, while a virtual one is a metadata-only change and
|
||||
-- the index that follows builds INPLACE. On a `user` table of any size with
|
||||
-- read replicas attached, that is the difference between a routine change and an
|
||||
-- outage. The ALGORITHM/LOCK clauses are spelled out so a server that cannot
|
||||
-- honour them refuses the statement instead of quietly copying the table.
|
||||
--
|
||||
-- Matching is on the canonical address so provider aliases
|
||||
-- (`[email protected]` vs `[email protected]`) collide. `clean_email` is
|
||||
-- written on every modern write path; the COALESCE covers rows old enough to
|
||||
-- predate the column. Run the `clean_email` backfill before this migration or
|
||||
-- alias collisions among those rows go unnoticed.
|
||||
--
|
||||
-- Idempotent: both steps are guarded on INFORMATION_SCHEMA so the directory
|
||||
-- replays safely.
|
||||
--
|
||||
-- If the index creation fails with ER_DUP_ENTRY, the DB already contains
|
||||
-- duplicate owners. Collapse them first (admin → One-off Jobs → Collapse
|
||||
-- Duplicate Emails) — there is no safe automatic merge of two accounts.
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_owned_email()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'user'
|
||||
AND COLUMN_NAME = 'owned_email'
|
||||
) THEN
|
||||
ALTER TABLE `user`
|
||||
ADD COLUMN `owned_email` VARCHAR(256)
|
||||
CHARACTER SET latin1 COLLATE latin1_swedish_ci
|
||||
GENERATED ALWAYS AS (
|
||||
CASE
|
||||
WHEN `email` IS NOT NULL
|
||||
AND (`email_confirmed` = 1 OR `password` IS NOT NULL)
|
||||
THEN COALESCE(`clean_email`, LOWER(`email`))
|
||||
ELSE NULL
|
||||
END
|
||||
) VIRTUAL,
|
||||
ALGORITHM=INSTANT;
|
||||
END IF;
|
||||
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'user'
|
||||
AND INDEX_NAME = 'idx_user_owned_email'
|
||||
) THEN
|
||||
ALTER TABLE `user`
|
||||
ADD UNIQUE KEY `idx_user_owned_email` (`owned_email`),
|
||||
ALGORITHM=INPLACE, LOCK=NONE;
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_owned_email();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
|
||||
|
||||
-- One Puter account per external identity. OIDCStore.link already assumes this
|
||||
-- constraint exists — it catches the unique violation to tell "re-linking the
|
||||
-- same account" apart from "this sub belongs to someone else" — but the table
|
||||
-- never actually had it, so two concurrent first-time logins could each create
|
||||
-- an account and each link the same sub. Subsequent logins then resolved to
|
||||
-- whichever row came back first.
|
||||
--
|
||||
-- Dedupe `user_oidc_providers` before applying this: keep the lowest `id` per
|
||||
-- (provider, provider_sub) and point it at the account the collapse job kept.
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_oidc_sub_unique()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'user_oidc_providers'
|
||||
AND INDEX_NAME = 'idx_user_oidc_provider_sub'
|
||||
) THEN
|
||||
ALTER TABLE `user_oidc_providers`
|
||||
ADD UNIQUE KEY `idx_user_oidc_provider_sub` (`provider`, `provider_sub`);
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_oidc_sub_unique();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
|
||||
@@ -0,0 +1,106 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Grow `share` from a pending-email-invite table into the index of active
|
||||
-- shares. See sqlite/0067_share_entries.sql for the column rationale.
|
||||
--
|
||||
-- Idempotent: columns go through _puter_add_col (from mig_1), indexes and
|
||||
-- foreign keys through the guarded procedure below. There is no per-file
|
||||
-- applied-state tracking, so every statement must tolerate a re-run.
|
||||
|
||||
CALL _puter_add_col('share', 'holder_user_id', '`holder_user_id` int unsigned DEFAULT NULL');
|
||||
CALL _puter_add_col('share', 'fsentry_id', '`fsentry_id` int unsigned DEFAULT NULL');
|
||||
CALL _puter_add_col('share', 'mode', '`mode` varchar(20) DEFAULT NULL');
|
||||
CALL _puter_add_col('share', 'applied_at', '`applied_at` timestamp NULL DEFAULT NULL');
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_share_index_constraints()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'share'
|
||||
AND INDEX_NAME = 'idx_share_holder'
|
||||
) THEN
|
||||
ALTER TABLE `share` ADD INDEX `idx_share_holder` (`holder_user_id`, `id`);
|
||||
END IF;
|
||||
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'share'
|
||||
AND INDEX_NAME = 'idx_share_fsentry'
|
||||
) THEN
|
||||
ALTER TABLE `share` ADD INDEX `idx_share_fsentry` (`fsentry_id`);
|
||||
END IF;
|
||||
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'share'
|
||||
AND INDEX_NAME = 'idx_share_holder_entry_issuer'
|
||||
) THEN
|
||||
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_holder_entry_issuer`
|
||||
(`holder_user_id`, `fsentry_id`, `issuer_user_id`);
|
||||
END IF;
|
||||
|
||||
-- Retiring a deleted node's grants looks them up by permission text
|
||||
-- (`fs:<uuid>` and `fs:<uuid>:%`); the subject lives in that text rather
|
||||
-- than a column, so no foreign key can cascade it. The primary key is
|
||||
-- (issuer, holder, permission), which never puts `permission` first, so
|
||||
-- without this both the equality and the left-anchored LIKE degrade to a
|
||||
-- full scan on the delete path. The column is ascii varchar(255), well
|
||||
-- inside InnoDB's key limit, so it is indexed whole.
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'user_to_user_permissions'
|
||||
AND INDEX_NAME = 'idx_user_to_user_permissions_permission'
|
||||
) THEN
|
||||
ALTER TABLE `user_to_user_permissions`
|
||||
ADD INDEX `idx_user_to_user_permissions_permission` (`permission`);
|
||||
END IF;
|
||||
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'share'
|
||||
AND CONSTRAINT_NAME = 'share_holder_user_fk'
|
||||
) THEN
|
||||
ALTER TABLE `share` ADD CONSTRAINT `share_holder_user_fk`
|
||||
FOREIGN KEY (`holder_user_id`) REFERENCES `user` (`id`)
|
||||
ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
END IF;
|
||||
|
||||
-- The cascade that retires a share with its file.
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'share'
|
||||
AND CONSTRAINT_NAME = 'share_fsentry_fk'
|
||||
) THEN
|
||||
ALTER TABLE `share` ADD CONSTRAINT `share_fsentry_fk`
|
||||
FOREIGN KEY (`fsentry_id`) REFERENCES `fsentries` (`id`)
|
||||
ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_share_index_constraints();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
|
||||
@@ -0,0 +1,35 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- One user refusing contact from another. See sqlite/0068_user-block.sql for
|
||||
-- the rationale. `created_at` is unix seconds.
|
||||
--
|
||||
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
|
||||
-- so the directory can replay safely.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS `user_block` (
|
||||
`id` INT NOT NULL AUTO_INCREMENT,
|
||||
`blocker_user_id` INT UNSIGNED NOT NULL,
|
||||
`blocked_user_id` INT UNSIGNED NOT NULL,
|
||||
`created_at` BIGINT NOT NULL,
|
||||
PRIMARY KEY (`id`),
|
||||
UNIQUE KEY `idx_user_block_pair` (`blocker_user_id`, `blocked_user_id`),
|
||||
CONSTRAINT `fk_user_block_blocker` FOREIGN KEY (`blocker_user_id`)
|
||||
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
CONSTRAINT `fk_user_block_blocked` FOREIGN KEY (`blocked_user_id`)
|
||||
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
@@ -0,0 +1,61 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Widen `notification.shown` and `notification.acknowledged` to hold the unix
|
||||
-- second the store has been writing to them since the backend rework.
|
||||
--
|
||||
-- Both arrived from the v1 schema as `tinyint(1)`, where they were flags. The
|
||||
-- rework changed the writes to timestamps and sqlite (`INTEGER`) and postgres
|
||||
-- (`bigint`) took them; only mysql was left too narrow, so every `markShown`
|
||||
-- and `markAcknowledged` there fails with ER_WARN_DATA_OUT_OF_RANGE and the
|
||||
-- column stays NULL. Dismissing a notification therefore never sticks, and one
|
||||
-- already delivered is re-sent on every reconnect.
|
||||
--
|
||||
-- No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a legacy
|
||||
-- `1` keeps meaning "yes" once widened.
|
||||
--
|
||||
-- Guarded on the current type rather than run unconditionally. Changing a
|
||||
-- column type copies the table, and this one grows without bound — a migration
|
||||
-- directory that replays on every boot must not rebuild it every time.
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_widen_notification_stamps()
|
||||
BEGIN
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'notification'
|
||||
AND COLUMN_NAME = 'shown'
|
||||
AND DATA_TYPE = 'tinyint'
|
||||
) THEN
|
||||
ALTER TABLE `notification` MODIFY `shown` BIGINT DEFAULT NULL;
|
||||
END IF;
|
||||
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'notification'
|
||||
AND COLUMN_NAME = 'acknowledged'
|
||||
AND DATA_TYPE = 'tinyint'
|
||||
) THEN
|
||||
ALTER TABLE `notification` MODIFY `acknowledged` BIGINT DEFAULT NULL;
|
||||
END IF;
|
||||
END //
|
||||
DELIMITER ;
|
||||
CALL _puter_widen_notification_stamps();
|
||||
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
|
||||
@@ -0,0 +1,22 @@
|
||||
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
|
||||
-- `dangerous`. No migration creates them — they were added by hand back when
|
||||
-- hardcoded permissions were keyed by group name. That map is now a flat
|
||||
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
|
||||
-- so a group nothing looks up grants nothing, and the rows are dead weight.
|
||||
--
|
||||
-- Guarded, and deliberately so: `user_to_group_permissions.group_id` and
|
||||
-- `jct_user_group.group_id` both cascade on delete, so removing a group that
|
||||
-- still carries permissions or members would silently revoke them from every
|
||||
-- member. Only a group with neither is dropped. A group that survives this
|
||||
-- migration has dependents and needs a deliberate decision, not a sweep --
|
||||
-- query `user_to_group_permissions` by `group_id` to see what it holds.
|
||||
--
|
||||
-- The joins rather than `NOT IN` subqueries: MySQL will not read from the table
|
||||
-- it is deleting from inside a subquery on the same statement.
|
||||
DELETE g FROM `group` g
|
||||
LEFT JOIN `user_to_group_permissions` p ON p.`group_id` = g.`id`
|
||||
LEFT JOIN `jct_user_group` j ON j.`group_id` = g.`id`
|
||||
WHERE JSON_UNQUOTE(JSON_EXTRACT(g.`extra`, '$.name'))
|
||||
IN ('freeai', 'experimental', 'dangerous')
|
||||
AND p.`group_id` IS NULL
|
||||
AND j.`group_id` IS NULL;
|
||||
@@ -0,0 +1,40 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Enforce "at most one account owns an email address". Mirrors SQLite
|
||||
-- migration 0066 and MySQL migration 21; see those for the full rationale.
|
||||
--
|
||||
-- In short: `user.email` is deliberately not UNIQUE because several rows may
|
||||
-- hold the same address while unconfirmed. What must not happen is two rows
|
||||
-- both owning it — confirmed, or holding a password and so able to drive
|
||||
-- password recovery for that inbox. The application checks for an owner before
|
||||
-- every write, but a check and a write are not one operation.
|
||||
--
|
||||
-- If this fails, the DB already contains duplicate owners; collapse them first
|
||||
-- (admin → One-off Jobs → Collapse Duplicate Emails).
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_owned_email
|
||||
ON "user" (COALESCE(clean_email, LOWER(email)))
|
||||
WHERE email IS NOT NULL
|
||||
AND (email_confirmed = TRUE OR password IS NOT NULL);
|
||||
|
||||
-- One Puter account per external identity. OIDCStore.link already assumes this
|
||||
-- constraint exists — it catches the unique violation to tell "re-linking the
|
||||
-- same account" apart from "this sub belongs to someone else" — but the table
|
||||
-- never actually had it.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_provider_sub
|
||||
ON user_oidc_providers (provider, provider_sub);
|
||||
@@ -0,0 +1,42 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Grow share from a pending-email-invite table into the index of active
|
||||
-- shares. See sqlite/0067_share_entries.sql for the column rationale.
|
||||
|
||||
ALTER TABLE share
|
||||
ADD COLUMN IF NOT EXISTS holder_user_id integer
|
||||
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
ADD COLUMN IF NOT EXISTS fsentry_id integer
|
||||
REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
ADD COLUMN IF NOT EXISTS mode varchar(20),
|
||||
ADD COLUMN IF NOT EXISTS applied_at timestamp;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_share_holder
|
||||
ON share (holder_user_id, id);
|
||||
CREATE INDEX IF NOT EXISTS idx_share_fsentry
|
||||
ON share (fsentry_id);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_share_holder_entry_issuer
|
||||
ON share (holder_user_id, fsentry_id, issuer_user_id);
|
||||
|
||||
-- Retiring a deleted node's grants looks them up by permission text, and the
|
||||
-- primary key is (issuer, holder, permission) — `permission` is never the
|
||||
-- leading column, so the delete path would otherwise scan the table. The
|
||||
-- text_pattern_ops class is what makes the left-anchored LIKE a range scan
|
||||
-- under a non-C collation; plain equality still uses it.
|
||||
CREATE INDEX IF NOT EXISTS idx_user_to_user_permissions_permission
|
||||
ON user_to_user_permissions (permission text_pattern_ops);
|
||||
@@ -0,0 +1,33 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- One user refusing contact from another. See sqlite/0068_user-block.sql for
|
||||
-- the rationale. `created_at` is unix seconds.
|
||||
--
|
||||
-- Idempotent via IF NOT EXISTS.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS user_block (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
blocker_user_id integer NOT NULL
|
||||
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
blocked_user_id integer NOT NULL
|
||||
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
created_at BIGINT NOT NULL
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_block_pair
|
||||
ON user_block (blocker_user_id, blocked_user_id);
|
||||
@@ -0,0 +1,20 @@
|
||||
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
|
||||
-- `dangerous`. No migration creates them — they were added by hand back when
|
||||
-- hardcoded permissions were keyed by group name. That map is now a flat
|
||||
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
|
||||
-- so a group nothing looks up grants nothing, and the rows are dead weight.
|
||||
--
|
||||
-- Guarded, and deliberately so: user_to_group_permissions.group_id and
|
||||
-- jct_user_group.group_id both cascade on delete, so removing a group that
|
||||
-- still carries permissions or members would silently revoke them from every
|
||||
-- member. Only a group with neither is dropped. A group that survives this
|
||||
-- migration has dependents and needs a deliberate decision, not a sweep --
|
||||
-- query user_to_group_permissions by group_id to see what it holds.
|
||||
DELETE FROM "group" g
|
||||
WHERE g.extra ->> 'name' IN ('freeai', 'experimental', 'dangerous')
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM user_to_group_permissions p WHERE p.group_id = g.id
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM jct_user_group j WHERE j.group_id = g.id
|
||||
);
|
||||
@@ -0,0 +1,53 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Enforce "at most one account owns an email address".
|
||||
--
|
||||
-- `user.email` is deliberately not UNIQUE: several rows may legitimately hold
|
||||
-- the same address while unconfirmed (admin-provisioned placeholders, signups
|
||||
-- that were never confirmed, temp accounts on their way to becoming real). What
|
||||
-- must never happen is two rows both *owning* an address — owning meaning the
|
||||
-- row is confirmed, or holds a password and so can drive password recovery for
|
||||
-- that inbox.
|
||||
--
|
||||
-- Signup, save-account, change-email, OIDC and admin provisioning each check for
|
||||
-- an owner before writing, but a check and an insert are not one operation: two
|
||||
-- requests can both read "free" and both write. This index is what actually
|
||||
-- holds the invariant; the application checks just produce a nicer error most of
|
||||
-- the time.
|
||||
--
|
||||
-- Matching is on the canonical address so provider aliases
|
||||
-- (`[email protected]` vs `[email protected]`) collide. `clean_email` is
|
||||
-- written on every modern write path; the COALESCE covers rows old enough to
|
||||
-- predate the column.
|
||||
--
|
||||
-- If this CREATE fails, the DB already contains duplicate owners. Collapse them
|
||||
-- first (admin → One-off Jobs → Collapse Duplicate Emails) — there is no safe
|
||||
-- automatic merge of two accounts.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_owned_email
|
||||
ON user(COALESCE(clean_email, lower(email)))
|
||||
WHERE email IS NOT NULL
|
||||
AND (email_confirmed = 1 OR password IS NOT NULL);
|
||||
|
||||
-- One Puter account per external identity. OIDCStore.link already assumes this
|
||||
-- constraint exists — it catches the unique violation to tell "re-linking the
|
||||
-- same account" apart from "this sub belongs to someone else" — but the table
|
||||
-- never actually had it, so two concurrent first-time logins could each create
|
||||
-- an account and each link the same sub. Subsequent logins then resolved to
|
||||
-- whichever row came back first.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_provider_sub
|
||||
ON user_oidc_providers(provider, provider_sub);
|
||||
@@ -0,0 +1,60 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Grow `share` from a pending-email-invite table into the index of active
|
||||
-- shares. Permissions stay the source of truth for access; this is what makes
|
||||
-- shares listable and gives them a lifecycle.
|
||||
--
|
||||
-- - `holder_user_id` : NULL while an invite awaits signup.
|
||||
-- - `fsentry_id` : the shared node. ON DELETE CASCADE retires the share
|
||||
-- with the file, which permissions alone don't do.
|
||||
-- - `mode` : see|list|read|write|manage. Unconstrained on purpose —
|
||||
-- ACLService owns the mode set and adding one shouldn't
|
||||
-- need a three-dialect migration.
|
||||
-- - `applied_at` : set when an invite is claimed. Claiming updates the
|
||||
-- row rather than deleting it, so the share stays
|
||||
-- queryable.
|
||||
|
||||
ALTER TABLE `share` ADD COLUMN `holder_user_id` INTEGER DEFAULT NULL
|
||||
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
ALTER TABLE `share` ADD COLUMN `fsentry_id` INTEGER DEFAULT NULL
|
||||
REFERENCES `fsentries` (`id`) ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
ALTER TABLE `share` ADD COLUMN `mode` TEXT DEFAULT NULL;
|
||||
ALTER TABLE `share` ADD COLUMN `applied_at` TIMESTAMP DEFAULT NULL;
|
||||
|
||||
-- "Shared with me", keyset-paginated: ORDER BY ends in `id` as the tiebreaker.
|
||||
CREATE INDEX IF NOT EXISTS `idx_share_holder`
|
||||
ON `share` (`holder_user_id`, `id`);
|
||||
|
||||
-- Who has access to one node — also how an owner sees a manage-delegate's
|
||||
-- re-grants, which the issuer/holder permission tables can't answer.
|
||||
CREATE INDEX IF NOT EXISTS `idx_share_fsentry` ON `share` (`fsentry_id`);
|
||||
|
||||
-- One row per (holder, node, issuer). Pending invites have a NULL
|
||||
-- holder_user_id and so aren't covered here; dedup for those belongs with the
|
||||
-- invite flow.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS `idx_share_holder_entry_issuer`
|
||||
ON `share` (`holder_user_id`, `fsentry_id`, `issuer_user_id`);
|
||||
|
||||
-- Retiring a deleted node's grants looks them up by permission text
|
||||
-- (`fs:<uuid>` and `fs:<uuid>:%`), and the subject only lives in that text —
|
||||
-- no column, so no foreign key can cascade it. The table's primary key is
|
||||
-- (issuer, holder, permission), which never puts `permission` first, so
|
||||
-- without this the equality and the left-anchored LIKE both degrade to a full
|
||||
-- scan on the delete path. With it, each is a range scan.
|
||||
CREATE INDEX IF NOT EXISTS `idx_user_to_user_permissions_permission`
|
||||
ON `user_to_user_permissions` (`permission`);
|
||||
@@ -0,0 +1,32 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- One referral code, one account. The mysql and postgres baselines have
|
||||
-- declared `user.referral_code` UNIQUE since the column was introduced; the
|
||||
-- sqlite baseline only declared the column, so the invariant held on hosted
|
||||
-- deployments and not on self-hosted ones.
|
||||
--
|
||||
-- Codes are minted by picking at random and writing, so this index is not a
|
||||
-- nicety: it is what makes "did anyone already take this code" a decision the
|
||||
-- database makes once, instead of a read the next mint can race. A caller that
|
||||
-- loses the race sees a unique violation and picks again.
|
||||
--
|
||||
-- NULL is distinct from NULL in a sqlite unique index, so accounts that have
|
||||
-- never asked for a code (the majority) are unaffected.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_referral_code
|
||||
ON user(referral_code)
|
||||
WHERE referral_code IS NOT NULL;
|
||||
@@ -0,0 +1,39 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- One user refusing contact from another. Sharing is what reads it today —
|
||||
-- a blocked sender's share is refused outright, and their pending invite is
|
||||
-- dropped when the blocker confirms the address it was aimed at — but the
|
||||
-- table is deliberately not share-specific: the same answer serves anything
|
||||
-- else one person can push at another.
|
||||
--
|
||||
-- Rows are read on the share path by exact (blocker, blocked) pair, which the
|
||||
-- unique index answers as a point lookup; the same index's leading column
|
||||
-- serves "who have I blocked". `created_at` is unix seconds, matching
|
||||
-- `app_feedback`.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS `user_block` (
|
||||
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
"blocker_user_id" INTEGER NOT NULL
|
||||
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
"blocked_user_id" INTEGER NOT NULL
|
||||
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
|
||||
"created_at" INTEGER NOT NULL -- unix seconds
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS `idx_user_block_pair`
|
||||
ON `user_block` (`blocker_user_id`, `blocked_user_id`);
|
||||
@@ -0,0 +1,33 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
|
||||
-- `dangerous`. No migration creates them — they were added by hand back when
|
||||
-- hardcoded permissions were keyed by group name. That map is now a flat
|
||||
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
|
||||
-- so a group nothing looks up grants nothing, and the rows are dead weight.
|
||||
--
|
||||
-- Guarded, and deliberately so: `user_to_group_permissions.group_id` and
|
||||
-- `jct_user_group.group_id` both cascade on delete, so removing a group that
|
||||
-- still carries permissions or members would silently revoke them from every
|
||||
-- member. Only a group with neither is dropped. A group that survives this
|
||||
-- migration has dependents and needs a deliberate decision, not a sweep --
|
||||
-- query `user_to_group_permissions` by `group_id` to see what it holds.
|
||||
DELETE FROM `group`
|
||||
WHERE json_extract(`extra`, '$.name') IN ('freeai', 'experimental', 'dangerous')
|
||||
AND `id` NOT IN (SELECT `group_id` FROM `user_to_group_permissions`)
|
||||
AND `id` NOT IN (SELECT `group_id` FROM `jct_user_group`);
|
||||
@@ -3,18 +3,19 @@
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import { createServer, type Server } from 'node:http';
|
||||
@@ -302,6 +303,35 @@ describe('DDBClient — batch writes', () => {
|
||||
const stored = await client.query(TABLE, { pk: 'bulk' });
|
||||
expect(stored.Items).toHaveLength(60);
|
||||
});
|
||||
|
||||
it('reports no consumed capacity for an empty delete batch', async () => {
|
||||
await expect(client.batchDel([])).resolves.toEqual({
|
||||
ConsumedCapacity: [],
|
||||
});
|
||||
});
|
||||
|
||||
it('batch-deletes across chunks, leaving unrelated items alone', async () => {
|
||||
const items = Array.from({ length: 30 }, (_, index) => ({
|
||||
table: TABLE,
|
||||
item: { pk: 'bulk-del', sk: `item-${index}`, index },
|
||||
}));
|
||||
await client.batchPut(items);
|
||||
|
||||
// Delete all but the last item — 29 keys still exercises chunking
|
||||
// paths shared with batchPut while proving deletion is targeted.
|
||||
const result = await client.batchDel(
|
||||
items.slice(0, -1).map(({ table, item }) => ({
|
||||
table,
|
||||
key: { pk: item.pk, sk: item.sk },
|
||||
})),
|
||||
);
|
||||
|
||||
expect(result.ConsumedCapacity).toHaveLength(1);
|
||||
expect(result.ConsumedCapacity[0].TableName).toBe(TABLE);
|
||||
|
||||
const stored = await client.query(TABLE, { pk: 'bulk-del' });
|
||||
expect(stored.Items?.map((item) => item.sk)).toEqual(['item-29']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — expired item sweep', () => {
|
||||
|
||||
@@ -45,6 +45,11 @@ import { HttpError } from '../../core/http';
|
||||
import type { IConfig, IDynamoConfig } from '../../types';
|
||||
import { Span } from '../../util/span.js';
|
||||
import { PuterClient } from '../types';
|
||||
import {
|
||||
clampStoredNumber,
|
||||
isRepairableMarshallError,
|
||||
repairForMarshall,
|
||||
} from './marshallRepair.js';
|
||||
|
||||
const LOCAL_DYNAMO_MEMORY_PREFIX = ':memory:';
|
||||
const localDynaliteEndpointPromises = new Map<string, Promise<string>>();
|
||||
@@ -114,6 +119,49 @@ const sleep = async (ms: number) => {
|
||||
await new Promise((resolve) => setTimeout(resolve, ms));
|
||||
};
|
||||
|
||||
// One caller writing an out-of-range number usually keeps doing it, so the
|
||||
// warning is per-target and throttled rather than one line per write.
|
||||
const REPAIR_WARNING_INTERVAL_MS = 60_000;
|
||||
const lastRepairWarningAt = new Map<string, number>();
|
||||
|
||||
const warnRepaired = (target: string, message: string): void => {
|
||||
const now = Date.now();
|
||||
const lastWarnedAt = lastRepairWarningAt.get(target) ?? 0;
|
||||
if (now - lastWarnedAt < REPAIR_WARNING_INTERVAL_MS) return;
|
||||
|
||||
lastRepairWarningAt.set(target, now);
|
||||
console.warn(
|
||||
`[ddb] clamped an out-of-range value in ${target}: ${message}`,
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Send a write, and if the payload was rejected while being encoded for a value
|
||||
* the store cannot represent, clamp it and send it once more.
|
||||
*
|
||||
* Nothing is inspected on the way in: the payload of a successful write is
|
||||
* never walked, so this costs one `try` on the hot path. The repair only runs
|
||||
* on the failure, and only retries when it actually changed something —
|
||||
* otherwise the original error stands.
|
||||
*/
|
||||
const sendRepairingValues = async <TPayload, TResult>(
|
||||
payload: TPayload,
|
||||
send: (payload: TPayload) => Promise<TResult>,
|
||||
describe: () => string,
|
||||
): Promise<TResult> => {
|
||||
try {
|
||||
return await send(payload);
|
||||
} catch (error) {
|
||||
if (!isRepairableMarshallError(error)) throw error;
|
||||
|
||||
const repaired = repairForMarshall(payload);
|
||||
if (!repaired.changed) throw error;
|
||||
|
||||
warnRepaired(describe(), (error as Error).message);
|
||||
return send(repaired.value as TPayload);
|
||||
}
|
||||
};
|
||||
|
||||
export class DDBClient extends PuterClient {
|
||||
#documentClient: DynamoDBDocumentClient | null = null;
|
||||
#localInitPromise: Promise<void> | null = null;
|
||||
@@ -171,14 +219,19 @@ export class DDBClient extends PuterClient {
|
||||
|
||||
@Span('ddb.put', (table: string) => ({ 'db.table': table }))
|
||||
async put<T extends Record<string, unknown>>(table: string, item: T) {
|
||||
const command = new PutCommand({
|
||||
TableName: table,
|
||||
Item: item,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
});
|
||||
|
||||
const client = await this.#getDocumentClient();
|
||||
return client.send(command);
|
||||
return sendRepairingValues(
|
||||
item,
|
||||
(itemToWrite) =>
|
||||
client.send(
|
||||
new PutCommand({
|
||||
TableName: table,
|
||||
Item: itemToWrite,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
}),
|
||||
),
|
||||
() => `a put to ${table}`,
|
||||
);
|
||||
}
|
||||
|
||||
@Span('ddb.batchGet', (params: unknown[]) => ({
|
||||
@@ -222,6 +275,37 @@ export class DDBClient extends PuterClient {
|
||||
'db.batch_size': params.length,
|
||||
}))
|
||||
async batchPut(params: { table: string; item: Record<string, unknown> }[]) {
|
||||
return this.#batchWrite(
|
||||
params.map(({ table, item }) => ({
|
||||
table,
|
||||
request: { PutRequest: { Item: item } },
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
@Span('ddb.batchDel', (params: unknown[]) => ({
|
||||
'db.batch_size': params.length,
|
||||
}))
|
||||
async batchDel(params: { table: string; key: Record<string, unknown> }[]) {
|
||||
return this.#batchWrite(
|
||||
params.map(({ table, key }) => ({
|
||||
table,
|
||||
request: { DeleteRequest: { Key: key } },
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
// Shared BatchWriteItem plumbing for batchPut/batchDel: 25-item chunks,
|
||||
// UnprocessedItems retried with capped exponential backoff, consumed
|
||||
// capacity accumulated per table across every request.
|
||||
async #batchWrite(
|
||||
params: {
|
||||
table: string;
|
||||
request: NonNullable<
|
||||
BatchWriteCommandInput['RequestItems']
|
||||
>[string][number];
|
||||
}[],
|
||||
) {
|
||||
const consumedCapacityByTable = new Map<string, number>();
|
||||
if (params.length === 0) {
|
||||
return { ConsumedCapacity: [] };
|
||||
@@ -258,11 +342,7 @@ export class DDBClient extends PuterClient {
|
||||
let requestItems = chunk.reduce(
|
||||
(acc, curr) => {
|
||||
const tableRequests = acc[curr.table] ?? [];
|
||||
tableRequests.push({
|
||||
PutRequest: {
|
||||
Item: curr.item,
|
||||
},
|
||||
});
|
||||
tableRequests.push(curr.request);
|
||||
acc[curr.table] = tableRequests;
|
||||
return acc;
|
||||
},
|
||||
@@ -278,11 +358,17 @@ export class DDBClient extends PuterClient {
|
||||
break;
|
||||
}
|
||||
|
||||
const response = await client.send(
|
||||
new BatchWriteCommand({
|
||||
RequestItems: requestItems,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
}),
|
||||
const response = await sendRepairingValues(
|
||||
requestItems,
|
||||
(itemsToWrite) =>
|
||||
client.send(
|
||||
new BatchWriteCommand({
|
||||
RequestItems: itemsToWrite,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
}),
|
||||
),
|
||||
() =>
|
||||
`a batch write to ${Object.keys(requestItems).join(', ')}`,
|
||||
);
|
||||
accumulateConsumedCapacity(
|
||||
response.ConsumedCapacity as
|
||||
@@ -328,11 +414,18 @@ export class DDBClient extends PuterClient {
|
||||
}
|
||||
|
||||
@Span('ddb.del', (table: string) => ({ 'db.table': table }))
|
||||
async del<T extends Record<string, unknown>>(table: string, key: T) {
|
||||
async del<T extends Record<string, unknown>>(
|
||||
table: string,
|
||||
key: T,
|
||||
opts?: { returnOld?: boolean },
|
||||
) {
|
||||
const command = new DeleteCommand({
|
||||
TableName: table,
|
||||
Key: key,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
// ALL_OLD makes the delete an atomic claim: exactly one caller
|
||||
// gets the attributes back.
|
||||
...(opts?.returnOld ? { ReturnValues: 'ALL_OLD' as const } : {}),
|
||||
});
|
||||
|
||||
const client = await this.#getDocumentClient();
|
||||
@@ -423,20 +516,27 @@ export class DDBClient extends PuterClient {
|
||||
!!expressionValues && Object.keys(expressionValues).length > 0;
|
||||
const hasNames =
|
||||
!!expressionNames && Object.keys(expressionNames).length > 0;
|
||||
const command = new UpdateCommand({
|
||||
TableName: table,
|
||||
Key: key,
|
||||
UpdateExpression: expression,
|
||||
...(hasValues
|
||||
? { ExpressionAttributeValues: expressionValues }
|
||||
: {}),
|
||||
...(hasNames ? { ExpressionAttributeNames: expressionNames } : {}),
|
||||
ReturnValues: 'ALL_NEW',
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
});
|
||||
|
||||
const client = await this.#getDocumentClient();
|
||||
return client.send(command);
|
||||
return sendRepairingValues(
|
||||
expressionValues,
|
||||
(valuesToWrite) =>
|
||||
client.send(
|
||||
new UpdateCommand({
|
||||
TableName: table,
|
||||
Key: key,
|
||||
UpdateExpression: expression,
|
||||
...(hasValues
|
||||
? { ExpressionAttributeValues: valuesToWrite }
|
||||
: {}),
|
||||
...(hasNames
|
||||
? { ExpressionAttributeNames: expressionNames }
|
||||
: {}),
|
||||
ReturnValues: 'ALL_NEW',
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
}),
|
||||
),
|
||||
() => `an update to ${table}`,
|
||||
);
|
||||
}
|
||||
|
||||
async createTableIfNotExists(
|
||||
@@ -538,6 +638,9 @@ export class DDBClient extends PuterClient {
|
||||
marshallOptions: {
|
||||
removeUndefinedValues: true,
|
||||
},
|
||||
unmarshallOptions: {
|
||||
wrapNumbers: clampStoredNumber,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -565,6 +668,9 @@ export class DDBClient extends PuterClient {
|
||||
marshallOptions: {
|
||||
removeUndefinedValues: true,
|
||||
},
|
||||
unmarshallOptions: {
|
||||
wrapNumbers: clampStoredNumber,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
clampStoredNumber,
|
||||
isRepairableMarshallError,
|
||||
MAX_STORED_NUMBER,
|
||||
MIN_STORED_NUMBER,
|
||||
repairForMarshall,
|
||||
} from './marshallRepair.ts';
|
||||
|
||||
describe('isRepairableMarshallError', () => {
|
||||
it('recognizes the out-of-range and special-value failures', () => {
|
||||
for (const message of [
|
||||
'Number 1.6515584833071455e+55 is greater than Number.MAX_SAFE_INTEGER. Use NumberValue from @aws-sdk/lib-dynamodb.',
|
||||
'Number -1e+55 is lesser than Number.MIN_SAFE_INTEGER. Use NumberValue from @aws-sdk/lib-dynamodb.',
|
||||
'Special numeric value NaN is not allowed',
|
||||
'Special numeric value Infinity is not allowed',
|
||||
]) {
|
||||
expect(isRepairableMarshallError(new Error(message))).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('leaves every other failure alone', () => {
|
||||
for (const error of [
|
||||
new Error('Unsupported type passed: [object Symbol].'),
|
||||
new Error('ValidationException'),
|
||||
new Error(''),
|
||||
undefined,
|
||||
'not an error',
|
||||
]) {
|
||||
expect(isRepairableMarshallError(error)).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('repairForMarshall', () => {
|
||||
it('clamps a number past the safe range in both directions', () => {
|
||||
expect(repairForMarshall(1.6515584833071455e55)).toEqual({
|
||||
value: MAX_STORED_NUMBER,
|
||||
changed: true,
|
||||
});
|
||||
expect(repairForMarshall(-1e55)).toEqual({
|
||||
value: MIN_STORED_NUMBER,
|
||||
changed: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('clamps infinities and nulls NaN', () => {
|
||||
expect(repairForMarshall(Infinity).value).toBe(MAX_STORED_NUMBER);
|
||||
expect(repairForMarshall(-Infinity).value).toBe(MIN_STORED_NUMBER);
|
||||
expect(repairForMarshall(NaN)).toEqual({ value: null, changed: true });
|
||||
});
|
||||
|
||||
it('repairs numbers nested in objects and arrays', () => {
|
||||
const repaired = repairForMarshall({
|
||||
profile: { netWorth: 1e55, level: 29 },
|
||||
scores: [1, [2, -1e55]],
|
||||
});
|
||||
|
||||
expect(repaired.changed).toBe(true);
|
||||
expect(repaired.value).toEqual({
|
||||
profile: { netWorth: MAX_STORED_NUMBER, level: 29 },
|
||||
scores: [1, [2, MIN_STORED_NUMBER]],
|
||||
});
|
||||
});
|
||||
|
||||
it('returns the original payload untouched when nothing is out of range', () => {
|
||||
const payload = {
|
||||
a: MAX_STORED_NUMBER,
|
||||
b: [1, 2, { c: 'three' }],
|
||||
d: null,
|
||||
};
|
||||
|
||||
const repaired = repairForMarshall(payload);
|
||||
expect(repaired.changed).toBe(false);
|
||||
expect(repaired.value).toBe(payload);
|
||||
});
|
||||
|
||||
it('does not mutate the payload it repairs', () => {
|
||||
const nested = { netWorth: 1e55 };
|
||||
const payload = { nested };
|
||||
|
||||
repairForMarshall(payload);
|
||||
|
||||
expect(nested.netWorth).toBe(1e55);
|
||||
});
|
||||
|
||||
it('keeps a `__proto__` key as data', () => {
|
||||
const payload = JSON.parse('{"__proto__":{"big":1e55}}');
|
||||
|
||||
const repaired = repairForMarshall(payload) as {
|
||||
value: Record<string, unknown>;
|
||||
changed: boolean;
|
||||
};
|
||||
|
||||
expect(repaired.changed).toBe(true);
|
||||
expect(Object.getPrototypeOf(repaired.value)).toBe(Object.prototype);
|
||||
expect(
|
||||
Object.getOwnPropertyDescriptor(repaired.value, '__proto__')?.value,
|
||||
).toEqual({ big: MAX_STORED_NUMBER });
|
||||
});
|
||||
|
||||
it('leaves values it does not own alone', () => {
|
||||
const date = new Date(0);
|
||||
expect(repairForMarshall(date).value).toBe(date);
|
||||
expect(repairForMarshall('str')).toEqual({
|
||||
value: 'str',
|
||||
changed: false,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('clampStoredNumber', () => {
|
||||
it('decodes a number inside the range unchanged', () => {
|
||||
expect(clampStoredNumber('42')).toBe(42);
|
||||
expect(clampStoredNumber('-0.5')).toBe(-0.5);
|
||||
expect(clampStoredNumber(String(MAX_STORED_NUMBER))).toBe(
|
||||
MAX_STORED_NUMBER,
|
||||
);
|
||||
});
|
||||
|
||||
it('clamps a stored number past the range instead of decoding a BigInt', () => {
|
||||
expect(clampStoredNumber('18014398509481982')).toBe(MAX_STORED_NUMBER);
|
||||
expect(clampStoredNumber('-18014398509481982')).toBe(MIN_STORED_NUMBER);
|
||||
expect(clampStoredNumber('1e400')).toBe(MAX_STORED_NUMBER);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,129 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* A stored number keeps its exact value only within the IEEE-754 safe integer
|
||||
* range; a magnitude past this cannot round-trip, and the document client
|
||||
* refuses to marshall one rather than write a value that reads back wrong.
|
||||
*/
|
||||
export const MAX_STORED_NUMBER = Number.MAX_SAFE_INTEGER;
|
||||
export const MIN_STORED_NUMBER = Number.MIN_SAFE_INTEGER;
|
||||
|
||||
/**
|
||||
* Marshalling failures a retry can fix by clamping: an out-of-range number, or
|
||||
* a special numeric value (`NaN`, `±Infinity`) that has no representation at
|
||||
* all. Matched on the message because they are plain `Error`s thrown while
|
||||
* encoding the payload — no error name or code separates them from anything
|
||||
* else, and they never reach the wire, so there is no response to inspect.
|
||||
*/
|
||||
const REPAIRABLE_MARSHALL_ERRORS = [
|
||||
/is greater than Number\.MAX_SAFE_INTEGER/,
|
||||
/is lesser than Number\.MIN_SAFE_INTEGER/,
|
||||
/^Special numeric value /,
|
||||
];
|
||||
|
||||
export const isRepairableMarshallError = (error: unknown): boolean => {
|
||||
const message = (error as Error | undefined)?.message;
|
||||
if (typeof message !== 'string') return false;
|
||||
return REPAIRABLE_MARSHALL_ERRORS.some((pattern) => pattern.test(message));
|
||||
};
|
||||
|
||||
/**
|
||||
* The same bound, applied to what comes back. A stored number past the safe
|
||||
* range — left by an older write, or reached by a counter incremented up to it
|
||||
* — is otherwise decoded as a `BigInt`, which no JSON response can carry. This
|
||||
* hands back the same kind of value a write accepts, and does less work per
|
||||
* number than the default decoding it replaces.
|
||||
*/
|
||||
export const clampStoredNumber = (stored: string): number => {
|
||||
const num = Number(stored);
|
||||
if (num > MAX_STORED_NUMBER) return MAX_STORED_NUMBER;
|
||||
if (num < MIN_STORED_NUMBER) return MIN_STORED_NUMBER;
|
||||
return num;
|
||||
};
|
||||
|
||||
export interface MarshallRepair {
|
||||
value: unknown;
|
||||
/** False when nothing in the payload was out of range — nothing to retry. */
|
||||
changed: boolean;
|
||||
}
|
||||
|
||||
const unchanged = (value: unknown): MarshallRepair => ({
|
||||
value,
|
||||
changed: false,
|
||||
});
|
||||
|
||||
// Matches the shapes the marshaller walks into as a map: object literals and
|
||||
// null-prototype objects, but not class instances (Date, Buffer, Set, …),
|
||||
// which it encodes by their own rules.
|
||||
const isPlainObject = (value: unknown): value is Record<string, unknown> =>
|
||||
typeof value === 'object' &&
|
||||
value !== null &&
|
||||
(value.constructor === Object || value.constructor === undefined);
|
||||
|
||||
const repairNumber = (num: number): MarshallRepair => {
|
||||
// No representation for these; JSON already turns them into null, so a
|
||||
// caller that got one this far did not send it as JSON.
|
||||
if (Number.isNaN(num)) return { value: null, changed: true };
|
||||
if (num > MAX_STORED_NUMBER)
|
||||
return { value: MAX_STORED_NUMBER, changed: true };
|
||||
if (num < MIN_STORED_NUMBER)
|
||||
return { value: MIN_STORED_NUMBER, changed: true };
|
||||
return unchanged(num);
|
||||
};
|
||||
|
||||
/**
|
||||
* Clamp every number a payload holds into the range the store accepts,
|
||||
* returning the original object untouched when nothing needed it — callers hold
|
||||
* on to the values they pass (caches, retries), so a repair copies rather than
|
||||
* mutating in place.
|
||||
*
|
||||
* Only run after a marshalling failure. Walking a payload this way costs as
|
||||
* much as marshalling it, which is exactly why writes are not checked up
|
||||
* front.
|
||||
*/
|
||||
export const repairForMarshall = (value: unknown): MarshallRepair => {
|
||||
if (typeof value === 'number') return repairNumber(value);
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
let changed = false;
|
||||
const repairedEntries = value.map((entry) => {
|
||||
const repaired = repairForMarshall(entry);
|
||||
changed = changed || repaired.changed;
|
||||
return repaired.value;
|
||||
});
|
||||
return changed ? { value: repairedEntries, changed } : unchanged(value);
|
||||
}
|
||||
|
||||
if (isPlainObject(value)) {
|
||||
let changed = false;
|
||||
const repairedEntries = Object.entries(value).map(([key, entry]) => {
|
||||
const repaired = repairForMarshall(entry);
|
||||
changed = changed || repaired.changed;
|
||||
return [key, repaired.value] as const;
|
||||
});
|
||||
// `fromEntries` defines each property rather than assigning it, so a
|
||||
// `__proto__` key in the payload stays data here.
|
||||
return changed
|
||||
? { value: Object.fromEntries(repairedEntries), changed }
|
||||
: unchanged(value);
|
||||
}
|
||||
|
||||
return unchanged(value);
|
||||
};
|
||||
@@ -54,14 +54,22 @@ const sentMessage = (info: unknown) =>
|
||||
};
|
||||
|
||||
describe('EmailClient — transport lifecycle', () => {
|
||||
it('reports as configured once a transport is wired up', () => {
|
||||
it('stops sending once the transport is shut down', async () => {
|
||||
const client = startClient();
|
||||
expect(client.isConfigured).toBe(true);
|
||||
expect(
|
||||
await client.sendRaw({ to: '[email protected]', subject: 'up', text: 'x' }),
|
||||
).not.toBeNull();
|
||||
|
||||
client.onServerShutdown();
|
||||
expect(client.isConfigured).toBe(false);
|
||||
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
expect(
|
||||
await client.sendRaw({ to: '[email protected]', subject: 'down', text: 'x' }),
|
||||
).toBeNull();
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it('warns and stays unconfigured when no transport is set', () => {
|
||||
it('warns at boot when no transport is set', () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const client = new EmailClient({
|
||||
port: 0,
|
||||
@@ -69,7 +77,6 @@ describe('EmailClient — transport lifecycle', () => {
|
||||
} as unknown as IConfig);
|
||||
client.onServerStart();
|
||||
|
||||
expect(client.isConfigured).toBe(false);
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining('no email transport configured'),
|
||||
);
|
||||
@@ -238,6 +245,139 @@ describe('EmailClient — template rendering', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('EmailClient — share notification templates', () => {
|
||||
/** Captures both parts of a send without touching the transport. */
|
||||
const renderShare = async (
|
||||
template: 'file_shared_with_you' | 'file_shared_invite',
|
||||
values: Record<string, unknown>,
|
||||
) => {
|
||||
const client = startClient();
|
||||
const captured: { html: string; text: string } = { html: '', text: '' };
|
||||
vi.spyOn(client, 'sendRaw').mockImplementation(async (options) => {
|
||||
captured.html = options.html ?? '';
|
||||
captured.text = options.text ?? '';
|
||||
return null;
|
||||
});
|
||||
await client.send('[email protected]', template, values);
|
||||
return captured;
|
||||
};
|
||||
|
||||
const HOLDER = {
|
||||
recipient: 'alice',
|
||||
subject_line: 'bob shared notes.md with you',
|
||||
shares: [
|
||||
{ sender: 'bob', what: 'notes.md' },
|
||||
{ sender: 'carol', what: '3 items — a.txt, b.txt, +1 more' },
|
||||
],
|
||||
link: 'https://puter.test',
|
||||
unsubscribe_uuid: null,
|
||||
};
|
||||
|
||||
it('sends a plain-text alternative beside the html', async () => {
|
||||
const { html, text } = await renderShare(
|
||||
'file_shared_with_you',
|
||||
HOLDER,
|
||||
);
|
||||
|
||||
expect(html).toContain('<!DOCTYPE html>');
|
||||
// Every sender reaches both parts, so a text-only client loses nothing.
|
||||
for (const part of [html, text]) {
|
||||
expect(part).toContain('bob');
|
||||
expect(part).toContain('notes.md');
|
||||
expect(part).toContain('carol');
|
||||
expect(part).toContain('+1 more');
|
||||
}
|
||||
expect(text).not.toContain('<');
|
||||
});
|
||||
|
||||
it('escapes item names in the html and leaves them raw in the text', async () => {
|
||||
const { html, text } = await renderShare('file_shared_with_you', {
|
||||
...HOLDER,
|
||||
shares: [{ sender: 'bob', what: 'r&d "notes".md' }],
|
||||
});
|
||||
|
||||
expect(html).toContain('r&d "notes".md');
|
||||
expect(text).toContain('r&d "notes".md');
|
||||
});
|
||||
|
||||
it('renders as a responsive single column with no remote assets', async () => {
|
||||
const { html } = await renderShare('file_shared_with_you', HOLDER);
|
||||
|
||||
expect(html).toContain(
|
||||
'<meta name="viewport" content="width=device-width, initial-scale=1" />',
|
||||
);
|
||||
expect(html).toContain('@media only screen and (max-width: 600px)');
|
||||
expect(html).toContain('@media (prefers-color-scheme: dark)');
|
||||
expect(html).toContain('max-width: 600px');
|
||||
// Images are the one thing a client can refuse to load, so the design
|
||||
// does without them — the layout can't depend on a blocked asset.
|
||||
expect(html).not.toContain('<img');
|
||||
// The colors have to survive a client that drops the stylesheet.
|
||||
expect(html).toContain('style="background-color: #ffffff;');
|
||||
});
|
||||
|
||||
it('opens with a preview line that is hidden in the body', async () => {
|
||||
const { html } = await renderShare('file_shared_with_you', HOLDER);
|
||||
|
||||
const preheader = html.indexOf('Waiting for you under Shared with me.');
|
||||
expect(preheader).toBeGreaterThan(-1);
|
||||
expect(html.slice(0, preheader)).toContain('mso-hide: all');
|
||||
expect(preheader).toBeLessThan(html.indexOf('Hi alice,'));
|
||||
});
|
||||
|
||||
it('points the call to action at the app and nowhere else', async () => {
|
||||
const { html, text } = await renderShare(
|
||||
'file_shared_with_you',
|
||||
HOLDER,
|
||||
);
|
||||
|
||||
expect(html).toContain('href="https://puter.test"');
|
||||
expect(html).toContain('>Open Puter</a>');
|
||||
expect(text).toContain('Open Puter: https://puter.test');
|
||||
});
|
||||
|
||||
it('separates senders without a rule above the first', async () => {
|
||||
const { html } = await renderShare('file_shared_with_you', HOLDER);
|
||||
|
||||
// One rule between two senders, none leading the list.
|
||||
expect(html.split('border-top: 1px solid').length - 1).toBe(1);
|
||||
});
|
||||
|
||||
it('offers the unsubscribe link only to a recipient who has an account', async () => {
|
||||
const withAccount = await renderShare('file_shared_with_you', {
|
||||
...HOLDER,
|
||||
unsubscribe_uuid: 'a-uuid',
|
||||
});
|
||||
expect(withAccount.html).toContain(
|
||||
'href="https://puter.test/unsubscribe?user_uuid=a-uuid"',
|
||||
);
|
||||
expect(withAccount.text).toContain(
|
||||
'https://puter.test/unsubscribe?user_uuid=a-uuid',
|
||||
);
|
||||
|
||||
const anonymous = await renderShare('file_shared_with_you', HOLDER);
|
||||
expect(anonymous.html).not.toContain('/unsubscribe');
|
||||
expect(anonymous.text).not.toContain('/unsubscribe');
|
||||
});
|
||||
|
||||
it('tells an invited address what to do with it', async () => {
|
||||
const { html, text } = await renderShare('file_shared_invite', {
|
||||
email: '[email protected]',
|
||||
subject_line: 'bob shared notes.md with you on Puter',
|
||||
shares: [{ sender: 'bob', what: 'notes.md' }],
|
||||
link: 'https://puter.test',
|
||||
});
|
||||
|
||||
for (const part of [html, text]) {
|
||||
expect(part).toContain('[email protected]');
|
||||
expect(part).toContain('Create your free account');
|
||||
// An invite has no account to unsubscribe from.
|
||||
expect(part).not.toContain('/unsubscribe');
|
||||
}
|
||||
expect(html).toContain('href="https://puter.test"');
|
||||
});
|
||||
});
|
||||
|
||||
describe('EmailClient.clean', () => {
|
||||
const clean = (email: string) => startClient().clean(email);
|
||||
|
||||
|
||||
@@ -22,7 +22,11 @@ import handlebars, { template } from 'handlebars';
|
||||
import nodemailer from 'nodemailer';
|
||||
import type { IConfig } from '../../types';
|
||||
import { PuterClient } from '../types';
|
||||
import { EMAIL_TEMPLATES, type EmailTemplateName } from './templates';
|
||||
import {
|
||||
EMAIL_TEMPLATES,
|
||||
type EmailTemplate,
|
||||
type EmailTemplateName,
|
||||
} from './templates';
|
||||
|
||||
/** Attachment shape passed through to the underlying transport. */
|
||||
export interface EmailAttachment {
|
||||
@@ -64,6 +68,7 @@ export type EmailValidator = (email: string) => Promise<boolean> | boolean;
|
||||
interface CompiledTemplate {
|
||||
subject: ReturnType<typeof template>;
|
||||
html: ReturnType<typeof template>;
|
||||
text?: ReturnType<typeof template>;
|
||||
}
|
||||
|
||||
// -- Clean-email rules ------------------------------------------------
|
||||
@@ -173,6 +178,7 @@ export class EmailClient extends PuterClient {
|
||||
to,
|
||||
subject: compiled.subject(values),
|
||||
html: compiled.html(values),
|
||||
...(compiled.text ? { text: compiled.text(values) } : {}),
|
||||
...(options.replyTo ? { replyTo: options.replyTo } : {}),
|
||||
});
|
||||
}
|
||||
@@ -183,7 +189,8 @@ export class EmailClient extends PuterClient {
|
||||
*
|
||||
* Returns the transport's send result, or `null` when no transport is
|
||||
* configured (the send is a no-op in that case — callers that must not
|
||||
* silently drop mail should check `isConfigured` first).
|
||||
* silently drop mail should check `config.email` before composing, or the
|
||||
* `null` return after).
|
||||
*/
|
||||
async sendRaw(options: SendMailOptions) {
|
||||
if (!this.transport) {
|
||||
@@ -199,11 +206,6 @@ export class EmailClient extends PuterClient {
|
||||
});
|
||||
}
|
||||
|
||||
/** Whether an SMTP transport is configured (sends are no-ops otherwise). */
|
||||
get isConfigured(): boolean {
|
||||
return this.transport !== null;
|
||||
}
|
||||
|
||||
// -- Public API: clean / validate ---------------------------------
|
||||
|
||||
/**
|
||||
@@ -289,7 +291,11 @@ export class EmailClient extends PuterClient {
|
||||
}
|
||||
|
||||
private compileTemplates(): void {
|
||||
for (const [name, template] of Object.entries(EMAIL_TEMPLATES)) {
|
||||
// Widened to the interface: the literal map keeps a distinct type per
|
||||
// template, and only some of them carry a `text` part.
|
||||
const templates: Record<EmailTemplateName, EmailTemplate> =
|
||||
EMAIL_TEMPLATES;
|
||||
for (const [name, template] of Object.entries(templates)) {
|
||||
this.compiledTemplates[name as EmailTemplateName] = {
|
||||
// Subjects are plain-text headers: HTML-escaping would put
|
||||
// literal entities in front of the recipient (& etc.).
|
||||
@@ -300,6 +306,15 @@ export class EmailClient extends PuterClient {
|
||||
noEscape: true,
|
||||
}),
|
||||
html: handlebars.compile(dedent(template.html)),
|
||||
// Same reasoning as the subject: a text part is not HTML, so
|
||||
// escaping would show entities to the reader.
|
||||
...(template.text
|
||||
? {
|
||||
text: handlebars.compile(dedent(template.text), {
|
||||
noEscape: true,
|
||||
}),
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -29,8 +29,184 @@
|
||||
export interface EmailTemplate {
|
||||
subject: string;
|
||||
html: string;
|
||||
/**
|
||||
* Optional plain-text alternative. Where a template has one it goes out
|
||||
* beside the HTML as multipart/alternative, which is what plain-text
|
||||
* clients, screen readers and spam scoring all prefer to a machine
|
||||
* down-conversion of the markup.
|
||||
*/
|
||||
text?: string;
|
||||
}
|
||||
|
||||
// -- Shared layout ----------------------------------------------------
|
||||
|
||||
/**
|
||||
* Colors, in one place so the two share templates can't drift apart. Every one
|
||||
* of them is also written inline further down: clients that drop `<style>`
|
||||
* (Gmail reading a non-Gmail account, Outlook.com) would otherwise render an
|
||||
* unstyled page, so the stylesheet only carries what inline CSS can't express —
|
||||
* the small-screen and dark-mode overrides.
|
||||
*
|
||||
* `ACCENT` is the GUI's primary blue darkened until white text on it clears
|
||||
* 4.5:1, since a button label is small text however bold it is.
|
||||
*/
|
||||
const INK = '#101828';
|
||||
const INK_SOFT = '#5b6472';
|
||||
const INK_FAINT = '#667085';
|
||||
const PAGE_BG = '#f2f4f7';
|
||||
const CARD_BG = '#ffffff';
|
||||
const CARD_BORDER = '#e4e7ec';
|
||||
const RULE = '#e6e9ee';
|
||||
const PANEL_BG = '#f8f9fb';
|
||||
const PANEL_BORDER = '#eceff3';
|
||||
const ACCENT = '#0a76cc';
|
||||
|
||||
const FONT =
|
||||
"-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,'Helvetica Neue',Arial,sans-serif";
|
||||
|
||||
/** Pads the preview line so body text can't leak into it. */
|
||||
const PREHEADER_FILL = '͏‌ '.repeat(60);
|
||||
|
||||
/**
|
||||
* The document both share notifications render into: one 600px column that goes
|
||||
* full width on a phone, laid out with tables and `bgcolor` because Outlook
|
||||
* renders through Word and ignores anything more modern.
|
||||
*
|
||||
* `preheader` is the line the inbox shows beside the subject — it should add to
|
||||
* the subject, not repeat it. `content` is the card's rows; `footer` sits below
|
||||
* the card, where the reason-for-receiving and unsubscribe belong.
|
||||
*/
|
||||
const shareEmailLayout = (parts: {
|
||||
preheader: string;
|
||||
content: string;
|
||||
footer: string;
|
||||
}): string => `
|
||||
<!DOCTYPE html>
|
||||
<html lang="en" dir="ltr" xmlns:o="urn:schemas-microsoft-com:office:office">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="x-apple-disable-message-reformatting" />
|
||||
<meta name="color-scheme" content="light dark" />
|
||||
<meta name="supported-color-schemes" content="light dark" />
|
||||
<title>{{subject_line}}</title>
|
||||
<!--[if mso]>
|
||||
<noscript><xml><o:OfficeDocumentSettings><o:PixelsPerInch>96</o:PixelsPerInch></o:OfficeDocumentSettings></xml></noscript>
|
||||
<![endif]-->
|
||||
<style>
|
||||
body { margin: 0; padding: 0; width: 100% !important; -webkit-text-size-adjust: 100%; -ms-text-size-adjust: 100%; }
|
||||
table { border-spacing: 0; mso-table-lspace: 0pt; mso-table-rspace: 0pt; }
|
||||
td { mso-line-height-rule: exactly; }
|
||||
a[x-apple-data-detectors] { color: inherit !important; text-decoration: none !important; font-size: inherit !important; font-family: inherit !important; font-weight: inherit !important; line-height: inherit !important; }
|
||||
@media only screen and (max-width: 600px) {
|
||||
.col { width: 100% !important; }
|
||||
.pad { padding-left: 24px !important; padding-right: 24px !important; }
|
||||
.pad-y { padding-top: 28px !important; padding-bottom: 28px !important; }
|
||||
.h1 { font-size: 22px !important; line-height: 30px !important; }
|
||||
.btn { width: 100% !important; }
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
.page { background-color: #0e1116 !important; }
|
||||
.card { background-color: #171b22 !important; border-color: #272d38 !important; }
|
||||
.ink { color: #e9ebef !important; }
|
||||
.ink-soft { color: #9aa3b2 !important; }
|
||||
.ink-soft a { color: #9aa3b2 !important; }
|
||||
.rule { border-color: #272d38 !important; }
|
||||
.panel { background-color: #1d222a !important; border-color: #2b323d !important; }
|
||||
.mark { color: #4da3f2 !important; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body class="page" style="margin: 0; padding: 0; width: 100%; background-color: ${PAGE_BG};">
|
||||
<div style="display: none; font-size: 1px; line-height: 1px; max-height: 0; max-width: 0; opacity: 0; overflow: hidden; mso-hide: all;">${parts.preheader}${PREHEADER_FILL}</div>
|
||||
<div role="article" aria-roledescription="email" lang="en">
|
||||
<table role="presentation" class="page" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%; background-color: ${PAGE_BG};">
|
||||
<tr>
|
||||
<td align="center" style="padding: 32px 12px;">
|
||||
<!--[if mso]><table role="presentation" width="600" cellpadding="0" cellspacing="0" border="0"><tr><td><![endif]-->
|
||||
<table role="presentation" class="col" width="600" cellpadding="0" cellspacing="0" border="0" style="width: 100%; max-width: 600px; text-align: left;">
|
||||
<tr>
|
||||
<td class="pad mark" style="padding: 0 4px 14px; font-family: ${FONT}; font-size: 15px; line-height: 20px; font-weight: 700; letter-spacing: 0.01em; color: ${ACCENT};">Puter</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="card" style="background-color: ${CARD_BG}; border: 1px solid ${CARD_BORDER}; border-radius: 14px;">
|
||||
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">
|
||||
<tr>
|
||||
<td class="pad pad-y" style="padding: 36px 36px 34px;">
|
||||
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">${parts.content}
|
||||
</table>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="pad ink-soft" style="padding: 20px 4px 0; font-family: ${FONT}; font-size: 13px; line-height: 20px; color: ${INK_FAINT};">${parts.footer}
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
<!--[if mso]></td></tr></table><![endif]-->
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
`;
|
||||
|
||||
/** The greeting line above the heading. */
|
||||
const greetingRow = (text: string): string => `
|
||||
<tr>
|
||||
<td class="ink-soft" style="padding: 0 0 6px; font-family: ${FONT}; font-size: 15px; line-height: 22px; color: ${INK_SOFT};">${text}</td>
|
||||
</tr>`;
|
||||
|
||||
const headingRow = (text: string): string => `
|
||||
<tr>
|
||||
<td class="ink h1" style="padding: 0 0 18px; font-family: ${FONT}; font-size: 24px; line-height: 32px; font-weight: 700; letter-spacing: -0.01em; color: ${INK};">${text}</td>
|
||||
</tr>`;
|
||||
|
||||
/**
|
||||
* One row per sender, hairline-separated. The wording comes pre-composed from
|
||||
* the digest (`digestLines`), so the list stays a list however many senders and
|
||||
* items fold into it.
|
||||
*/
|
||||
const SHARE_LIST_ROW = `
|
||||
<tr>
|
||||
<td class="panel" style="padding: 6px 18px; background-color: ${PANEL_BG}; border: 1px solid ${PANEL_BORDER}; border-radius: 10px;">
|
||||
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">
|
||||
{{#each shares}}
|
||||
<tr>
|
||||
<td class="ink rule" style="padding: 12px 0;{{#unless @first}} border-top: 1px solid ${RULE};{{/unless}} font-family: ${FONT}; font-size: 16px; line-height: 24px; color: ${INK};"><strong style="font-weight: 600;">{{this.sender}}</strong> shared {{this.what}}</td>
|
||||
</tr>
|
||||
{{/each}}
|
||||
</table>
|
||||
</td>
|
||||
</tr>`;
|
||||
|
||||
/** A body paragraph. `padding` lets a caller tune the rhythm around it. */
|
||||
const textRow = (text: string, padding = '18px 0 0'): string => `
|
||||
<tr>
|
||||
<td class="ink-soft" style="padding: ${padding}; font-family: ${FONT}; font-size: 15px; line-height: 23px; color: ${INK_SOFT};">${text}</td>
|
||||
</tr>`;
|
||||
|
||||
/**
|
||||
* The call to action. Padding sits on the cell and the color on `bgcolor` so
|
||||
* Outlook still draws a real button (square-cornered, which is fine); the table
|
||||
* goes full width under 600px so the tap target spans the card.
|
||||
*/
|
||||
const buttonRow = (label: string): string => `
|
||||
<tr>
|
||||
<td style="padding: 26px 0 0;">
|
||||
<table role="presentation" class="btn" cellpadding="0" cellspacing="0" border="0" style="border-collapse: separate;">
|
||||
<tr>
|
||||
<td align="center" bgcolor="${ACCENT}" style="background-color: ${ACCENT}; border-radius: 10px; padding: 14px 26px;">
|
||||
<a href="{{link}}" style="display: inline-block; font-family: ${FONT}; font-size: 16px; line-height: 20px; font-weight: 600; color: #ffffff; text-decoration: none;">${label}</a>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
</td>
|
||||
</tr>`;
|
||||
|
||||
export const EMAIL_TEMPLATES = {
|
||||
'approved-for-listing': {
|
||||
subject: '🎉 Your app has been approved for listing!',
|
||||
@@ -183,32 +359,85 @@ immediately</p>
|
||||
<p>Puter</p>
|
||||
`,
|
||||
},
|
||||
share_by_username: {
|
||||
subject: 'Puter share from {{susername}}',
|
||||
html: `
|
||||
<p>Hi there {{rusername}},</p>
|
||||
<p>You've received a share from {{susername}} on Puter.</p>
|
||||
<p>Go to puter.com to check it out.</p>
|
||||
{{#if message}}
|
||||
<p>The following message was included:</p>
|
||||
<blockquote>{{message}}</blockquote>
|
||||
{{/if}}
|
||||
<p>Sincerely,</p>
|
||||
<p>Puter</p>
|
||||
/**
|
||||
* A digest: shares to one recipient are held briefly and merged, so
|
||||
* `shares` may carry several senders. The subject is composed by the
|
||||
* service (see `digestSubject`), which owns the grouped wording.
|
||||
*/
|
||||
file_shared_with_you: {
|
||||
subject: '{{subject_line}}',
|
||||
html: shareEmailLayout({
|
||||
preheader: 'Waiting for you under Shared with me.',
|
||||
content:
|
||||
greetingRow('Hi{{#if recipient}} {{recipient}}{{/if}},') +
|
||||
headingRow('Shared with you') +
|
||||
SHARE_LIST_ROW +
|
||||
buttonRow('Open Puter') +
|
||||
textRow(
|
||||
'Shared items live under <span style="font-weight: 600;">Shared</span> in your files.',
|
||||
'24px 0 0',
|
||||
),
|
||||
footer: `You're receiving this because someone shared with your Puter account.{{#if unsubscribe_uuid}}
|
||||
<br /><a href="{{link}}/unsubscribe?user_uuid={{unsubscribe_uuid}}" style="color: inherit; text-decoration: underline;">Unsubscribe from notification emails</a>{{/if}}`,
|
||||
}),
|
||||
text: `
|
||||
Hi{{#if recipient}} {{recipient}}{{/if}},
|
||||
|
||||
Shared with you on Puter:
|
||||
{{#each shares}}
|
||||
- {{this.sender}} shared {{this.what}}
|
||||
{{/each}}
|
||||
|
||||
Open Puter: {{link}}
|
||||
|
||||
Shared items live under "Shared" in your files.
|
||||
|
||||
--
|
||||
You're receiving this because someone shared with your Puter account.
|
||||
{{#if unsubscribe_uuid}}Unsubscribe from notification emails:
|
||||
{{link}}/unsubscribe?user_uuid={{unsubscribe_uuid}}{{/if}}
|
||||
`,
|
||||
},
|
||||
share_by_email: {
|
||||
subject: 'share by email',
|
||||
html: `
|
||||
<p>Hi there,</p>
|
||||
<p>You've received a share from {{sender_name}} on Puter:</p>
|
||||
<p><a href="{{link}}">{{link}}</a></p>
|
||||
{{#if message}}
|
||||
<p>The following message was included:</p>
|
||||
<blockquote>{{message}}</blockquote>
|
||||
{{/if}}
|
||||
<p>Sincerely,</p>
|
||||
<p>Puter</p>
|
||||
// The only way to reach someone with no account. Same digest shape.
|
||||
file_shared_invite: {
|
||||
subject: '{{subject_line}}',
|
||||
html: shareEmailLayout({
|
||||
preheader: 'Claim it with a free Puter account.',
|
||||
content:
|
||||
greetingRow('Hi there,') +
|
||||
headingRow('Shared with you on Puter') +
|
||||
SHARE_LIST_ROW +
|
||||
textRow(
|
||||
'There\'s no Puter account for <span style="font-weight: 600;">{{email}}</span> yet. Create one with this address, confirm it, and everything above will be waiting for you. It\'s free and takes about a minute.',
|
||||
) +
|
||||
buttonRow('Create your free account') +
|
||||
textRow(
|
||||
'Already on Puter? Add {{email}} to your account and confirm it to get the same access.',
|
||||
'24px 0 0',
|
||||
),
|
||||
footer: `You're receiving this because someone shared with {{email}}. Nothing is shared until the address is confirmed, so you can ignore this email and nothing happens.`,
|
||||
}),
|
||||
text: `
|
||||
Hi there,
|
||||
|
||||
Shared with you on Puter:
|
||||
{{#each shares}}
|
||||
- {{this.sender}} shared {{this.what}}
|
||||
{{/each}}
|
||||
|
||||
There's no Puter account for {{email}} yet. Create one with this
|
||||
address, confirm it, and everything above will be waiting for you.
|
||||
It's free and takes about a minute.
|
||||
|
||||
Create your free account: {{link}}
|
||||
|
||||
Already on Puter? Add {{email}} to your account and confirm it to get
|
||||
the same access.
|
||||
|
||||
--
|
||||
You're receiving this because someone shared with {{email}}. Nothing is
|
||||
shared until the address is confirmed, so you can ignore this email
|
||||
and nothing happens.
|
||||
`,
|
||||
},
|
||||
} satisfies Record<string, EmailTemplate>;
|
||||
|
||||
@@ -32,6 +32,24 @@ type GuiEvent<R = Record<string, unknown>> = {
|
||||
response: R;
|
||||
};
|
||||
|
||||
/**
|
||||
* Extension-augmentable half of {@link EventMap}. Extensions that emit their own
|
||||
* events declare the payload here by declaration merging, so both the emitter
|
||||
* and every listener are typed against the same shape:
|
||||
*
|
||||
* declare module '@heyputer/backend/clients/event/types' {
|
||||
* interface IExtensionEventMap {
|
||||
* 'my.thing.happened': { thingId: string };
|
||||
* }
|
||||
* }
|
||||
*
|
||||
* Deliberately member-less and index-signature-free: an index signature here
|
||||
* would widen `keyof EventMap` to `string` and silently disable key checking on
|
||||
* every `emit` in the tree.
|
||||
*/
|
||||
// eslint-disable-next-line @typescript-eslint/no-empty-object-type
|
||||
export interface IExtensionEventMap {}
|
||||
|
||||
export type EventMap = {
|
||||
// ---- Server lifecycle ----
|
||||
serverStart: Record<string, never>;
|
||||
@@ -142,6 +160,19 @@ export type EventMap = {
|
||||
'puter.signup.validate': {
|
||||
allow: boolean;
|
||||
email?: string;
|
||||
/**
|
||||
* The address in the same canonical form the `email.validate` hook was
|
||||
* given (`cleanEmail`), so a handler can correlate the two hooks on one
|
||||
* key. Aliases (`[email protected]`, `[email protected]`) differ from
|
||||
* `email` here.
|
||||
*/
|
||||
clean_email?: string;
|
||||
/**
|
||||
* True for a temp (frictionless, no-password) signup. Those carry a
|
||||
* synthetic `<username>@gmail.com` and never reach email validation, so
|
||||
* a handler must not draw conclusions from `email`.
|
||||
*/
|
||||
is_temp?: boolean;
|
||||
ip?: string | null;
|
||||
source?: 'oidc';
|
||||
req?: unknown;
|
||||
@@ -290,7 +321,17 @@ export type EventMap = {
|
||||
sourceObjectKey: string;
|
||||
copyObjectKey: string;
|
||||
};
|
||||
'fs.move.node': { node: FSEntry; fromPath: string; toPath: string };
|
||||
/**
|
||||
* `fromUserId` is the owner the node had before the move. A move into
|
||||
* someone else's tree re-owns the row, and `node` already carries the new
|
||||
* owner — listeners that care about the hand-over need the old one.
|
||||
*/
|
||||
'fs.move.node': {
|
||||
node: FSEntry;
|
||||
fromPath: string;
|
||||
toPath: string;
|
||||
fromUserId?: number;
|
||||
};
|
||||
'fs.remove.node': { node: FSEntry; entry: FSEntry; target: FSEntry };
|
||||
'fs.write.file': { node: FSEntry; entry: FSEntry; target: FSEntry };
|
||||
'fs.storage.upload-progress': {
|
||||
@@ -338,12 +379,39 @@ export type EventMap = {
|
||||
// is pricing workers that came into existence, not deploys.
|
||||
'worker.create': { actor: Actor; workerName: string };
|
||||
|
||||
// The same announcement for worker code that lives inside a hosted site
|
||||
// and is deployed on demand rather than by an explicit call. Such a worker
|
||||
// has no row of ours to key on, so what stands in for "genuinely new" is
|
||||
// the absence of any prior record that it ran — which is why it is
|
||||
// announced separately, and identified by where its source lives rather
|
||||
// than by a worker name.
|
||||
'worker.dynamic.create': {
|
||||
actor: Actor;
|
||||
subdomain: string;
|
||||
worker: string;
|
||||
};
|
||||
|
||||
// ---- Outer / GUI broadcast ----
|
||||
'outer.cacheUpdate': {
|
||||
cacheKey: string[];
|
||||
data?: unknown;
|
||||
ttlSeconds?: number;
|
||||
};
|
||||
/**
|
||||
* Permission cache generations were bumped, so peer regions must bump their
|
||||
* own — the counter is per-cluster, so a local bump says nothing to them.
|
||||
* Carries the actors, not the values: the numbers only have to change.
|
||||
*/
|
||||
'outer.permission.generationBumped': { actorUids: string[] };
|
||||
/**
|
||||
* Flat permission entries were deleted. Grant-path flat entries carry no
|
||||
* expiry, so without this a revoke never lands in a peer region whose KV
|
||||
* table isn't replicated. Revoke-only: a grant that fails to replicate just
|
||||
* denies there, which is the safe direction.
|
||||
*/
|
||||
'outer.permission.flatInvalidated': {
|
||||
entries: Array<{ holderUserId: number; permission: string }>;
|
||||
};
|
||||
'outer.fs.write-hash': { hash: string; uuid: string };
|
||||
/**
|
||||
* Cache keys the KV read cache must stop serving, because the entries
|
||||
@@ -367,8 +435,10 @@ export type EventMap = {
|
||||
}>;
|
||||
|
||||
// ---- Subdomains ----
|
||||
'subdomain.delete': { subdomain: string };
|
||||
'subdomain.update': { subdomain: string };
|
||||
// `uid` is the row uuid — for `delete` it is the only surviving handle a
|
||||
// listener can use to find state keyed to the row after it is gone.
|
||||
'subdomain.delete': { subdomain: string; uid?: string };
|
||||
'subdomain.update': { subdomain: string; uid?: string };
|
||||
'site.htmlServed': {
|
||||
subdomain: string;
|
||||
entry: unknown;
|
||||
@@ -460,7 +530,7 @@ export type EventMap = {
|
||||
* have all dropped that answer.
|
||||
*/
|
||||
'outer.pubsub.metering.credits-changed': { userUuid: string };
|
||||
};
|
||||
} & IExtensionEventMap;
|
||||
|
||||
/**
|
||||
* Phase of a request/method lifecycle. `reject` is emitted when a `before`
|
||||
@@ -540,10 +610,11 @@ export type EventKey = keyof EventMap & string;
|
||||
// Generates a wildcard for every non-final dot-separated prefix of K.
|
||||
export type WildcardPrefixes<K extends string> =
|
||||
K extends `${infer Head}.${infer Tail}`
|
||||
? | `${Head}.*`
|
||||
| (Tail extends `${string}.${string}`
|
||||
? `${Head}.${WildcardPrefixes<Tail>}`
|
||||
: never)
|
||||
?
|
||||
| `${Head}.*`
|
||||
| (Tail extends `${string}.${string}`
|
||||
? `${Head}.${WildcardPrefixes<Tail>}`
|
||||
: never)
|
||||
: never;
|
||||
|
||||
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
|
||||
|
||||
@@ -30,7 +30,7 @@ describe('PreludeClient', () => {
|
||||
expect(makeClient().isConfigured()).toBe(false);
|
||||
});
|
||||
|
||||
describe('isCountrySupported (€0.07 cap)', () => {
|
||||
describe('isCountrySupported (€0.20 cap)', () => {
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
it('allows revenue markets up to the cap (incl. the priciest)', () => {
|
||||
@@ -40,6 +40,17 @@ describe('PreludeClient', () => {
|
||||
expect(client.isCountrySupported('us')).toBe(true); // case-insensitive
|
||||
});
|
||||
|
||||
it('allows the markets the old €0.07 cap excluded', () => {
|
||||
// Raising the cap to €0.20 brought ~89 countries into the phone
|
||||
// gate. These pin the new range so a future change to the default
|
||||
// has to say so out loud.
|
||||
expect(client.isCountrySupported('UA')).toBe(true); // €0.0940
|
||||
expect(client.isCountrySupported('PH')).toBe(true); // €0.1237
|
||||
expect(client.isCountrySupported('EG')).toBe(true); // €0.1561
|
||||
expect(client.isCountrySupported('NG')).toBe(true); // €0.1980
|
||||
expect(client.isCountrySupported('WS')).toBe(true); // €0.2000, at the cap
|
||||
});
|
||||
|
||||
it('rejects countries above the cap, with no SMS, or unknown', () => {
|
||||
expect(client.isCountrySupported('PK')).toBe(false); // €0.3548
|
||||
expect(client.isCountrySupported('ID')).toBe(false); // €0.2430
|
||||
@@ -63,9 +74,7 @@ describe('PreludeClient', () => {
|
||||
});
|
||||
|
||||
it('createVerification POSTs the phone target + ip signal with bearer auth', async () => {
|
||||
fetchMock.mockResolvedValue(
|
||||
okJson({ id: 'vrf_1', status: 'success' }),
|
||||
);
|
||||
fetchMock.mockResolvedValue(okJson({ id: 'vrf_1', status: 'success' }));
|
||||
const client = makeClient('sk_test');
|
||||
|
||||
const res = await client.createVerification('+14155550123', {
|
||||
@@ -81,7 +90,11 @@ describe('PreludeClient', () => {
|
||||
target: { type: 'phone_number', value: '+14155550123' },
|
||||
// Defaults to RCS (cheaper); Prelude falls back to SMS. locale is
|
||||
// hardcoded to en-US so the message text is always English.
|
||||
options: { code_size: 6, preferred_channel: 'rcs', locale: 'en-US' },
|
||||
options: {
|
||||
code_size: 6,
|
||||
preferred_channel: 'rcs',
|
||||
locale: 'en-US',
|
||||
},
|
||||
signals: { ip: '203.0.113.7' },
|
||||
});
|
||||
});
|
||||
@@ -196,9 +209,9 @@ describe('PreludeClient', () => {
|
||||
|
||||
it('throws (does not call fetch) when not configured', async () => {
|
||||
const client = makeClient();
|
||||
await expect(
|
||||
client.createVerification('+14155550123'),
|
||||
).rejects.toThrow(/not configured/i);
|
||||
await expect(client.createVerification('+14155550123')).rejects.toThrow(
|
||||
/not configured/i,
|
||||
);
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
|
||||
@@ -50,12 +50,15 @@ export type PreludeChannel =
|
||||
export type PreludeDeliveryChannel = PreludeChannel | 'silent' | 'voice';
|
||||
/**
|
||||
* Default per-SMS cost ceiling (EUR). Countries whose Prelude SMS rate exceeds
|
||||
* this — or that have no SMS channel — are not offered phone verification. The
|
||||
* cap covers every realistic revenue market (priciest are Germany €0.0598 and
|
||||
* Saudi Arabia €0.0638) while excluding the expensive, high-fraud long tail.
|
||||
* Override per-deployment with `config.prelude.maxSmsCostEur`.
|
||||
* this — or that have no SMS channel — are not offered phone verification.
|
||||
* Raised from €0.07 to €0.20: the old cap sat only marginally above the
|
||||
* priciest revenue markets (Germany €0.0598, Saudi Arabia €0.0638), so any rate
|
||||
* drift silently withdrew the phone gate from a real market — and the abuse
|
||||
* harness now routes far more legitimate signups into the SMS band, where an
|
||||
* unavailable gate is a dead end rather than a mild inconvenience. Override
|
||||
* per-deployment with `config.prelude.maxSmsCostEur`.
|
||||
*/
|
||||
const DEFAULT_MAX_SMS_COST_EUR = 0.07;
|
||||
const DEFAULT_MAX_SMS_COST_EUR = 0.2;
|
||||
|
||||
/** Status returned by Prelude when creating/retrying a verification. */
|
||||
export type PreludeCreateStatus =
|
||||
|
||||
@@ -38,7 +38,11 @@ import type { Actor } from '../../core/actor.js';
|
||||
import { Context, runWithContext } from '../../core/context.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { requireUserActorGate } from '../../core/http/middleware/gates.js';
|
||||
import type { TokenSource } from '../../core/http/types.js';
|
||||
import {
|
||||
ROUTES_METADATA_KEY,
|
||||
type CollectedRoute,
|
||||
type TokenSource,
|
||||
} from '../../core/http/types.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
@@ -346,6 +350,188 @@ describe('puter.signup.validate event', () => {
|
||||
|
||||
// ── Signup flow ─────────────────────────────────────────────────────
|
||||
|
||||
// ── Concurrent claims on one address ────────────────────────────────
|
||||
//
|
||||
// The duplicate checks these flows run are not atomic with the writes that
|
||||
// follow them — the validate hook and bcrypt sit in between, and both are slow
|
||||
// enough for a second request to pass the same check. These tests fire the
|
||||
// requests together and assert the address still ends up on exactly one row.
|
||||
|
||||
describe('concurrent claims on one email address', () => {
|
||||
const uniq = () => Math.random().toString(36).slice(2, 10);
|
||||
|
||||
const countOwners = async (email: string): Promise<number> => {
|
||||
// `email_confirmed` is TINYINT on MySQL and BOOLEAN on Postgres, so the
|
||||
// literal has to come from the client rather than be hardcoded.
|
||||
const isTrue = server.clients.db.booleanLiteral(true);
|
||||
const rows = (await server.clients.db.read(
|
||||
`SELECT COUNT(*) AS n FROM \`user\` WHERE \`email\` = ? AND (\`email_confirmed\` = ${isTrue} OR \`password\` IS NOT NULL)`,
|
||||
[email],
|
||||
)) as Array<{ n: number }>;
|
||||
return Number(rows[0]?.n ?? 0);
|
||||
};
|
||||
|
||||
it('lets only one of two simultaneous signups take the address', async () => {
|
||||
const email = `race-${uniq()}@test.local`;
|
||||
const signup = (username: string) =>
|
||||
controller.handleSignup(
|
||||
makeReq({ username, email, password: 'correct-horse-battery' }),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
// Distinct usernames on purpose: the UNIQUE on `username` would
|
||||
// otherwise be what rejects the second request, and the email race
|
||||
// would go untested.
|
||||
const results = await Promise.allSettled([
|
||||
signup(`r_a_${uniq()}`),
|
||||
signup(`r_b_${uniq()}`),
|
||||
]);
|
||||
|
||||
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
|
||||
expect(await countOwners(email)).toBe(1);
|
||||
|
||||
const rejected = results.find((r) => r.status === 'rejected') as
|
||||
| PromiseRejectedResult
|
||||
| undefined;
|
||||
expect(rejected?.reason).toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('lets only one of many simultaneous signups take the address', async () => {
|
||||
const email = `race-many-${uniq()}@test.local`;
|
||||
const results = await Promise.allSettled(
|
||||
Array.from({ length: 5 }, () =>
|
||||
controller.handleSignup(
|
||||
makeReq({
|
||||
username: `r_m_${uniq()}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
);
|
||||
|
||||
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
|
||||
expect(await countOwners(email)).toBe(1);
|
||||
});
|
||||
|
||||
it('rejects a signup racing an admin-provisioned placeholder claim', async () => {
|
||||
const email = `race-pseudo-${uniq()}@test.local`;
|
||||
// The placeholder shape signup is allowed to convert: unconfirmed,
|
||||
// no password. Two signups both see it as claimable.
|
||||
await server.stores.user.create({
|
||||
username: `r_p_${uniq()}`,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email,
|
||||
clean_email: email,
|
||||
});
|
||||
|
||||
const results = await Promise.allSettled([
|
||||
controller.handleSignup(
|
||||
makeReq({
|
||||
username: `r_p_a_${uniq()}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
),
|
||||
controller.handleSignup(
|
||||
makeReq({
|
||||
username: `r_p_b_${uniq()}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
),
|
||||
]);
|
||||
|
||||
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
|
||||
expect(await countOwners(email)).toBe(1);
|
||||
});
|
||||
|
||||
it('reports a duplicate address as a 400, not a constraint error', async () => {
|
||||
const email = `dupe-${uniq()}@test.local`;
|
||||
await controller.handleSignup(
|
||||
makeReq({
|
||||
username: `d_a_${uniq()}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
// The message has to be the one the pre-check produces — a user who
|
||||
// loses the race should not be able to tell.
|
||||
await expect(
|
||||
controller.handleSignup(
|
||||
makeReq({
|
||||
username: `d_b_${uniq()}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
message:
|
||||
'This email already exists in our database. Please use another one.',
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses to give a placeholder row a password for a taken address', async () => {
|
||||
const email = `recover-${uniq()}@test.local`;
|
||||
await controller.handleSignup(
|
||||
makeReq({
|
||||
username: `rec_own_${uniq()}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
// An unconfirmed, password-less row is allowed to sit on the same
|
||||
// address — but password recovery accepts a username, so it can be
|
||||
// driven for this row rather than for the account that owns the
|
||||
// address. Setting a password here would make it a second account able
|
||||
// to recover that inbox.
|
||||
const placeholderName = `rec_ph_${uniq()}`;
|
||||
const placeholder = await server.stores.user.create({
|
||||
username: placeholderName,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email,
|
||||
clean_email: email,
|
||||
});
|
||||
const token = uuidv4();
|
||||
await server.stores.user.update(placeholder.id, {
|
||||
pass_recovery_token: token,
|
||||
});
|
||||
|
||||
const jwt = server.services.token.sign(
|
||||
'otp',
|
||||
{
|
||||
token,
|
||||
user_uid: placeholder.uuid,
|
||||
email,
|
||||
purpose: 'pass-recovery',
|
||||
},
|
||||
{ expiresIn: '1h' },
|
||||
);
|
||||
|
||||
await expect(
|
||||
controller.handleSetPassUsingToken(
|
||||
makeReq({ token: jwt, password: 'another-strong-password' }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
message:
|
||||
'This email is already in use. Recover the account that uses it instead.',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('AuthController.handleSignup', () => {
|
||||
const uniq = () => Math.random().toString(36).slice(2, 10);
|
||||
|
||||
@@ -1499,59 +1685,6 @@ describe('AuthController grant flows', () => {
|
||||
} as Actor;
|
||||
});
|
||||
|
||||
it('grant-user-user: rejects missing target_username/permission with 400', async () => {
|
||||
await expect(
|
||||
controller.handleGrantUserUser(
|
||||
makeReq({ permission: 'fs:read' }, { actor: issuerActor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('grant-user-user: persists the permission and PermissionService.check sees it', async () => {
|
||||
const permission = `service:test-grant-${uuidv4()}:ii:read`;
|
||||
// The controller calls PermissionService.grantUserUserPermission,
|
||||
// which gates on `manage:<permission>` for non-system actors. Pre-
|
||||
// bootstrap the manage flag directly through the permission store
|
||||
// (the system actor would skip this gate, but its in-memory shape
|
||||
// has no user.id, so it can't issue grants). Then the controller
|
||||
// call exercises persist + check end-to-end.
|
||||
await server.stores.permission.setFlatUserPerm(
|
||||
issuer.id,
|
||||
`manage:${permission}`,
|
||||
{
|
||||
permission: `manage:${permission}`,
|
||||
deleted: false,
|
||||
issuer_user_id: issuer.id,
|
||||
} as never,
|
||||
);
|
||||
|
||||
const res = makeRes();
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserUser(
|
||||
makeReq(
|
||||
{
|
||||
target_username: target.username,
|
||||
permission,
|
||||
extra: { reason: 'unit-test' },
|
||||
},
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
res,
|
||||
),
|
||||
);
|
||||
expect(res.body).toEqual({});
|
||||
|
||||
// The target now sees the permission via the user-to-user grant.
|
||||
const targetActor = {
|
||||
user: { ...target, email_confirmed: true },
|
||||
} as Actor;
|
||||
const granted = await server.services.permission
|
||||
.check(targetActor, permission)
|
||||
.catch(() => false);
|
||||
expect(granted).toBeTruthy();
|
||||
});
|
||||
|
||||
it('grant-user-app: persists a user→app permission grant', async () => {
|
||||
// Create an app row owned by the issuer so the grant has somewhere
|
||||
// to land.
|
||||
@@ -2039,20 +2172,6 @@ describe('AuthController grant flows', () => {
|
||||
expect(await storedPermissions()).not.toContain(NOTHING);
|
||||
});
|
||||
|
||||
it('grant-user-group: 404 when the group does not exist', async () => {
|
||||
await expect(
|
||||
controller.handleGrantUserGroup(
|
||||
makeReq(
|
||||
{
|
||||
group_uid: `does-not-exist-${uuidv4()}`,
|
||||
permission: 'service:foo:ii:read',
|
||||
},
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
});
|
||||
});
|
||||
|
||||
// ── Token grants: get-user-app-token / check-app ───────────────────
|
||||
@@ -3235,8 +3354,10 @@ describe('AuthController.handleConfirmPhone', () => {
|
||||
});
|
||||
|
||||
describe('AuthController.handleCardVerificationSetup', () => {
|
||||
it('short-circuits to verified when the gate is not set', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
it('short-circuits to verified for an account with a card on file', async () => {
|
||||
const { actor } = await makeUserAndActor({
|
||||
card_fingerprint: 'fp_existing',
|
||||
});
|
||||
const res = makeRes();
|
||||
await controller.handleCardVerificationSetup(
|
||||
makeReq({}, { actor }),
|
||||
@@ -3245,6 +3366,21 @@ describe('AuthController.handleCardVerificationSetup', () => {
|
||||
expect(res.body).toMatchObject({ card_verified: true });
|
||||
});
|
||||
|
||||
it('starts the flow for an account that was never asked and never verified', async () => {
|
||||
// A route requiring a verified card (`requireCardVerified`) sends a
|
||||
// user here with the gate clear and no card on file. Answering
|
||||
// "already verified" on the flag alone would bounce them between a
|
||||
// dialog reporting success and a route that keeps refusing; reaching
|
||||
// the extension (absent here, hence 503) is the flow actually running.
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleCardVerificationSetup(
|
||||
makeReq({}, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 503 });
|
||||
});
|
||||
|
||||
it('throws 403 when the account is suspended', async () => {
|
||||
const { actor } = await makeUserAndActor({
|
||||
requires_card_verification: 1,
|
||||
@@ -3345,8 +3481,10 @@ describe('AuthController.handleCardVerificationConfirm', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('short-circuits to verified when the gate is not set', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
it('short-circuits to verified for an account with a card on file', async () => {
|
||||
const { actor } = await makeUserAndActor({
|
||||
card_fingerprint: 'fp_existing',
|
||||
});
|
||||
const res = makeRes();
|
||||
await controller.handleCardVerificationConfirm(
|
||||
makeReq({ setup_intent_id: 'seti_1' }, { actor }),
|
||||
@@ -3355,6 +3493,18 @@ describe('AuthController.handleCardVerificationConfirm', () => {
|
||||
expect(res.body).toMatchObject({ card_verified: true });
|
||||
});
|
||||
|
||||
it('runs the confirm for an account that was never asked and never verified', async () => {
|
||||
// Same reason as setup: the pending flag says nobody asked, not that
|
||||
// a card was checked, so a voluntary verification has to go through.
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleCardVerificationConfirm(
|
||||
makeReq({ setup_intent_id: 'seti_1' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 503 });
|
||||
});
|
||||
|
||||
it('throws 409 when phone verification must be completed first', async () => {
|
||||
const { actor } = await makeUserAndActor({
|
||||
requires_card_verification: 1,
|
||||
@@ -3847,6 +3997,151 @@ describe('AuthController.handleConfirmEmail', () => {
|
||||
});
|
||||
expect(after!.email_confirmed).toBeFalsy();
|
||||
});
|
||||
|
||||
it('strips a rival placeholder before confirming, not after', async () => {
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const email = user.email as string;
|
||||
|
||||
// A placeholder sitting on the same address. Confirming first and
|
||||
// demoting second would momentarily leave two rows owning it, which
|
||||
// the unique index rejects — turning a legitimate confirmation into a
|
||||
// 500.
|
||||
const rival = await server.stores.user.create({
|
||||
username: `rival_${Math.random().toString(36).slice(2, 10)}`,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email,
|
||||
clean_email: email,
|
||||
});
|
||||
|
||||
const refreshed = await server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
});
|
||||
const res = makeRes();
|
||||
await controller.handleConfirmEmail(
|
||||
makeReq({ code: refreshed!.email_confirm_code! }, { actor }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.body).toMatchObject({ email_confirmed: true });
|
||||
const confirmed = await server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(confirmed!.email_confirmed).toBe(true);
|
||||
const strippedRival = await server.stores.user.getById(rival.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(strippedRival!.email).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses when another account already confirmed the address', async () => {
|
||||
const email = `owned-${uniq()}@test.local`;
|
||||
|
||||
// Confirmed with no password is the shape an identity provider
|
||||
// creates, and it is what the old rival check (which also demanded a
|
||||
// password) let through. Demoting it would take the address off an
|
||||
// account that proved it owns the inbox.
|
||||
const owner = await server.stores.user.create({
|
||||
username: `owner_${uniq()}`,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email,
|
||||
clean_email: email,
|
||||
email_confirmed: true,
|
||||
});
|
||||
|
||||
// A second, unconfirmed row on the same address holding a confirm
|
||||
// code — the legacy duplicate this path used to resolve in its favour.
|
||||
const claimant = await server.stores.user.create({
|
||||
username: `claim_${uniq()}`,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email,
|
||||
clean_email: email,
|
||||
email_confirm_code: '123456',
|
||||
});
|
||||
const actor = {
|
||||
user: {
|
||||
id: claimant.id,
|
||||
uuid: claimant.uuid,
|
||||
username: claimant.username,
|
||||
email,
|
||||
email_confirmed: false,
|
||||
},
|
||||
} as Actor;
|
||||
|
||||
await expect(
|
||||
controller.handleConfirmEmail(
|
||||
makeReq({ code: '123456' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
|
||||
const untouched = await server.stores.user.getById(owner.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(untouched!.email).toBe(email);
|
||||
expect(untouched!.email_confirmed).toBe(true);
|
||||
const stillUnconfirmed = await server.stores.user.getById(claimant.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(stillUnconfirmed!.email_confirmed).toBeFalsy();
|
||||
});
|
||||
});
|
||||
|
||||
describe('AuthController.handleSaveAccount address conflicts', () => {
|
||||
it('refuses to promote a temp account onto a taken address', async () => {
|
||||
const email = `save-${Math.random().toString(36).slice(2, 10)}@test.local`;
|
||||
await controller.handleSignup(
|
||||
makeReq({
|
||||
username: `save_own_${Math.random().toString(36).slice(2, 10)}`,
|
||||
email,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
const tempRes = makeRes();
|
||||
await controller.handleSignup(makeReq({ is_temp: true }), tempRes);
|
||||
const tempUser = (
|
||||
tempRes.body as { user: { username: string; uuid: string } }
|
||||
).user;
|
||||
const tempRow = await server.stores.user.getByUuid(tempUser.uuid);
|
||||
const actor = {
|
||||
user: {
|
||||
id: tempRow!.id,
|
||||
uuid: tempRow!.uuid,
|
||||
username: tempRow!.username,
|
||||
email: null,
|
||||
email_confirmed: false,
|
||||
},
|
||||
} as Actor;
|
||||
|
||||
await expect(
|
||||
controller.handleSaveAccount(
|
||||
makeReq(
|
||||
{
|
||||
username: `save_new_${Math.random().toString(36).slice(2, 10)}`,
|
||||
email,
|
||||
password: 'another-strong-password',
|
||||
},
|
||||
{ actor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'email_already_in_use',
|
||||
});
|
||||
|
||||
// The temp row must be left alone — a failed promotion that already
|
||||
// wrote the username would strand the account half-converted.
|
||||
const untouched = await server.stores.user.getById(tempRow!.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(untouched!.email).toBeNull();
|
||||
expect(untouched!.password).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
// ── Password recovery flow ──────────────────────────────────────────
|
||||
@@ -4453,16 +4748,24 @@ describe('AuthController permission revokes', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('revoke-user-group: 400 on missing group_uid/permission', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleRevokeUserGroup(
|
||||
makeReq({ permission: 'fs:read' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
// No in-tree caller, so a later cleanup would read it as dead: pin the gate.
|
||||
it('revoke-user-user: stays registered as a user-actor route', () => {
|
||||
const proto = Object.getPrototypeOf(controller) as Record<
|
||||
string,
|
||||
CollectedRoute[] | undefined
|
||||
>;
|
||||
const route = (proto[ROUTES_METADATA_KEY] ?? []).find(
|
||||
(r) => r.path === '/auth/revoke-user-user',
|
||||
);
|
||||
expect(route, '/auth/revoke-user-user is not registered').toBeDefined();
|
||||
expect(route!.method).toBe('post');
|
||||
expect(route!.options).toMatchObject({
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
});
|
||||
});
|
||||
|
||||
// Why the deprecated route exists: older grants stay withdrawable.
|
||||
it('revoke-user-user: round-trips a grant + revoke without throwing', async () => {
|
||||
const { actor: issuerActor, user: issuer } = await makeUserAndActor();
|
||||
const { user: target } = await makeUserAndActor();
|
||||
@@ -4476,24 +4779,17 @@ describe('AuthController permission revokes', () => {
|
||||
issuer_user_id: issuer.id,
|
||||
} as never,
|
||||
);
|
||||
// Grant first.
|
||||
// Through the service: the grant route is retired, the revoke is not.
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserUser(
|
||||
makeReq(
|
||||
{ target_username: target.username, permission },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
server.services.permission.grantUserUserPermission(
|
||||
issuerActor,
|
||||
target.username,
|
||||
permission,
|
||||
),
|
||||
);
|
||||
|
||||
// Now revoke — must complete without throwing and return {}.
|
||||
// We don't re-assert the post-revoke `check()` answer here: the
|
||||
// Redis-mock scan cache is process-wide, and intervening grants
|
||||
// from other tests have repeatedly been observed to leave the
|
||||
// cached `true` answer in place even after a successful revoke.
|
||||
// Verifying the controller path rather than the cache eviction
|
||||
// semantics keeps this test focused.
|
||||
// Asserts the controller path only: the process-wide Redis-mock scan
|
||||
// cache makes a post-revoke `check()` unreliable across tests.
|
||||
const res = makeRes();
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleRevokeUserUser(
|
||||
@@ -4569,9 +4865,10 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
|
||||
);
|
||||
|
||||
// A user→user grant must show up under `myself_to_user` for the
|
||||
// issuer and `user_to_myself` for the holder. Grants gate on
|
||||
// `manage:<permission>`, so bootstrap that flag first (mirrors the
|
||||
// grant-user-user persistence test).
|
||||
// issuer and `user_to_myself` for the holder. Seeded through the
|
||||
// service: the grant route is retired, but the listing still has to
|
||||
// report grants made before that, and the ones sharing writes.
|
||||
// Grants gate on `manage:<permission>`, so bootstrap that flag first.
|
||||
const { user: holder, actor: holderActor } = await makeUserAndActor();
|
||||
const userPermission = 'service:tl-user:ii:read';
|
||||
await server.stores.permission.setFlatUserPerm(
|
||||
@@ -4584,16 +4881,11 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
|
||||
} as never,
|
||||
);
|
||||
await inCtx(actor, () =>
|
||||
controller.handleGrantUserUser(
|
||||
makeReq(
|
||||
{
|
||||
target_username: holder.username,
|
||||
permission: userPermission,
|
||||
extra: {},
|
||||
},
|
||||
{ actor },
|
||||
),
|
||||
makeRes(),
|
||||
server.services.permission.grantUserUserPermission(
|
||||
actor,
|
||||
holder.username,
|
||||
userPermission,
|
||||
{},
|
||||
),
|
||||
);
|
||||
|
||||
@@ -5022,6 +5314,33 @@ describe('AuthController 2FA flows', () => {
|
||||
).toHaveLength(10);
|
||||
});
|
||||
|
||||
it('a recovery code minted by configure-2fa setup satisfies /login/recovery-code', async () => {
|
||||
const { user, actor } = await makeUserAndActor({ email_confirmed: 1 });
|
||||
const setupRes = makeRes();
|
||||
await controller.handleConfigure2fa(
|
||||
makeReq({}, { actor, params: { action: 'setup' } }),
|
||||
setupRes,
|
||||
);
|
||||
const { codes } = setupRes.body as { codes: string[] };
|
||||
|
||||
await controller.handleConfigure2fa(
|
||||
makeReq({}, { actor, params: { action: 'enable' } }),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
const otpJwt = server.services.token.sign(
|
||||
'otp',
|
||||
{ user_uid: user.uuid, purpose: 'otp-login' },
|
||||
{ expiresIn: '5m' },
|
||||
);
|
||||
const res = makeRes();
|
||||
await controller.handleLoginRecoveryCode(
|
||||
makeReq({ token: otpJwt, code: codes[0] }),
|
||||
res,
|
||||
);
|
||||
expect(isCompleteLoginResponse(res.body)).toBe(true);
|
||||
});
|
||||
|
||||
it('configure-2fa setup: 409 when 2FA is already enabled', async () => {
|
||||
const { actor } = await makeUserAndActor({ otp_enabled: 1 });
|
||||
await expect(
|
||||
@@ -5116,130 +5435,6 @@ describe('AuthController.handleGetDevProfile', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── Group endpoints ────────────────────────────────────────────────
|
||||
|
||||
describe('AuthController group endpoints', () => {
|
||||
it('group/create: rejects non-object extra/metadata with 400', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGroupCreate(
|
||||
makeReq({ extra: ['x'] }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
await expect(
|
||||
controller.handleGroupCreate(
|
||||
makeReq({ metadata: ['x'] }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('group/create + add-users + remove-users: full owner-driven lifecycle', async () => {
|
||||
const { actor: owner } = await makeUserAndActor();
|
||||
const { user: target } = await makeUserAndActor();
|
||||
|
||||
// Create.
|
||||
const createRes = makeRes();
|
||||
await controller.handleGroupCreate(
|
||||
makeReq({ metadata: { name: 'g' } }, { actor: owner }),
|
||||
createRes,
|
||||
);
|
||||
const { uid } = createRes.body as { uid: string };
|
||||
expect(typeof uid).toBe('string');
|
||||
|
||||
// Add.
|
||||
const addRes = makeRes();
|
||||
await controller.handleGroupAddUsers(
|
||||
makeReq({ uid, users: [target.username] }, { actor: owner }),
|
||||
addRes,
|
||||
);
|
||||
expect(addRes.body).toEqual({});
|
||||
|
||||
// Remove.
|
||||
const remRes = makeRes();
|
||||
await controller.handleGroupRemoveUsers(
|
||||
makeReq({ uid, users: [target.username] }, { actor: owner }),
|
||||
remRes,
|
||||
);
|
||||
expect(remRes.body).toEqual({});
|
||||
});
|
||||
|
||||
it('group/add-users: 400 on missing uid or non-array users', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGroupAddUsers(
|
||||
makeReq({ users: ['x'] }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
await expect(
|
||||
controller.handleGroupAddUsers(
|
||||
makeReq({ uid: 'g-1' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('group/add-users: 404 on unknown uid; 403 when caller doesn’t own the group', async () => {
|
||||
const { actor: a1 } = await makeUserAndActor();
|
||||
const { actor: a2 } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGroupAddUsers(
|
||||
makeReq(
|
||||
{ uid: `does-not-exist-${uuidv4()}`, users: [] },
|
||||
{ actor: a1 },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
|
||||
// Group owned by a1; a2 tries to add → 403.
|
||||
const createRes = makeRes();
|
||||
await controller.handleGroupCreate(
|
||||
makeReq({}, { actor: a1 }),
|
||||
createRes,
|
||||
);
|
||||
const { uid } = createRes.body as { uid: string };
|
||||
await expect(
|
||||
controller.handleGroupAddUsers(
|
||||
makeReq({ uid, users: [] }, { actor: a2 }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403 });
|
||||
});
|
||||
|
||||
it('group/list: forwards to GroupStore listByOwner/listByMember (or surfaces the source-side method-name mismatch)', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
const res = makeRes();
|
||||
try {
|
||||
await controller.handleGroupList(makeReq({}, { actor }), res);
|
||||
const body = res.body as {
|
||||
owned_groups: unknown[];
|
||||
in_groups: unknown[];
|
||||
};
|
||||
expect(Array.isArray(body.owned_groups)).toBe(true);
|
||||
expect(Array.isArray(body.in_groups)).toBe(true);
|
||||
} catch (e) {
|
||||
// The handler calls `stores.group.listByOwner(...)`, but the
|
||||
// GroupStore implementation may expose a differently-named
|
||||
// method. Surface the mismatch so a future GroupStore rename
|
||||
// re-enables the assertion above.
|
||||
expect((e as Error).message).toMatch(
|
||||
/listByOwner|listByMember|is not a function/,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('group/public-groups: returns {user, temp} from config', async () => {
|
||||
const res = makeRes();
|
||||
await controller.handleGroupPublicGroups(makeReq({}), res);
|
||||
const body = res.body as { user: string | null; temp: string | null };
|
||||
expect(body).toHaveProperty('user');
|
||||
expect(body).toHaveProperty('temp');
|
||||
});
|
||||
});
|
||||
|
||||
// ── GUI token + session sync cookie ────────────────────────────────
|
||||
|
||||
describe('AuthController.handleGetGuiToken + handleSessionSyncCookie', () => {
|
||||
@@ -6125,16 +6320,6 @@ describe('AuthController grant/revoke additional branches', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('grant-user-group: 400 on missing group_uid', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGrantUserGroup(
|
||||
makeReq({ permission: 'fs:read' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('revoke-user-app: 400 when permission is "*" but app_uid is missing', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
@@ -6252,58 +6437,6 @@ describe('AuthController.handleGetDevProfile additional branches', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('AuthController group endpoints: additional branches', () => {
|
||||
it('group/remove-users: 400 on missing uid', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGroupRemoveUsers(
|
||||
makeReq({ users: ['x'] }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('group/remove-users: 400 on non-array users', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGroupRemoveUsers(
|
||||
makeReq({ uid: 'g-1' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('group/remove-users: 404 on unknown uid', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleGroupRemoveUsers(
|
||||
makeReq(
|
||||
{ uid: `does-not-exist-${uuidv4()}`, users: [] },
|
||||
{ actor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
});
|
||||
|
||||
it('group/remove-users: 403 when caller does not own the group', async () => {
|
||||
const { actor: a1 } = await makeUserAndActor();
|
||||
const { actor: a2 } = await makeUserAndActor();
|
||||
const createRes = makeRes();
|
||||
await controller.handleGroupCreate(
|
||||
makeReq({}, { actor: a1 }),
|
||||
createRes,
|
||||
);
|
||||
const { uid } = createRes.body as { uid: string };
|
||||
await expect(
|
||||
controller.handleGroupRemoveUsers(
|
||||
makeReq({ uid, users: [] }, { actor: a2 }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('AuthController.handleGetGuiToken / handleSessionSyncCookie additional branches', () => {
|
||||
it('get-gui-token: 404 when actor has a session but the user row is gone', async () => {
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
|
||||
@@ -52,6 +52,8 @@ import {
|
||||
createSecret as otpCreateSecret,
|
||||
verify as verifyOtp,
|
||||
} from '../../services/auth/OTPUtil.js';
|
||||
import type { UserRow } from '../../stores/user/UserStore.js';
|
||||
import { isOwnedEmailConflict } from '../../stores/user/UserStore.js';
|
||||
import { sessionCookieFlags } from '../../util/cookieFlags.js';
|
||||
import { cleanEmail, isBlockedEmail } from '../../util/email.js';
|
||||
import { generate_identifier } from '../../util/identifier.js';
|
||||
@@ -116,7 +118,7 @@ const TWO_FACTOR_LIMIT = {
|
||||
key: 'user',
|
||||
} as const;
|
||||
|
||||
/** Permission and membership writes. Never called in a loop by a client. */
|
||||
/** Permission writes. Never called in a loop by a client. */
|
||||
const GRANT_LIMIT = {
|
||||
scope: 'auth-grant',
|
||||
limit: 60,
|
||||
@@ -156,7 +158,7 @@ const ANTI_CSRF_MINT_LIMIT = {
|
||||
key: 'user',
|
||||
} as const;
|
||||
|
||||
/** Settings-page reads — enumerating sessions, permissions, groups. */
|
||||
/** Settings-page reads — enumerating sessions and permissions. */
|
||||
const AUTH_LIST_LIMIT = {
|
||||
scope: 'auth-list',
|
||||
limit: 120,
|
||||
@@ -177,6 +179,21 @@ const CARD_FALLBACK_OPEN_TTL_SECONDS = 24 * 60 * 60;
|
||||
// How long a failed-SMS-send record stays readable by its error_id — long
|
||||
// enough to cover the typical support round-trip.
|
||||
const SMS_SEND_ERROR_TTL_SECONDS = 7 * 24 * 60 * 60;
|
||||
|
||||
/**
|
||||
* Whether this account has already been through the card flow. Both halves
|
||||
* matter: the pending flag being clear only says nobody asked, while the
|
||||
* fingerprint is the artifact a completed check leaves behind. A route that
|
||||
* requires a verified card (`requireCardVerified`) sends users here with the
|
||||
* flag clear and no card on file, and answering "already verified" on the flag
|
||||
* alone would bounce them between a dialog that reports success and a route
|
||||
* that keeps refusing.
|
||||
*/
|
||||
const cardAlreadyVerified = (user: {
|
||||
card_fingerprint?: string | null;
|
||||
requires_card_verification?: boolean;
|
||||
}): boolean =>
|
||||
Boolean(user.card_fingerprint) && !user.requires_card_verification;
|
||||
const RESERVED_USERNAMES = new Set([
|
||||
'admin',
|
||||
'administrator',
|
||||
@@ -775,11 +792,9 @@ export class AuthController extends PuterController {
|
||||
if (this.config.disable_user_signup) {
|
||||
let claimable = false;
|
||||
if (!is_temp) {
|
||||
const existing =
|
||||
(await this.stores.user.getByEmail(body.email)) ??
|
||||
(await this.stores.user.getByCleanEmail(
|
||||
cleanEmail(body.email),
|
||||
));
|
||||
const existing = await this.stores.user.findEmailOwner(
|
||||
body.email,
|
||||
);
|
||||
claimable = Boolean(
|
||||
existing &&
|
||||
!existing.email_confirmed &&
|
||||
@@ -817,28 +832,18 @@ export class AuthController extends PuterController {
|
||||
// password to an OIDC account, the owner logs in via OIDC and
|
||||
// uses the authenticated change-password flow.
|
||||
//
|
||||
// Match on both raw `email` and canonical `clean_email` so
|
||||
// Matching runs against both raw `email` and canonical `clean_email` so
|
||||
// gmail-style aliases (`[email protected]` vs
|
||||
// `[email protected]`) collapse to the same account.
|
||||
let pseudo_user = null;
|
||||
if (!is_temp) {
|
||||
const canonical = cleanEmail(body.email);
|
||||
const existing =
|
||||
(await this.stores.user.getByEmail(body.email)) ??
|
||||
(await this.stores.user.getByCleanEmail(canonical));
|
||||
if (existing) {
|
||||
// Confirmed account (regardless of credential type) → reject.
|
||||
if (existing.email_confirmed || existing.password !== null) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This email already exists in our database. Please use another one.',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
// Password-null AND unconfirmed → treat as pseudo.
|
||||
pseudo_user = existing;
|
||||
}
|
||||
}
|
||||
//
|
||||
// This is the cheap early check: it keeps an obvious duplicate from
|
||||
// paying for the validate hook and a bcrypt round. It is NOT the
|
||||
// guarantee — everything between here and the insert widens the window,
|
||||
// so the check runs again against the primary immediately before the
|
||||
// write, and the unique index catches whatever still slips through.
|
||||
let pseudo_user = is_temp
|
||||
? null
|
||||
: await this.#resolveSignupEmailClaim(body.email);
|
||||
|
||||
// Extension-level validation gate. Abuse-prevention extensions
|
||||
// inspect the incoming signup and can:
|
||||
@@ -857,6 +862,15 @@ export class AuthController extends PuterController {
|
||||
req.socket?.remoteAddress ||
|
||||
null) as string | null,
|
||||
email: body.email,
|
||||
// The same canonical form `email.validate` was given, so a check
|
||||
// in the abuse harness can look up the verdict that hook cached
|
||||
// for this address. Without it an alias (`[email protected]`,
|
||||
// `[email protected]`) reaches the two hooks under two different keys.
|
||||
clean_email: cleanEmail(body.email),
|
||||
// Temp signups carry a synthetic `<username>@gmail.com` and skip
|
||||
// #validateEmail entirely, so an email check must know not to
|
||||
// reason about the address at all.
|
||||
is_temp,
|
||||
allow: true,
|
||||
no_temp_user: false,
|
||||
requires_email_confirmation: false,
|
||||
@@ -945,24 +959,54 @@ export class AuthController extends PuterController {
|
||||
.slice(0, 19)
|
||||
.replace('T', ' ');
|
||||
|
||||
// Re-run the claim against the primary now that the slow work is done.
|
||||
// The check above ran before the validate hook (network round-trips to
|
||||
// the abuse listeners) and before bcrypt — hundreds of milliseconds in
|
||||
// which a concurrent signup can take the address, or claim the very
|
||||
// placeholder row we were about to convert.
|
||||
if (!is_temp) {
|
||||
pseudo_user = await this.#resolveSignupEmailClaim(body.email, {
|
||||
force: true,
|
||||
});
|
||||
}
|
||||
|
||||
let user;
|
||||
if (pseudo_user) {
|
||||
// -- Pseudo-user claim (convert the placeholder row) --
|
||||
await this.stores.user.update(pseudo_user.id, {
|
||||
username: body.username,
|
||||
password: password_hash,
|
||||
uuid: user_uuid,
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
email_confirmed: 0,
|
||||
requires_email_confirmation: 1,
|
||||
last_activity_ts: signupSqlTs,
|
||||
...(validateEvent.reputation != null
|
||||
? { reputation: validateEvent.reputation }
|
||||
: {}),
|
||||
requires_phone_verification: force_phone_verification ? 1 : 0,
|
||||
requires_card_verification: force_card_verification ? 1 : 0,
|
||||
});
|
||||
//
|
||||
// Guarded, not a plain update: the address never changes hands here
|
||||
// (the row already holds it), so the unique index has nothing to
|
||||
// catch. Two signups that both read this row as claimable would
|
||||
// otherwise both "succeed", the second overwriting the first's
|
||||
// username and password on a row the first was already given a
|
||||
// session for.
|
||||
const claimed = await this.stores.user.claimPlaceholder(
|
||||
pseudo_user.id,
|
||||
{
|
||||
username: body.username,
|
||||
password: password_hash,
|
||||
uuid: user_uuid,
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
email_confirmed: 0,
|
||||
requires_email_confirmation: 1,
|
||||
last_activity_ts: signupSqlTs,
|
||||
...(validateEvent.reputation != null
|
||||
? { reputation: validateEvent.reputation }
|
||||
: {}),
|
||||
requires_phone_verification: force_phone_verification
|
||||
? 1
|
||||
: 0,
|
||||
requires_card_verification: force_card_verification ? 1 : 0,
|
||||
},
|
||||
);
|
||||
if (!claimed) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This email already exists in our database. Please use another one.',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
|
||||
// Move from temp group to regular user group
|
||||
if (this.config.default_temp_group) {
|
||||
@@ -994,37 +1038,51 @@ export class AuthController extends PuterController {
|
||||
const clientIp = req.ip || req.socket?.remoteAddress || null;
|
||||
const proxyIpChain = req.headers['x-forwarded-for'];
|
||||
|
||||
user = await this.stores.user.create({
|
||||
username: body.username,
|
||||
uuid: user_uuid,
|
||||
password: password_hash,
|
||||
email: is_temp ? null : body.email,
|
||||
clean_email: is_temp ? null : cleanEmail(body.email),
|
||||
free_storage: this.config.storage_capacity ?? null,
|
||||
requires_email_confirmation:
|
||||
!is_temp || force_email_confirmation,
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
audit_metadata: {
|
||||
ip: clientIp,
|
||||
ip_fwd: proxyIpChain,
|
||||
user_agent: req.headers?.['user-agent'],
|
||||
origin: req.headers?.origin,
|
||||
fingerprint,
|
||||
},
|
||||
signup_ip: clientIp,
|
||||
signup_ip_forwarded: proxyIpChain,
|
||||
signup_user_agent: req.headers?.['user-agent'] ?? null,
|
||||
signup_origin: (req.headers?.origin as string | null) ?? null,
|
||||
signup_server: (this.config as { serverId?: string }).serverId,
|
||||
referrer: req.body.referrer ?? null,
|
||||
last_activity_ts: signupSqlTs,
|
||||
reputation: validateEvent.reputation,
|
||||
// Phone collected later in the verification dialog (null now).
|
||||
phone: null,
|
||||
requires_phone_verification: force_phone_verification,
|
||||
requires_card_verification: force_card_verification,
|
||||
} as never);
|
||||
try {
|
||||
user = await this.stores.user.create({
|
||||
username: body.username,
|
||||
uuid: user_uuid,
|
||||
password: password_hash,
|
||||
email: is_temp ? null : body.email,
|
||||
clean_email: is_temp ? null : cleanEmail(body.email),
|
||||
free_storage: this.config.storage_capacity ?? null,
|
||||
requires_email_confirmation:
|
||||
!is_temp || force_email_confirmation,
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
audit_metadata: {
|
||||
ip: clientIp,
|
||||
ip_fwd: proxyIpChain,
|
||||
user_agent: req.headers?.['user-agent'],
|
||||
origin: req.headers?.origin,
|
||||
fingerprint,
|
||||
},
|
||||
signup_ip: clientIp,
|
||||
signup_ip_forwarded: proxyIpChain,
|
||||
signup_user_agent: req.headers?.['user-agent'] ?? null,
|
||||
signup_origin:
|
||||
(req.headers?.origin as string | null) ?? null,
|
||||
signup_server: (this.config as { serverId?: string })
|
||||
.serverId,
|
||||
referrer: req.body.referrer ?? null,
|
||||
last_activity_ts: signupSqlTs,
|
||||
reputation: validateEvent.reputation,
|
||||
// Phone collected later in the verification dialog (null now).
|
||||
phone: null,
|
||||
requires_phone_verification: force_phone_verification,
|
||||
requires_card_verification: force_card_verification,
|
||||
} as never);
|
||||
} catch (e) {
|
||||
// Lost the race to another signup between the re-check above and
|
||||
// this insert. The index is the only thing that can see that, so
|
||||
// translate it into the answer the pre-check would have given.
|
||||
if (!isOwnedEmailConflict(e)) throw e;
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This email already exists in our database. Please use another one.',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
|
||||
// Add to default group
|
||||
const defaultGroup = is_temp
|
||||
@@ -1101,8 +1159,7 @@ export class AuthController extends PuterController {
|
||||
is_temp: user!.password === null && user!.email === null,
|
||||
ip:
|
||||
(req?.headers?.['x-forwarded-for'] as
|
||||
| string
|
||||
| undefined) ||
|
||||
string | undefined) ||
|
||||
(
|
||||
req as unknown as {
|
||||
connection?: { remoteAddress?: string };
|
||||
@@ -1275,6 +1332,34 @@ export class AuthController extends PuterController {
|
||||
// after signup but before confirmation.
|
||||
await this.#validateEmail(user.email!);
|
||||
|
||||
// An account that already confirmed this address proved access to the
|
||||
// inbox, and revoking it below would hand the address to whoever
|
||||
// confirmed second. Refuse instead — a duplicate this old is data to
|
||||
// repair, not a race to resolve.
|
||||
const canonical = cleanEmail(user.email!);
|
||||
const confirmedRival = await this.stores.user.findConfirmedOtherByEmail(
|
||||
user.id,
|
||||
user.email!,
|
||||
canonical,
|
||||
);
|
||||
if (confirmedRival) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This email was confirmed on a different account.',
|
||||
{ legacyCode: 'email_already_in_use' as never },
|
||||
);
|
||||
}
|
||||
|
||||
// Revoke the address from every remaining (unconfirmed) account holding
|
||||
// it, THEN confirm this one. Only one row may own an address, so
|
||||
// confirming first would momentarily create a second owner — which the
|
||||
// unique index rejects, turning a legitimate confirmation into a 500.
|
||||
await this.stores.user.unconfirmOthersByEmail(
|
||||
user.id,
|
||||
user.email!,
|
||||
canonical,
|
||||
);
|
||||
|
||||
await this.stores.user.update(user.id, {
|
||||
email_confirmed: 1,
|
||||
requires_email_confirmation: 0,
|
||||
@@ -1282,16 +1367,6 @@ export class AuthController extends PuterController {
|
||||
email_confirm_token: null,
|
||||
});
|
||||
|
||||
// Revoke confirmation from any other accounts sharing this
|
||||
// email so only the account whose owner just proved inbox
|
||||
// access retains verified status.
|
||||
const canonical = cleanEmail(user.email!);
|
||||
await this.stores.user.unconfirmOthersByEmail(
|
||||
user.id,
|
||||
user.email!,
|
||||
canonical,
|
||||
);
|
||||
|
||||
await promoteToVerifiedGroup(this.stores.group, this.config, user);
|
||||
|
||||
try {
|
||||
@@ -1864,7 +1939,7 @@ export class AuthController extends PuterController {
|
||||
// Phone normally comes first, but the fallback lets a phone-gated user
|
||||
// in once they've exhausted SMS attempts.
|
||||
const fallbackEligible = await this.isCardFallbackEligible(user);
|
||||
if (!user.requires_card_verification && !fallbackEligible) {
|
||||
if (cardAlreadyVerified(user) && !fallbackEligible) {
|
||||
res.json({ card_verified: true });
|
||||
return;
|
||||
}
|
||||
@@ -1984,7 +2059,7 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
// Same fallback exception as setup: card may come before phone.
|
||||
const fallbackEligible = await this.isCardFallbackEligible(user);
|
||||
if (!user.requires_card_verification && !fallbackEligible) {
|
||||
if (cardAlreadyVerified(user) && !fallbackEligible) {
|
||||
res.json({ card_verified: true });
|
||||
return;
|
||||
}
|
||||
@@ -2278,10 +2353,26 @@ export class AuthController extends PuterController {
|
||||
|
||||
// Atomic check: only update if the recovery token still matches
|
||||
const password_hash = await bcrypt.hash(password, 8);
|
||||
const result = await this.clients.db.write(
|
||||
'UPDATE `user` SET `password` = ?, `pass_recovery_token` = NULL, `change_email_confirm_token` = NULL WHERE `id` = ? AND `pass_recovery_token` = ?',
|
||||
[password_hash, user.id, decoded.token],
|
||||
);
|
||||
let result;
|
||||
try {
|
||||
result = await this.clients.db.write(
|
||||
'UPDATE `user` SET `password` = ?, `pass_recovery_token` = NULL, `change_email_confirm_token` = NULL WHERE `id` = ? AND `pass_recovery_token` = ?',
|
||||
[password_hash, user.id, decoded.token],
|
||||
);
|
||||
} catch (e) {
|
||||
if (!isOwnedEmailConflict(e)) throw e;
|
||||
// Recovery can be requested by username, so this row may be an
|
||||
// unconfirmed placeholder that shares its address with a real
|
||||
// account. Giving it a password would make it a second account able
|
||||
// to drive recovery for that inbox, which is the thing the address
|
||||
// constraint exists to stop. The inbox owner has an account
|
||||
// already — they should be recovering that one.
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This email is already in use. Recover the account that uses it instead.',
|
||||
{ legacyCode: 'email_already_in_use' as never },
|
||||
);
|
||||
}
|
||||
const affected =
|
||||
(result as { affectedRows?: number; changes?: number })
|
||||
?.affectedRows ??
|
||||
@@ -2442,10 +2533,7 @@ export class AuthController extends PuterController {
|
||||
// aliases — which is also why the caller has to be excluded: an
|
||||
// alias of your own current address resolves back to you, and
|
||||
// "already in use" about yourself is nonsense.
|
||||
const canonical = cleanEmail(new_email);
|
||||
const existing =
|
||||
(await this.stores.user.getByEmail(new_email)) ??
|
||||
(await this.stores.user.getByCleanEmail(canonical));
|
||||
const existing = await this.stores.user.findEmailOwner(new_email);
|
||||
if (
|
||||
existing &&
|
||||
existing.id !== req.actor!.user.id &&
|
||||
@@ -2546,7 +2634,7 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
|
||||
const rows = (await this.clients.db.read(
|
||||
'SELECT * FROM `user` WHERE `change_email_confirm_token` = ? LIMIT 1',
|
||||
'SELECT * FROM `user` WHERE `change_email_confirm_token` = ? ORDER BY `id` ASC LIMIT 1',
|
||||
[decoded.token],
|
||||
)) as Array<Record<string, unknown>>;
|
||||
const user = rows[0] as
|
||||
@@ -2567,11 +2655,12 @@ export class AuthController extends PuterController {
|
||||
|
||||
// Re-check nobody claimed the new email meanwhile. Match raw +
|
||||
// canonical; block if any real account (confirmed OR
|
||||
// password-holding) already owns it.
|
||||
// password-holding) already owns it. Read the primary — the request
|
||||
// that took the address may have landed moments ago.
|
||||
const canonical = cleanEmail(newEmail);
|
||||
const owner =
|
||||
(await this.stores.user.getByEmail(newEmail)) ??
|
||||
(await this.stores.user.getByCleanEmail(canonical));
|
||||
const owner = await this.stores.user.findEmailOwner(newEmail, {
|
||||
force: true,
|
||||
});
|
||||
if (
|
||||
owner &&
|
||||
owner.id !== user.id &&
|
||||
@@ -2582,15 +2671,30 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
await this.stores.user.update(user.id, {
|
||||
email: newEmail,
|
||||
clean_email: cleanEmail(newEmail),
|
||||
unconfirmed_change_email: null,
|
||||
change_email_confirm_token: null,
|
||||
pass_recovery_token: null,
|
||||
email_confirmed: 1,
|
||||
requires_email_confirmation: 0,
|
||||
});
|
||||
// Strip the address off any unconfirmed placeholder still holding it
|
||||
// before taking it, so this row is the only owner.
|
||||
await this.stores.user.unconfirmOthersByEmail(
|
||||
user.id,
|
||||
newEmail,
|
||||
canonical,
|
||||
);
|
||||
|
||||
try {
|
||||
await this.stores.user.update(user.id, {
|
||||
email: newEmail,
|
||||
clean_email: canonical,
|
||||
unconfirmed_change_email: null,
|
||||
change_email_confirm_token: null,
|
||||
pass_recovery_token: null,
|
||||
email_confirmed: 1,
|
||||
requires_email_confirmation: 0,
|
||||
});
|
||||
} catch (e) {
|
||||
if (!isOwnedEmailConflict(e)) throw e;
|
||||
throw new HttpError(400, 'This email is already in use.', {
|
||||
legacyCode: 'email_already_in_use' as never,
|
||||
});
|
||||
}
|
||||
|
||||
await this.stores.oidc.unlinkAllByUserId(user.id);
|
||||
|
||||
@@ -2693,9 +2797,7 @@ export class AuthController extends PuterController {
|
||||
// reject on ANY confirmed account (OIDC accounts have
|
||||
// password=null but are real) — not just password-holders.
|
||||
const canonical = cleanEmail(email);
|
||||
const existingEmail =
|
||||
(await this.stores.user.getByEmail(email)) ??
|
||||
(await this.stores.user.getByCleanEmail(canonical));
|
||||
const existingEmail = await this.stores.user.findEmailOwner(email);
|
||||
if (
|
||||
existingEmail &&
|
||||
existingEmail.id !== user.id &&
|
||||
@@ -2711,16 +2813,38 @@ export class AuthController extends PuterController {
|
||||
const email_confirm_code = String(crypto.randomInt(100000, 1000000));
|
||||
const email_confirm_token = uuidv4();
|
||||
|
||||
await this.stores.user.update(user.id, {
|
||||
username,
|
||||
email,
|
||||
clean_email: cleanEmail(email),
|
||||
password: password_hash,
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
email_confirmed: 0,
|
||||
requires_email_confirmation: 1,
|
||||
// bcrypt above is slow enough for someone else to take the address in
|
||||
// the meantime, so re-check against the primary before the write.
|
||||
const raced = await this.stores.user.findEmailOwner(email, {
|
||||
force: true,
|
||||
});
|
||||
if (
|
||||
raced &&
|
||||
raced.id !== user.id &&
|
||||
(raced.email_confirmed || raced.password !== null)
|
||||
) {
|
||||
throw new HttpError(400, 'This email is already in use.', {
|
||||
legacyCode: 'email_already_in_use' as never,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
await this.stores.user.update(user.id, {
|
||||
username,
|
||||
email,
|
||||
clean_email: canonical,
|
||||
password: password_hash,
|
||||
email_confirm_code,
|
||||
email_confirm_token,
|
||||
email_confirmed: 0,
|
||||
requires_email_confirmation: 1,
|
||||
});
|
||||
} catch (e) {
|
||||
if (!isOwnedEmailConflict(e)) throw e;
|
||||
throw new HttpError(400, 'This email is already in use.', {
|
||||
legacyCode: 'email_already_in_use' as never,
|
||||
});
|
||||
}
|
||||
|
||||
// Rename the user's FS home so `/<temp>/Desktop` etc.
|
||||
// become `/<new>/Desktop`. Without this cascade, any
|
||||
@@ -2850,30 +2974,6 @@ export class AuthController extends PuterController {
|
||||
|
||||
// -- Permission grants -------------------------------------------
|
||||
|
||||
@Post('/auth/grant-user-user', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
rateLimit: GRANT_LIMIT,
|
||||
})
|
||||
async handleGrantUserUser(req: Request, res: Response): Promise<void> {
|
||||
const { target_username, permission, extra, meta } = req.body;
|
||||
if (!target_username || !permission) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'Missing `target_username` or `permission`',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
await this.services.permission.grantUserUserPermission(
|
||||
req.actor!,
|
||||
target_username,
|
||||
permission,
|
||||
extra,
|
||||
meta,
|
||||
);
|
||||
res.json({});
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared input validation for the user-app grant/revoke handlers, which
|
||||
* accept a caller-supplied `origin` as an alternative to `app_uid`. All
|
||||
@@ -3094,35 +3194,13 @@ export class AuthController extends PuterController {
|
||||
res.json({});
|
||||
}
|
||||
|
||||
@Post('/auth/grant-user-group', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
rateLimit: GRANT_LIMIT,
|
||||
})
|
||||
async handleGrantUserGroup(req: Request, res: Response): Promise<void> {
|
||||
const { group_uid, permission, extra, meta } = req.body;
|
||||
if (!group_uid || !permission) {
|
||||
throw new HttpError(400, 'Missing `group_uid` or `permission`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const group = await this.stores.group.getByUid(group_uid);
|
||||
if (!group)
|
||||
throw new HttpError(404, 'Group not found', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
await this.services.permission.grantUserGroupPermission(
|
||||
req.actor!,
|
||||
group,
|
||||
permission,
|
||||
extra,
|
||||
meta,
|
||||
);
|
||||
res.json({});
|
||||
}
|
||||
|
||||
// -- Permission revokes ------------------------------------------
|
||||
|
||||
/**
|
||||
* @deprecated Use `puter.fs.unshare()`, which withdraws the share row and
|
||||
* the grant together. Kept for direct HTTP callers: the grant side is
|
||||
* retired, but access it left behind has to stay withdrawable.
|
||||
*/
|
||||
@Post('/auth/revoke-user-user', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
@@ -3190,27 +3268,6 @@ export class AuthController extends PuterController {
|
||||
res.json({});
|
||||
}
|
||||
|
||||
@Post('/auth/revoke-user-group', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
rateLimit: GRANT_LIMIT,
|
||||
})
|
||||
async handleRevokeUserGroup(req: Request, res: Response): Promise<void> {
|
||||
const { group_uid, permission, meta } = req.body;
|
||||
if (!group_uid || !permission) {
|
||||
throw new HttpError(400, 'Missing `group_uid` or `permission`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
await this.services.permission.revokeUserGroupPermission(
|
||||
req.actor!,
|
||||
{ uid: group_uid } as never,
|
||||
permission,
|
||||
meta,
|
||||
);
|
||||
res.json({});
|
||||
}
|
||||
|
||||
// -- Permission checks -------------------------------------------
|
||||
|
||||
@Post('/auth/check-permissions', {
|
||||
@@ -3885,147 +3942,6 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
// -- Group management --------------------------------------------
|
||||
|
||||
@Post('/group/create', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
// Creates a persistent row per call with no quota behind it, so it
|
||||
// sits on the hour-scale budget rather than the grant one.
|
||||
rateLimit: { ...CREDENTIAL_MINT_LIMIT, scope: 'group-create' },
|
||||
})
|
||||
async handleGroupCreate(req: Request, res: Response): Promise<void> {
|
||||
const extra = req.body.extra ?? {};
|
||||
const metadata = req.body.metadata ?? {};
|
||||
if (typeof extra !== 'object' || Array.isArray(extra))
|
||||
throw new HttpError(400, '`extra` must be an object', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
if (typeof metadata !== 'object' || Array.isArray(metadata))
|
||||
throw new HttpError(400, '`metadata` must be an object', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const uid = await this.stores.group.create({
|
||||
ownerUserId: req.actor!.user.id,
|
||||
extra: {},
|
||||
metadata,
|
||||
} as never);
|
||||
res.json({ uid });
|
||||
}
|
||||
|
||||
@Post('/group/add-users', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
rateLimit: GRANT_LIMIT,
|
||||
})
|
||||
async handleGroupAddUsers(req: Request, res: Response): Promise<void> {
|
||||
const { uid, users } = req.body ?? {};
|
||||
if (!uid)
|
||||
throw new HttpError(400, 'Missing `uid`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
if (!Array.isArray(users))
|
||||
throw new HttpError(400, '`users` must be an array', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const group = await this.stores.group.getByUid(uid);
|
||||
if (!group)
|
||||
throw new HttpError(404, 'Group not found', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
if (
|
||||
(group as { owner_user_id?: number }).owner_user_id !==
|
||||
req.actor!.user.id
|
||||
)
|
||||
throw new HttpError(403, 'Forbidden', {
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
|
||||
await this.stores.group.addUsers(uid, users);
|
||||
// New members inherit the group's permissions immediately, not
|
||||
// after the permission-cache TTL.
|
||||
await this.services.permission.bumpPermissionCacheForUsernames(users);
|
||||
res.json({});
|
||||
}
|
||||
|
||||
@Post('/group/remove-users', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
rateLimit: GRANT_LIMIT,
|
||||
})
|
||||
async handleGroupRemoveUsers(req: Request, res: Response): Promise<void> {
|
||||
const { uid, users } = req.body ?? {};
|
||||
if (!uid)
|
||||
throw new HttpError(400, 'Missing `uid`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
if (!Array.isArray(users))
|
||||
throw new HttpError(400, '`users` must be an array', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const group = await this.stores.group.getByUid(uid);
|
||||
if (!group)
|
||||
throw new HttpError(404, 'Group not found', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
if (
|
||||
(group as { owner_user_id?: number }).owner_user_id !==
|
||||
req.actor!.user.id
|
||||
)
|
||||
throw new HttpError(403, 'Forbidden', {
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
|
||||
await this.stores.group.removeUsers(uid, users);
|
||||
// Removed members must lose the group's permissions immediately,
|
||||
// not after the permission-cache TTL.
|
||||
await this.services.permission.bumpPermissionCacheForUsernames(users);
|
||||
res.json({});
|
||||
}
|
||||
|
||||
@Get('/group/list', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
rateLimit: AUTH_LIST_LIMIT,
|
||||
})
|
||||
async handleGroupList(req: Request, res: Response): Promise<void> {
|
||||
const userId = req.actor!.user.id!;
|
||||
const [owned, member] = await Promise.all([
|
||||
this.stores.group.listGroupsWithOwner(userId),
|
||||
this.stores.group.listGroupsWithMember(userId),
|
||||
]);
|
||||
res.json({
|
||||
owned_groups: owned,
|
||||
in_groups: member,
|
||||
});
|
||||
}
|
||||
|
||||
@Get('/group/public-groups', {
|
||||
subdomain: 'api',
|
||||
// The only unauthenticated route in the group set, so IP is the
|
||||
// only key available — and that makes the bucket an aggregate:
|
||||
// one office, campus or carrier gateway is a single key for
|
||||
// everybody behind it, and each of them reads this once while
|
||||
// bootstrapping. Sized for that population of real people rather
|
||||
// than one browser, and no wider: this sits next to the sign-in
|
||||
// surface, so it stays a real bound on enumeration.
|
||||
rateLimit: {
|
||||
scope: 'public-groups',
|
||||
limit: 1_200,
|
||||
window: 60_000,
|
||||
key: 'ip',
|
||||
},
|
||||
})
|
||||
async handleGroupPublicGroups(_req: Request, res: Response): Promise<void> {
|
||||
res.json({
|
||||
user: this.config.default_user_group ?? null,
|
||||
temp: this.config.default_temp_group ?? null,
|
||||
});
|
||||
}
|
||||
|
||||
// -- Session helpers ---------------------------------------------
|
||||
|
||||
@Get('/get-gui-token', {
|
||||
@@ -4426,57 +4342,7 @@ export class AuthController extends PuterController {
|
||||
// -- Private helpers ----------------------------------------------
|
||||
|
||||
async #cascadeDeleteUser(userId: number): Promise<void> {
|
||||
// Capture the identifiers downstream teardown needs before the row is
|
||||
// gone — the marketplace extension cancels the user's Stripe
|
||||
// subscriptions off `user.delete`, keyed by uuid / customer id.
|
||||
let userUuid: string | undefined;
|
||||
let stripeCustomerId: string | null = null;
|
||||
try {
|
||||
const rows = (await this.clients.db.read(
|
||||
'SELECT `uuid`, `stripe_customer_id` FROM `user` WHERE `id` = ?',
|
||||
[userId],
|
||||
)) as Array<{ uuid?: string; stripe_customer_id?: string | null }>;
|
||||
userUuid = rows[0]?.uuid;
|
||||
stripeCustomerId = rows[0]?.stripe_customer_id ?? null;
|
||||
} catch (e) {
|
||||
console.warn('[cascade-delete-user] identifier lookup failed:', e);
|
||||
}
|
||||
|
||||
try {
|
||||
await this.services.fs.removeAllForUser(userId);
|
||||
} catch (e) {
|
||||
// Proceed with user-row delete anyway — orphaned fsentries are
|
||||
// better than a resurrected account.
|
||||
console.warn('[cascade-delete-user] fs cleanup failed:', e);
|
||||
}
|
||||
|
||||
// Sessions FK is SET NULL, so delete explicitly to avoid dangling rows.
|
||||
await this.clients.db.write(
|
||||
'DELETE FROM `sessions` WHERE `user_id` = ?',
|
||||
[userId],
|
||||
);
|
||||
await this.clients.db.write('DELETE FROM `user` WHERE `id` = ?', [
|
||||
userId,
|
||||
]);
|
||||
await this.stores.user.invalidateById(userId);
|
||||
|
||||
// Fire-and-forget: let listeners purge external state tied to the
|
||||
// account (Stripe subscriptions are cancelled immediately, without
|
||||
// proration). Emitted after the row delete — listeners key off the
|
||||
// payload, not the DB row.
|
||||
try {
|
||||
this.clients.event?.emit(
|
||||
'user.delete',
|
||||
{
|
||||
user_id: userId,
|
||||
user_uuid: userUuid,
|
||||
stripe_customer_id: stripeCustomerId,
|
||||
},
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
// ignore — event emission shouldn't block deletion
|
||||
}
|
||||
await this.services.userAccount.cascadeDelete(userId);
|
||||
}
|
||||
|
||||
async #generateRandomUsername(): Promise<string> {
|
||||
@@ -4495,6 +4361,34 @@ export class AuthController extends PuterController {
|
||||
return username;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether a signup may take `email`, and hand back the placeholder
|
||||
* row it should convert instead of inserting a new one.
|
||||
*
|
||||
* Throws when a live account already owns the address. Returns the
|
||||
* unconfirmed, password-less pseudo row when one exists (admin
|
||||
* pre-provisioning — signup claims it), or null when the address is free.
|
||||
*
|
||||
* Called twice per signup: once early, to fail fast before the validate
|
||||
* hook and bcrypt, and once against the primary immediately before the
|
||||
* write.
|
||||
*/
|
||||
async #resolveSignupEmailClaim(
|
||||
email: string,
|
||||
opts: { force?: boolean } = {},
|
||||
): Promise<UserRow | null> {
|
||||
const existing = await this.stores.user.findEmailOwner(email, opts);
|
||||
if (!existing) return null;
|
||||
if (existing.email_confirmed || existing.password !== null) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This email already exists in our database. Please use another one.',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return existing;
|
||||
}
|
||||
|
||||
/**
|
||||
* Config-blocklist + extension-driven email validation. Config blocklist
|
||||
* (suffix match on cleaned email) blocks first; then the `email.validate`
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/* eslint-disable @typescript-eslint/no-explicit-any */
|
||||
import { EventEmitter } from 'node:events';
|
||||
import type { Request, RequestHandler, Response } from 'express';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { isHttpError } from '../../core/http/HttpError.js';
|
||||
import { configureRateLimit } from '../../core/http/middleware/rateLimit.js';
|
||||
import { DriverController } from './DriverController.js';
|
||||
|
||||
// The subscription requirement a driver declares per method, exercised
|
||||
// through `#handleCall` the same way the concurrent test does: a synthetic
|
||||
// driver carrying only the field under test, plus a stub metering service
|
||||
// that answers with a fixed plan.
|
||||
|
||||
const captureCallHandler = (controller: DriverController): RequestHandler => {
|
||||
let handler: RequestHandler | undefined;
|
||||
const fakeRouter = {
|
||||
post: (path: string, _opts: unknown, h: RequestHandler) => {
|
||||
if (path === '/call') handler = h;
|
||||
return fakeRouter;
|
||||
},
|
||||
get: () => fakeRouter,
|
||||
use: () => fakeRouter,
|
||||
};
|
||||
controller.registerRoutes(
|
||||
fakeRouter as unknown as Parameters<
|
||||
typeof controller.registerRoutes
|
||||
>[0],
|
||||
);
|
||||
if (!handler) throw new Error('failed to capture POST /call handler');
|
||||
return handler;
|
||||
};
|
||||
|
||||
class StubRes extends EventEmitter {
|
||||
statusCode = 200;
|
||||
body: unknown = undefined;
|
||||
status(code: number) {
|
||||
this.statusCode = code;
|
||||
return this;
|
||||
}
|
||||
json(body: unknown) {
|
||||
this.body = body;
|
||||
return this;
|
||||
}
|
||||
setHeader() {
|
||||
return this;
|
||||
}
|
||||
type() {
|
||||
return this;
|
||||
}
|
||||
send() {
|
||||
return this;
|
||||
}
|
||||
}
|
||||
|
||||
const makeReq = (method: string, tag: string): Request =>
|
||||
({
|
||||
body: { interface: 'test-iface', method },
|
||||
actor: { user: { uuid: `u-${tag}`, username: 'u' } },
|
||||
headers: { 'user-agent': tag },
|
||||
query: {},
|
||||
ip: '127.0.0.1',
|
||||
socket: { remoteAddress: '127.0.0.1' },
|
||||
}) as unknown as Request;
|
||||
|
||||
const syntheticDriver = {
|
||||
driverInterface: 'test-iface',
|
||||
driverName: 'test-driver',
|
||||
isDefault: true,
|
||||
requireSubscription: {
|
||||
methods: { paid: true, premium: ['business', 'pro'] },
|
||||
},
|
||||
onServerStart() {},
|
||||
onServerPrepareShutdown() {},
|
||||
onServerShutdown() {},
|
||||
free: async () => 'free-ok',
|
||||
paid: async () => 'paid-ok',
|
||||
premium: async () => 'premium-ok',
|
||||
};
|
||||
|
||||
const buildController = (plan: string | null) => {
|
||||
const clients = { alarm: { create: () => {} } };
|
||||
const services = plan
|
||||
? {
|
||||
metering: {
|
||||
getActorSubscription: vi.fn().mockResolvedValue({ id: plan }),
|
||||
},
|
||||
}
|
||||
: {};
|
||||
return new DriverController(
|
||||
{} as any,
|
||||
clients as any,
|
||||
{} as any,
|
||||
services as any,
|
||||
{ syntheticDriver } as any,
|
||||
);
|
||||
};
|
||||
|
||||
const call = (plan: string | null, method: string, tag: string) => {
|
||||
const handler = captureCallHandler(buildController(plan));
|
||||
const res = new StubRes();
|
||||
return runWithContext({}, () =>
|
||||
handler(makeReq(method, tag), res as unknown as Response, () => {}),
|
||||
).then(() => res);
|
||||
};
|
||||
|
||||
const expect402 = (promise: Promise<unknown>) =>
|
||||
expect(promise).rejects.toSatisfy(
|
||||
(e) =>
|
||||
isHttpError(e) &&
|
||||
(e as { statusCode: number }).statusCode === 402 &&
|
||||
(e as { legacyCode?: string }).legacyCode ===
|
||||
'subscription_required',
|
||||
);
|
||||
|
||||
describe('DriverController — per-method subscription requirement', () => {
|
||||
beforeEach(() => {
|
||||
configureRateLimit();
|
||||
});
|
||||
|
||||
it('leaves a method that declares nothing open to a free account', async () => {
|
||||
const res = await call('user_free', 'free', 'open');
|
||||
expect(res.body).toMatchObject({ success: true, result: 'free-ok' });
|
||||
});
|
||||
|
||||
it('rejects a free account on a method that requires a subscription', async () => {
|
||||
await expect402(call('user_free', 'paid', 'paid-free'));
|
||||
});
|
||||
|
||||
it('admits a subscriber on that same method', async () => {
|
||||
const res = await call('basic', 'paid', 'paid-sub');
|
||||
expect(res.body).toMatchObject({ success: true, result: 'paid-ok' });
|
||||
});
|
||||
|
||||
it('honours a per-method plan allowlist', async () => {
|
||||
await expect402(call('basic', 'premium', 'premium-basic'));
|
||||
const res = await call('pro', 'premium', 'premium-pro');
|
||||
expect(res.body).toMatchObject({ success: true, result: 'premium-ok' });
|
||||
});
|
||||
|
||||
it('enforces nothing on a deployment with no metering service', async () => {
|
||||
const res = await call(null, 'paid', 'no-metering');
|
||||
expect(res.body).toMatchObject({ success: true, result: 'paid-ok' });
|
||||
});
|
||||
});
|
||||
@@ -212,6 +212,30 @@ const makeRes = (): MockRes => {
|
||||
return res;
|
||||
};
|
||||
|
||||
/**
|
||||
* Run the call route and return the rejection, or null when it resolves.
|
||||
* Lets a test assert on *which* gate rejected without depending on how far
|
||||
* an admitted call gets afterwards.
|
||||
*/
|
||||
const callForError = async (
|
||||
routes: Captured,
|
||||
req: Request,
|
||||
actor: Actor,
|
||||
): Promise<{ statusCode?: number; legacyCode?: string } | null> => {
|
||||
try {
|
||||
await runWithContext({ actor }, () =>
|
||||
routes['POST /call'](
|
||||
req,
|
||||
makeRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
);
|
||||
return null;
|
||||
} catch (e) {
|
||||
return e as { statusCode?: number; legacyCode?: string };
|
||||
}
|
||||
};
|
||||
|
||||
const makeReq = (body: Record<string, unknown> = {}, actor?: Actor): Request =>
|
||||
({
|
||||
body,
|
||||
@@ -338,8 +362,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
it('admits a bare session ("root" token) actor on the AI drivers', async () => {
|
||||
// Privileged ("godmode") apps call AI with the user's own session
|
||||
// token, so the AI drivers no longer set `noUserSession`: the
|
||||
// session actor reaches the permission scan (403 forbidden — no
|
||||
// perms granted) instead of the credential-shape rejection.
|
||||
// session actor must get past the credential-shape gate.
|
||||
const actor = await makeUserActor();
|
||||
const req = makeReq(
|
||||
{
|
||||
@@ -349,18 +372,8 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
},
|
||||
actor,
|
||||
);
|
||||
await expect(
|
||||
runWithContext({ actor }, () =>
|
||||
routes['POST /call'](
|
||||
req,
|
||||
makeRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
const err = await callForError(routes, req, actor);
|
||||
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
|
||||
});
|
||||
|
||||
it('admits app and access-token actors on the AI drivers (they fail later on permission, not credential shape)', async () => {
|
||||
@@ -385,29 +398,15 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
},
|
||||
actor,
|
||||
);
|
||||
await expect(
|
||||
runWithContext({ actor }, () =>
|
||||
routes['POST /call'](
|
||||
req,
|
||||
makeRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
// Fresh actors hold no service:* perms, so the call still
|
||||
// 403s — but at the permission scan, proving the
|
||||
// credential-shape gate let the delegated actor through.
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
const err = await callForError(routes, req, actor);
|
||||
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
|
||||
}
|
||||
});
|
||||
|
||||
it('admits a user-scoped worker session on the AI drivers', async () => {
|
||||
// Workers deployed without an app binding authenticate as a user
|
||||
// actor whose session row is kind='worker' — they must keep their
|
||||
// AI access (they fail later on permission here, not on
|
||||
// credential shape).
|
||||
// AI access rather than be rejected for their credential shape.
|
||||
const base = await makeUserActor();
|
||||
const actor: Actor = {
|
||||
...base,
|
||||
@@ -421,24 +420,13 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
},
|
||||
actor,
|
||||
);
|
||||
await expect(
|
||||
runWithContext({ actor }, () =>
|
||||
routes['POST /call'](
|
||||
req,
|
||||
makeRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
const err = await callForError(routes, req, actor);
|
||||
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
|
||||
});
|
||||
|
||||
it('still allows bare session actors on drivers without the noUserSession flag', async () => {
|
||||
// puter-kvstore doesn't set the flag; the session actor reaches
|
||||
// the permission scan (403 forbidden — no perms granted) instead
|
||||
// of being rejected for its credential shape.
|
||||
// puter-kvstore doesn't set the flag, so the session actor must get
|
||||
// past the credential-shape gate.
|
||||
const actor = await makeUserActor();
|
||||
const req = makeReq(
|
||||
{
|
||||
@@ -448,24 +436,24 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
},
|
||||
actor,
|
||||
);
|
||||
await expect(
|
||||
runWithContext({ actor }, () =>
|
||||
routes['POST /call'](
|
||||
req,
|
||||
makeRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
const err = await callForError(routes, req, actor);
|
||||
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
|
||||
});
|
||||
|
||||
it('rejects with 403 when the actor lacks the service permission', async () => {
|
||||
// A fresh user has no service:* perms → permission.check returns
|
||||
// false → 403.
|
||||
const actor = await makeUserActor();
|
||||
// A scoped access token holds only the permissions written to its
|
||||
// own row, so it is the actor shape that can still be denied — the
|
||||
// `driver`/`service` grants every user inherits are resolved from
|
||||
// the issuer only for full-access tokens.
|
||||
const base = await makeUserActor();
|
||||
const actor: Actor = {
|
||||
...base,
|
||||
accessToken: {
|
||||
uid: `tok-${uuidv4()}`,
|
||||
issuer: base,
|
||||
fullAccess: false,
|
||||
},
|
||||
};
|
||||
const req = makeReq(
|
||||
{
|
||||
interface: 'puter-kvstore',
|
||||
@@ -474,15 +462,8 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
},
|
||||
actor,
|
||||
);
|
||||
await expect(
|
||||
runWithContext({ actor }, () =>
|
||||
routes['POST /call'](
|
||||
req,
|
||||
makeRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403 });
|
||||
const err = await callForError(routes, req, actor);
|
||||
expect(err).toMatchObject({ statusCode: 403 });
|
||||
});
|
||||
|
||||
it('returns the wrapped {success, result, service} envelope on a successful call', async () => {
|
||||
|
||||
@@ -36,7 +36,9 @@ import {
|
||||
resolveDriverMeta,
|
||||
resolveDriverMethodConcurrent,
|
||||
resolveDriverMethodRateLimit,
|
||||
resolveDriverMethodRequireSubscription,
|
||||
} from '../../drivers/meta.js';
|
||||
import { assertActorHasSubscription } from '../../services/metering/enforcement.js';
|
||||
import type { PermissionService } from '../../services/permission/PermissionService.js';
|
||||
import { PermissionUtil } from '../../services/permission/permissionUtil.js';
|
||||
import type { WithLifecycle } from '../../types';
|
||||
@@ -297,8 +299,7 @@ export class DriverController extends PuterController {
|
||||
|
||||
if (req.actor) {
|
||||
const permService = this.services.permission as unknown as
|
||||
| PermissionService
|
||||
| undefined;
|
||||
PermissionService | undefined;
|
||||
if (permService) {
|
||||
// Build via PermissionUtil.join so any `:` in a driver or
|
||||
// interface name is escaped — raw interpolation would let a
|
||||
@@ -326,6 +327,25 @@ export class DriverController extends PuterController {
|
||||
}
|
||||
}
|
||||
|
||||
// Subscriber-only methods. Declared per-driver
|
||||
// (`@Driver({ requireSubscription })`) because `/drivers/call` is a
|
||||
// single route and a route option would apply to every driver at once.
|
||||
// Checked before the rate limit — the same order the route chain uses
|
||||
// — so a caller whose plan never included the method is told that
|
||||
// rather than spending a bucket on it.
|
||||
const subscriptionRequirement = resolveDriverMethodRequireSubscription(
|
||||
driverMeta?.requireSubscription,
|
||||
method,
|
||||
);
|
||||
if (subscriptionRequirement !== undefined) {
|
||||
await assertActorHasSubscription(
|
||||
this.services.metering,
|
||||
req.actor,
|
||||
subscriptionRequirement,
|
||||
this.config,
|
||||
);
|
||||
}
|
||||
|
||||
// Per-method rate-limit and concurrent specs both live on the
|
||||
// driver's resolved meta (set by `@Driver({ rateLimit, concurrent })`
|
||||
// or imperative fields). Rate-limit is single-shot; concurrent
|
||||
|
||||
@@ -55,6 +55,8 @@ afterAll(async () => {
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
// Back to the self-hosted no-SMTP baseline the server booted with.
|
||||
delete liveConfig().email;
|
||||
});
|
||||
|
||||
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
|
||||
@@ -101,11 +103,15 @@ const makeApp = async (
|
||||
|
||||
// Feedback is only offered when the deployment can deliver it (email
|
||||
// transport configured); most tests want that baseline without asserting
|
||||
// anything about the mail itself.
|
||||
const mockEmailConfigured = () =>
|
||||
vi.spyOn(server.clients.email, 'isConfigured', 'get').mockReturnValue(
|
||||
true,
|
||||
);
|
||||
// anything about the mail itself. The service reads `config.email` — the
|
||||
// same live object every layer holds — so the helper writes it there and
|
||||
// the global afterEach clears it.
|
||||
const liveConfig = () =>
|
||||
(server.clients.email as unknown as { config: Record<string, unknown> })
|
||||
.config;
|
||||
const mockEmailConfigured = () => {
|
||||
liveConfig().email = { jsonTransport: true };
|
||||
};
|
||||
|
||||
const confirmOwnerEmail = async (userId: number) => {
|
||||
await server.clients.db.write(
|
||||
|
||||
@@ -25,7 +25,15 @@ import type { Actor } from '../../core/actor.js';
|
||||
import { Context } from '../../core/context.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { Controller, Get, Post } from '../../core/http/decorators.js';
|
||||
import { assertNormalized } from '../../services/fs/resolveNode.js';
|
||||
import {
|
||||
assertNormalized,
|
||||
isOwnersTrash,
|
||||
} from '../../services/fs/resolveNode.js';
|
||||
import {
|
||||
maskEntryPath,
|
||||
maskPathForRequest,
|
||||
resolveSharePath,
|
||||
} from '../../services/fs/sharePathMask.js';
|
||||
import type {
|
||||
PreparedBatchWrite,
|
||||
UploadedBatchWriteItem,
|
||||
@@ -144,7 +152,7 @@ export class FSController extends PuterController {
|
||||
const userId = this.#getActorUserId(req);
|
||||
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
|
||||
const requestBody = this.#withGuiMetadata(req.body, req.body);
|
||||
requestBody.fileMetadata = this.#normalizeFileMetadataPath(
|
||||
requestBody.fileMetadata = await this.#normalizeFileMetadataPath(
|
||||
req,
|
||||
requestBody.fileMetadata,
|
||||
requestBody,
|
||||
@@ -215,7 +223,7 @@ export class FSController extends PuterController {
|
||||
req.body,
|
||||
);
|
||||
normalizedRequestBody.fileMetadata =
|
||||
this.#normalizeFileMetadataPath(
|
||||
await this.#normalizeFileMetadataPath(
|
||||
req,
|
||||
normalizedRequestBody.fileMetadata,
|
||||
normalizedRequestBody,
|
||||
@@ -435,7 +443,7 @@ export class FSController extends PuterController {
|
||||
const userId = this.#getActorUserId(req);
|
||||
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
|
||||
const requestBody = this.#withGuiMetadata(req.body, req.body);
|
||||
requestBody.fileMetadata = this.#normalizeFileMetadataPath(
|
||||
requestBody.fileMetadata = await this.#normalizeFileMetadataPath(
|
||||
req,
|
||||
requestBody.fileMetadata,
|
||||
requestBody,
|
||||
@@ -449,7 +457,7 @@ export class FSController extends PuterController {
|
||||
await this.#assertWriteAccess(req, requestBody.fileMetadata, {
|
||||
pathAlreadyNormalized: true,
|
||||
});
|
||||
const normalizedPath = this.#normalizePath(
|
||||
const normalizedPath = await this.#resolveClientPath(
|
||||
requestBody.fileMetadata.path,
|
||||
);
|
||||
const uploadTracker = await this.#createUploadTracker(
|
||||
@@ -555,7 +563,7 @@ export class FSController extends PuterController {
|
||||
...parsedManifest,
|
||||
items: parsedManifest.items.map((item) => ({
|
||||
...item,
|
||||
fileMetadata: this.#normalizeFileMetadataPath(
|
||||
fileMetadata: this.#expandFileMetadataPath(
|
||||
req,
|
||||
item.fileMetadata,
|
||||
item,
|
||||
@@ -586,7 +594,12 @@ export class FSController extends PuterController {
|
||||
...item,
|
||||
fileMetadata:
|
||||
await this.#resolveAssociatedAppMetadata(
|
||||
item.fileMetadata,
|
||||
{
|
||||
...item.fileMetadata,
|
||||
path: await this.#unmaskPath(
|
||||
item.fileMetadata.path,
|
||||
),
|
||||
},
|
||||
item,
|
||||
appUidLookupCache,
|
||||
userId,
|
||||
@@ -831,7 +844,7 @@ export class FSController extends PuterController {
|
||||
req.body,
|
||||
);
|
||||
normalizedRequestBody.fileMetadata =
|
||||
this.#normalizeFileMetadataPath(
|
||||
await this.#normalizeFileMetadataPath(
|
||||
req,
|
||||
normalizedRequestBody.fileMetadata,
|
||||
normalizedRequestBody,
|
||||
@@ -994,7 +1007,7 @@ export class FSController extends PuterController {
|
||||
uuid: entry.uuid,
|
||||
uid: entry.uid ?? entry.uuid,
|
||||
parentUid: entry.parentUid ?? null,
|
||||
path: entry.path,
|
||||
path: maskEntryPath(entry),
|
||||
name: entry.name,
|
||||
isDir: entry.isDir,
|
||||
isShortcut: entry.isShortcut,
|
||||
@@ -1110,7 +1123,6 @@ export class FSController extends PuterController {
|
||||
const rootChildren = await listRootEntries(
|
||||
actor,
|
||||
this.stores.fsEntry,
|
||||
this.services.permission,
|
||||
);
|
||||
const rootSuggestions =
|
||||
await this.services.suggestedApps.getSuggestedAppsForEntries(
|
||||
@@ -1385,7 +1397,7 @@ export class FSController extends PuterController {
|
||||
// would compute a wrong parent for `~/...` inputs (e.g. dirname of
|
||||
// `/~/Documents/foo` is `/~/Documents`, not `/<username>/Documents`).
|
||||
const username = this.#getActorUsername(req);
|
||||
const path = this.#normalizePath(rawPath, username);
|
||||
const path = await this.#resolveClientPath(rawPath, username);
|
||||
if (path === '/')
|
||||
throw new HttpError(400, 'Cannot mkdir at root', {
|
||||
legacyCode: 'bad_request',
|
||||
@@ -1426,7 +1438,7 @@ export class FSController extends PuterController {
|
||||
});
|
||||
|
||||
const username = this.#getActorUsername(req);
|
||||
const path = this.#normalizePath(rawPath, username);
|
||||
const path = await this.#resolveClientPath(rawPath, username);
|
||||
if (path === '/')
|
||||
throw new HttpError(400, 'Cannot touch root', {
|
||||
legacyCode: 'bad_request',
|
||||
@@ -1467,7 +1479,8 @@ export class FSController extends PuterController {
|
||||
const entry = await this.#resolveEntryForRequest(body);
|
||||
await this.#assertAccess(actor, entry.path, 'write');
|
||||
|
||||
const renamed = await this.services.fs.rename(entry, newName);
|
||||
const userId = this.#getActorUserId(req);
|
||||
const renamed = await this.services.fs.rename(userId, entry, newName);
|
||||
this.#emitGuiItemUpdated(renamed);
|
||||
res.json(this.#toClientEntry(renamed));
|
||||
}
|
||||
@@ -1511,7 +1524,9 @@ export class FSController extends PuterController {
|
||||
await this.#resolveEntryForRequest(destinationRef);
|
||||
|
||||
await this.#assertAccess(actor, source.path, 'write');
|
||||
await this.#assertAccess(actor, destinationParent.path, 'write');
|
||||
if (!isOwnersTrash(source, destinationParent)) {
|
||||
await this.#assertAccess(actor, destinationParent.path, 'write');
|
||||
}
|
||||
|
||||
const moved = await this.services.fs.move(userId, {
|
||||
source,
|
||||
@@ -1623,7 +1638,7 @@ export class FSController extends PuterController {
|
||||
const ref = {
|
||||
path:
|
||||
rawPath !== undefined
|
||||
? mod.expandTildePath(rawPath, username)
|
||||
? await this.#resolveClientPath(rawPath, username)
|
||||
: undefined,
|
||||
uid:
|
||||
typeof source.uid === 'string'
|
||||
@@ -2178,7 +2193,8 @@ export class FSController extends PuterController {
|
||||
// the ActorUser type. Access via the escape hatch until a proper
|
||||
// storage-quota mechanism is in place.
|
||||
const actorUser = req.actor?.user as
|
||||
Record<string, unknown> | undefined;
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
|
||||
const candidates = [
|
||||
this.#toStorageCapacityCandidate(actorUser?.free_storage),
|
||||
@@ -2191,6 +2207,18 @@ export class FSController extends PuterController {
|
||||
return Math.max(...candidates);
|
||||
}
|
||||
|
||||
/**
|
||||
* `#normalizePath`, plus turning a masked share path back into the owner's
|
||||
* real one. Every client-authored path goes through here.
|
||||
*/
|
||||
async #resolveClientPath(path: string, username?: string): Promise<string> {
|
||||
return resolveSharePath(
|
||||
this.stores.fsEntry,
|
||||
Context.get('actor'),
|
||||
this.#normalizePath(path, username),
|
||||
);
|
||||
}
|
||||
|
||||
#normalizePath(path: string, username?: string): string {
|
||||
const trimmedPath = path.trim();
|
||||
if (trimmedPath.length === 0) {
|
||||
@@ -2221,7 +2249,7 @@ export class FSController extends PuterController {
|
||||
return normalizedPath;
|
||||
}
|
||||
|
||||
#normalizeFileMetadataPath(
|
||||
#expandFileMetadataPath(
|
||||
req: Request,
|
||||
fileMetadata: FSEntryWriteInput | undefined,
|
||||
fallbackSource?: unknown,
|
||||
@@ -2236,13 +2264,38 @@ export class FSController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
const username = this.#getActorUsername(req);
|
||||
return {
|
||||
...resolvedFileMetadata,
|
||||
path: this.#normalizePath(resolvedFileMetadata.path, username),
|
||||
path: this.#normalizePath(
|
||||
resolvedFileMetadata.path,
|
||||
this.#getActorUsername(req),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/** As above, plus turning a masked share path into the owner's real one. */
|
||||
async #normalizeFileMetadataPath(
|
||||
req: Request,
|
||||
fileMetadata: FSEntryWriteInput | undefined,
|
||||
fallbackSource?: unknown,
|
||||
): Promise<FSEntryWriteInput> {
|
||||
const expanded = this.#expandFileMetadataPath(
|
||||
req,
|
||||
fileMetadata,
|
||||
fallbackSource,
|
||||
);
|
||||
return { ...expanded, path: await this.#unmaskPath(expanded.path) };
|
||||
}
|
||||
|
||||
/** The un-masking half, for callers that already expanded the path. */
|
||||
async #unmaskPath(path: string): Promise<string> {
|
||||
return resolveSharePath(
|
||||
this.stores.fsEntry,
|
||||
Context.get('actor'),
|
||||
path,
|
||||
);
|
||||
}
|
||||
|
||||
#extractGuiMetadata(
|
||||
input: unknown,
|
||||
fallback: WriteGuiMetadata | undefined,
|
||||
@@ -2321,7 +2374,7 @@ export class FSController extends PuterController {
|
||||
}
|
||||
const normalizedFileMetadata = options?.pathAlreadyNormalized
|
||||
? fileMetadata
|
||||
: this.#normalizeFileMetadataPath(req, fileMetadata);
|
||||
: await this.#normalizeFileMetadataPath(req, fileMetadata);
|
||||
if (!normalizedFileMetadata) {
|
||||
throw new HttpError(400, 'Missing path', {
|
||||
legacyCode: 'bad_request',
|
||||
@@ -2484,14 +2537,19 @@ export class FSController extends PuterController {
|
||||
requestBody: SignedWriteRequest,
|
||||
response: SignedWriteResponse,
|
||||
): Promise<void> {
|
||||
const normalizedPath = this.#normalizePath(
|
||||
const normalizedPath = await this.#resolveClientPath(
|
||||
requestBody.fileMetadata.path,
|
||||
);
|
||||
// This goes to the *acting* user, not the owner, so a recipient writing
|
||||
// into a shared folder would otherwise be handed the owner's real path
|
||||
// — the one thing the masking exists to withhold. There is no row yet,
|
||||
// so the mask is built from the path the caller already named.
|
||||
const publishedPath = maskPathForRequest(normalizedPath);
|
||||
const pendingResponse = {
|
||||
id: response.objectKey,
|
||||
uid: response.objectKey,
|
||||
uuid: response.objectKey,
|
||||
path: normalizedPath,
|
||||
path: publishedPath,
|
||||
name: pathPosix.basename(normalizedPath),
|
||||
is_dir: false,
|
||||
content_type: response.contentType,
|
||||
@@ -2774,7 +2832,10 @@ export class FSController extends PuterController {
|
||||
user_id: userId,
|
||||
userId: userId,
|
||||
item_uid: itemUid,
|
||||
item_path: itemPath,
|
||||
// Progress is reported to the acting user, who is not the
|
||||
// owner when the write lands in a shared folder — publish
|
||||
// the path in the form they addressed it by.
|
||||
item_path: maskPathForRequest(itemPath),
|
||||
...this.#toEventGuiMetadata(guiMetadata),
|
||||
},
|
||||
},
|
||||
|
||||
@@ -1166,6 +1166,115 @@ describe('LegacyFSController.move', () => {
|
||||
};
|
||||
expect(removedPayload.response?.uid).toBe(replaced.uuid);
|
||||
});
|
||||
|
||||
it('lets a share recipient trash an item into the owner’s trash', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const ownerName = owner.actor.user!.username!;
|
||||
const sharedPath = `/${ownerName}/Documents/Contents`;
|
||||
|
||||
await withActor(owner.actor, () =>
|
||||
controller.mkdir(
|
||||
makeReq({ body: { path: sharedPath }, actor: owner.actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
await withActor(owner.actor, () =>
|
||||
controller.touch(
|
||||
makeReq({
|
||||
body: { path: `${sharedPath}/note.txt` },
|
||||
actor: owner.actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
const shared = (await server.stores.fsEntry.getEntryByPath(sharedPath))!;
|
||||
await server.services.acl.setUserUser(
|
||||
owner.actor,
|
||||
holder.actor,
|
||||
{
|
||||
path: shared.path,
|
||||
resolveAncestors: () =>
|
||||
server.services.fs.getAncestorChain(shared.path),
|
||||
},
|
||||
'write',
|
||||
);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(holder.actor, () =>
|
||||
controller.move(
|
||||
makeReq({
|
||||
body: {
|
||||
source: `${sharedPath}/note.txt`,
|
||||
destination: `/${ownerName}/Trash`,
|
||||
},
|
||||
actor: holder.actor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
);
|
||||
|
||||
// The move landed in the owner's trash, but the recipient is told so
|
||||
// in masked form — the owner's real layout stays theirs.
|
||||
const body = captured.body as { moved: { uid: string; path: string } };
|
||||
expect(body.moved.path).toBe(
|
||||
`/${ownerName}/${body.moved.uid}/note.txt`,
|
||||
);
|
||||
const moved = await server.stores.fsEntry.getEntryByUuid(
|
||||
body.moved.uid,
|
||||
);
|
||||
expect(moved!.path).toBe(`/${ownerName}/Trash/note.txt`);
|
||||
});
|
||||
|
||||
it('refuses a share recipient moving an item into their own trash', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const ownerName = owner.actor.user!.username!;
|
||||
const holderName = holder.actor.user!.username!;
|
||||
const sharedPath = `/${ownerName}/Documents/Shared`;
|
||||
|
||||
await withActor(owner.actor, () =>
|
||||
controller.mkdir(
|
||||
makeReq({ body: { path: sharedPath }, actor: owner.actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
await withActor(owner.actor, () =>
|
||||
controller.touch(
|
||||
makeReq({
|
||||
body: { path: `${sharedPath}/theirs.txt` },
|
||||
actor: owner.actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
const shared = (await server.stores.fsEntry.getEntryByPath(sharedPath))!;
|
||||
await server.services.acl.setUserUser(
|
||||
owner.actor,
|
||||
holder.actor,
|
||||
{
|
||||
path: shared.path,
|
||||
resolveAncestors: () =>
|
||||
server.services.fs.getAncestorChain(shared.path),
|
||||
},
|
||||
'write',
|
||||
);
|
||||
|
||||
await expect(
|
||||
withActor(holder.actor, () =>
|
||||
controller.move(
|
||||
makeReq({
|
||||
body: {
|
||||
source: `${sharedPath}/theirs.txt`,
|
||||
destination: `/${holderName}/Trash`,
|
||||
},
|
||||
actor: holder.actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403 });
|
||||
});
|
||||
});
|
||||
|
||||
// ── search ──────────────────────────────────────────────────────────
|
||||
@@ -1586,8 +1695,11 @@ describe('LegacyFSController.writeFile (write IDOR)', () => {
|
||||
),
|
||||
);
|
||||
|
||||
// The write landed on the signed file …
|
||||
expect((captured.body as { path?: string }).path).toBe(target);
|
||||
// The write landed on the signed file — reported masked, since the
|
||||
// attacker doesn't own it …
|
||||
expect((captured.body as { path?: string }).path).toBe(
|
||||
`/${victim.actor.user!.username}/${entry!.uuid}/secret.txt`,
|
||||
);
|
||||
// … and never created the attacker-named sibling.
|
||||
const siblingEntry =
|
||||
await server.stores.fsEntry.getEntryByPath(sibling);
|
||||
@@ -1982,6 +2094,50 @@ describe('LegacyFSController.requestAppRootDir', () => {
|
||||
expect(body.path).toBe(`/${username}/AppData/${appUid}`);
|
||||
expect(body.is_dir).toBe(true);
|
||||
});
|
||||
|
||||
// `check: true` answers the question without acting on the answer, so that
|
||||
// asking whether the access is held is not itself a filesystem write.
|
||||
it('answers check:true without creating the directory', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const username = actor.user!.username!;
|
||||
const appUid = 'app-check-only';
|
||||
const appActor = makeActor({ ...actor, app: { uid: appUid } });
|
||||
const rootPath = `/${username}/AppData/${appUid}`;
|
||||
const { res, captured } = makeRes();
|
||||
|
||||
await withActor(appActor, () =>
|
||||
controller.requestAppRootDir(
|
||||
makeReq({
|
||||
body: { app_uid: appUid, access: 'read', check: true },
|
||||
actor: appActor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
);
|
||||
|
||||
expect(captured.body).toEqual({ allowed: true });
|
||||
expect(
|
||||
await server.stores.fsEntry.getEntryByPath(rootPath),
|
||||
).toBeFalsy();
|
||||
});
|
||||
|
||||
// The guard runs first either way, so a caller that may not claim it is
|
||||
// refused rather than told it is allowed.
|
||||
it('still refuses check:true from a caller that is not the app', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
controller.requestAppRootDir(
|
||||
makeReq({
|
||||
body: { app_uid: 'app-xyz', check: true },
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403 });
|
||||
});
|
||||
});
|
||||
|
||||
// ── checkAppAcl ─────────────────────────────────────────────────────
|
||||
|
||||
@@ -37,7 +37,11 @@ import type { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import type { ACLService } from '../../services/acl/ACLService.js';
|
||||
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
|
||||
import type { SignedFile } from '../../util/fileSigning.js';
|
||||
import { verifySignature } from '../../util/fileSigning.js';
|
||||
import {
|
||||
NON_OWNER_SIGNATURE_TTL_SECONDS,
|
||||
verifySignature,
|
||||
} from '../../util/fileSigning.js';
|
||||
import { maskEntryPath } from '../../services/fs/sharePathMask.js';
|
||||
import {
|
||||
buildHostedBackingDenial,
|
||||
hostedIndexUrlBackingIsUnavailable,
|
||||
@@ -66,6 +70,7 @@ import {
|
||||
asRecord,
|
||||
assertAccess,
|
||||
assertCanCreate,
|
||||
assertCanMoveInto,
|
||||
getBoolean,
|
||||
getString,
|
||||
loadLegacyAssociatedApps,
|
||||
@@ -484,7 +489,6 @@ export class LegacyFSController extends PuterController {
|
||||
const rootChildren = await listRootEntries(
|
||||
actor,
|
||||
this.stores.fsEntry,
|
||||
this.services.permission,
|
||||
);
|
||||
const rootSuggestions =
|
||||
await this.services.suggestedApps.getSuggestedAppsForEntries(
|
||||
@@ -790,12 +794,12 @@ export class LegacyFSController extends PuterController {
|
||||
source.path,
|
||||
'write',
|
||||
);
|
||||
await assertAccess(
|
||||
await assertCanMoveInto(
|
||||
this.services.acl,
|
||||
this.services.fs,
|
||||
actor,
|
||||
destinationParent.path,
|
||||
'write',
|
||||
source,
|
||||
destinationParent,
|
||||
);
|
||||
|
||||
// The v1 wire contract reports the entry an overwrite replaced
|
||||
@@ -944,7 +948,8 @@ export class LegacyFSController extends PuterController {
|
||||
'write',
|
||||
);
|
||||
|
||||
const renamed = await this.services.fs.rename(entry, newName);
|
||||
const userId = this.#getActorUserId(req);
|
||||
const renamed = await this.services.fs.rename(userId, entry, newName);
|
||||
await this.#emitGuiEvent('outer.gui.item.updated', renamed);
|
||||
res.json(await toLegacyEntry(this.clients.event, renamed));
|
||||
};
|
||||
@@ -1331,6 +1336,11 @@ export class LegacyFSController extends PuterController {
|
||||
);
|
||||
let finalAction: 'read' | 'write' = 'read';
|
||||
if (action === 'write') {
|
||||
// The real path, not the masked one: ACL reads the path
|
||||
// string itself for its short-circuits (own home, an app's
|
||||
// AppData under another user), and a mask hides the
|
||||
// `AppData/<appUid>` shape those match on. Nothing here
|
||||
// publishes the descriptor, so there is nothing to hide.
|
||||
const writeOk = await this.services.acl.check(
|
||||
actor,
|
||||
{
|
||||
@@ -1354,12 +1364,20 @@ export class LegacyFSController extends PuterController {
|
||||
);
|
||||
}
|
||||
|
||||
const signed = signEntry(entry, signingCfg);
|
||||
const signed = signEntry(entry, signingCfg, {
|
||||
actorUserId: actor.user?.id,
|
||||
});
|
||||
if (finalAction !== 'write') {
|
||||
const { write_url: _, ...rest } = signed;
|
||||
result.signatures.push({ ...rest, path: entry.path });
|
||||
result.signatures.push({
|
||||
...rest,
|
||||
path: maskEntryPath(entry),
|
||||
});
|
||||
} else {
|
||||
result.signatures.push({ ...signed, path: entry.path });
|
||||
result.signatures.push({
|
||||
...signed,
|
||||
path: maskEntryPath(entry),
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Silently skip unresolvable items.
|
||||
@@ -1465,8 +1483,10 @@ export class LegacyFSController extends PuterController {
|
||||
'outer.gui.item.added',
|
||||
uploadResult.fsEntry,
|
||||
);
|
||||
const signed = signEntry(uploadResult.fsEntry, signingCfg);
|
||||
res.json({ ...signed, path: uploadResult.fsEntry.path });
|
||||
const signed = signEntry(uploadResult.fsEntry, signingCfg, {
|
||||
actorUserId: callerActor.user?.id,
|
||||
});
|
||||
res.json({ ...signed, path: maskEntryPath(uploadResult.fsEntry) });
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1494,7 +1514,12 @@ export class LegacyFSController extends PuterController {
|
||||
dedupeName: true,
|
||||
});
|
||||
await this.#emitGuiEvent('outer.gui.item.added', entry);
|
||||
res.json({ ...signEntry(entry, signingCfg), path: entry.path });
|
||||
res.json({
|
||||
...signEntry(entry, signingCfg, {
|
||||
actorUserId: callerActor.user?.id,
|
||||
}),
|
||||
path: maskEntryPath(entry),
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (operation === 'rename') {
|
||||
@@ -1511,9 +1536,18 @@ export class LegacyFSController extends PuterController {
|
||||
targetEntry.path,
|
||||
'write',
|
||||
);
|
||||
const renamed = await this.services.fs.rename(targetEntry, newName);
|
||||
const renamed = await this.services.fs.rename(
|
||||
userId,
|
||||
targetEntry,
|
||||
newName,
|
||||
);
|
||||
await this.#emitGuiEvent('outer.gui.item.updated', renamed);
|
||||
res.json({ ...signEntry(renamed, signingCfg), path: renamed.path });
|
||||
res.json({
|
||||
...signEntry(renamed, signingCfg, {
|
||||
actorUserId: callerActor.user?.id,
|
||||
}),
|
||||
path: maskEntryPath(renamed),
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (operation === 'delete' || operation === 'trash') {
|
||||
@@ -1581,7 +1615,12 @@ export class LegacyFSController extends PuterController {
|
||||
? { old_path: targetEntry.path }
|
||||
: undefined,
|
||||
);
|
||||
res.json({ ...signEntry(result, signingCfg), path: result.path });
|
||||
res.json({
|
||||
...signEntry(result, signingCfg, {
|
||||
actorUserId: callerActor.user?.id,
|
||||
}),
|
||||
path: maskEntryPath(result),
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1643,11 +1682,18 @@ export class LegacyFSController extends PuterController {
|
||||
// Directory: return a signed listing of direct children.
|
||||
// The caller only proved read access, so strip write_url from
|
||||
// each child to prevent privilege escalation via /writeFile.
|
||||
//
|
||||
// Child paths are the owner's real ones — there is no actor here to
|
||||
// mask for. Accepted: a signature is minted and handed out by the
|
||||
// owner, so the layout it reveals is theirs to reveal, and the child
|
||||
// signatures expire; the paths are the only part that outlives them.
|
||||
if (entry.isDir) {
|
||||
const children = await this.services.fs.listDirectory(entry.uuid);
|
||||
const signedChildren = children.map((child) => {
|
||||
const { write_url: _, ...rest } = signEntry(child, signingCfg);
|
||||
return { ...rest, path: child.path };
|
||||
const { write_url: _, ...rest } = signEntry(child, signingCfg, {
|
||||
ttlSeconds: NON_OWNER_SIGNATURE_TTL_SECONDS,
|
||||
});
|
||||
return { ...rest, path: maskEntryPath(child) };
|
||||
});
|
||||
res.json(signedChildren);
|
||||
return;
|
||||
@@ -1741,6 +1787,7 @@ export class LegacyFSController extends PuterController {
|
||||
// read. `/writeFile`'s ACL re-check would still block the write, but
|
||||
// returning `write_url` to a read-only caller is the same
|
||||
// privilege-leak shape that `/sign` and `/readdir` strip.
|
||||
// Real path — see the note on the other `acl.check` above.
|
||||
const writeOk = await this.services.acl.check(
|
||||
actor,
|
||||
{
|
||||
@@ -1754,7 +1801,7 @@ export class LegacyFSController extends PuterController {
|
||||
const suggested =
|
||||
(await this.services.suggestedApps?.getSuggestedApps({
|
||||
name: entry.name,
|
||||
path: entry.path,
|
||||
path: maskEntryPath(entry),
|
||||
})) ?? [];
|
||||
|
||||
let token: string | null = null;
|
||||
@@ -1777,12 +1824,14 @@ export class LegacyFSController extends PuterController {
|
||||
}
|
||||
|
||||
const signingCfg = signingConfigFromAppConfig(this.config);
|
||||
const signed = signEntry(entry, signingCfg);
|
||||
const signed = signEntry(entry, signingCfg, {
|
||||
actorUserId: actor.user?.id,
|
||||
});
|
||||
const signature = writeOk
|
||||
? { ...signed, path: entry.path }
|
||||
? { ...signed, path: maskEntryPath(entry) }
|
||||
: (() => {
|
||||
const { write_url: _, ...rest } = signed;
|
||||
return { ...rest, path: entry.path };
|
||||
return { ...rest, path: maskEntryPath(entry) };
|
||||
})();
|
||||
res.json({
|
||||
signature,
|
||||
@@ -1794,6 +1843,12 @@ export class LegacyFSController extends PuterController {
|
||||
/**
|
||||
* POST /auth/request-app-root-dir — an app-under-user requests stat on its
|
||||
* own app root directory. The app must own itself.
|
||||
*
|
||||
* `check: true` answers whether the caller may claim it and stops there,
|
||||
* for callers asking the question rather than acting on the answer —
|
||||
* otherwise `puter.perms.check('appRootDir', …)` would provision a
|
||||
* directory just by being asked. Answering is the whole response: a caller
|
||||
* that may not claim it gets the 403 below either way.
|
||||
*/
|
||||
requestAppRootDir = async (req: Request, res: Response): Promise<void> => {
|
||||
const actor = this.#requireActor(req);
|
||||
@@ -1820,6 +1875,11 @@ export class LegacyFSController extends PuterController {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
|
||||
if (getBoolean(body, 'check')) {
|
||||
res.json({ allowed: true });
|
||||
return;
|
||||
}
|
||||
|
||||
const rootPath = `/${username}/AppData/${appUid}`;
|
||||
// Auto-create the AppData/<uid> tree on first call.
|
||||
const entry = await this.services.fs.mkdir(userId, {
|
||||
@@ -2296,12 +2356,12 @@ export class LegacyFSController extends PuterController {
|
||||
source.path,
|
||||
'write',
|
||||
);
|
||||
await assertAccess(
|
||||
await assertCanMoveInto(
|
||||
this.services.acl,
|
||||
this.services.fs,
|
||||
actor,
|
||||
destinationParent.path,
|
||||
'write',
|
||||
source,
|
||||
destinationParent,
|
||||
);
|
||||
const moved = await this.services.fs.move(userId, {
|
||||
source,
|
||||
|
||||
@@ -24,11 +24,13 @@ import {
|
||||
getBoolean,
|
||||
getString,
|
||||
loadLegacyAssociatedApps,
|
||||
signEntry,
|
||||
signEntryThumbnail,
|
||||
signingConfigFromAppConfig,
|
||||
toLegacyEntry,
|
||||
} from './legacyFsHelpers.js';
|
||||
import type { FSEntry } from '../../stores/fs/FSEntry.js';
|
||||
import { NON_OWNER_SIGNATURE_TTL_SECONDS } from '../../util/fileSigning.js';
|
||||
|
||||
const entryWithApp = (associatedAppId: number): FSEntry =>
|
||||
({ associatedAppId }) as unknown as FSEntry;
|
||||
@@ -444,3 +446,48 @@ describe('loadLegacyAssociatedApps short-circuit', () => {
|
||||
expect(seen).toEqual([[4, 5]]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('signEntry', () => {
|
||||
const cfg = { secret: 'test-secret', apiBaseUrl: 'https://api.test' };
|
||||
const entry = {
|
||||
uuid: 'e-1',
|
||||
name: 'x.txt',
|
||||
isDir: false,
|
||||
size: 1,
|
||||
accessed: 0,
|
||||
modified: 0,
|
||||
created: 0,
|
||||
userId: 7,
|
||||
};
|
||||
// signFile ceils its own clock read, so allow a second of slack.
|
||||
const inSeconds = (expires: number) =>
|
||||
expires - Math.ceil(Date.now() / 1000);
|
||||
|
||||
it('leaves an owner’s signature effectively permanent', () => {
|
||||
const signed = signEntry(entry, cfg, { actorUserId: 7 });
|
||||
expect(inSeconds(signed.expires)).toBeGreaterThan(
|
||||
NON_OWNER_SIGNATURE_TTL_SECONDS * 10,
|
||||
);
|
||||
});
|
||||
|
||||
it('expires a signature over someone else’s entry', () => {
|
||||
const signed = signEntry(entry, cfg, { actorUserId: 8 });
|
||||
const ttl = inSeconds(signed.expires);
|
||||
expect(ttl).toBeLessThanOrEqual(NON_OWNER_SIGNATURE_TTL_SECONDS);
|
||||
expect(ttl).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('keeps the old default when no actor is supplied', () => {
|
||||
expect(inSeconds(signEntry(entry, cfg).expires)).toBeGreaterThan(
|
||||
NON_OWNER_SIGNATURE_TTL_SECONDS * 10,
|
||||
);
|
||||
});
|
||||
|
||||
it('honours an explicit ttl over the ownership check', () => {
|
||||
const signed = signEntry(entry, cfg, {
|
||||
actorUserId: 7,
|
||||
ttlSeconds: 30,
|
||||
});
|
||||
expect(inSeconds(signed.expires)).toBeLessThanOrEqual(30);
|
||||
});
|
||||
});
|
||||
@@ -30,10 +30,16 @@ import { HttpError } from '../../core/http/HttpError.js';
|
||||
import {
|
||||
resolveNode,
|
||||
normalizeAbsolutePath,
|
||||
isOwnersTrash,
|
||||
joinChildPath,
|
||||
expandTildePath,
|
||||
} from '../../services/fs/resolveNode.js';
|
||||
import {
|
||||
maskEntryPath,
|
||||
resolveSharePath,
|
||||
} from '../../services/fs/sharePathMask.js';
|
||||
import {
|
||||
NON_OWNER_SIGNATURE_TTL_SECONDS,
|
||||
signFile,
|
||||
type SigningConfig,
|
||||
type SignedFile,
|
||||
@@ -93,6 +99,22 @@ export function getBoolean(
|
||||
// username, read from the ALS-backed Context. Legacy clients send tilde-
|
||||
// rooted paths (e.g. `~/AppData/<app-uid>/...`); FSController does the same
|
||||
// expansion via its own `#normalizePath` helper.
|
||||
/**
|
||||
* Everything a client-authored path needs before it names a row: `~` expanded,
|
||||
* and a masked share path turned back into the owner's real one.
|
||||
*/
|
||||
export async function expandClientPath(
|
||||
fsEntryStore: FSEntryStore,
|
||||
raw: string,
|
||||
username?: string,
|
||||
): Promise<string> {
|
||||
return resolveSharePath(
|
||||
fsEntryStore,
|
||||
Context.get('actor'),
|
||||
expandTildePath(raw, username),
|
||||
);
|
||||
}
|
||||
|
||||
export async function resolveV1Selector(
|
||||
fsEntryStore: FSEntryStore,
|
||||
raw: unknown,
|
||||
@@ -106,7 +128,7 @@ export async function resolveV1Selector(
|
||||
if (typeof raw === 'string') {
|
||||
const isPath = raw.startsWith('/') || raw.startsWith('~');
|
||||
const ref = isPath
|
||||
? { path: expandTildePath(raw, username) }
|
||||
? { path: await expandClientPath(fsEntryStore, raw, username) }
|
||||
: { uid: raw };
|
||||
const entry = await resolveNode(fsEntryStore, ref, { required: true });
|
||||
if (!entry)
|
||||
@@ -138,7 +160,7 @@ export async function resolveV1Selector(
|
||||
const ref = {
|
||||
path:
|
||||
rawPath !== undefined
|
||||
? expandTildePath(rawPath, username)
|
||||
? await expandClientPath(fsEntryStore, rawPath, username)
|
||||
: undefined,
|
||||
uid:
|
||||
typeof record.uid === 'string'
|
||||
@@ -260,6 +282,24 @@ export async function assertCanCreate(
|
||||
await assertAccess(aclService, fsService, actor, parentForCheck, 'write');
|
||||
}
|
||||
|
||||
/** `write` on the destination parent, unless it is the entry's own Trash. */
|
||||
export async function assertCanMoveInto(
|
||||
aclService: ACLService,
|
||||
fsService: FSService,
|
||||
actor: Actor,
|
||||
source: FSEntry,
|
||||
destinationParent: FSEntry,
|
||||
): Promise<void> {
|
||||
if (isOwnersTrash(source, destinationParent)) return;
|
||||
await assertAccess(
|
||||
aclService,
|
||||
fsService,
|
||||
actor,
|
||||
destinationParent.path,
|
||||
'write',
|
||||
);
|
||||
}
|
||||
|
||||
// -- Response shaping ------------------------------------------------
|
||||
|
||||
type AppRowLookup = {
|
||||
@@ -411,7 +451,10 @@ export async function toLegacyEntry(
|
||||
appsById?: Map<number, Record<string, unknown>>;
|
||||
} = {},
|
||||
): Promise<Record<string, unknown>> {
|
||||
const dirname = pathPosix.dirname(entry.path);
|
||||
// Someone else's entry is published under its masked path; the owner's
|
||||
// real one, and everything above the share, stays server-side.
|
||||
const publishedPath = maskEntryPath(entry);
|
||||
const dirname = pathPosix.dirname(publishedPath);
|
||||
const mimeType = fsEntryMimeType(entry);
|
||||
|
||||
const pathComponents = entry.path.split('/');
|
||||
@@ -424,7 +467,7 @@ export async function toLegacyEntry(
|
||||
uuid: entry.uuid,
|
||||
parent_id: entry.parentUid,
|
||||
parent_uid: entry.parentUid,
|
||||
path: entry.path,
|
||||
path: publishedPath,
|
||||
dirname,
|
||||
dirpath: dirname,
|
||||
name: entry.name,
|
||||
@@ -520,7 +563,13 @@ export function signingConfigFromAppConfig(config: IConfig): SigningConfig {
|
||||
return { secret, apiBaseUrl };
|
||||
}
|
||||
|
||||
/** Convenience wrapper: turn an FSEntry into a signed-file response object. */
|
||||
/**
|
||||
* Convenience wrapper: turn an FSEntry into a signed-file response object.
|
||||
*
|
||||
* Pass `actorUserId` so a signature over someone else's entry — a shared file —
|
||||
* expires rather than outliving the share (see
|
||||
* NON_OWNER_SIGNATURE_TTL_SECONDS).
|
||||
*/
|
||||
export function signEntry(
|
||||
entry: {
|
||||
uuid: string;
|
||||
@@ -530,8 +579,19 @@ export function signEntry(
|
||||
accessed: number | null;
|
||||
modified: number;
|
||||
created: number | null;
|
||||
userId?: number;
|
||||
},
|
||||
config: SigningConfig,
|
||||
opts: { actorUserId?: number; ttlSeconds?: number } = {},
|
||||
): SignedFile {
|
||||
return signFile(entry as Parameters<typeof signFile>[0], config);
|
||||
const isForeign =
|
||||
typeof opts.actorUserId === 'number' &&
|
||||
typeof entry.userId === 'number' &&
|
||||
entry.userId !== opts.actorUserId;
|
||||
const ttlSeconds =
|
||||
opts.ttlSeconds ??
|
||||
(isForeign ? NON_OWNER_SIGNATURE_TTL_SECONDS : undefined);
|
||||
return signFile(entry as Parameters<typeof signFile>[0], config, {
|
||||
...(ttlSeconds === undefined ? {} : { ttlSeconds }),
|
||||
});
|
||||
}
|
||||
@@ -64,8 +64,7 @@ export class NotificationController extends PuterController {
|
||||
});
|
||||
|
||||
const notifService = this.services.notification as unknown as
|
||||
| NotificationService
|
||||
| undefined;
|
||||
NotificationService | undefined;
|
||||
if (notifService?.markAcknowledged) {
|
||||
await notifService.markAcknowledged(uid, userId);
|
||||
} else {
|
||||
@@ -115,8 +114,7 @@ export class NotificationController extends PuterController {
|
||||
});
|
||||
|
||||
const notifService = this.services.notification as unknown as
|
||||
| NotificationService
|
||||
| undefined;
|
||||
NotificationService | undefined;
|
||||
if (notifService?.markShown) {
|
||||
await notifService.markShown(uid, userId);
|
||||
} else {
|
||||
|
||||
@@ -661,6 +661,7 @@ if (window.opener) {
|
||||
provider: string,
|
||||
userinfo: { sub: string; email?: unknown; [k: string]: unknown },
|
||||
referrer?: string | null,
|
||||
attempt = 0,
|
||||
): Promise<
|
||||
| { error: string; code?: string; requestCode?: string }
|
||||
| {
|
||||
@@ -679,8 +680,12 @@ if (window.opener) {
|
||||
const claimedEmail =
|
||||
typeof userinfo.email === 'string' ? userinfo.email : null;
|
||||
if (claimedEmail) {
|
||||
const byEmail =
|
||||
await this.services.oidc.findUserByEmail(claimedEmail);
|
||||
// On the retry after a lost race, read the primary — the winning
|
||||
// row may be younger than the replica snapshot.
|
||||
const byEmail = await this.services.oidc.findUserByEmail(
|
||||
claimedEmail,
|
||||
{ force: attempt > 0 },
|
||||
);
|
||||
if (byEmail) {
|
||||
if (!byEmail.email_confirmed) {
|
||||
return {
|
||||
@@ -707,6 +712,18 @@ if (window.opener) {
|
||||
userinfo as { sub: string; email?: string },
|
||||
referrer,
|
||||
);
|
||||
// A concurrent callback (a second tab, a provider retry) created the
|
||||
// account between step 2 and the insert. Nothing went wrong for the
|
||||
// user — start over and we'll find the winner at step 1 or 2. One retry
|
||||
// only: a second miss means something other than a race is going on.
|
||||
if (outcome.raced && attempt === 0) {
|
||||
return this.#resolveOrCreateOIDCUser(
|
||||
provider,
|
||||
userinfo,
|
||||
referrer,
|
||||
attempt + 1,
|
||||
);
|
||||
}
|
||||
if (!outcome.success || !outcome.user) {
|
||||
return {
|
||||
error: outcome.error ?? 'Account creation failed.',
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
|
||||
/**
|
||||
* The vendor-compatible wire routes are subscriber-only, over real HTTP.
|
||||
*
|
||||
* The gate's decision is unit-tested in
|
||||
* `services/metering/enforcement.test.ts`; what this covers is that the route
|
||||
* option reaches the middleware chain on these four routes, that it sits behind
|
||||
* the credential gates (a session token is still turned away for being the
|
||||
* wrong credential, not for the plan), and that the model catalogue stays open
|
||||
* — clients fetch it before they have any reason to have authenticated at all.
|
||||
*/
|
||||
describe('AI wire routes require a subscription', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
const WIRE_ROUTES = [
|
||||
'/puterai/openai/v1/chat/completions',
|
||||
'/puterai/openai/v1/completions',
|
||||
'/puterai/openai/v1/responses',
|
||||
'/puterai/anthropic/v1/messages',
|
||||
];
|
||||
|
||||
/**
|
||||
* The plan the test user is on. A single resolver reads this, since
|
||||
* resolvers are registered for the life of the service and can't be taken
|
||||
* back off — flipping a variable is what keeps one test's plan out of the
|
||||
* next one's.
|
||||
*/
|
||||
let plan: string | null = null;
|
||||
let userUuid = '';
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
const row = await env.server.stores.user.getByUsername(
|
||||
env.users.user.username,
|
||||
);
|
||||
userUuid = row!.uuid;
|
||||
// Paid plans are registered by an extension in production, so a
|
||||
// resolver alone resolves to nothing here — the policy has to exist
|
||||
// for the id to be found.
|
||||
env.server.services.metering.registerPolicy({
|
||||
id: 'business',
|
||||
monthUsageAllowance: 45 * 1_000_000 * 100,
|
||||
monthlyStorageAllowance: 1024 ** 3,
|
||||
});
|
||||
env.server.services.metering.registerSubscriptionResolver((actor) =>
|
||||
actor.user?.uuid === userUuid ? plan : null,
|
||||
);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
const call = (route: string, token: string) =>
|
||||
fetch(new URL(route, env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Authorization: `Bearer ${token}`,
|
||||
Origin: env.apiOrigin,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: 'fake',
|
||||
provider: 'fake-chat',
|
||||
prompt: 'hi',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
max_tokens: 16,
|
||||
}),
|
||||
});
|
||||
|
||||
/** Put the test user on a plan for the duration of one block. */
|
||||
const withSubscription = async <T>(
|
||||
id: string | null,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<T> => {
|
||||
const metering = env.server.services.metering;
|
||||
plan = id;
|
||||
metering.invalidateActorSubscription(userUuid);
|
||||
try {
|
||||
return await fn();
|
||||
} finally {
|
||||
plan = null;
|
||||
metering.invalidateActorSubscription(userUuid);
|
||||
}
|
||||
};
|
||||
|
||||
it('turns a free account away with 402 subscription_required', async () => {
|
||||
for (const route of WIRE_ROUTES) {
|
||||
const res = await call(route, env.users.user.apiToken);
|
||||
expect(res.status, route).toBe(402);
|
||||
expect(await res.json(), route).toMatchObject({
|
||||
code: 'subscription_required',
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('lets a subscriber past the gate', async () => {
|
||||
await withSubscription('business', async () => {
|
||||
for (const route of WIRE_ROUTES) {
|
||||
const res = await call(route, env.users.user.apiToken);
|
||||
// Whether the fake provider answers is the controller's
|
||||
// business; all this asserts is that the plan gate is not
|
||||
// what stopped the request.
|
||||
expect(res.status, route).not.toBe(402);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
it('answers a session token on credential shape, not plan', async () => {
|
||||
// Credential gates run first, so a caller with the wrong token shape
|
||||
// never learns anything about the account's plan.
|
||||
const res = await call(WIRE_ROUTES[0]!, env.users.user.token);
|
||||
expect(res.status).toBe(403);
|
||||
expect(await res.json()).toMatchObject({
|
||||
code: 'app_or_api_token_required',
|
||||
});
|
||||
});
|
||||
|
||||
it('leaves the model catalogue open', async () => {
|
||||
const res = await fetch(new URL('/puterai/chat/models', env.apiOrigin));
|
||||
expect(res.status).toBe(200);
|
||||
});
|
||||
|
||||
it('leaves the video proxy outside the plan gate', async () => {
|
||||
// Unsigned, so this is turned away on the signature — the point is
|
||||
// that a free account gets that answer rather than a 402. The proxy
|
||||
// delivers a video the account already paid to generate.
|
||||
const res = await fetch(
|
||||
new URL(
|
||||
'/puterai/video/proxy?url=https://x.test/v.mp4',
|
||||
env.apiOrigin,
|
||||
),
|
||||
);
|
||||
expect(res.status).not.toBe(402);
|
||||
});
|
||||
});
|
||||
@@ -195,7 +195,9 @@ describe('PuterAIController.registerRoutes', () => {
|
||||
// `requireUserActor` keeps apps out, `allowFullAccessToken` admits
|
||||
// the PAT, and `noUserSession` rejects the account session ("root")
|
||||
// token. Each route also carries `requireVerified` (fresh accounts
|
||||
// must confirm their email before using the AI wire surface) and
|
||||
// must confirm their email before using the AI wire surface),
|
||||
// `requireSubscription` (the vendor-compatible surface is a paid
|
||||
// feature; free accounts reach the same models via /drivers/call) and
|
||||
// the shared per-tier AI rate-limit / concurrency policy — these
|
||||
// routes bypass the `/drivers/call` dispatch (where the
|
||||
// driver-declared limits are enforced), so without the route gates
|
||||
@@ -214,6 +216,7 @@ describe('PuterAIController.registerRoutes', () => {
|
||||
allowFullAccessToken: true,
|
||||
noUserSession: true,
|
||||
requireVerified: true,
|
||||
requireSubscription: true,
|
||||
rateLimit: {
|
||||
...AI_RATE_LIMIT.default,
|
||||
scope: 'driver:puter-chat-completion:complete',
|
||||
@@ -244,6 +247,38 @@ describe('PuterAIController.registerRoutes', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('leaves the catalogue and the video proxy outside the plan gate', () => {
|
||||
const calls: Array<{ path: string; opts: RouteOptions }> = [];
|
||||
const router = {
|
||||
post: vi.fn(() => router),
|
||||
get: vi.fn((path: string, opts: RouteOptions) => {
|
||||
calls.push({ path, opts });
|
||||
return router;
|
||||
}),
|
||||
};
|
||||
|
||||
controller.registerRoutes(router as never);
|
||||
|
||||
// The catalogue is fetched before a client has any reason to
|
||||
// authenticate, and the proxy streams a video the account already
|
||||
// paid to generate to whoever holds the signed link — neither has a
|
||||
// plan to read, let alone one to charge for.
|
||||
const openPaths = [
|
||||
'/puterai/chat/models',
|
||||
'/puterai/chat/models/details',
|
||||
'/puterai/image/models',
|
||||
'/puterai/image/models/details',
|
||||
'/puterai/video/models',
|
||||
'/puterai/video/models/details',
|
||||
'/puterai/video/proxy',
|
||||
];
|
||||
for (const path of openPaths) {
|
||||
const route = calls.find((c) => c.path === path);
|
||||
expect(route, path).toBeDefined();
|
||||
expect(route?.opts.requireSubscription, path).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('keys the proxy-route AI limits by user uuid so they share the /drivers/call buckets', () => {
|
||||
const calls: Array<{ path: string; opts: RouteOptions }> = [];
|
||||
const router = {
|
||||
|
||||
@@ -49,7 +49,11 @@ const GEMINI_DOWNLOAD_BASE =
|
||||
* All routes live on `subdomain: 'api'` and require a full-access API token
|
||||
* minted from the dashboard (user-scoped worker tokens also pass — workers are
|
||||
* never treated as root tokens). Apps, scoped tokens, and account session
|
||||
* ("root") tokens are rejected.
|
||||
* ("root") tokens are rejected. The vendor-compatible routes additionally
|
||||
* require the account to be on a paid plan. The model listings stay open —
|
||||
* clients fetch the catalogue before they have any reason to authenticate — and
|
||||
* so does the video proxy, whose signed URLs are handed out by a generation the
|
||||
* account already paid for.
|
||||
*/
|
||||
export class PuterAIController extends PuterController {
|
||||
registerRoutes(router: PuterRouter): void {
|
||||
@@ -79,6 +83,13 @@ export class PuterAIController extends PuterController {
|
||||
allowFullAccessToken: true,
|
||||
noUserSession: true,
|
||||
requireVerified: true,
|
||||
// The vendor-compatible wire surface is a paid feature: it is
|
||||
// what makes Puter a drop-in for a metered vendor API, and an
|
||||
// account on a free plan reaches the same models through
|
||||
// `/drivers/call` (and puter.js) without it. Free accounts get
|
||||
// 402 `subscription_required` here rather than a rate limit,
|
||||
// so the answer is "upgrade", not "slow down".
|
||||
requireSubscription: true,
|
||||
rateLimit: {
|
||||
...AI_RATE_LIMIT.default!,
|
||||
scope: aiPolicyScope,
|
||||
@@ -166,7 +177,9 @@ export class PuterAIController extends PuterController {
|
||||
// -- Video URL proxy -----------------------------------------
|
||||
// Reverse-proxies AI-generated video URLs that can't be given
|
||||
// directly to the client (auth-gated provider downloads). The
|
||||
// URL itself is HMAC-signed, so no additional auth gate.
|
||||
// URL itself is HMAC-signed, so no additional auth gate — and no
|
||||
// plan gate either: it delivers a video the account already paid to
|
||||
// generate, to whoever holds the link.
|
||||
router.get(
|
||||
'/puterai/video/proxy',
|
||||
{
|
||||
@@ -574,8 +587,7 @@ export class PuterAIController extends PuterController {
|
||||
text: extractTextContent(
|
||||
(
|
||||
messageResult.message as
|
||||
| Record<string, unknown>
|
||||
| undefined
|
||||
Record<string, unknown> | undefined
|
||||
)?.content,
|
||||
),
|
||||
index: 0,
|
||||
|
||||
@@ -0,0 +1,554 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
|
||||
/**
|
||||
* Route-level coverage for the sharing endpoints. The service unit tests drive
|
||||
* the semantics; this suite exists to catch a route that was never registered,
|
||||
* a gate that rejects a legitimate request, and anything the response shape
|
||||
* leaks.
|
||||
*/
|
||||
describe('share endpoints over HTTP', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
}, 120_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
const post = (path: string, token: string, body: unknown) =>
|
||||
fetch(new URL(path, env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${token}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
const get = (path: string, token: string, params: Record<string, string>) => {
|
||||
const url = new URL(path, env.apiOrigin);
|
||||
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
|
||||
return fetch(url, { headers: { authorization: `Bearer ${token}` } });
|
||||
};
|
||||
|
||||
/** A file in the owner's home. Written directly — these tests are about
|
||||
* the share routes, not the upload path. */
|
||||
const makeFile = async (owner: { username: string }) => {
|
||||
const uid = crypto.randomUUID();
|
||||
const name = `share-http-${uid.slice(0, 8)}.txt`;
|
||||
const path = `/${owner.username}/${name}`;
|
||||
const user = await env.server.stores.user.getByUsername(owner.username);
|
||||
await env.server.clients.db.write(
|
||||
'INSERT INTO `fsentries` (`uuid`, `name`, `path`, `user_id`, `is_dir`, `modified`) VALUES (?, ?, ?, ?, 0, ?)',
|
||||
[uid, name, path, user!.id, Math.floor(Date.now() / 1000)],
|
||||
);
|
||||
return { uid, path, name };
|
||||
};
|
||||
|
||||
// Masking hides where an item sits, not who may open it.
|
||||
it('will not let a masked path reach an unshared sibling', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const shared = await makeFile(owner);
|
||||
const secret = await makeFile(owner);
|
||||
|
||||
await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: shared.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
|
||||
const read = (path: string) =>
|
||||
fetch(
|
||||
`${env.apiOrigin}/read?${new URLSearchParams({ file: path })}`,
|
||||
{
|
||||
headers: {
|
||||
authorization: `Bearer ${recipient.token}`,
|
||||
origin: env.apiOrigin,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
const masked = `/${owner.username}/${shared.uid}/${shared.name}`;
|
||||
expect((await read(masked)).status).toBe(200);
|
||||
|
||||
for (const attempt of [
|
||||
// The shared item's root, renamed to the sibling.
|
||||
`/${owner.username}/${shared.uid}/${secret.name}`,
|
||||
// The sibling's own uuid, as if it had leaked.
|
||||
`/${owner.username}/${secret.uid}/${secret.name}`,
|
||||
// Back out of the root the uuid vouched for.
|
||||
`/${owner.username}/${shared.uid}/${shared.name}/../${secret.name}`,
|
||||
// The owner's real path, named outright.
|
||||
secret.path,
|
||||
]) {
|
||||
const res = await read(attempt);
|
||||
expect(res.status, `reachable: ${attempt}`).not.toBe(200);
|
||||
}
|
||||
});
|
||||
|
||||
it('shares an item, lists it for the recipient, then revokes it', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const file = await makeFile(owner);
|
||||
|
||||
const shareRes = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(shareRes.status).toBe(200);
|
||||
const shareBody = (await shareRes.json()) as {
|
||||
status: string;
|
||||
results: Array<{ status: string; mode?: string; name?: string }>;
|
||||
};
|
||||
expect(shareBody.status).toBe('success');
|
||||
expect(shareBody.results[0].mode).toBe('read');
|
||||
expect(shareBody.results[0].name).toBe(file.name);
|
||||
|
||||
const listRes = await get(
|
||||
'/share/shared-with-me',
|
||||
recipient.token,
|
||||
{ includeTotal: 'true' },
|
||||
);
|
||||
expect(listRes.status).toBe(200);
|
||||
const listed = (await listRes.json()) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
total?: number;
|
||||
};
|
||||
const row = listed.items.find((i) => i.uid_entry === file.uid);
|
||||
expect(row).toBeDefined();
|
||||
expect(row?.issuer).toBe(owner.username);
|
||||
expect(row?.mode).toBe('read');
|
||||
expect(typeof listed.total).toBe('number');
|
||||
|
||||
// Nothing internal rides along in the response.
|
||||
for (const key of ['issuer_user_id', 'holder_user_id', 'fsentry_id']) {
|
||||
expect(row).not.toHaveProperty(key);
|
||||
}
|
||||
|
||||
const revokeRes = await post('/share/revoke', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
});
|
||||
expect(revokeRes.status).toBe(200);
|
||||
expect(await revokeRes.json()).toMatchObject({ revoked: 1 });
|
||||
|
||||
const afterRes = await get('/share/shared-with-me', recipient.token, {});
|
||||
const after = (await afterRes.json()) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
};
|
||||
expect(after.items.find((i) => i.uid_entry === file.uid)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('revokes every item in the request, not just the first', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const fileA = await makeFile(owner);
|
||||
const fileB = await makeFile(owner);
|
||||
|
||||
for (const file of [fileA, fileB]) {
|
||||
const res = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
}
|
||||
|
||||
// A truncated revoke is a silent security failure: the caller is told
|
||||
// "success" while items after the first keep their grants.
|
||||
const revokeRes = await post('/share/revoke', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: fileA.uid }, { uid: fileB.uid }],
|
||||
});
|
||||
expect(revokeRes.status).toBe(200);
|
||||
expect(await revokeRes.json()).toMatchObject({
|
||||
status: 'success',
|
||||
revoked: 2,
|
||||
});
|
||||
|
||||
const afterRes = await get('/share/shared-with-me', recipient.token, {});
|
||||
const after = (await afterRes.json()) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
};
|
||||
for (const file of [fileA, fileB]) {
|
||||
expect(
|
||||
after.items.find((i) => i.uid_entry === file.uid),
|
||||
).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('accepts tilde-rooted paths the way the FS routes do', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const file = await makeFile(owner);
|
||||
const tildePath = `~/${file.path.split('/').pop()}`;
|
||||
|
||||
const shareRes = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [tildePath],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(shareRes.status).toBe(200);
|
||||
expect(await shareRes.json()).toMatchObject({ status: 'success' });
|
||||
|
||||
const listRes = await get('/share/shares', owner.token, {
|
||||
path: tildePath,
|
||||
});
|
||||
expect(listRes.status).toBe(200);
|
||||
const listed = (await listRes.json()) as {
|
||||
items: Array<{ holder: string }>;
|
||||
};
|
||||
expect(
|
||||
listed.items.some((i) => i.holder === recipient.username),
|
||||
).toBe(true);
|
||||
|
||||
const revokeRes = await post('/share/revoke', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [tildePath],
|
||||
});
|
||||
expect(revokeRes.status).toBe(200);
|
||||
expect(await revokeRes.json()).toMatchObject({ revoked: 1 });
|
||||
});
|
||||
|
||||
it('tells the recipient over the wire, once for the whole batch', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const [a, b] = [await makeFile(owner), await makeFile(owner)];
|
||||
|
||||
// These accounts are shared between tests, so both things that would
|
||||
// otherwise decide this outcome are cleared first: the budgets that
|
||||
// silence a repeat interruption, and any notification still open for
|
||||
// this recipient, which a new share folds into instead of creating one.
|
||||
// What's left under test is the batching.
|
||||
const [issuer, holder] = await Promise.all([
|
||||
env.server.stores.user.getByUsername(owner.username),
|
||||
env.server.stores.user.getByUsername(recipient.username),
|
||||
]);
|
||||
await env.server.clients.redis.del(
|
||||
`rate:share:notify:pair:${issuer!.id}:${holder!.id}`,
|
||||
`rate:share:notify:pair-day:${issuer!.id}:${holder!.id}`,
|
||||
`rate:share:notify:to:${holder!.id}`,
|
||||
`rate:share:notify:to-day:${holder!.id}`,
|
||||
);
|
||||
for (const row of await env.server.stores.notification.listByUserId(
|
||||
holder!.id,
|
||||
{ filter: 'unacknowledged' },
|
||||
)) {
|
||||
await env.server.stores.notification.markAcknowledged(
|
||||
row.uid,
|
||||
holder!.id,
|
||||
);
|
||||
}
|
||||
|
||||
const seen: Array<{ userIds: number[]; payload: Record<string, unknown> }> = [];
|
||||
const notification = env.server.services.notification;
|
||||
const original = notification.notify.bind(notification);
|
||||
notification.notify = async (userIds, payload) => {
|
||||
seen.push({ userIds, payload });
|
||||
return original(userIds, payload);
|
||||
};
|
||||
|
||||
try {
|
||||
const res = await post('/share', owner.token, {
|
||||
items: [{ path: a.path }, { path: b.path }],
|
||||
recipients: [{ username: recipient.username }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
|
||||
// Delivery is off the response path, so it may land just after.
|
||||
await vi.waitFor(() => expect(seen.length).toBeGreaterThan(0), {
|
||||
timeout: 5000,
|
||||
});
|
||||
} finally {
|
||||
notification.notify = original;
|
||||
}
|
||||
|
||||
// Two items, one recipient — one notification carrying the count.
|
||||
expect(seen).toHaveLength(1);
|
||||
expect(seen[0].payload).toMatchObject({
|
||||
source: 'sharing',
|
||||
fields: { count: 2 },
|
||||
});
|
||||
});
|
||||
|
||||
it('reports per-pair outcomes when only some recipients resolve', async () => {
|
||||
const owner = env.users.user;
|
||||
const file = await makeFile(owner);
|
||||
|
||||
const res = await post('/share', owner.token, {
|
||||
recipients: [env.users.other.username, 'nosuchuser-zzz'],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
const body = (await res.json()) as {
|
||||
status: string;
|
||||
results: Array<{ status: string; recipient: string }>;
|
||||
};
|
||||
expect(body.status).toBe('mixed');
|
||||
expect(body.results).toHaveLength(2);
|
||||
expect(
|
||||
body.results.find((r) => r.recipient === 'nosuchuser-zzz')?.status,
|
||||
).toBe('error');
|
||||
});
|
||||
|
||||
it('lists who can reach an item for its owner', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const file = await makeFile(owner);
|
||||
|
||||
await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'write',
|
||||
});
|
||||
|
||||
const res = await get('/share/shares', owner.token, { uid: file.uid });
|
||||
expect(res.status).toBe(200);
|
||||
const body = (await res.json()) as {
|
||||
items: Array<{ holder: string; mode: string }>;
|
||||
};
|
||||
expect(body.items).toHaveLength(1);
|
||||
expect(body.items[0].holder).toBe(recipient.username);
|
||||
expect(body.items[0].mode).toBe('write');
|
||||
});
|
||||
|
||||
it('hides an item from a stranger asking who can reach it', async () => {
|
||||
const owner = env.users.user;
|
||||
const file = await makeFile(owner);
|
||||
|
||||
const res = await get('/share/shares', env.users.other.token, {
|
||||
uid: file.uid,
|
||||
});
|
||||
expect(res.status).toBe(404);
|
||||
});
|
||||
|
||||
|
||||
it('runs a duplicated pair once, and answers for both positions', async () => {
|
||||
const owner = env.users.user;
|
||||
const file = await makeFile(owner);
|
||||
const invitee = `dup-${crypto.randomUUID().slice(0, 8)}@puter.local`;
|
||||
|
||||
// The same pair twice raced itself into two pending rows; it must
|
||||
// execute once, with the one outcome reported in both positions.
|
||||
const res = await post('/share', owner.token, {
|
||||
recipients: [invitee],
|
||||
items: [{ uid: file.uid }, { uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
const body = (await res.json()) as {
|
||||
status: string;
|
||||
results: Array<{ status: string; uid?: string }>;
|
||||
};
|
||||
expect(body.results).toHaveLength(2);
|
||||
expect(body.results[0].status).toBe('pending');
|
||||
expect(body.results[1]).toEqual(body.results[0]);
|
||||
|
||||
const rows = await env.server.stores.share.listPendingByEmail(invitee);
|
||||
expect(rows).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('rejects an unauthenticated share', async () => {
|
||||
const res = await fetch(new URL('/share', env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({ recipients: ['x'], items: ['y'] }),
|
||||
});
|
||||
expect(res.status).toBeGreaterThanOrEqual(400);
|
||||
expect(res.status).toBeLessThan(500);
|
||||
});
|
||||
|
||||
it('caps how many recipients one request can reach', async () => {
|
||||
const owner = env.users.user;
|
||||
const file = await makeFile(owner);
|
||||
const many = Array.from({ length: 64 }, (_, i) => `user-${i}`);
|
||||
|
||||
const res = await post('/share', owner.token, {
|
||||
recipients: many,
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(res.status).toBe(400);
|
||||
expect(await res.json()).toMatchObject({
|
||||
code: 'too_many_recipients',
|
||||
});
|
||||
});
|
||||
|
||||
it('caps how many items one request can carry', async () => {
|
||||
const owner = env.users.user;
|
||||
const many = Array.from({ length: 128 }, () => ({
|
||||
uid: crypto.randomUUID(),
|
||||
}));
|
||||
|
||||
const res = await post('/share', owner.token, {
|
||||
recipients: [env.users.other.username],
|
||||
items: many,
|
||||
mode: 'read',
|
||||
});
|
||||
expect(res.status).toBe(400);
|
||||
expect(await res.json()).toMatchObject({ code: 'too_many_items' });
|
||||
});
|
||||
|
||||
it('rejects a request with no recipients or no items', async () => {
|
||||
const owner = env.users.user;
|
||||
const file = await makeFile(owner);
|
||||
|
||||
expect(
|
||||
(await post('/share', owner.token, { items: [{ uid: file.uid }] }))
|
||||
.status,
|
||||
).toBe(400);
|
||||
expect(
|
||||
(
|
||||
await post('/share', owner.token, {
|
||||
recipients: [env.users.other.username],
|
||||
})
|
||||
).status,
|
||||
).toBe(400);
|
||||
});
|
||||
|
||||
it('blocks a sender, refuses their share, then unblocks them', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const file = await makeFile(owner);
|
||||
|
||||
const blocked = await post('/share/blocks', recipient.token, {
|
||||
username: owner.username,
|
||||
});
|
||||
expect(blocked.status).toBe(200);
|
||||
expect(await blocked.json()).toMatchObject({
|
||||
username: owner.username,
|
||||
blocked: true,
|
||||
created: true,
|
||||
});
|
||||
|
||||
const refused = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
// A per-pair outcome, so the envelope reports it rather than the status.
|
||||
expect(refused.status).toBe(200);
|
||||
expect(await refused.json()).toMatchObject({
|
||||
status: 'aborted',
|
||||
results: [{ status: 'error', code: 'recipient_not_accepting_shares' }],
|
||||
});
|
||||
|
||||
const listed = await get('/share/blocks', recipient.token, {});
|
||||
expect(listed.status).toBe(200);
|
||||
const body = (await listed.json()) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
};
|
||||
const row = body.items.find((i) => i.username === owner.username);
|
||||
expect(row).toBeDefined();
|
||||
expect(typeof row?.created_at).toBe('number');
|
||||
// Nothing internal rides along.
|
||||
for (const key of ['blocker_user_id', 'blocked_user_id', 'id']) {
|
||||
expect(row).not.toHaveProperty(key);
|
||||
}
|
||||
|
||||
const unblocked = await fetch(new URL('/share/blocks', env.apiOrigin), {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${recipient.token}`,
|
||||
},
|
||||
body: JSON.stringify({ username: owner.username }),
|
||||
});
|
||||
expect(unblocked.status).toBe(200);
|
||||
expect(await unblocked.json()).toMatchObject({ unblocked: true });
|
||||
|
||||
const shared = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(await shared.json()).toMatchObject({ status: 'success' });
|
||||
});
|
||||
|
||||
it('requires a username to block', async () => {
|
||||
const res = await post('/share/blocks', env.users.other.token, {});
|
||||
expect(res.status).toBe(400);
|
||||
expect(await res.json()).toMatchObject({ code: 'bad_request' });
|
||||
});
|
||||
|
||||
it('refuses every sender while the blanket switch is on', async () => {
|
||||
const owner = env.users.user;
|
||||
const recipient = env.users.other;
|
||||
const file = await makeFile(owner);
|
||||
const del = (body: unknown) =>
|
||||
fetch(new URL('/share/blocks', env.apiOrigin), {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${recipient.token}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
const on = await post('/share/blocks', recipient.token, { all: true });
|
||||
expect(on.status).toBe(200);
|
||||
expect(await on.json()).toMatchObject({ all: true, blocked: true });
|
||||
|
||||
const listed = await get('/share/blocks', recipient.token, {});
|
||||
expect(await listed.json()).toMatchObject({ all: true, items: [] });
|
||||
|
||||
const refused = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(await refused.json()).toMatchObject({
|
||||
status: 'aborted',
|
||||
results: [
|
||||
{ status: 'error', code: 'recipient_not_accepting_shares' },
|
||||
],
|
||||
});
|
||||
|
||||
const off = await del({ all: true });
|
||||
expect(off.status).toBe(200);
|
||||
expect(await off.json()).toMatchObject({ all: false, blocked: false });
|
||||
|
||||
const shared = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode: 'read',
|
||||
});
|
||||
expect(await shared.json()).toMatchObject({ status: 'success' });
|
||||
|
||||
// Cleaned up so a later assertion on this recipient isn't reading
|
||||
// state this test left behind.
|
||||
expect((await del({ all: true })).status).toBe(200);
|
||||
});
|
||||
|
||||
it('keeps one caller\'s blocklist out of another\'s', async () => {
|
||||
const res = await get('/share/blocks', env.users.admin.token, {});
|
||||
const body = (await res.json()) as { all: boolean; items: unknown[] };
|
||||
expect(body).toEqual({ all: false, items: [] });
|
||||
});
|
||||
});
|
||||
@@ -17,381 +17,593 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
// import type { Request, Response } from 'express';
|
||||
// import { HttpError } from '../../core/http/HttpError.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import type { Request, Response } from 'express';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
|
||||
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
|
||||
import type {
|
||||
ResolvedShare,
|
||||
ShareRecipient,
|
||||
ShareTarget,
|
||||
} from '../../services/share/ShareService.js';
|
||||
import { expandTildePath } from '../../services/fs/resolveNode.js';
|
||||
import { signEntryThumbnail } from '../fs/legacyFsHelpers.js';
|
||||
import { runWithConcurrencyLimitSettled } from '../../util/concurrency.js';
|
||||
import { normalizeLimit } from '../../util/pagination.js';
|
||||
import { PuterController } from '../types.js';
|
||||
|
||||
// const SHARE_TOKEN_TYPE = 'share';
|
||||
// const SHARE_TOKEN_EXPIRY = '14d';
|
||||
/**
|
||||
* Two windows: a burst ceiling, and a daily one so a slow drip can't add up to
|
||||
* a mail-merge. Neither bounds _shares_ — one request carries many — which is
|
||||
* what `ShareService`'s per-day quota is for.
|
||||
*/
|
||||
const SHARE_LIMIT = [
|
||||
{ scope: 'share:mutate', limit: 60, window: 60_000, key: 'user' as const },
|
||||
{
|
||||
scope: 'share:mutate-daily',
|
||||
limit: 500,
|
||||
window: 24 * 60 * 60_000,
|
||||
key: 'user' as const,
|
||||
},
|
||||
];
|
||||
|
||||
const SHARE_LIST_LIMIT = {
|
||||
scope: 'share:list',
|
||||
limit: 600,
|
||||
window: 60_000,
|
||||
key: 'user' as const,
|
||||
};
|
||||
|
||||
/** Distinct (holder, item) pairs run together; see the note on grouping below. */
|
||||
const SHARE_CONCURRENCY = 8;
|
||||
const LIST_LIMIT_CAP = 200;
|
||||
|
||||
/**
|
||||
* Share link endpoints — check, apply, and request access to pending shares.
|
||||
* The main `POST /share` creation endpoint is also here.
|
||||
*
|
||||
* Shares are permission grants addressed to an email. When the recipient
|
||||
* doesn't have a Puter account yet, the share row lives in the `share` table
|
||||
* until they sign up and apply it. When they DO have an account, permissions
|
||||
* are granted immediately and no row is stored.
|
||||
* Caps on one request's fan-out. Recipients matter most: that number is how
|
||||
* many people a single call can reach, so it stays small by default and only
|
||||
* moves by configuration.
|
||||
*/
|
||||
export const DEFAULT_MAX_RECIPIENTS = 10;
|
||||
export const DEFAULT_MAX_ITEMS = 50;
|
||||
|
||||
/**
|
||||
* A success carries the created share; a failure carries why. `pending` is an
|
||||
* invite: recorded, but not access anyone holds yet.
|
||||
*/
|
||||
interface ShareOutcome {
|
||||
recipient: string;
|
||||
status: 'success' | 'error' | 'pending';
|
||||
path?: string;
|
||||
uid?: string;
|
||||
mode?: string;
|
||||
uid_entry?: string;
|
||||
is_dir?: boolean;
|
||||
issuer?: string | null;
|
||||
holder?: string | null;
|
||||
created_at?: unknown;
|
||||
message?: string;
|
||||
code?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sharing endpoints. `ShareService` owns the semantics; this layer parses
|
||||
* input, bounds fan-out, and shapes responses.
|
||||
*
|
||||
* Apps and tokens are admitted rather than gated out, because `ShareService`
|
||||
* bounds them properly: authority to share comes from the user behind the
|
||||
* actor, and the actor must additionally reach the node in its own right. An
|
||||
* app therefore shares its own AppData and the files it was given, and nothing
|
||||
* else its user happens to own.
|
||||
*/
|
||||
@Controller('/share')
|
||||
export class ShareController extends PuterController {
|
||||
registerRoutes(_router: PuterRouter): void {
|
||||
// const api = { subdomain: 'api' } as const;
|
||||
// router.post('/sharelink/check', api, this.#check);
|
||||
// router.post(
|
||||
// '/sharelink/apply',
|
||||
// { ...api, requireAuth: true },
|
||||
// this.#apply,
|
||||
// );
|
||||
// router.post(
|
||||
// '/sharelink/request',
|
||||
// { ...api, requireAuth: true },
|
||||
// this.#request,
|
||||
// );
|
||||
// router.post('/share', { ...api, requireAuth: true }, this.#share);
|
||||
/**
|
||||
* POST /share — grant `mode` on one or more items to one or more
|
||||
* recipients. Partial success is the contract: each pair reports its own
|
||||
* outcome and the envelope summarizes.
|
||||
*/
|
||||
@Post('', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIMIT,
|
||||
})
|
||||
async createShares(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const body = this.#body(req);
|
||||
const recipients = this.#recipients(body);
|
||||
const items = this.#items(body, actor);
|
||||
const mode = typeof body.mode === 'string' ? body.mode : 'read';
|
||||
|
||||
// Every (recipient, item) pair is a distinct (holder, entry) key, so
|
||||
// they can run together. Two writes to the *same* pair could not —
|
||||
// setUserUser is a read-modify-write, and a duplicated invite races
|
||||
// itself into two pending rows — so duplicates in the request execute
|
||||
// once and every original position reports that one outcome.
|
||||
const pairs = recipients.flatMap((recipient) =>
|
||||
items.map((item) => ({ recipient, item })),
|
||||
);
|
||||
const { unique, indexOf } = this.#dedupePairs(pairs);
|
||||
|
||||
const settledUnique = await runWithConcurrencyLimitSettled(
|
||||
unique,
|
||||
SHARE_CONCURRENCY,
|
||||
async ({ recipient, item }) => {
|
||||
const share = await this.services.share.share(actor, {
|
||||
...item,
|
||||
recipient,
|
||||
mode: mode as never,
|
||||
});
|
||||
return share;
|
||||
},
|
||||
);
|
||||
const settled = indexOf.map((i) => settledUnique[i]);
|
||||
|
||||
const results: ShareOutcome[] = await Promise.all(
|
||||
settled.map(async (outcome, index) => {
|
||||
const { recipient, item } = pairs[index];
|
||||
const label = recipient.email ?? recipient.username ?? '';
|
||||
if (outcome.status === 'fulfilled') {
|
||||
// The whole share, not just an acknowledgement, so a caller
|
||||
// needn't re-read to learn what it created.
|
||||
const share = outcome.value as ResolvedShare;
|
||||
return {
|
||||
...(await this.#toClientShare(share)),
|
||||
recipient: label,
|
||||
status: share.pending ? 'pending' : 'success',
|
||||
};
|
||||
}
|
||||
return {
|
||||
recipient: label,
|
||||
...(item.path ? { path: item.path } : {}),
|
||||
status: 'error',
|
||||
...this.#errorShape(outcome.reason),
|
||||
};
|
||||
}),
|
||||
);
|
||||
|
||||
// Off the response path: a share that landed must not be reported as
|
||||
// failed because telling the recipient didn't. Fanned out from the
|
||||
// unique outcomes, so a duplicated pair is not counted twice.
|
||||
void this.services.shareNotification.notifyShared(
|
||||
actor,
|
||||
settledUnique
|
||||
.filter((o) => o.status === 'fulfilled')
|
||||
.map((o) => (o as PromiseFulfilledResult<ResolvedShare>).value),
|
||||
);
|
||||
|
||||
const succeeded = results.filter((r) => r.status !== 'error').length;
|
||||
res.json({
|
||||
status:
|
||||
succeeded === results.length
|
||||
? 'success'
|
||||
: succeeded > 0
|
||||
? 'mixed'
|
||||
: 'aborted',
|
||||
results,
|
||||
});
|
||||
}
|
||||
|
||||
// // -- POST /sharelink/check ---------------------------------------
|
||||
// // Public — verify a share token from an email link.
|
||||
/**
|
||||
* POST /share/revoke — withdraw recipients' access to items. Same fan-out
|
||||
* contract as POST /share: every (recipient, item) pair is its own revoke
|
||||
* with its own outcome. Silently dropping pairs after the first would leave
|
||||
* access standing that the caller believes is gone.
|
||||
*/
|
||||
@Post('/revoke', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIMIT,
|
||||
})
|
||||
async revokeShare(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const body = this.#body(req);
|
||||
const recipients = this.#recipients(body);
|
||||
const items = this.#items(body, actor);
|
||||
|
||||
// #check = async (req: Request, res: Response): Promise<void> => {
|
||||
// const token = req.body?.token;
|
||||
// if (typeof token !== 'string' || token.length === 0) {
|
||||
// throw new HttpError(400, 'Missing `token`');
|
||||
// }
|
||||
const pairs = recipients.flatMap((recipient) =>
|
||||
items.map((item) => ({ recipient, item })),
|
||||
);
|
||||
const { unique, indexOf } = this.#dedupePairs(pairs);
|
||||
|
||||
// let decoded: { uid?: string; type?: string };
|
||||
// try {
|
||||
// decoded = this.services.token.verify(SHARE_TOKEN_TYPE, token);
|
||||
// } catch {
|
||||
// throw new HttpError(400, 'Invalid or expired share token');
|
||||
// }
|
||||
// if (decoded.type !== `token:${SHARE_TOKEN_TYPE}` || !decoded.uid) {
|
||||
// throw new HttpError(400, 'Invalid share token');
|
||||
// }
|
||||
const settledUnique = await runWithConcurrencyLimitSettled(
|
||||
unique,
|
||||
SHARE_CONCURRENCY,
|
||||
({ recipient, item }) =>
|
||||
this.services.share.unshare(actor, { ...item, recipient }),
|
||||
);
|
||||
const settled = indexOf.map((i) => settledUnique[i]);
|
||||
|
||||
// const share = await this.stores.share.getByUid(decoded.uid);
|
||||
// if (!share) throw new HttpError(404, 'Share not found or expired');
|
||||
// Totalled over the unique outcomes: a pair listed twice revoked its
|
||||
// grants once.
|
||||
const revoked = settledUnique.reduce(
|
||||
(total, outcome) =>
|
||||
outcome.status === 'fulfilled'
|
||||
? total + outcome.value.revoked
|
||||
: total,
|
||||
0,
|
||||
);
|
||||
const results: ShareOutcome[] = settled.map((outcome, index) => {
|
||||
const { recipient, item } = pairs[index];
|
||||
const label = recipient.email ?? recipient.username ?? '';
|
||||
if (outcome.status === 'fulfilled') {
|
||||
return {
|
||||
recipient: label,
|
||||
...(item.path ? { path: item.path } : {}),
|
||||
...(item.uid ? { uid: item.uid } : {}),
|
||||
status: 'success',
|
||||
};
|
||||
}
|
||||
return {
|
||||
recipient: label,
|
||||
...(item.path ? { path: item.path } : {}),
|
||||
...(item.uid ? { uid: item.uid } : {}),
|
||||
status: 'error',
|
||||
...this.#errorShape(outcome.reason),
|
||||
};
|
||||
});
|
||||
|
||||
// res.json({
|
||||
// $: 'api:share',
|
||||
// uid: share.uid,
|
||||
// email: share.recipient_email,
|
||||
// });
|
||||
// };
|
||||
const succeeded = results.filter((r) => r.status === 'success').length;
|
||||
res.json({
|
||||
status:
|
||||
succeeded === results.length
|
||||
? 'success'
|
||||
: succeeded > 0
|
||||
? 'mixed'
|
||||
: 'aborted',
|
||||
revoked,
|
||||
results,
|
||||
});
|
||||
}
|
||||
|
||||
// // -- POST /sharelink/apply ---------------------------------------
|
||||
// // Auth required — apply a pending share's permissions to the caller.
|
||||
/**
|
||||
* GET /share/shared-with-me — paginated listing of what others have shared
|
||||
* with the caller.
|
||||
*/
|
||||
@Get('/shared-with-me', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIST_LIMIT,
|
||||
})
|
||||
async listSharedWithMe(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const query = this.#query(req);
|
||||
|
||||
// #apply = async (req: Request, res: Response): Promise<void> => {
|
||||
// const uid = req.body?.uid;
|
||||
// if (typeof uid !== 'string') throw new HttpError(400, 'Missing `uid`');
|
||||
const page = await this.services.share.listSharedWithMe(actor, {
|
||||
limit: normalizeLimit(query.limit, { cap: LIST_LIMIT_CAP }),
|
||||
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
|
||||
includeTotal: query.includeTotal === 'true',
|
||||
});
|
||||
|
||||
// const actor = req.actor;
|
||||
// if (!actor?.user) throw new HttpError(401, 'Unauthorized');
|
||||
res.json({
|
||||
items: await Promise.all(
|
||||
page.items.map((share) => this.#toClientShare(share)),
|
||||
),
|
||||
...(page.cursor ? { cursor: page.cursor } : {}),
|
||||
...(page.total !== undefined ? { total: page.total } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
// const share = await this.stores.share.getByUid(uid);
|
||||
// if (!share) throw new HttpError(404, 'Share not found or expired');
|
||||
/** GET /share/shares — who can reach one item. */
|
||||
@Get('/shares', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIST_LIMIT,
|
||||
})
|
||||
async listSharesOf(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const query = this.#query(req);
|
||||
const target: ShareTarget = {};
|
||||
if (typeof query.uid === 'string') target.uid = query.uid;
|
||||
if (typeof query.path === 'string')
|
||||
target.path = expandTildePath(query.path, actor.user?.username);
|
||||
if (!target.uid && !target.path) {
|
||||
throw new HttpError(400, 'one of `uid` or `path` is required', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
|
||||
// // Issuer must still exist
|
||||
// const issuer = await this.stores.user.getById(share.issuer_user_id);
|
||||
// if (!issuer)
|
||||
// throw new HttpError(410, 'Share expired — issuer account gone');
|
||||
const shares = await this.services.share.listSharesOf(actor, target);
|
||||
res.json({
|
||||
items: await Promise.all(
|
||||
shares.map((share) => this.#toClientShare(share)),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
// // Email must be confirmed
|
||||
// if (
|
||||
// actor.user.requires_email_confirmation &&
|
||||
// !actor.user.email_confirmed
|
||||
// ) {
|
||||
// throw new HttpError(
|
||||
// 403,
|
||||
// 'Please confirm your email before applying shares',
|
||||
// );
|
||||
// }
|
||||
// -- Blocking -----------------------------------------------------
|
||||
|
||||
// // Recipient email must match
|
||||
// if (
|
||||
// !actor.user.email ||
|
||||
// actor.user.email.toLowerCase() !==
|
||||
// share.recipient_email.toLowerCase()
|
||||
// ) {
|
||||
// throw new HttpError(
|
||||
// 403,
|
||||
// 'This share was sent to a different email address',
|
||||
// );
|
||||
// }
|
||||
/**
|
||||
* GET /share/blocks — who the caller is refusing shares from, and whether
|
||||
* they are refusing everyone.
|
||||
*/
|
||||
@Get('/blocks', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIST_LIMIT,
|
||||
})
|
||||
async listBlocks(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const { all, items } =
|
||||
await this.services.share.listBlockedSenders(actor);
|
||||
res.json({
|
||||
all,
|
||||
items: items.map((item) => ({
|
||||
username: item.username,
|
||||
created_at: item.createdAt,
|
||||
})),
|
||||
});
|
||||
}
|
||||
|
||||
// // Grant each permission
|
||||
// const issuerActor = {
|
||||
// user: {
|
||||
// id: issuer.id,
|
||||
// uuid: issuer.uuid,
|
||||
// username: issuer.username,
|
||||
// email: issuer.email ?? null,
|
||||
// suspended: false,
|
||||
// email_confirmed: true,
|
||||
// requires_email_confirmation: false,
|
||||
// },
|
||||
// } as import('../../core/actor.js').Actor;
|
||||
// const data = (share.data ?? {}) as {
|
||||
// permissions?: Array<{
|
||||
// permission: string;
|
||||
// extra?: Record<string, unknown>;
|
||||
// }>;
|
||||
// };
|
||||
// for (const perm of data.permissions ?? []) {
|
||||
// try {
|
||||
// await this.services.permission.grantUserUserPermission(
|
||||
// issuerActor,
|
||||
// actor.user.username ?? '',
|
||||
// perm.permission,
|
||||
// perm.extra ?? {},
|
||||
// );
|
||||
// } catch (err) {
|
||||
// console.warn('[share] grant failed for', perm.permission, err);
|
||||
// }
|
||||
// }
|
||||
/**
|
||||
* POST /share/blocks — stop accepting shares. `{ all: true }` refuses
|
||||
* everyone; `{ username }` refuses one person. Idempotent either way:
|
||||
* blocking twice is the state the caller asked for.
|
||||
*
|
||||
* Access already granted is untouched — `POST /share/revoke` is what
|
||||
* withdraws that.
|
||||
*/
|
||||
@Post('/blocks', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIMIT,
|
||||
})
|
||||
async createBlock(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const body = this.#body(req);
|
||||
if (this.#isBlockAll(body)) {
|
||||
const { all } = await this.services.share.setBlockAllSenders(
|
||||
actor,
|
||||
true,
|
||||
);
|
||||
res.json({ all, blocked: true });
|
||||
return;
|
||||
}
|
||||
const { username, created } = await this.services.share.blockSender(
|
||||
actor,
|
||||
this.#username(body),
|
||||
);
|
||||
res.json({ username, blocked: true, created });
|
||||
}
|
||||
|
||||
// // Share consumed — delete it
|
||||
// await this.stores.share.deleteByUid(uid);
|
||||
/**
|
||||
* DELETE /share/blocks — accept shares again. `{ all: true }` lifts the
|
||||
* blanket refusal, leaving the per-sender list as it was.
|
||||
*/
|
||||
@Delete('/blocks', {
|
||||
subdomain: 'api',
|
||||
requireVerified: true,
|
||||
rateLimit: SHARE_LIMIT,
|
||||
})
|
||||
async deleteBlock(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const body = this.#body(req);
|
||||
if (this.#isBlockAll(body)) {
|
||||
const { all } = await this.services.share.setBlockAllSenders(
|
||||
actor,
|
||||
false,
|
||||
);
|
||||
res.json({ all, blocked: false });
|
||||
return;
|
||||
}
|
||||
const { username, unblocked } = await this.services.share.unblockSender(
|
||||
actor,
|
||||
this.#username(body),
|
||||
);
|
||||
res.json({ username, blocked: false, unblocked });
|
||||
}
|
||||
|
||||
// res.json({ $: 'api:status-report', status: 'success' });
|
||||
// };
|
||||
// -- Helpers ------------------------------------------------------
|
||||
|
||||
// // -- POST /sharelink/request -------------------------------------
|
||||
// // Auth required — notify the issuer that someone is requesting access.
|
||||
/**
|
||||
* Collapse repeated (recipient, item) pairs to one execution. `indexOf`
|
||||
* maps every original position onto its unique pair, so a request that
|
||||
* names the same pair twice still gets a result in both positions — the
|
||||
* same result, which is the truth of what happened.
|
||||
*/
|
||||
#dedupePairs<T extends { recipient: ShareRecipient; item: ShareTarget }>(
|
||||
pairs: T[],
|
||||
): { unique: T[]; indexOf: number[] } {
|
||||
const unique: T[] = [];
|
||||
const seen = new Map<string, number>();
|
||||
const indexOf = pairs.map((pair) => {
|
||||
const key = JSON.stringify([
|
||||
pair.recipient.email ?? null,
|
||||
pair.recipient.username ?? null,
|
||||
pair.item.uid ?? null,
|
||||
pair.item.path ?? null,
|
||||
]);
|
||||
let at = seen.get(key);
|
||||
if (at === undefined) {
|
||||
at = unique.length;
|
||||
seen.set(key, at);
|
||||
unique.push(pair);
|
||||
}
|
||||
return at;
|
||||
});
|
||||
return { unique, indexOf };
|
||||
}
|
||||
|
||||
// #request = async (req: Request, res: Response): Promise<void> => {
|
||||
// const uid = req.body?.uid;
|
||||
// if (typeof uid !== 'string') throw new HttpError(400, 'Missing `uid`');
|
||||
/**
|
||||
* Whether this call is about the blanket switch rather than one person.
|
||||
* Only the literal `true` counts, so a client sending `all: false`
|
||||
* alongside a username still means the username.
|
||||
*/
|
||||
#isBlockAll(body: Record<string, unknown>): boolean {
|
||||
return body.all === true;
|
||||
}
|
||||
|
||||
// const actor = req.actor;
|
||||
// if (!actor?.user) throw new HttpError(401, 'Unauthorized');
|
||||
#username(body: Record<string, unknown>): string {
|
||||
const username =
|
||||
typeof body.username === 'string' ? body.username.trim() : '';
|
||||
if (!username) {
|
||||
throw new HttpError(400, '`username` is required', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
return username;
|
||||
}
|
||||
|
||||
// const share = await this.stores.share.getByUid(uid);
|
||||
// if (!share) throw new HttpError(404, 'Share not found or expired');
|
||||
/**
|
||||
* Only ever the username — never the internal id, and never an email the
|
||||
* caller didn't already supply.
|
||||
*
|
||||
* `thumbnail` is stored as an `s3://bucket/key` URI, so it is swapped for a
|
||||
* signed URL rather than emitted: the raw value names internal storage and
|
||||
* no client can render it.
|
||||
*/
|
||||
async #toClientShare(share: ResolvedShare) {
|
||||
const thumbnail =
|
||||
share.thumbnail === undefined
|
||||
? undefined
|
||||
: await signEntryThumbnail(
|
||||
this.clients.event,
|
||||
share.entryUid,
|
||||
share.thumbnail,
|
||||
);
|
||||
return {
|
||||
uid: share.uid,
|
||||
mode: share.mode,
|
||||
path: share.path,
|
||||
// A share listing has no fsentry behind it for a client to stat.
|
||||
...(share.name === undefined ? {} : { name: share.name }),
|
||||
...(share.type === undefined ? {} : { type: share.type }),
|
||||
...(thumbnail === undefined ? {} : { thumbnail }),
|
||||
...(share.owner === undefined
|
||||
? {}
|
||||
: { owner: share.owner.username }),
|
||||
...(share.pending
|
||||
? { pending: true, recipient_email: share.recipientEmail }
|
||||
: {}),
|
||||
uid_entry: share.entryUid,
|
||||
is_dir: share.isDir,
|
||||
issuer: share.issuer.username,
|
||||
holder: share.holder.username,
|
||||
created_at: share.createdAt,
|
||||
issued_by_app: share.issuedByApp ?? null,
|
||||
inherited_from: share.inheritedFrom ?? null,
|
||||
modified: share.modified,
|
||||
size: share.size,
|
||||
};
|
||||
}
|
||||
|
||||
// const issuer = await this.stores.user.getById(share.issuer_user_id);
|
||||
// if (!issuer)
|
||||
// throw new HttpError(410, 'Share expired — issuer account gone');
|
||||
#requireActor(req: Request): Actor {
|
||||
const actor = req.actor;
|
||||
if (!actor?.user)
|
||||
throw new HttpError(401, 'Unauthorized', {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
return actor;
|
||||
}
|
||||
|
||||
// // If caller IS the intended recipient (confirmed email matches),
|
||||
// // they should just /apply instead.
|
||||
// if (
|
||||
// actor.user.email_confirmed &&
|
||||
// actor.user.email?.toLowerCase() ===
|
||||
// share.recipient_email.toLowerCase()
|
||||
// ) {
|
||||
// throw new HttpError(
|
||||
// 400,
|
||||
// 'You are the intended recipient — use /sharelink/apply instead',
|
||||
// );
|
||||
// }
|
||||
#body(req: Request): Record<string, unknown> {
|
||||
const body = req.body;
|
||||
if (!body || typeof body !== 'object' || Array.isArray(body)) {
|
||||
throw new HttpError(400, 'body must be an object', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
return body as Record<string, unknown>;
|
||||
}
|
||||
|
||||
// // Notify the issuer
|
||||
// if (this.services.notification) {
|
||||
// await this.services.notification.notify([issuer.id], {
|
||||
// source: 'sharing',
|
||||
// title: `User ${actor.user.username} is trying to open a share you sent to ${share.recipient_email}`,
|
||||
// template: 'user-requesting-share',
|
||||
// fields: {
|
||||
// username: actor.user.username,
|
||||
// intended_recipient: share.recipient_email,
|
||||
// permissions:
|
||||
// (share.data as Record<string, unknown>)?.permissions ??
|
||||
// [],
|
||||
// },
|
||||
// });
|
||||
// }
|
||||
#query(req: Request): Record<string, unknown> {
|
||||
return (req.query ?? {}) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
// res.json({ $: 'api:status-report', status: 'success' });
|
||||
// };
|
||||
#recipients(body: Record<string, unknown>): ShareRecipient[] {
|
||||
const raw = body.recipients ?? body.recipient;
|
||||
const list = Array.isArray(raw) ? raw : [raw];
|
||||
const out: ShareRecipient[] = [];
|
||||
for (const entry of list) {
|
||||
if (typeof entry === 'string') {
|
||||
const value = entry.trim();
|
||||
if (!value) continue;
|
||||
out.push(
|
||||
value.includes('@')
|
||||
? { email: value }
|
||||
: { username: value },
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (entry && typeof entry === 'object') {
|
||||
const rec = entry as Record<string, unknown>;
|
||||
const email =
|
||||
typeof rec.email === 'string' ? rec.email.trim() : '';
|
||||
const username =
|
||||
typeof rec.username === 'string' ? rec.username.trim() : '';
|
||||
if (email || username) {
|
||||
out.push(email ? { email } : { username });
|
||||
}
|
||||
}
|
||||
}
|
||||
if (out.length === 0) {
|
||||
throw new HttpError(400, '`recipients` is required', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const max = this.config.share_max_recipients ?? DEFAULT_MAX_RECIPIENTS;
|
||||
if (out.length > max) {
|
||||
throw new HttpError(400, `at most ${max} recipients per request`, {
|
||||
legacyCode: 'too_many_recipients',
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// // -- POST /share -------------------------------------------------
|
||||
// // Auth required — create shares for recipients (users or emails).
|
||||
#items(body: Record<string, unknown>, actor: Actor): ShareTarget[] {
|
||||
const username = actor.user?.username;
|
||||
const raw = body.items ?? body.item ?? body.path ?? body.uid;
|
||||
const list = Array.isArray(raw) ? raw : [raw];
|
||||
const out: ShareTarget[] = [];
|
||||
for (const entry of list) {
|
||||
if (typeof entry === 'string') {
|
||||
const value = entry.trim();
|
||||
if (!value) continue;
|
||||
// Tilde-rooted strings are paths, as the legacy FS routes treat them.
|
||||
const isPath = value.startsWith('/') || value.startsWith('~');
|
||||
out.push(
|
||||
isPath
|
||||
? { path: expandTildePath(value, username) }
|
||||
: { uid: value },
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (entry && typeof entry === 'object') {
|
||||
const item = entry as Record<string, unknown>;
|
||||
const path =
|
||||
typeof item.path === 'string'
|
||||
? expandTildePath(item.path, username)
|
||||
: '';
|
||||
const uid = typeof item.uid === 'string' ? item.uid : '';
|
||||
if (path || uid) out.push(path ? { path } : { uid });
|
||||
}
|
||||
}
|
||||
if (out.length === 0) {
|
||||
throw new HttpError(400, '`items` is required', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const max = this.config.share_max_items ?? DEFAULT_MAX_ITEMS;
|
||||
if (out.length > max) {
|
||||
throw new HttpError(400, `at most ${max} items per request`, {
|
||||
legacyCode: 'too_many_items',
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// #share = async (req: Request, res: Response): Promise<void> => {
|
||||
// const actor = req.actor;
|
||||
// if (!actor?.user) throw new HttpError(401, 'Unauthorized');
|
||||
|
||||
// const body = req.body ?? {};
|
||||
// let recipients = body.recipients;
|
||||
// let shares = body.shares;
|
||||
// const dryRun = !!body.dry_run;
|
||||
|
||||
// if (!recipients) throw new HttpError(400, 'Missing `recipients`');
|
||||
// if (!shares) throw new HttpError(400, 'Missing `shares`');
|
||||
// if (!Array.isArray(recipients)) recipients = [recipients];
|
||||
// if (!Array.isArray(shares)) shares = [shares];
|
||||
|
||||
// // Build the permissions list from share declarations.
|
||||
// const permissions = this.#resolvePermissions(shares as unknown[]);
|
||||
|
||||
// const recipientResults: unknown[] = [];
|
||||
|
||||
// for (const recipient of recipients as unknown[]) {
|
||||
// const recipientStr =
|
||||
// typeof recipient === 'string' ? recipient.trim() : '';
|
||||
// if (!recipientStr) {
|
||||
// recipientResults.push({
|
||||
// $: 'error',
|
||||
// message: 'empty recipient',
|
||||
// });
|
||||
// continue;
|
||||
// }
|
||||
|
||||
// try {
|
||||
// // Try username first
|
||||
// const targetUser =
|
||||
// (await this.stores.user.getByUsername(recipientStr)) ??
|
||||
// (recipientStr.includes('@')
|
||||
// ? await this.stores.user.getByEmail(recipientStr)
|
||||
// : null);
|
||||
|
||||
// if (targetUser) {
|
||||
// // Direct grant — user exists
|
||||
// if (!dryRun) {
|
||||
// for (const perm of permissions) {
|
||||
// try {
|
||||
// await this.services.permission.grantUserUserPermission(
|
||||
// actor,
|
||||
// targetUser.username ?? '',
|
||||
// perm.permission,
|
||||
// perm.extra ?? {},
|
||||
// );
|
||||
// } catch (err) {
|
||||
// console.warn(
|
||||
// '[share] grant to user failed',
|
||||
// perm.permission,
|
||||
// err,
|
||||
// );
|
||||
// }
|
||||
// }
|
||||
|
||||
// // Notify
|
||||
// if (this.services.notification) {
|
||||
// await this.services.notification.notify(
|
||||
// [targetUser.id],
|
||||
// {
|
||||
// source: 'sharing',
|
||||
// title: `${actor.user.username} shared items with you`,
|
||||
// template: 'file-shared-with-you',
|
||||
// fields: {
|
||||
// username: actor.user.username,
|
||||
// permissions: permissions.map(
|
||||
// (p) => p.permission,
|
||||
// ),
|
||||
// },
|
||||
// },
|
||||
// );
|
||||
// }
|
||||
// }
|
||||
// recipientResults.push({
|
||||
// $: 'api:status-report',
|
||||
// status: 'success',
|
||||
// });
|
||||
// } else if (recipientStr.includes('@')) {
|
||||
// // Email recipient — store pending share
|
||||
// if (!dryRun) {
|
||||
// const share = await this.stores.share.create({
|
||||
// issuerUserId: actor.user.id,
|
||||
// recipientEmail: recipientStr.toLowerCase(),
|
||||
// data: {
|
||||
// permissions,
|
||||
// metadata: body.metadata ?? {},
|
||||
// },
|
||||
// });
|
||||
|
||||
// // Sign a share token (14-day expiry)
|
||||
// const token = this.services.token.sign(
|
||||
// SHARE_TOKEN_TYPE,
|
||||
// {
|
||||
// type: `token:${SHARE_TOKEN_TYPE}`,
|
||||
// uid: share.uid,
|
||||
// },
|
||||
// { expiresIn: SHARE_TOKEN_EXPIRY },
|
||||
// );
|
||||
|
||||
// // Email the share link
|
||||
// const origin = `https://${this.config.domain ?? 'puter.com'}`;
|
||||
// try {
|
||||
// await this.clients.email.sendRaw({
|
||||
// to: recipientStr,
|
||||
// subject: `${actor.user.username} shared something with you on Puter`,
|
||||
// html: `<p>${actor.user.username} shared items with you.</p><p><a href="${origin}?share_token=${encodeURIComponent(token)}">Click here to accept</a></p>`,
|
||||
// });
|
||||
// } catch (err) {
|
||||
// console.warn('[share] email send failed', err);
|
||||
// }
|
||||
// }
|
||||
// recipientResults.push({
|
||||
// $: 'api:status-report',
|
||||
// status: 'success',
|
||||
// });
|
||||
// } else {
|
||||
// recipientResults.push({
|
||||
// $: 'error',
|
||||
// message: 'User not found',
|
||||
// });
|
||||
// }
|
||||
// } catch (err) {
|
||||
// recipientResults.push({ $: 'error', message: String(err) });
|
||||
// }
|
||||
// }
|
||||
|
||||
// const allOk = recipientResults.every(
|
||||
// (r: unknown) => (r as Record<string, unknown>).status === 'success',
|
||||
// );
|
||||
// const anyOk = recipientResults.some(
|
||||
// (r: unknown) => (r as Record<string, unknown>).status === 'success',
|
||||
// );
|
||||
|
||||
// res.json({
|
||||
// $: 'api:share',
|
||||
// $version: 'v0.0.0',
|
||||
// status: allOk ? 'success' : anyOk ? 'mixed' : 'aborted',
|
||||
// recipients: recipientResults,
|
||||
// ...(dryRun ? { dry_run: true } : {}),
|
||||
// });
|
||||
// };
|
||||
|
||||
// // -- Helpers ------------------------------------------------------
|
||||
|
||||
// /**
|
||||
// * Convert share declarations into a flat permission list.
|
||||
// * Supports `fs-share` ({ path, access }) and `app-share` ({ uid, name }).
|
||||
// */
|
||||
// #resolvePermissions(
|
||||
// shares: unknown[],
|
||||
// ): Array<{ permission: string; extra?: Record<string, unknown> }> {
|
||||
// const perms: Array<{
|
||||
// permission: string;
|
||||
// extra?: Record<string, unknown>;
|
||||
// }> = [];
|
||||
|
||||
// for (const share of shares) {
|
||||
// if (!share || typeof share !== 'object') continue;
|
||||
// const s = share as Record<string, unknown>;
|
||||
|
||||
// if (s.$ === 'fs-share' || s.type === 'fs-share' || s.path) {
|
||||
// const path = String(s.path ?? '');
|
||||
// const access = String(s.access ?? 'read');
|
||||
// if (path) {
|
||||
// perms.push({ permission: `fs:${path}:${access}` });
|
||||
// }
|
||||
// } else if (
|
||||
// s.$ === 'app-share' ||
|
||||
// s.type === 'app-share' ||
|
||||
// s.uid ||
|
||||
// s.name
|
||||
// ) {
|
||||
// const appUid = String(s.uid ?? s.name ?? '');
|
||||
// if (appUid) {
|
||||
// perms.push({ permission: `app:uid#${appUid}:access` });
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
|
||||
// return perms;
|
||||
// }
|
||||
/**
|
||||
* Report a failure without widening what the caller already knew. The
|
||||
* service already decides 404-vs-403; anything unrecognized becomes a
|
||||
* generic error rather than leaking an internal message.
|
||||
*/
|
||||
#errorShape(reason: unknown): { message: string; code?: string } {
|
||||
if (!isHttpError(reason) || reason.statusCode >= 500) {
|
||||
return { message: 'Request failed' };
|
||||
}
|
||||
const code = reason.legacyCode ?? reason.code;
|
||||
return {
|
||||
message: reason.message || 'Request failed',
|
||||
...(code ? { code } : {}),
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -280,15 +280,19 @@ export class StaticPagesController extends PuterController {
|
||||
(user.clean_email as string | null | undefined) ??
|
||||
String(user.email ?? '').toLowerCase();
|
||||
|
||||
const [dupe] = (await this.clients.db.read(
|
||||
`SELECT EXISTS(
|
||||
SELECT 1 FROM \`user\` WHERE (\`email\` = ? OR \`clean_email\` = ?)
|
||||
AND \`email_confirmed\` = ${this.clients.db.booleanLiteral(true)}
|
||||
AND \`password\` IS NOT NULL
|
||||
) AS email_exists`,
|
||||
[user.email, cleanEmail],
|
||||
)) as Array<{ email_exists: number }>;
|
||||
if (dupe?.email_exists) {
|
||||
// An account that already confirmed this address proved access
|
||||
// to the inbox. The strip below would take it away from them,
|
||||
// so refuse here instead. Password-less accounts count: an
|
||||
// identity provider verified the address for those, and they
|
||||
// are exactly what the old `password IS NOT NULL` clause let
|
||||
// through.
|
||||
const confirmedRival =
|
||||
await this.stores.user.findConfirmedOtherByEmail(
|
||||
user.id as number,
|
||||
user.email as string,
|
||||
cleanEmail,
|
||||
);
|
||||
if (confirmedRival) {
|
||||
res.send(
|
||||
err('This email was confirmed on a different account.'),
|
||||
);
|
||||
@@ -302,6 +306,15 @@ export class StaticPagesController extends PuterController {
|
||||
[user.email],
|
||||
);
|
||||
|
||||
// Take the address off every remaining row before confirming
|
||||
// this one. The check above leaves only unconfirmed rows, and
|
||||
// only one row may own an address once this one is confirmed.
|
||||
await this.stores.user.unconfirmOthersByEmail(
|
||||
user.id,
|
||||
user.email as string,
|
||||
cleanEmail,
|
||||
);
|
||||
|
||||
await this.stores.user.update(user.id, {
|
||||
email_confirmed: 1,
|
||||
requires_email_confirmation: 0,
|
||||
|
||||
@@ -1,19 +1,21 @@
|
||||
// This suite tests basic features of puter webdav. it is not a comprehensive webdav test suite unlike litmus
|
||||
// but rather it performs some common sense checks to ensure that WebDAV support isn't irrevocably broken in puter
|
||||
import type { Request, Response } from 'express';
|
||||
import type { Request, RequestHandler, Response } from 'express';
|
||||
import { Readable, Writable } from 'node:stream';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { hash as bcryptHash } from 'bcrypt';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import type { RouteDescriptor } from '../../core/http/types.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import type { WebDAVController } from './WebDAVController.js';
|
||||
|
||||
let server: PuterServer;
|
||||
let controller: WebDAVController;
|
||||
let dispatchMiddleware: Function;
|
||||
let routes: RouteDescriptor[];
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
@@ -21,12 +23,60 @@ beforeAll(async () => {
|
||||
|
||||
const router = new PuterRouter();
|
||||
controller.registerRoutes(router);
|
||||
|
||||
// WebDAVController registers a single `use()` middleware on the `dav`
|
||||
// subdomain. Grab it to call directly in tests.
|
||||
dispatchMiddleware = router.routes[0]!.handler;
|
||||
routes = router.routes;
|
||||
});
|
||||
|
||||
/** Run one handler; true when it called `next` instead of answering. */
|
||||
const runHandler = async (
|
||||
handler: RequestHandler,
|
||||
req: Request,
|
||||
res: Response,
|
||||
): Promise<boolean> => {
|
||||
let advanced = false;
|
||||
await handler(req, res, (() => {
|
||||
advanced = true;
|
||||
}) as never);
|
||||
return advanced;
|
||||
};
|
||||
|
||||
/**
|
||||
* Stand in for express's router: walk the controller's collected routes in
|
||||
* order and run the first one that matches the request's method, plus any
|
||||
* `middleware` it declares.
|
||||
*
|
||||
* The options the real server materializes into gates — subdomain, rate limit,
|
||||
* concurrency — are deliberately not applied here; those belong to the
|
||||
* materializer's own tests, and reproducing them would make every case below
|
||||
* share one DAV budget. `server.test.ts` covers the wiring on a real port.
|
||||
*/
|
||||
const dispatchMiddleware = async (
|
||||
req: Request,
|
||||
res: Response,
|
||||
next: () => void,
|
||||
): Promise<void> => {
|
||||
const method = req.method.toLowerCase();
|
||||
for (const route of routes) {
|
||||
if (
|
||||
route.method !== 'use' &&
|
||||
route.method !== 'all' &&
|
||||
route.method !== method
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
let advanced = true;
|
||||
for (const handler of [
|
||||
...(route.options.middleware ?? []),
|
||||
route.handler,
|
||||
]) {
|
||||
advanced = await runHandler(handler, req, res);
|
||||
if (!advanced) break;
|
||||
}
|
||||
// A chain that stopped short of `next` answered the request.
|
||||
if (!advanced) return;
|
||||
}
|
||||
next();
|
||||
};
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
@@ -151,10 +201,45 @@ const makeUser = async () => {
|
||||
|
||||
describe('WebDAVController', () => {
|
||||
describe('route registration', () => {
|
||||
it('registers a single catch-all use() route', () => {
|
||||
it('registers one catch-all per DAV verb, all on the dav subdomain', () => {
|
||||
const router = new PuterRouter();
|
||||
controller.registerRoutes(router);
|
||||
expect(router.routes.length).toBeGreaterThanOrEqual(1);
|
||||
|
||||
for (const verb of [
|
||||
'options',
|
||||
'head',
|
||||
'get',
|
||||
'propfind',
|
||||
'proppatch',
|
||||
'mkcol',
|
||||
'put',
|
||||
'delete',
|
||||
'copy',
|
||||
'move',
|
||||
'lock',
|
||||
'unlock',
|
||||
]) {
|
||||
const route = router.routes.find((r) => r.method === verb);
|
||||
expect(route, verb).toBeDefined();
|
||||
expect(route!.path).toBe('/{*splat}');
|
||||
expect(route!.options.subdomain).toBe('dav');
|
||||
// The gates the materializer installs ahead of the handler.
|
||||
expect(route!.options.rateLimit).toBeDefined();
|
||||
expect(route!.options.concurrent).toBeDefined();
|
||||
}
|
||||
|
||||
// `all` catches the verbs we don't implement, so it has to come
|
||||
// after every route that does.
|
||||
const allIndex = router.routes.findIndex((r) => r.method === 'all');
|
||||
expect(allIndex).toBe(router.routes.length - 1);
|
||||
});
|
||||
|
||||
it('registers HEAD ahead of GET so express does not fold the two', () => {
|
||||
const router = new PuterRouter();
|
||||
controller.registerRoutes(router);
|
||||
const head = router.routes.findIndex((r) => r.method === 'head');
|
||||
const get = router.routes.findIndex((r) => r.method === 'get');
|
||||
expect(head).toBeLessThan(get);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -240,6 +325,48 @@ describe('WebDAVController', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('CORS preflight', () => {
|
||||
it('answers a browser preflight before the auth gate', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'OPTIONS',
|
||||
path: '/someone/.vscode/settings.json',
|
||||
headers: {
|
||||
origin: 'https://code.puter.com',
|
||||
'access-control-request-method': 'PROPFIND',
|
||||
},
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
// A non-2xx here is what made every browser DAV client give up
|
||||
// before it could send a token.
|
||||
expect(captured.statusCode).toBe(200);
|
||||
expect(captured.headers['dav']).toContain('1');
|
||||
expect(captured.headers['allow']).toContain('PROPFIND');
|
||||
expect(captured.headers['access-control-max-age']).toBe('86400');
|
||||
expect(captured.headers['www-authenticate']).toBeUndefined();
|
||||
});
|
||||
|
||||
it('still authenticates an OPTIONS that is not a preflight', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'OPTIONS',
|
||||
// An Origin alone isn't a preflight — only the
|
||||
// `Access-Control-Request-Method` probe is.
|
||||
headers: { origin: 'https://code.puter.com' },
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
);
|
||||
expect(captured.statusCode).toBe(401);
|
||||
// macOS opens a mount on this reply, so it keeps the DAV header.
|
||||
expect(captured.headers['dav']).toContain('1');
|
||||
});
|
||||
});
|
||||
|
||||
describe('pending-verification gate', () => {
|
||||
// WebDAV must enforce the same gate every other authenticated route
|
||||
// gets from requireVerifiedAccount — it dispatches off a single use()
|
||||
@@ -1122,6 +1249,54 @@ describe('WebDAVController verbs', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('reports timestamps in the present, not the epoch', async () => {
|
||||
// Entry timestamps are stored as Unix seconds. Handing those to
|
||||
// `Date` unscaled reads them as milliseconds and dates every file
|
||||
// to January 1970, which clients sort and sync against.
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/dated.txt`;
|
||||
await putFile(actor, path, 'x');
|
||||
|
||||
const captured = await dispatch({
|
||||
method: 'PROPFIND',
|
||||
path,
|
||||
actor,
|
||||
headers: { depth: '0' },
|
||||
});
|
||||
const xml = captured.body as string;
|
||||
const modified = xml.match(
|
||||
/<D:getlastmodified>(.*?)<\/D:getlastmodified>/,
|
||||
)?.[1];
|
||||
expect(modified).toBeDefined();
|
||||
const skew = Math.abs(
|
||||
new Date(modified as string).getTime() - Date.now(),
|
||||
);
|
||||
expect(skew).toBeLessThan(5 * 60 * 1000);
|
||||
});
|
||||
|
||||
it('emits the same ETag for an entry over PROPFIND and GET', async () => {
|
||||
// A client that sees the validator change between the two treats
|
||||
// the file as modified and re-downloads it on every pass.
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/etag-match.txt`;
|
||||
await putFile(actor, path, 'x');
|
||||
|
||||
const propfind = await dispatch({
|
||||
method: 'PROPFIND',
|
||||
path,
|
||||
actor,
|
||||
headers: { depth: '0' },
|
||||
});
|
||||
const propfindEtag = (propfind.body as string).match(
|
||||
/<D:getetag>(.*?)<\/D:getetag>/,
|
||||
)?.[1];
|
||||
|
||||
const get = await dispatch({ method: 'GET', path, actor });
|
||||
|
||||
expect(propfindEtag).toBeDefined();
|
||||
expect(get.headers['etag']).toBe(propfindEtag);
|
||||
});
|
||||
|
||||
it('omits children at depth 0', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
await putFile(actor, `/${username}/Documents/hidden.txt`, 'x');
|
||||
@@ -1166,6 +1341,68 @@ describe('WebDAVController verbs', () => {
|
||||
'<D:getcontenttype>text/markdown</D:getcontenttype>',
|
||||
);
|
||||
});
|
||||
|
||||
it("answers a share root's parent with a virtual collection", async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
await dispatch({
|
||||
method: 'MKCOL',
|
||||
path: `/${owner.username}/Documents/Album`,
|
||||
actor: owner.actor,
|
||||
});
|
||||
const root = (await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}/Documents/Album`,
|
||||
))!;
|
||||
await server.services.acl.setUserUser(
|
||||
owner.actor as never,
|
||||
holder.actor as never,
|
||||
{
|
||||
path: root.path,
|
||||
resolveAncestors: () =>
|
||||
server.services.fs.getAncestorChain(root.path),
|
||||
},
|
||||
'read',
|
||||
);
|
||||
|
||||
// `/<owner>/<uuid>` is where Up from a share root lands; it is not
|
||||
// a real path, so it answers as a virtual collection whose only
|
||||
// member is the share root at its masked path.
|
||||
const captured = await dispatch({
|
||||
method: 'PROPFIND',
|
||||
path: `/${owner.username}/${root.uuid}`,
|
||||
actor: holder.actor,
|
||||
});
|
||||
expect(captured.statusCode).toBe(207);
|
||||
const xml = captured.body as string;
|
||||
expect(xml).toContain(
|
||||
`<D:href>/${owner.username}/${root.uuid}/</D:href>`,
|
||||
);
|
||||
expect(xml).toContain(
|
||||
`<D:href>/${owner.username}/${root.uuid}/Album/</D:href>`,
|
||||
);
|
||||
// The owner's real tree never shows through.
|
||||
expect(xml).not.toContain('/Documents/');
|
||||
});
|
||||
|
||||
it("keeps a share root's parent a 404 for a caller with no share", async () => {
|
||||
const owner = await makeUser();
|
||||
const stranger = await makeUser();
|
||||
await dispatch({
|
||||
method: 'MKCOL',
|
||||
path: `/${owner.username}/Documents/Private`,
|
||||
actor: owner.actor,
|
||||
});
|
||||
const root = (await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}/Documents/Private`,
|
||||
))!;
|
||||
|
||||
const captured = await dispatch({
|
||||
method: 'PROPFIND',
|
||||
path: `/${owner.username}/${root.uuid}`,
|
||||
actor: stranger.actor,
|
||||
});
|
||||
expect(captured.statusCode).toBe(404);
|
||||
});
|
||||
});
|
||||
|
||||
describe('PROPPATCH', () => {
|
||||
@@ -1653,4 +1890,368 @@ describe('WebDAVController verbs', () => {
|
||||
expect(captured.body).toBe('Lock token does not match this path');
|
||||
});
|
||||
});
|
||||
|
||||
describe('shared-path masking', () => {
|
||||
/** A `Secrets/plan.txt` folder in `owner`'s Documents, shared with `to`. */
|
||||
const shareFolder = async (
|
||||
owner: Awaited<ReturnType<typeof makeUser>>,
|
||||
to: Awaited<ReturnType<typeof makeUser>>,
|
||||
) => {
|
||||
const dirUuid = uuidv4();
|
||||
const dirPath = `/${owner.username}/Documents/Secrets`;
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const parent = (await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}/Documents`,
|
||||
))!;
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `fsentries` (`uuid`, `name`, `path`, `user_id`, `is_dir`, `modified`, `parent_id`, `parent_uid`) VALUES (?, ?, ?, ?, 1, ?, ?, ?)',
|
||||
[
|
||||
dirUuid,
|
||||
'Secrets',
|
||||
dirPath,
|
||||
owner.userId,
|
||||
now,
|
||||
parent.id,
|
||||
parent.uuid,
|
||||
],
|
||||
);
|
||||
const dir = (await server.stores.fsEntry.getEntryByPath(dirPath))!;
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `fsentries` (`uuid`, `name`, `path`, `user_id`, `is_dir`, `modified`, `parent_id`, `parent_uid`) VALUES (?, ?, ?, ?, 0, ?, ?, ?)',
|
||||
[
|
||||
uuidv4(),
|
||||
'plan.txt',
|
||||
`${dirPath}/plan.txt`,
|
||||
owner.userId,
|
||||
now,
|
||||
dir.id,
|
||||
dir.uuid,
|
||||
],
|
||||
);
|
||||
|
||||
await runWithContext({ actor: owner.actor as never }, () =>
|
||||
server.services.share.share(owner.actor as never, {
|
||||
uid: dirUuid,
|
||||
recipient: { username: to.username! },
|
||||
mode: 'read',
|
||||
}),
|
||||
);
|
||||
return { dirUuid, dirPath };
|
||||
};
|
||||
|
||||
/**
|
||||
* PROPFIND under the ALS wrap a real request gets, pre-auth actor
|
||||
* empty.
|
||||
*/
|
||||
const propfind = async (path: string, actor: unknown, depth = '1') => {
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext({ actor: undefined }, () =>
|
||||
dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'PROPFIND',
|
||||
path,
|
||||
headers: { depth },
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
),
|
||||
);
|
||||
return captured;
|
||||
};
|
||||
|
||||
// The DAV controller authenticates in-controller, after the request
|
||||
// context snapshotted an empty `req.actor` — so it has to place the
|
||||
// actor into the context itself or outbound masking silently turns
|
||||
// off and PROPFIND lists the owner's real paths.
|
||||
it('lists a shared folder under its masked path, not the owner’s real one', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
const { dirUuid } = await shareFolder(owner, recipient);
|
||||
|
||||
const masked = `/${owner.username}/${dirUuid}/Secrets`;
|
||||
const captured = await propfind(masked, recipient.actor);
|
||||
|
||||
expect(captured.statusCode).toBe(207);
|
||||
const xml = String(captured.body);
|
||||
expect(xml).toContain(`${masked}/plan.txt`);
|
||||
expect(xml).not.toContain('/Documents/Secrets');
|
||||
});
|
||||
|
||||
it('lists each sharer as a collection under the root', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
await shareFolder(owner, recipient);
|
||||
|
||||
const captured = await propfind('/', recipient.actor);
|
||||
expect(captured.statusCode).toBe(207);
|
||||
const xml = String(captured.body);
|
||||
// The recipient's own home, plus the owner who shared with them.
|
||||
expect(xml).toContain(`<D:href>/${recipient.username}/</D:href>`);
|
||||
expect(xml).toContain(`<D:href>/${owner.username}/</D:href>`);
|
||||
});
|
||||
|
||||
it('omits the sharers at depth 0', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
await shareFolder(owner, recipient);
|
||||
|
||||
const captured = await propfind('/', recipient.actor, '0');
|
||||
expect(captured.statusCode).toBe(207);
|
||||
expect(String(captured.body)).not.toContain(`/${owner.username}/`);
|
||||
});
|
||||
|
||||
it('answers a sharer’s collection with the shares they issued', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
const { dirUuid } = await shareFolder(owner, recipient);
|
||||
|
||||
const captured = await propfind(
|
||||
`/${owner.username}`,
|
||||
recipient.actor,
|
||||
);
|
||||
expect(captured.statusCode).toBe(207);
|
||||
const xml = String(captured.body);
|
||||
expect(xml).toContain(`<D:href>/${owner.username}/</D:href>`);
|
||||
// The `<uuid>` level, which the share root hangs off. It borrows
|
||||
// the shared item's name so a client has something to show.
|
||||
expect(xml).toContain(
|
||||
`<D:href>/${owner.username}/${dirUuid}/</D:href>`,
|
||||
);
|
||||
expect(xml).toContain('<D:displayname>Secrets</D:displayname>');
|
||||
// Nothing else of the owner's shows through — not their home
|
||||
// directory's other children, not the folder the share sits in.
|
||||
expect(xml).not.toContain('Documents');
|
||||
});
|
||||
|
||||
it('keeps a sharer’s collection a 403 for a caller with no share', async () => {
|
||||
const owner = await makeUser();
|
||||
const stranger = await makeUser();
|
||||
await shareFolder(owner, await makeUser());
|
||||
|
||||
const captured = await propfind(
|
||||
`/${owner.username}`,
|
||||
stranger.actor,
|
||||
);
|
||||
expect(captured.statusCode).toBe(403);
|
||||
});
|
||||
});
|
||||
|
||||
// The virtual collections invent two levels of path that no fsentry backs.
|
||||
// Everything here checks that they only ever widen to what was actually
|
||||
// shared — the rest of the owner's tree has to stay unreachable and
|
||||
// unnamed.
|
||||
describe('shared-space containment', () => {
|
||||
/**
|
||||
* `owner` shares `Shared/` (holding `in.txt`) with `to`, and keeps
|
||||
* `Private/` (holding `out.txt`) to themselves.
|
||||
*/
|
||||
const shareOneOfTwo = async (
|
||||
owner: Awaited<ReturnType<typeof makeUser>>,
|
||||
to: Awaited<ReturnType<typeof makeUser>>,
|
||||
) => {
|
||||
const mk = async (
|
||||
method: string,
|
||||
path: string,
|
||||
extra: Record<string, string> = {},
|
||||
) => {
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext({ actor: undefined }, () =>
|
||||
dispatchMiddleware(
|
||||
makeReq({
|
||||
method,
|
||||
path,
|
||||
actor: owner.actor,
|
||||
headers: extra,
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
),
|
||||
);
|
||||
return captured;
|
||||
};
|
||||
await mk('MKCOL', `/${owner.username}/Documents/Shared`);
|
||||
await mk('MKCOL', `/${owner.username}/Documents/Private`);
|
||||
|
||||
const shared = (await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}/Documents/Shared`,
|
||||
))!;
|
||||
const private_ = (await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}/Documents/Private`,
|
||||
))!;
|
||||
|
||||
await runWithContext({ actor: owner.actor as never }, () =>
|
||||
server.services.share.share(owner.actor as never, {
|
||||
uid: shared.uuid,
|
||||
recipient: { username: to.username! },
|
||||
mode: 'read',
|
||||
}),
|
||||
);
|
||||
return { shared, private: private_ };
|
||||
};
|
||||
|
||||
const propfind = async (path: string, actor: unknown, depth = '1') => {
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext({ actor: undefined }, () =>
|
||||
dispatchMiddleware(
|
||||
makeReq({
|
||||
method: 'PROPFIND',
|
||||
path,
|
||||
headers: { depth },
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
),
|
||||
);
|
||||
return captured;
|
||||
};
|
||||
|
||||
it('lists only the items that were shared, not the owner’s siblings', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const { shared, private: hidden } = await shareOneOfTwo(
|
||||
owner,
|
||||
holder,
|
||||
);
|
||||
|
||||
const captured = await propfind(`/${owner.username}`, holder.actor);
|
||||
expect(captured.statusCode).toBe(207);
|
||||
const xml = String(captured.body);
|
||||
expect(xml).toContain(shared.uuid);
|
||||
expect(xml).not.toContain(hidden.uuid);
|
||||
expect(xml).not.toContain('Private');
|
||||
});
|
||||
|
||||
it('does not recurse past the uuid level at Depth: infinity', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const { shared } = await shareOneOfTwo(owner, holder);
|
||||
|
||||
const captured = await propfind(
|
||||
`/${owner.username}`,
|
||||
holder.actor,
|
||||
'infinity',
|
||||
);
|
||||
expect(captured.statusCode).toBe(207);
|
||||
const xml = String(captured.body);
|
||||
// The share's own href lives one segment deeper; asking for the
|
||||
// world must not walk into it.
|
||||
expect(xml).toContain(`/${shared.uuid}/`);
|
||||
expect(xml).not.toContain(`/${shared.uuid}/Shared`);
|
||||
});
|
||||
|
||||
it('refuses every way out of a share', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const { shared, private: hidden } = await shareOneOfTwo(
|
||||
owner,
|
||||
holder,
|
||||
);
|
||||
const root = `/${owner.username}/${shared.uuid}/Shared`;
|
||||
|
||||
for (const escape of [
|
||||
// Up and sideways out of the shared subtree.
|
||||
`${root}/../Private`,
|
||||
`${root}/../../Documents`,
|
||||
// Percent-encoded, because the path is decoded before it is
|
||||
// resolved — the guard has to sit after that, not before.
|
||||
`${root}/%2e%2e/Private`,
|
||||
// The owner's real path, named directly rather than walked to.
|
||||
`/${owner.username}/Documents/Private`,
|
||||
`/${owner.username}/Documents`,
|
||||
// A mask pointing at an entry the holder was never given —
|
||||
// both a bad tail under a uuid they do hold, and the uuid of
|
||||
// one they don't.
|
||||
`/${owner.username}/${shared.uuid}/Private`,
|
||||
`/${owner.username}/${hidden.uuid}`,
|
||||
`/${owner.username}/${hidden.uuid}/Private`,
|
||||
]) {
|
||||
const captured = await propfind(escape, holder.actor);
|
||||
expect(captured.statusCode, escape).toBeGreaterThanOrEqual(400);
|
||||
expect(captured.statusCode, escape).toBeLessThan(500);
|
||||
expect(String(captured.body), escape).not.toContain('Private');
|
||||
}
|
||||
});
|
||||
|
||||
it('does not leak one holder’s shares to another', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const outsider = await makeUser();
|
||||
await shareOneOfTwo(owner, holder);
|
||||
|
||||
const captured = await propfind('/', outsider.actor);
|
||||
expect(captured.statusCode).toBe(207);
|
||||
expect(String(captured.body)).not.toContain(owner.username);
|
||||
});
|
||||
|
||||
it('drops the sharer once the share is revoked', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const { shared } = await shareOneOfTwo(owner, holder);
|
||||
|
||||
expect(String((await propfind('/', holder.actor)).body)).toContain(
|
||||
`/${owner.username}/`,
|
||||
);
|
||||
|
||||
await runWithContext({ actor: owner.actor as never }, () =>
|
||||
server.services.share.unshare(owner.actor as never, {
|
||||
uid: shared.uuid,
|
||||
recipient: { username: holder.username! },
|
||||
}),
|
||||
);
|
||||
|
||||
const root = await propfind('/', holder.actor);
|
||||
expect(String(root.body)).not.toContain(`/${owner.username}/`);
|
||||
// And the level it used to stand in for is a plain 403 again.
|
||||
const owned = await propfind(`/${owner.username}`, holder.actor);
|
||||
expect(owned.statusCode).toBe(403);
|
||||
});
|
||||
|
||||
it('refuses writes and reads aimed at the invented levels', async () => {
|
||||
const owner = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const { shared } = await shareOneOfTwo(owner, holder);
|
||||
|
||||
const attempt = async (method: string, path: string) => {
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext({ actor: undefined }, () =>
|
||||
dispatchMiddleware(
|
||||
makeReq({
|
||||
method,
|
||||
path,
|
||||
actor: holder.actor,
|
||||
headers: { 'content-length': '3' },
|
||||
}),
|
||||
res,
|
||||
noop,
|
||||
),
|
||||
);
|
||||
return captured.statusCode;
|
||||
};
|
||||
|
||||
// A collection, so GET is a 400 either way — what matters is that
|
||||
// neither the owner level nor the uuid level accepts a write.
|
||||
expect(await attempt('GET', `/${owner.username}`)).not.toBe(200);
|
||||
expect(
|
||||
await attempt('MKCOL', `/${owner.username}/intruder`),
|
||||
).toBeGreaterThanOrEqual(400);
|
||||
expect(
|
||||
await attempt(
|
||||
'MKCOL',
|
||||
`/${owner.username}/${shared.uuid}/intruder`,
|
||||
),
|
||||
).toBeGreaterThanOrEqual(400);
|
||||
expect(
|
||||
await attempt('DELETE', `/${owner.username}/${shared.uuid}`),
|
||||
).toBeGreaterThanOrEqual(400);
|
||||
|
||||
// Nothing landed in the owner's tree.
|
||||
expect(
|
||||
await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}/intruder`,
|
||||
),
|
||||
).toBeFalsy();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -27,9 +27,31 @@ import {
|
||||
assertNotSuspended,
|
||||
assertVerifiedAccount,
|
||||
} from '../../core/http/middleware/gates.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import {
|
||||
All,
|
||||
Controller,
|
||||
Copy,
|
||||
Delete,
|
||||
Get,
|
||||
Head,
|
||||
Lock,
|
||||
Mkcol,
|
||||
Move,
|
||||
Options,
|
||||
Propfind,
|
||||
Proppatch,
|
||||
Put,
|
||||
Unlock,
|
||||
} from '../../core/http/decorators.js';
|
||||
import type { RouteOptions } from '../../core/http/types.js';
|
||||
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
|
||||
import { expandTildePath } from '../../services/fs/resolveNode.js';
|
||||
import {
|
||||
maskEntryPath,
|
||||
parseMaskedSharePath,
|
||||
resolveSharePath,
|
||||
} from '../../services/fs/sharePathMask.js';
|
||||
import { Context } from '../../core/context.js';
|
||||
import type { FSEntry } from '../../stores/fs/FSEntry.js';
|
||||
import { toLegacyEntry } from '../fs/legacyFsHelpers.js';
|
||||
import { PuterController } from '../types.js';
|
||||
@@ -43,12 +65,8 @@ import {
|
||||
refreshLock,
|
||||
} from './locks.js';
|
||||
import { DAV_CONCURRENT, DAV_LIMIT } from '../fs/limits.js';
|
||||
import {
|
||||
acquireConcurrent,
|
||||
checkRateLimit,
|
||||
computeNetworkFingerprint,
|
||||
} from '../../core/http/middleware/rateLimit.js';
|
||||
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
|
||||
import type { ResolvedShare } from '../../services/share/ShareService.js';
|
||||
|
||||
const DAV_HEADERS = {
|
||||
DAV: '1, 2, ordered-collections',
|
||||
@@ -62,11 +80,41 @@ const ALLOW_METHODS =
|
||||
const MACOS_JUNK_REGEX = /(?:^\.DS_Store$|^\._)/;
|
||||
|
||||
/**
|
||||
* Verbs that move file content or duplicate it in the object store, and so are
|
||||
* refused to an account with nothing left of its budget. HEAD is here with GET
|
||||
* because a client asking for headers is a client about to fetch the body.
|
||||
* Every DAV verb mounts on the same catch-all: on a DAV host the path _is_ the
|
||||
* resource, so there is nothing else to route on. `{*splat}` rather than plain
|
||||
* `*splat` because only the braced form also matches `/`, which is the first
|
||||
* collection a client PROPFINDs.
|
||||
*/
|
||||
const CREDIT_GATED_DAV_METHODS = new Set(['GET', 'HEAD', 'PUT', 'COPY']);
|
||||
const DAV_ROUTE = '/{*splat}';
|
||||
|
||||
/**
|
||||
* Everything the route table declares. The subdomain keeps DAV off every other
|
||||
* host, and both limits key on the network fingerprint because nothing has
|
||||
* authenticated yet when they run — which is the point of putting them here
|
||||
* rather than inside the handler. A DAV client sends credentials on every
|
||||
* request anyway, so there is no unauthenticated browsing phase that a per-user
|
||||
* key would protect.
|
||||
*/
|
||||
const DAV_ROUTE_OPTIONS: RouteOptions = {
|
||||
subdomain: 'dav',
|
||||
rateLimit: DAV_LIMIT,
|
||||
concurrent: DAV_CONCURRENT,
|
||||
};
|
||||
|
||||
/** How long a browser may cache a DAV preflight. */
|
||||
const PREFLIGHT_MAX_AGE = '86400';
|
||||
|
||||
/** How many shared items the virtual share collections list. */
|
||||
const SHARE_LISTING_CAP = 200;
|
||||
|
||||
/** What `#context` resolved for one request, handed to every verb handler. */
|
||||
interface DavContext {
|
||||
actor: Actor;
|
||||
/** The real path being addressed, with `~` and any share mask resolved. */
|
||||
davPath: string;
|
||||
/** Lock token from `If:` / `Lock-Token:`, when the client sent one. */
|
||||
lockToken: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* WebDAV controller — full RFC 4918 surface on the `dav.*` subdomain.
|
||||
@@ -79,93 +127,162 @@ const CREDIT_GATED_DAV_METHODS = new Set(['GET', 'HEAD', 'PUT', 'COPY']);
|
||||
* `-token` username for token-based auth). Falls back to the global authProbe's
|
||||
* `req.actor` if a session cookie is present.
|
||||
*/
|
||||
@Controller()
|
||||
export class WebDAVController extends PuterController {
|
||||
registerRoutes(router: PuterRouter): void {
|
||||
// Single catch-all on the `dav` subdomain. We dispatch by req.method
|
||||
// inside the handler because WebDAV uses non-standard HTTP verbs that
|
||||
// Express doesn't have first-class router methods for in all versions.
|
||||
//
|
||||
// The rate limit is applied inside the handler rather than through
|
||||
// `RouteOptions`. For a `use` mount the subdomain check lives in the
|
||||
// handler wrapper, not in the middleware chain — so a `rateLimit`
|
||||
// here would run for every request on every subdomain and count
|
||||
// non-DAV traffic against the DAV budget.
|
||||
router.use(
|
||||
{ subdomain: 'dav' },
|
||||
async (req: Request, res: Response, _next) => {
|
||||
try {
|
||||
if (!(await this.#admit(req, res))) return;
|
||||
await this.#dispatch(req, res);
|
||||
} catch (err) {
|
||||
if (err instanceof HttpError) {
|
||||
res.status(err.statusCode).send(err.message);
|
||||
return;
|
||||
}
|
||||
console.error('[webdav] unhandled error', err);
|
||||
res.status(500).send('Internal Server Error');
|
||||
}
|
||||
// Don't call next — we always handle or error.
|
||||
},
|
||||
);
|
||||
// -- Routes ------------------------------------------------------
|
||||
//
|
||||
// One route per verb, every one of them the same catch-all. Declaration
|
||||
// order is registration order: HEAD before GET, because express falls a
|
||||
// HEAD request back onto a GET route when it meets one first, and `@All`
|
||||
// last, because it matches any method.
|
||||
//
|
||||
// `DAV_ROUTE_OPTIONS` is the whole declarative half of the chain. The
|
||||
// credential work happens in `#serve` instead, which is what keeps the
|
||||
// limits ahead of it: HTTP Basic means a bcrypt compare per request, and a
|
||||
// caller already over budget shouldn't get one.
|
||||
|
||||
@Options(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
options(req: Request, res: Response): Promise<void> {
|
||||
// Before the auth gate, and only for a real preflight — see
|
||||
// `answerPreflight`.
|
||||
if (answerPreflight(req, res)) return Promise.resolve();
|
||||
return this.#serve(req, res, () => this.#options(res));
|
||||
}
|
||||
|
||||
// GET/HEAD/PUT/COPY move file content or duplicate it in the object store,
|
||||
// so they carry the same budget gate the FS routes declare with
|
||||
// `requireCredits`: DAV serves the same files over a metered host, and
|
||||
// leaving it out would make mounting the drive the way around enforcement.
|
||||
// The verbs that only describe or remove things stay open, as they do over
|
||||
// HTTP. HEAD is metered with GET because a client asking for headers is a
|
||||
// client about to fetch the body.
|
||||
//
|
||||
// `requireCredits` itself can't be used for this: it reads `req.actor`, and
|
||||
// on the DAV host nothing has authenticated by the time route options run.
|
||||
|
||||
@Head(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
head(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#get(req, res, ctx, true), {
|
||||
credits: true,
|
||||
});
|
||||
}
|
||||
|
||||
@Get(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
get(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#get(req, res, ctx, false), {
|
||||
credits: true,
|
||||
});
|
||||
}
|
||||
|
||||
@Propfind(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
propfind(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#propfind(req, res, ctx));
|
||||
}
|
||||
|
||||
@Proppatch(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
proppatch(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#proppatch(res, ctx));
|
||||
}
|
||||
|
||||
@Mkcol(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
mkcol(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#mkcol(req, res, ctx));
|
||||
}
|
||||
|
||||
@Put(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
put(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#put(req, res, ctx), {
|
||||
credits: true,
|
||||
});
|
||||
}
|
||||
|
||||
@Delete(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
delete(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#delete(res, ctx));
|
||||
}
|
||||
|
||||
@Copy(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
copy(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#copy(req, res, ctx), {
|
||||
credits: true,
|
||||
});
|
||||
}
|
||||
|
||||
@Move(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
move(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#move(req, res, ctx));
|
||||
}
|
||||
|
||||
@Lock(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
lock(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#lock(req, res, ctx));
|
||||
}
|
||||
|
||||
@Unlock(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
unlock(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, (ctx) => this.#unlock(req, res, ctx));
|
||||
}
|
||||
|
||||
/** POST, TRACE, an extension verb we don't implement. */
|
||||
@All(DAV_ROUTE, DAV_ROUTE_OPTIONS)
|
||||
unsupported(req: Request, res: Response): Promise<void> {
|
||||
return this.#serve(req, res, () => {
|
||||
res.status(405)
|
||||
.set('Allow', ALLOW_METHODS)
|
||||
.send('Method Not Allowed');
|
||||
});
|
||||
}
|
||||
|
||||
// -- Request plumbing ---------------------------------------------
|
||||
|
||||
/**
|
||||
* Rate + concurrency gate for the whole DAV surface. Returns false when the
|
||||
* request was rejected (429 already sent).
|
||||
*
|
||||
* Runs before `#dispatch` authenticates, so it keys on the network
|
||||
* fingerprint rather than an actor. That is the coarser bucket, but a DAV
|
||||
* client sends credentials on every request anyway — there is no
|
||||
* unauthenticated browsing phase to protect a per-user key from.
|
||||
* Authenticate, gate, resolve the target path, and run `handler` — or
|
||||
* answer the request instead when any of that fails. A thrown `HttpError`
|
||||
* becomes the plain-text reply a DAV client reads; nothing calls `next`,
|
||||
* because a request that gets here is ours to answer.
|
||||
*/
|
||||
async #admit(req: Request, res: Response): Promise<boolean> {
|
||||
const key = computeNetworkFingerprint(req);
|
||||
if (
|
||||
!(await checkRateLimit(
|
||||
`${DAV_LIMIT.scope}:${key}`,
|
||||
DAV_LIMIT.limit,
|
||||
DAV_LIMIT.window,
|
||||
))
|
||||
) {
|
||||
res.status(429).send('Too many requests.');
|
||||
return false;
|
||||
async #serve(
|
||||
req: Request,
|
||||
res: Response,
|
||||
handler: (ctx: DavContext) => void | Promise<void>,
|
||||
opts: { credits?: boolean } = {},
|
||||
): Promise<void> {
|
||||
try {
|
||||
const ctx = await this.#context(req, res, opts);
|
||||
if (!ctx) return; // 401 already sent
|
||||
await handler(ctx);
|
||||
} catch (err) {
|
||||
sendDavError(res, err);
|
||||
}
|
||||
const slot = await acquireConcurrent(
|
||||
`${DAV_CONCURRENT.scope}:${key}`,
|
||||
DAV_CONCURRENT.limit,
|
||||
);
|
||||
if (!slot.ok) {
|
||||
res.status(429).send('Too many concurrent requests.');
|
||||
return false;
|
||||
}
|
||||
// `finish` and `close` can both fire; release is once-only.
|
||||
res.once('finish', () => void slot.release());
|
||||
res.once('close', () => void slot.release());
|
||||
return true;
|
||||
}
|
||||
|
||||
async #dispatch(req: Request, res: Response): Promise<void> {
|
||||
// Authenticate
|
||||
/** The actor, path and lock token for one request; null once 401'd. */
|
||||
async #context(
|
||||
req: Request,
|
||||
res: Response,
|
||||
opts: { credits?: boolean },
|
||||
): Promise<DavContext | null> {
|
||||
const actor = await this.#resolveActor(req, res);
|
||||
if (!actor) return; // 401 already sent
|
||||
if (!actor) return null;
|
||||
|
||||
// Apply the same suspension + pending-verification gates every other
|
||||
// authenticated route gets from `requireAuthGate` / `requireVerifiedAccount`.
|
||||
// WebDAV dispatches every method off a single `router.use` with no route
|
||||
// options, so that middleware is never inserted into its chain — without
|
||||
// these calls a suspended account (or one still pending email / phone /
|
||||
// card verification) could read, write, and delete its entire filesystem
|
||||
// over the `dav` subdomain, bypassing the gates. Both throw a 403
|
||||
// HttpError, surfaced by the catch in registerRoutes.
|
||||
// The same suspension + pending-verification gates every other
|
||||
// authenticated route gets from `requireAuthGate` /
|
||||
// `requireVerifiedAccount`. DAV authenticates itself rather than
|
||||
// through the auth probe, so that middleware is never in its chain —
|
||||
// without these calls a suspended account (or one still pending email /
|
||||
// phone / card verification) could read, write, and delete its entire
|
||||
// filesystem over the `dav` subdomain, bypassing the gates.
|
||||
assertNotSuspended(actor.user);
|
||||
assertVerifiedAccount(actor.user);
|
||||
|
||||
// And the same budget gate the FS routes declare with
|
||||
// `requireCredits`, for the verbs that move content — DAV serves the
|
||||
// same files over a metered host, so leaving it out would make mounting
|
||||
// the drive the way around enforcement. The verbs that only describe or
|
||||
// remove things stay open, as they do over HTTP.
|
||||
if (CREDIT_GATED_DAV_METHODS.has(req.method.toUpperCase())) {
|
||||
// The actor was absent when the auth probe ran, so anything that reads
|
||||
// it off the request or the context — shared-path masking, egress
|
||||
// metering, which bills the `dav` host — is blind until it's published
|
||||
// here.
|
||||
req.actor = actor;
|
||||
if (Context.current()) Context.set('actor', actor);
|
||||
|
||||
if (opts.credits) {
|
||||
await assertActorHasCredits(
|
||||
this.services.metering,
|
||||
actor,
|
||||
@@ -176,51 +293,20 @@ export class WebDAVController extends PuterController {
|
||||
// Expand `~`/`~/...` against the authenticated actor's username.
|
||||
// WebDAV doesn't standardize `~`, but some clients do — and the
|
||||
// pre-existing behaviour silently expanded it via the FS store.
|
||||
const davPath = expandTildePath(
|
||||
decodeURIComponent(req.path),
|
||||
actor.user.username,
|
||||
);
|
||||
const redis = this.clients.redis;
|
||||
const lockToken = extractLockToken(
|
||||
(req.headers['if'] as string | undefined) ??
|
||||
(req.headers['lock-token'] as string | undefined),
|
||||
const davPath = await resolveSharePath(
|
||||
this.stores.fsEntry,
|
||||
actor,
|
||||
expandTildePath(decodeURIComponent(req.path), actor.user.username),
|
||||
);
|
||||
|
||||
switch (req.method.toUpperCase()) {
|
||||
case 'OPTIONS':
|
||||
return this.#options(res);
|
||||
case 'HEAD':
|
||||
case 'GET':
|
||||
return this.#get(
|
||||
req,
|
||||
res,
|
||||
actor,
|
||||
davPath,
|
||||
req.method === 'HEAD',
|
||||
);
|
||||
case 'PROPFIND':
|
||||
return this.#propfind(req, res, actor, davPath);
|
||||
case 'PROPPATCH':
|
||||
return this.#proppatch(res, davPath, redis, lockToken);
|
||||
case 'MKCOL':
|
||||
return this.#mkcol(req, res, actor, davPath, redis, lockToken);
|
||||
case 'PUT':
|
||||
return this.#put(req, res, actor, davPath, redis, lockToken);
|
||||
case 'DELETE':
|
||||
return this.#delete(res, actor, davPath, redis, lockToken);
|
||||
case 'COPY':
|
||||
return this.#copy(req, res, actor, davPath, redis, lockToken);
|
||||
case 'MOVE':
|
||||
return this.#move(req, res, actor, davPath, redis, lockToken);
|
||||
case 'LOCK':
|
||||
return this.#lock(req, res, actor, davPath, redis, lockToken);
|
||||
case 'UNLOCK':
|
||||
return this.#unlock(req, res, davPath, redis);
|
||||
default:
|
||||
res.status(405)
|
||||
.set('Allow', ALLOW_METHODS)
|
||||
.send('Method Not Allowed');
|
||||
}
|
||||
return {
|
||||
actor,
|
||||
davPath,
|
||||
lockToken: extractLockToken(
|
||||
(req.headers['if'] as string | undefined) ??
|
||||
(req.headers['lock-token'] as string | undefined),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
// -- Auth ---------------------------------------------------------
|
||||
@@ -349,8 +435,7 @@ export class WebDAVController extends PuterController {
|
||||
async #get(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
{ actor, davPath }: DavContext,
|
||||
headOnly: boolean,
|
||||
): Promise<void> {
|
||||
const entry = await this.stores.fsEntry.getEntryByPath(davPath);
|
||||
@@ -363,15 +448,14 @@ export class WebDAVController extends PuterController {
|
||||
|
||||
await this.#assertRead(actor, davPath);
|
||||
|
||||
const etag = `"${entry.uuid}-${Math.floor(entry.modified ?? entry.created ?? 0)}"`;
|
||||
const modified = entry.modified ?? entry.created ?? 0;
|
||||
const etag = entryEtag(entry.uuid, modified);
|
||||
const size = entry.size ?? 0;
|
||||
|
||||
res.set({
|
||||
'Accept-Ranges': 'bytes',
|
||||
'Content-Length': String(size),
|
||||
'Last-Modified': new Date(
|
||||
entry.modified ?? entry.created ?? 0,
|
||||
).toUTCString(),
|
||||
'Last-Modified': entryDate(modified).toUTCString(),
|
||||
ETag: etag,
|
||||
});
|
||||
|
||||
@@ -399,43 +483,172 @@ export class WebDAVController extends PuterController {
|
||||
async #propfind(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
{ actor, davPath }: DavContext,
|
||||
): Promise<void> {
|
||||
const depth = req.headers.depth ?? '1';
|
||||
|
||||
const entry =
|
||||
davPath === '/'
|
||||
? null // root always exists
|
||||
: await this.stores.fsEntry.getEntryByPath(davPath);
|
||||
if (davPath !== '/' && !entry)
|
||||
if (davPath === '/') {
|
||||
await this.#assertRead(actor, davPath);
|
||||
this.#sendMultistatus(res, await this.#rootPropfind(actor, depth));
|
||||
return;
|
||||
}
|
||||
|
||||
const entry = await this.stores.fsEntry.getEntryByPath(davPath);
|
||||
|
||||
// The two levels a share mask invents — `/<owner>` and
|
||||
// `/<owner>/<uuid>` — have no entry behind them, so they answer as
|
||||
// virtual collections. Tried before the ACL check because `/<owner>`
|
||||
// _is_ a real path (the owner's home directory) that the recipient
|
||||
// can't read: a 403 there would hide every share they do have.
|
||||
const virtual =
|
||||
(await this.#shareOwnerPropfind(actor, davPath, entry, depth)) ??
|
||||
(entry
|
||||
? null
|
||||
: await this.#shareRootParentPropfind(actor, davPath, depth));
|
||||
if (virtual) {
|
||||
this.#sendMultistatus(res, virtual);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!entry) {
|
||||
throw new HttpError(404, 'Not Found', { legacyCode: 'not_found' });
|
||||
}
|
||||
|
||||
await this.#assertRead(actor, davPath);
|
||||
|
||||
const isDir = davPath === '/' || !!entry?.isDir;
|
||||
const responses = [propfindEntry(davPath, entry, isDir)];
|
||||
// `davPath` is the resolved real path; the href has to be the masked
|
||||
// one, like every child below it, or the self-entry names the owner's
|
||||
// real folder.
|
||||
const responses = [
|
||||
propfindEntry(maskEntryPath(entry), entry, entry.isDir),
|
||||
];
|
||||
|
||||
if (depth !== '0' && isDir && entry) {
|
||||
if (depth !== '0' && entry.isDir) {
|
||||
const children = await this.services.fs.listDirectory(
|
||||
entry.uuid,
|
||||
{},
|
||||
);
|
||||
for (const child of children) {
|
||||
responses.push(propfindEntry(child.path, child, child.isDir));
|
||||
}
|
||||
} else if (depth !== '0' && davPath === '/') {
|
||||
// Root: list top-level user directories
|
||||
const rootEntry = await this.stores.fsEntry.getEntryByPath(
|
||||
`/${actor.user!.username}`,
|
||||
);
|
||||
if (rootEntry) {
|
||||
responses.push(
|
||||
propfindEntry(rootEntry.path, rootEntry, rootEntry.isDir),
|
||||
propfindEntry(maskEntryPath(child), child, child.isDir),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
this.#sendMultistatus(res, responses);
|
||||
}
|
||||
|
||||
/**
|
||||
* The root collection: the caller's own home directory, plus one collection
|
||||
* per person who has shared something with them.
|
||||
*
|
||||
* A share is addressed `/<owner>/<uuid>/<name>`, where the uuid stands in
|
||||
* for the folder the owner keeps it in. Only the `<owner>` segment is a
|
||||
* child of the root, so without this level nothing shared is reachable from
|
||||
* a mount at all — which is why a Finder mount showed only the caller's own
|
||||
* files.
|
||||
*/
|
||||
async #rootPropfind(actor: Actor, depth: string | string[]) {
|
||||
const responses = [propfindEntry('/', null, true)];
|
||||
if (depth === '0') return responses;
|
||||
|
||||
const home = await this.stores.fsEntry.getEntryByPath(
|
||||
`/${actor.user!.username}`,
|
||||
);
|
||||
if (home) {
|
||||
responses.push(
|
||||
propfindEntry(maskEntryPath(home), home, home.isDir),
|
||||
);
|
||||
}
|
||||
|
||||
const owners = new Set<string>();
|
||||
for (const share of await this.#sharedWithMe(actor)) {
|
||||
const owner = share.owner?.username;
|
||||
// Self-owned entries never appear in the share index, but the home
|
||||
// directory above already covers them if one ever did.
|
||||
if (owner && owner !== actor.user!.username) owners.add(owner);
|
||||
}
|
||||
for (const owner of owners) {
|
||||
responses.push(propfindEntry(`/${owner}`, { name: owner }, true));
|
||||
}
|
||||
|
||||
return responses;
|
||||
}
|
||||
|
||||
/**
|
||||
* PROPFIND responses for the virtual collection at `/<owner>`, listing one
|
||||
* child per item that owner shared with the actor, or null when `davPath`
|
||||
* isn't a bare username, is the actor's own, is readable for real, or the
|
||||
* actor holds nothing of theirs — the caller then continues down the normal
|
||||
* path and gets the 403 or 404 it would otherwise have given.
|
||||
*
|
||||
* Children are the `<uuid>` level rather than the shared item itself: the
|
||||
* item's own href sits one segment deeper, and a collection may only report
|
||||
* its direct members. Nothing here reaches past that level, so `Depth:
|
||||
* infinity` on a sharer's collection still enumerates only what they
|
||||
* shared, and everything below it goes through the ordinary resolve + ACL
|
||||
* path.
|
||||
*/
|
||||
async #shareOwnerPropfind(
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
entry: FSEntry | null,
|
||||
depth: string | string[],
|
||||
): Promise<string[] | null> {
|
||||
const segments = davPath.split('/').filter(Boolean);
|
||||
if (segments.length !== 1) return null;
|
||||
const owner = segments[0]!;
|
||||
if (owner === actor.user?.username) return null;
|
||||
// Every provisioned user's home directory is a real entry and a
|
||||
// sharer is necessarily provisioned, so no directory here means no
|
||||
// share to stand in for. Checked before the share lookup so an
|
||||
// unknown name costs a 404 rather than a listing.
|
||||
if (!entry?.isDir) return null;
|
||||
// An owner who shared their whole home directory has a real listing
|
||||
// here, and it beats the virtual one.
|
||||
if (await this.#canRead(actor, davPath)) return null;
|
||||
|
||||
const shares = (await this.#sharedWithMe(actor)).filter(
|
||||
(share) => share.owner?.username === owner,
|
||||
);
|
||||
if (shares.length === 0) return null;
|
||||
|
||||
const responses = [propfindEntry(davPath, { name: owner }, true)];
|
||||
if (depth !== '0') {
|
||||
for (const share of shares) {
|
||||
responses.push(
|
||||
propfindEntry(
|
||||
`/${owner}/${share.entryUid}`,
|
||||
{
|
||||
name: share.name,
|
||||
uuid: share.entryUid,
|
||||
modified: share.modified,
|
||||
},
|
||||
true,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
return responses;
|
||||
}
|
||||
|
||||
/**
|
||||
* What has been shared with the actor, for the virtual collections above.
|
||||
* One page: WebDAV has no way to ask for the next one, so a caller holding
|
||||
* more shares than {@link SHARE_LISTING_CAP} sees the oldest that many.
|
||||
*
|
||||
* Listing also teaches the request's path masker every root it returns, so
|
||||
* an entry reached through one keeps that masked prefix on the way back out
|
||||
* instead of being masked against itself.
|
||||
*/
|
||||
async #sharedWithMe(actor: Actor): Promise<ResolvedShare[]> {
|
||||
const page = await this.services.share.listSharedWithMe(actor, {
|
||||
limit: SHARE_LISTING_CAP,
|
||||
});
|
||||
return page.items;
|
||||
}
|
||||
|
||||
#sendMultistatus(res: Response, responses: string[]): void {
|
||||
res.status(207)
|
||||
.set({ 'Content-Type': 'application/xml; charset=utf-8' })
|
||||
.send(wrapMultistatus(responses.join('\n')));
|
||||
@@ -445,16 +658,10 @@ export class WebDAVController extends PuterController {
|
||||
|
||||
async #proppatch(
|
||||
res: Response,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
lockToken: string | null,
|
||||
{ davPath, lockToken }: DavContext,
|
||||
): Promise<void> {
|
||||
if (
|
||||
!(await hasWritePermission(
|
||||
redis as import('ioredis').Cluster,
|
||||
davPath,
|
||||
lockToken,
|
||||
))
|
||||
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
|
||||
) {
|
||||
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
|
||||
}
|
||||
@@ -470,10 +677,7 @@ export class WebDAVController extends PuterController {
|
||||
async #mkcol(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
lockToken: string | null,
|
||||
{ actor, davPath, lockToken }: DavContext,
|
||||
): Promise<void> {
|
||||
if (davPath === '/')
|
||||
throw new HttpError(403, 'Cannot create at root', {
|
||||
@@ -488,11 +692,7 @@ export class WebDAVController extends PuterController {
|
||||
});
|
||||
}
|
||||
if (
|
||||
!(await hasWritePermission(
|
||||
redis as import('ioredis').Cluster,
|
||||
davPath,
|
||||
lockToken,
|
||||
))
|
||||
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
|
||||
) {
|
||||
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
|
||||
}
|
||||
@@ -520,10 +720,7 @@ export class WebDAVController extends PuterController {
|
||||
async #put(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
lockToken: string | null,
|
||||
{ actor, davPath, lockToken }: DavContext,
|
||||
): Promise<void> {
|
||||
const name = pathPosix.basename(davPath);
|
||||
if (MACOS_JUNK_REGEX.test(name)) {
|
||||
@@ -531,11 +728,7 @@ export class WebDAVController extends PuterController {
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!(await hasWritePermission(
|
||||
redis as import('ioredis').Cluster,
|
||||
davPath,
|
||||
lockToken,
|
||||
))
|
||||
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
|
||||
) {
|
||||
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
|
||||
}
|
||||
@@ -580,13 +773,11 @@ export class WebDAVController extends PuterController {
|
||||
);
|
||||
|
||||
const fe = writeResult.fsEntry;
|
||||
const etag = `"${fe.uuid}-${Math.floor(fe.modified ?? fe.created ?? 0)}"`;
|
||||
const modified = fe.modified ?? fe.created ?? 0;
|
||||
res.status(existing ? 204 : 201)
|
||||
.set({
|
||||
ETag: etag,
|
||||
'Last-Modified': new Date(
|
||||
fe.modified ?? fe.created ?? 0,
|
||||
).toUTCString(),
|
||||
ETag: entryEtag(fe.uuid, modified),
|
||||
'Last-Modified': entryDate(modified).toUTCString(),
|
||||
})
|
||||
.end();
|
||||
}
|
||||
@@ -595,17 +786,10 @@ export class WebDAVController extends PuterController {
|
||||
|
||||
async #delete(
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
lockToken: string | null,
|
||||
{ actor, davPath, lockToken }: DavContext,
|
||||
): Promise<void> {
|
||||
if (
|
||||
!(await hasWritePermission(
|
||||
redis as import('ioredis').Cluster,
|
||||
davPath,
|
||||
lockToken,
|
||||
))
|
||||
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
|
||||
) {
|
||||
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
|
||||
}
|
||||
@@ -626,18 +810,11 @@ export class WebDAVController extends PuterController {
|
||||
async #copy(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
lockToken: string | null,
|
||||
{ actor, davPath, lockToken }: DavContext,
|
||||
): Promise<void> {
|
||||
const destPath = this.#parseDestination(req);
|
||||
const destPath = await this.#parseDestination(req);
|
||||
if (
|
||||
!(await hasWritePermission(
|
||||
redis as import('ioredis').Cluster,
|
||||
destPath,
|
||||
lockToken,
|
||||
))
|
||||
!(await hasWritePermission(this.clients.redis, destPath, lockToken))
|
||||
) {
|
||||
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
|
||||
}
|
||||
@@ -684,13 +861,10 @@ export class WebDAVController extends PuterController {
|
||||
async #move(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
lockToken: string | null,
|
||||
{ actor, davPath, lockToken }: DavContext,
|
||||
): Promise<void> {
|
||||
const destPath = this.#parseDestination(req);
|
||||
const r = redis as import('ioredis').Cluster;
|
||||
const destPath = await this.#parseDestination(req);
|
||||
const r = this.clients.redis;
|
||||
if (!(await hasWritePermission(r, davPath, lockToken)))
|
||||
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
|
||||
if (!(await hasWritePermission(r, destPath, lockToken)))
|
||||
@@ -740,12 +914,9 @@ export class WebDAVController extends PuterController {
|
||||
async #lock(
|
||||
req: Request,
|
||||
res: Response,
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
headerToken: string | null,
|
||||
{ actor, davPath, lockToken: headerToken }: DavContext,
|
||||
): Promise<void> {
|
||||
const r = redis as import('ioredis').Cluster;
|
||||
const r = this.clients.redis;
|
||||
|
||||
// ACL must succeed before any lock state is touched — otherwise
|
||||
// an authenticated user could lock paths they don't own (e.g. `/`)
|
||||
@@ -807,10 +978,9 @@ export class WebDAVController extends PuterController {
|
||||
async #unlock(
|
||||
req: Request,
|
||||
res: Response,
|
||||
davPath: string,
|
||||
redis: unknown,
|
||||
{ davPath }: DavContext,
|
||||
): Promise<void> {
|
||||
const r = redis as import('ioredis').Cluster;
|
||||
const r = this.clients.redis;
|
||||
const tokenHeader = req.headers['lock-token'] as string | undefined;
|
||||
const token = extractLockToken(tokenHeader);
|
||||
if (!token)
|
||||
@@ -833,15 +1003,65 @@ export class WebDAVController extends PuterController {
|
||||
res.status(204).end();
|
||||
}
|
||||
|
||||
/**
|
||||
* PROPFIND responses for the virtual collection at `/<owner>/<uuid>`, or
|
||||
* null when `davPath` isn't that shape, the uuid doesn't resolve to an
|
||||
* entry of `owner`'s, or the actor cannot read the share — the caller falls
|
||||
* through to 404 in every null case, so an unauthorized probe learns
|
||||
* nothing about whether the uuid exists.
|
||||
*/
|
||||
async #shareRootParentPropfind(
|
||||
actor: Actor,
|
||||
davPath: string,
|
||||
depth: string | string[],
|
||||
): Promise<string[] | null> {
|
||||
const parsed = parseMaskedSharePath(davPath);
|
||||
if (!parsed || parsed.tail !== '') return null;
|
||||
if (parsed.ownerUsername === actor.user?.username) return null;
|
||||
|
||||
const root = await this.stores.fsEntry.getEntryByUuid(parsed.rootUuid);
|
||||
if (!root) return null;
|
||||
// Same guard as resolveSharePath: the mask names the owner, so the
|
||||
// uuid must be theirs.
|
||||
if (root.path.split('/')[1] !== parsed.ownerUsername) return null;
|
||||
|
||||
const allowed = await this.services.acl.check(
|
||||
actor,
|
||||
{
|
||||
path: root.path,
|
||||
resolveAncestors: () =>
|
||||
this.services.fs.getAncestorChain(root.path),
|
||||
},
|
||||
'read',
|
||||
);
|
||||
if (!allowed) return null;
|
||||
|
||||
const maskedRoot = `/${parsed.ownerUsername}/${root.uuid}/${root.name}`;
|
||||
// The uuid segment has no name of its own, so it borrows the shared
|
||||
// item's — a client that reads `displayname` shows something
|
||||
// recognizable rather than a bare uuid.
|
||||
const responses = [propfindEntry(davPath, { name: root.name }, true)];
|
||||
if (depth !== '0') {
|
||||
responses.push(propfindEntry(maskedRoot, root, root.isDir));
|
||||
}
|
||||
return responses;
|
||||
}
|
||||
|
||||
// -- ACL helpers -------------------------------------------------
|
||||
|
||||
async #canRead(actor: Actor, path: string): Promise<boolean> {
|
||||
return await this.services.acl.check(
|
||||
actor,
|
||||
{
|
||||
path,
|
||||
resolveAncestors: () => this.services.fs.getAncestorChain(path),
|
||||
},
|
||||
'read',
|
||||
);
|
||||
}
|
||||
|
||||
async #assertRead(actor: Actor, path: string): Promise<void> {
|
||||
const descriptor = {
|
||||
path,
|
||||
resolveAncestors: () => this.services.fs.getAncestorChain(path),
|
||||
};
|
||||
const ok = await this.services.acl.check(actor, descriptor, 'read');
|
||||
if (!ok)
|
||||
if (!(await this.#canRead(actor, path)))
|
||||
throw new HttpError(403, 'Permission denied', {
|
||||
legacyCode: 'permission_denied',
|
||||
});
|
||||
@@ -890,21 +1110,72 @@ export class WebDAVController extends PuterController {
|
||||
|
||||
// -- Misc helpers ------------------------------------------------
|
||||
|
||||
#parseDestination(req: Request): string {
|
||||
async #parseDestination(req: Request): Promise<string> {
|
||||
const dest = req.headers.destination as string | undefined;
|
||||
if (!dest)
|
||||
throw new HttpError(400, 'Missing Destination header', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
let raw: string;
|
||||
try {
|
||||
const url = new URL(dest, `http://${req.headers.host}`);
|
||||
return decodeURIComponent(url.pathname);
|
||||
raw = decodeURIComponent(url.pathname);
|
||||
} catch {
|
||||
return decodeURIComponent(dest);
|
||||
raw = decodeURIComponent(dest);
|
||||
}
|
||||
// The destination is addressed the same way as the request path, so a
|
||||
// client that browsed into a share names its target the same way too.
|
||||
return resolveSharePath(this.stores.fsEntry, Context.get('actor'), raw);
|
||||
}
|
||||
}
|
||||
|
||||
// -- Middleware helpers -----------------------------------------------
|
||||
|
||||
/**
|
||||
* Answer a CORS preflight before anything can reject it. True when this request
|
||||
* was one and has been answered.
|
||||
*
|
||||
* A browser sends a credential-less `OPTIONS` ahead of any cross-origin
|
||||
* PROPFIND/GET/PUT and treats a non-2xx reply as a failed preflight, never
|
||||
* sending the real request. The authenticated OPTIONS path answers 401, which
|
||||
* blocked every browser DAV client — the VS Code app's mount included — before
|
||||
* it could present a token.
|
||||
*
|
||||
* Only an actual preflight is answered here: it's the one OPTIONS that carries
|
||||
* `Access-Control-Request-Method`. A native client's plain OPTIONS still goes
|
||||
* through auth, so macOS keeps getting the 401-with-`DAV:` reply it opens a
|
||||
* mount with.
|
||||
*/
|
||||
function answerPreflight(req: Request, res: Response): boolean {
|
||||
if (!req.headers.origin || !req.headers['access-control-request-method']) {
|
||||
return false;
|
||||
}
|
||||
// `Allow-Origin`/`-Methods`/`-Headers` were set on the way in by the
|
||||
// server's CORS middleware. `Max-Age` is what belongs here specifically:
|
||||
// without it a browser re-preflights every URL every few seconds, doubling
|
||||
// the request count of a mount and its share of the DAV rate limit.
|
||||
res.status(200)
|
||||
.set({
|
||||
...DAV_HEADERS,
|
||||
Allow: ALLOW_METHODS,
|
||||
'Access-Control-Max-Age': PREFLIGHT_MAX_AGE,
|
||||
'Content-Length': '0',
|
||||
})
|
||||
.end();
|
||||
return true;
|
||||
}
|
||||
|
||||
/** DAV clients read a status and a plain-text reason, not our JSON error shape. */
|
||||
function sendDavError(res: Response, err: unknown): void {
|
||||
if (res.headersSent) return;
|
||||
if (err instanceof HttpError) {
|
||||
res.status(err.statusCode).send(err.message);
|
||||
return;
|
||||
}
|
||||
console.error('[webdav] unhandled error', err);
|
||||
res.status(500).send('Internal Server Error');
|
||||
}
|
||||
|
||||
// -- XML helpers ------------------------------------------------------
|
||||
|
||||
function escapeXml(text: string): string {
|
||||
@@ -920,9 +1191,42 @@ function wrapMultistatus(inner: string): string {
|
||||
return `<?xml version="1.0" encoding="utf-8"?>\n<D:multistatus xmlns:D="DAV:">\n${inner}\n</D:multistatus>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* FSEntry timestamps are Unix seconds. Entries with nothing stored fall back to
|
||||
* an ISO string literal, so both forms have to be accepted here.
|
||||
*/
|
||||
function toEpochSeconds(ts: number | string): number {
|
||||
return typeof ts === 'number'
|
||||
? Math.floor(ts)
|
||||
: Math.floor(new Date(ts).getTime() / 1000);
|
||||
}
|
||||
|
||||
/** `Date` takes milliseconds, so entry seconds must be scaled to format them. */
|
||||
function entryDate(ts: number | string): Date {
|
||||
return new Date(toEpochSeconds(ts) * 1000);
|
||||
}
|
||||
|
||||
/**
|
||||
* Opaque validator. Built from seconds so every verb emits the same ETag for an
|
||||
* entry — a client that gets one value from PROPFIND and another from GET
|
||||
* treats the resource as changed and re-fetches it on every pass.
|
||||
*/
|
||||
function entryEtag(uid: string, ts: number | string): string {
|
||||
return `"${uid}-${toEpochSeconds(ts)}"`;
|
||||
}
|
||||
|
||||
/**
|
||||
* What a `<D:response>` needs to describe one resource. An {@link FSEntry}
|
||||
* satisfies it; the virtual collections a share mask invents pass whatever they
|
||||
* know, which is sometimes only a name.
|
||||
*/
|
||||
type PropfindTarget = Partial<
|
||||
Pick<FSEntry, 'name' | 'uuid' | 'modified' | 'created' | 'size'>
|
||||
>;
|
||||
|
||||
function propfindEntry(
|
||||
href: string,
|
||||
entry: FSEntry | null,
|
||||
entry: PropfindTarget | null,
|
||||
isDir: boolean,
|
||||
): string {
|
||||
const encodedHref =
|
||||
@@ -932,14 +1236,13 @@ function propfindEntry(
|
||||
const created = entry?.created ?? '2025-01-01T00:00:00Z';
|
||||
const name = entry?.name ?? (pathPosix.basename(href) || '/');
|
||||
const uid = entry?.uuid ?? 'root';
|
||||
const modTs = Math.floor(new Date(modified as string).getTime());
|
||||
|
||||
let props = `
|
||||
<D:displayname>${escapeXml(String(name))}</D:displayname>
|
||||
<D:getlastmodified>${new Date(modified as string).toUTCString()}</D:getlastmodified>
|
||||
<D:creationdate>${new Date(created as string).toISOString()}</D:creationdate>
|
||||
<D:getlastmodified>${entryDate(modified).toUTCString()}</D:getlastmodified>
|
||||
<D:creationdate>${entryDate(created).toISOString()}</D:creationdate>
|
||||
<D:resourcetype>${isDir ? '<D:collection/>' : ''}</D:resourcetype>
|
||||
<D:getetag>"${uid}-${modTs}"</D:getetag>
|
||||
<D:getetag>${entryEtag(uid, modified)}</D:getetag>
|
||||
<D:supportedlock>
|
||||
<D:lockentry><D:lockscope><D:exclusive/></D:lockscope><D:locktype><D:write/></D:locktype></D:lockentry>
|
||||
<D:lockentry><D:lockscope><D:shared/></D:lockscope><D:locktype><D:write/></D:locktype></D:lockentry>
|
||||
@@ -949,7 +1252,7 @@ function propfindEntry(
|
||||
|
||||
if (!isDir && entry) {
|
||||
props += `\n <D:getcontentlength>${entry.size ?? 0}</D:getcontentlength>`;
|
||||
const mime = mimeFromExt(pathPosix.extname(entry.name));
|
||||
const mime = mimeFromExt(pathPosix.extname(name));
|
||||
props += `\n <D:getcontenttype>${escapeXml(mime)}</D:getcontenttype>`;
|
||||
}
|
||||
|
||||
|
||||
@@ -61,23 +61,22 @@ export type LegacyErrorCodes =
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
/**
|
||||
* Options accepted by `HttpError`. All optional.
|
||||
*/
|
||||
/** Options accepted by `HttpError`. All optional. */
|
||||
export interface HttpErrorOptions {
|
||||
/** Underlying error. Set as the standard `Error.cause`. */
|
||||
cause?: unknown;
|
||||
@@ -89,25 +88,38 @@ export interface HttpErrorOptions {
|
||||
legacyCode?: LegacyErrorCodes | (string & {});
|
||||
/**
|
||||
* Modern, structured error code. If both `legacyCode` and `code` are set,
|
||||
* the legacy one takes the `code` slot in the response body and `code`
|
||||
* is emitted as `errorCode`, so clients keying on either field find
|
||||
* what they expect.
|
||||
* the legacy one takes the `code` slot in the response body and `code` is
|
||||
* emitted as `errorCode`, so clients keying on either field find what they
|
||||
* expect.
|
||||
*/
|
||||
code?: string;
|
||||
/** Additional fields merged into the response body. */
|
||||
fields?: Record<string, unknown>;
|
||||
/**
|
||||
* Skip the terminal alarm gate for this error. Set it only where the call
|
||||
* site already knows the failure is expected and arrives in proportion to
|
||||
* traffic — an upstream rate limit on a zero-cost model, say. Never
|
||||
* serialized to the client.
|
||||
*/
|
||||
noAlarm?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The single error type controllers and services throw to surface an HTTP
|
||||
* failure. The terminal `errorHandler` middleware catches it, serializes a
|
||||
* JSON body, and sets the response status.
|
||||
* failure. The terminal `errorHandler` middleware catches it, serializes a JSON
|
||||
* body, and sets the response status.
|
||||
*
|
||||
* Usage:
|
||||
*
|
||||
* ```ts
|
||||
* throw new HttpError(404, 'Item not found');
|
||||
* throw new HttpError(409, 'Cannot overwrite directory', { legacyCode: 'is_directory' });
|
||||
* throw new HttpError(403, 'Forbidden', { legacyCode: 'forbidden', fields: { target } });
|
||||
* throw new HttpError(409, 'Cannot overwrite directory', {
|
||||
* legacyCode: 'is_directory',
|
||||
* });
|
||||
* throw new HttpError(403, 'Forbidden', {
|
||||
* legacyCode: 'forbidden',
|
||||
* fields: { target },
|
||||
* });
|
||||
* ```
|
||||
*
|
||||
* Express 5 forwards thrown errors (sync and async) to error-handling
|
||||
@@ -118,6 +130,7 @@ export class HttpError extends Error {
|
||||
readonly legacyCode?: LegacyErrorCodes | (string & {});
|
||||
readonly code?: string;
|
||||
readonly fields?: Record<string, unknown>;
|
||||
readonly noAlarm?: boolean;
|
||||
|
||||
constructor(
|
||||
statusCode: number,
|
||||
@@ -133,6 +146,7 @@ export class HttpError extends Error {
|
||||
this.legacyCode = options.legacyCode;
|
||||
this.code = options.code;
|
||||
this.fields = options.fields;
|
||||
this.noAlarm = options.noAlarm;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { Controller, Get, Propfind } from './decorators.ts';
|
||||
import { PuterRouter } from './PuterRouter.ts';
|
||||
|
||||
interface Registrable {
|
||||
registerRoutes: (router: PuterRouter) => void;
|
||||
}
|
||||
|
||||
const collect = (controller: Registrable) => {
|
||||
const router = new PuterRouter();
|
||||
controller.registerRoutes(router);
|
||||
return router.routes;
|
||||
};
|
||||
|
||||
@Controller('/demo')
|
||||
class DemoController {
|
||||
@Get('/thing')
|
||||
thing() {}
|
||||
|
||||
@Propfind('/{*splat}', { subdomain: 'dav' })
|
||||
browse() {}
|
||||
}
|
||||
|
||||
describe('route decorators', () => {
|
||||
it('collects each decorated method once, in declaration order', () => {
|
||||
const routes = collect(new DemoController() as unknown as Registrable);
|
||||
expect(routes.map((r) => r.method)).toEqual(['get', 'propfind']);
|
||||
expect(routes[1]!.options).toEqual({ subdomain: 'dav' });
|
||||
});
|
||||
|
||||
it('does not re-collect when a second instance is constructed', () => {
|
||||
// Method initializers run per instance but collect onto the shared
|
||||
// prototype, so without a guard a process that builds two servers
|
||||
// registers every route on the second one twice over.
|
||||
const first = collect(new DemoController() as unknown as Registrable);
|
||||
const second = collect(new DemoController() as unknown as Registrable);
|
||||
const third = collect(new DemoController() as unknown as Registrable);
|
||||
expect(second).toHaveLength(first.length);
|
||||
expect(third).toHaveLength(first.length);
|
||||
expect(third.map((r) => r.method)).toEqual(['get', 'propfind']);
|
||||
});
|
||||
|
||||
it('binds handlers to the instance that registered them', () => {
|
||||
@Controller()
|
||||
class Bound {
|
||||
marker = 'mine';
|
||||
seen: string | undefined;
|
||||
|
||||
@Get('/x')
|
||||
handle() {
|
||||
this.seen = this.marker;
|
||||
}
|
||||
}
|
||||
|
||||
const instance = new Bound();
|
||||
const [route] = collect(instance as unknown as Registrable);
|
||||
(route!.handler as unknown as () => void)();
|
||||
expect(instance.seen).toBe('mine');
|
||||
});
|
||||
});
|
||||
@@ -143,7 +143,21 @@ const createMethodDecorator = (method: Exclude<RouteMethod, 'use'>) => {
|
||||
const proto = Object.getPrototypeOf(
|
||||
this as object,
|
||||
) as DecoratedPrototype;
|
||||
getOrInitRoutes(proto).push({
|
||||
const routes = getOrInitRoutes(proto);
|
||||
// Initializers run once per *instance* but collect onto the
|
||||
// shared prototype, so a process that builds a second server
|
||||
// (every multi-server test file) would otherwise register
|
||||
// every route twice. The decorated method and the path literal
|
||||
// are the same references each time, so identity recognizes a
|
||||
// repeat without comparing option objects.
|
||||
const already = routes.some(
|
||||
(r) =>
|
||||
r.handler === (target as unknown as RequestHandler) &&
|
||||
r.method === method &&
|
||||
r.path === path,
|
||||
);
|
||||
if (already) return;
|
||||
routes.push({
|
||||
method,
|
||||
path,
|
||||
options,
|
||||
|
||||
@@ -21,7 +21,7 @@ import type { RequestHandler } from 'express';
|
||||
import type { IConfig } from '../../../types';
|
||||
import {
|
||||
assertActorHasCredits,
|
||||
type CreditMeteringLike,
|
||||
type CreditMetering,
|
||||
} from '../../../services/metering/enforcement.js';
|
||||
import '../expressAugmentation';
|
||||
|
||||
@@ -38,7 +38,7 @@ import '../expressAugmentation';
|
||||
* affordable on routes that are called hundreds of times a minute.
|
||||
*/
|
||||
export const requireCreditsGate = (
|
||||
metering: CreditMeteringLike | undefined,
|
||||
metering: CreditMetering | undefined,
|
||||
config: IConfig,
|
||||
): RequestHandler => {
|
||||
return (req, _res, next) => {
|
||||
|
||||
@@ -28,6 +28,8 @@ import {
|
||||
assertNotUserSession,
|
||||
noUserSessionGate,
|
||||
requireAuthGate,
|
||||
requireCardVerifiedGate,
|
||||
requirePhoneVerifiedGate,
|
||||
requireVerifiedAccount,
|
||||
requireNonAccessTokenGate,
|
||||
requireUserActorGate,
|
||||
@@ -803,3 +805,70 @@ describe('allowedAppIdsGate', () => {
|
||||
expectHttpError(got, 403, 'forbidden');
|
||||
});
|
||||
});
|
||||
|
||||
// ── requirePhoneVerifiedGate / requireCardVerifiedGate ──────────────
|
||||
|
||||
describe('requirePhoneVerifiedGate', () => {
|
||||
it('passes a user with a verified number on file', () => {
|
||||
const got = runGate(requirePhoneVerifiedGate(), {
|
||||
actor: { user: { uuid: 'u-1', phone: '+15550000000' } },
|
||||
});
|
||||
expect(got).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects a user who was never asked to verify a phone', () => {
|
||||
// The pending flag is clear, so the default-on account gate lets this
|
||||
// user through — an opt-in route asking for the factor itself must not.
|
||||
const got = runGate(requirePhoneVerifiedGate(), {
|
||||
actor: { user: { uuid: 'u-1' } },
|
||||
});
|
||||
expectHttpError(got, 403, 'phone_verification_required');
|
||||
});
|
||||
|
||||
it('rejects a user still mid-verification, stale number and all', () => {
|
||||
const got = runGate(requirePhoneVerifiedGate(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
phone: '+15550000000',
|
||||
requires_phone_verification: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
expectHttpError(got, 403, 'phone_verification_required');
|
||||
});
|
||||
|
||||
it('rejects when there is no actor at all', () => {
|
||||
const got = runGate(requirePhoneVerifiedGate(), {});
|
||||
expectHttpError(got, 403, 'phone_verification_required');
|
||||
});
|
||||
});
|
||||
|
||||
describe('requireCardVerifiedGate', () => {
|
||||
it('passes a user with a verified card on file', () => {
|
||||
const got = runGate(requireCardVerifiedGate(), {
|
||||
actor: { user: { uuid: 'u-1', card_fingerprint: 'fp_1' } },
|
||||
});
|
||||
expect(got).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects a user who never verified a card', () => {
|
||||
const got = runGate(requireCardVerifiedGate(), {
|
||||
actor: { user: { uuid: 'u-1' } },
|
||||
});
|
||||
expectHttpError(got, 403, 'card_verification_required');
|
||||
});
|
||||
|
||||
it('rejects a user still carrying the card gate', () => {
|
||||
const got = runGate(requireCardVerifiedGate(), {
|
||||
actor: {
|
||||
user: {
|
||||
uuid: 'u-1',
|
||||
card_fingerprint: 'fp_1',
|
||||
requires_card_verification: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
expectHttpError(got, 403, 'card_verification_required');
|
||||
});
|
||||
});
|
||||
@@ -378,17 +378,91 @@ export const assertVerifiedAccount = (
|
||||
403,
|
||||
'Please verify your phone number to continue',
|
||||
{
|
||||
legacyCode: 'phone_verification_required' as never,
|
||||
legacyCode: 'phone_verification_required',
|
||||
},
|
||||
);
|
||||
}
|
||||
if (user?.requires_card_verification) {
|
||||
throw new HttpError(403, 'Please verify your card to continue', {
|
||||
legacyCode: 'card_verification_required' as never,
|
||||
legacyCode: 'card_verification_required',
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// -- Per-verification gates ------------------------------------------
|
||||
//
|
||||
// `requireVerifiedAccount` above is the default-on gate: it turns away an
|
||||
// account that is *pending* a verification the abuse harness asked of it, and
|
||||
// says nothing about accounts that were never asked. The two gates below are
|
||||
// the opt-in counterparts of `requireVerified` (email) for the other two
|
||||
// factors: they require the verification to have actually happened, whether or
|
||||
// not this account was ever flagged for it.
|
||||
//
|
||||
// Neither factor has a boolean "verified" column — the pending flag is cleared
|
||||
// on success and the proof is the artifact left behind. `phone` is written only
|
||||
// once a code has been confirmed, and `card_fingerprint` only once a card has
|
||||
// been checked (both are also what `UserAccountService` reads as the
|
||||
// phone-verified / card-verified signals). Pending flag clear AND artifact
|
||||
// present is therefore the honest reading of "verified", and an account still
|
||||
// mid-flow fails on the flag rather than slipping through on a stale artifact.
|
||||
|
||||
/**
|
||||
* Reject unless the user has a phone number verified on file. 403
|
||||
* `phone_verification_required`, the same code the pending-verification gate
|
||||
* uses, so a client that already prompts for the SMS flow prompts here too.
|
||||
*/
|
||||
export const assertPhoneVerified = (
|
||||
user:
|
||||
{ phone?: unknown; requires_phone_verification?: unknown } | undefined,
|
||||
): void => {
|
||||
if (user?.phone && !user?.requires_phone_verification) return;
|
||||
throw new HttpError(403, 'Please verify your phone number to continue', {
|
||||
legacyCode: 'phone_verification_required',
|
||||
});
|
||||
};
|
||||
|
||||
/** Route-option form of {@link assertPhoneVerified} (`requirePhoneVerified`). */
|
||||
export const requirePhoneVerifiedGate = (): RequestHandler => {
|
||||
return (req, _res, next) => {
|
||||
try {
|
||||
assertPhoneVerified(req.actor?.user);
|
||||
} catch (err) {
|
||||
next(err);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Reject unless the user has a card verified on file. 403
|
||||
* `card_verification_required`, matching the pending-verification gate so the
|
||||
* client shows the card flow it already has.
|
||||
*/
|
||||
export const assertCardVerified = (
|
||||
user:
|
||||
| { card_fingerprint?: unknown; requires_card_verification?: unknown }
|
||||
| undefined,
|
||||
): void => {
|
||||
if (user?.card_fingerprint && !user?.requires_card_verification) return;
|
||||
throw new HttpError(403, 'Please verify your card to continue', {
|
||||
legacyCode: 'card_verification_required',
|
||||
});
|
||||
};
|
||||
|
||||
/** Route-option form of {@link assertCardVerified} (`requireCardVerified`). */
|
||||
export const requireCardVerifiedGate = (): RequestHandler => {
|
||||
return (req, _res, next) => {
|
||||
try {
|
||||
assertCardVerified(req.actor?.user);
|
||||
} catch (err) {
|
||||
next(err);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
};
|
||||
};
|
||||
|
||||
export const assertNotSuspended = (
|
||||
user: { suspended?: unknown } | undefined,
|
||||
): void => {
|
||||
|
||||
@@ -1206,3 +1206,166 @@ describe('createPuterSiteMiddleware — .puter_site_config', () => {
|
||||
expect(String(out.body)).toContain('Not Found');
|
||||
});
|
||||
});
|
||||
|
||||
// ── __workers/ (worker source) ──────────────────────────────────────
|
||||
//
|
||||
// `__workers/` at a site root holds server-side worker source. It is the
|
||||
// one thing under a site root that is NOT public: users' backend code can
|
||||
// carry trade secrets, so every route to those bytes — direct request,
|
||||
// folder fallback, custom error page — must dead-end in the same 404 a
|
||||
// missing path produces.
|
||||
|
||||
describe('createPuterSiteMiddleware — __workers/ is never served', () => {
|
||||
const setupSiteWithWorker = async () => {
|
||||
const owner = await makeUserWithHome();
|
||||
const homePath = `/${owner.username}`;
|
||||
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
|
||||
const sub = `dw-${Math.random().toString(36).slice(2, 8)}`;
|
||||
await server.stores.subdomain.create({
|
||||
userId: owner.id,
|
||||
subdomain: sub,
|
||||
rootDirId: homeEntry!.id,
|
||||
});
|
||||
await server.services.fs.mkdir(owner.id, {
|
||||
path: `${homePath}/__workers`,
|
||||
createMissingParents: true,
|
||||
});
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/__workers/matchmaking.worker.js`,
|
||||
Buffer.from('export default { secret: true };'),
|
||||
'application/javascript',
|
||||
);
|
||||
return { owner, homePath, sub };
|
||||
};
|
||||
|
||||
const request = async (sub: string, path: string) => {
|
||||
const mw = buildMiddleware();
|
||||
const { res, out } = makeRes();
|
||||
await mw(
|
||||
makeReq({ hostname: `${sub}.site.puter.localhost`, path }),
|
||||
res,
|
||||
vi.fn(),
|
||||
);
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
return out;
|
||||
};
|
||||
|
||||
it('404s a direct request for a worker source file that exists', async () => {
|
||||
const { sub } = await setupSiteWithWorker();
|
||||
const out = await request(sub, '/__workers/matchmaking.worker.js');
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(String(out.body)).not.toContain('secret');
|
||||
});
|
||||
|
||||
it('404s the folder itself — no index.html fallback into it', async () => {
|
||||
const { owner, homePath, sub } = await setupSiteWithWorker();
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/__workers/index.html`,
|
||||
Buffer.from('<html>listing</html>'),
|
||||
'text/html',
|
||||
);
|
||||
for (const path of ['/__workers', '/__workers/']) {
|
||||
const out = await request(sub, path);
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(String(out.body)).not.toContain('listing');
|
||||
}
|
||||
});
|
||||
|
||||
it('404s case variants and URL-encoded spellings of the folder', async () => {
|
||||
const { sub } = await setupSiteWithWorker();
|
||||
for (const path of [
|
||||
'/__Workers/matchmaking.worker.js',
|
||||
'/__WORKERS/matchmaking.worker.js',
|
||||
'/%5F%5Fworkers/matchmaking.worker.js',
|
||||
'/a/../__workers/matchmaking.worker.js',
|
||||
]) {
|
||||
const out = await request(sub, path);
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(String(out.body)).not.toContain('secret');
|
||||
}
|
||||
});
|
||||
|
||||
it('404s __workers/ at any depth — a site rooted above another site must not serve the inner one', async () => {
|
||||
// Same FS layout, but the serving site is rooted one level up:
|
||||
// the worker source now sits at /inner/__workers/… from its
|
||||
// point of view, and must be just as unreachable.
|
||||
const owner = await makeUserWithHome();
|
||||
const homePath = `/${owner.username}`;
|
||||
await server.services.fs.mkdir(owner.id, {
|
||||
path: `${homePath}/inner/__workers`,
|
||||
createMissingParents: true,
|
||||
});
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/inner/__workers/matchmaking.worker.js`,
|
||||
Buffer.from('export default { secret: true };'),
|
||||
'application/javascript',
|
||||
);
|
||||
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
|
||||
const sub = `dwo-${Math.random().toString(36).slice(2, 8)}`;
|
||||
await server.stores.subdomain.create({
|
||||
userId: owner.id,
|
||||
subdomain: sub,
|
||||
rootDirId: homeEntry!.id,
|
||||
});
|
||||
const out = await request(
|
||||
sub,
|
||||
'/inner/__workers/matchmaking.worker.js',
|
||||
);
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(String(out.body)).not.toContain('secret');
|
||||
});
|
||||
|
||||
it('refuses a custom error page that points into __workers/', async () => {
|
||||
// `errors.404.file` aimed at worker source would publish it on
|
||||
// every missing path — the rule is ignored and the default 404
|
||||
// page served instead.
|
||||
const { owner, homePath, sub } = await setupSiteWithWorker();
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/.puter_site_config`,
|
||||
Buffer.from(
|
||||
JSON.stringify({
|
||||
errors: {
|
||||
'404': {
|
||||
file: '/__workers/matchmaking.worker.js',
|
||||
status: 200,
|
||||
},
|
||||
},
|
||||
}),
|
||||
),
|
||||
'application/json',
|
||||
);
|
||||
const out = await request(sub, '/no-such-page');
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(String(out.body)).not.toContain('secret');
|
||||
expect(String(out.body)).toContain('Not Found');
|
||||
});
|
||||
|
||||
it('still applies an SPA fallback to __workers/ requests when the target is outside the folder', async () => {
|
||||
// The block hides the folder; it does not exempt those URLs from
|
||||
// the site's own 404 handling. A fallback pointing at a public
|
||||
// file keeps working — the fiction is "this path does not exist",
|
||||
// and this is exactly what a missing path serves.
|
||||
const { owner, homePath, sub } = await setupSiteWithWorker();
|
||||
const shell = Buffer.from('<html>spa-shell</html>');
|
||||
await writeFile(owner.id, `${homePath}/index.html`, shell, 'text/html');
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/.puter_site_config`,
|
||||
Buffer.from(
|
||||
JSON.stringify({
|
||||
errors: { '404': { file: '/index.html', status: 200 } },
|
||||
}),
|
||||
),
|
||||
'application/json',
|
||||
);
|
||||
const out = await request(sub, '/__workers/matchmaking.worker.js');
|
||||
expect(out.statusCode).toBe(200);
|
||||
const piped = out.body as Buffer | undefined;
|
||||
expect(Buffer.isBuffer(piped)).toBe(true);
|
||||
expect(piped!.equals(shell)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -71,6 +71,24 @@ import {
|
||||
* serving.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Reserved folder at a site root holding server-side worker source
|
||||
* (`__workers/<name>.worker.js`). Its contents are backend code — users' trade
|
||||
* secrets live there — so nothing under it is ever served: not directly, not
|
||||
* via the folder→index.html fallback, and not as a custom error page target.
|
||||
* Matched on the decoded, normalized path, and case-insensitively, so no
|
||||
* alternate spelling of the same entry can reach the bytes.
|
||||
*
|
||||
* Matched at ANY depth, not just the site root: site roots can nest, and a site
|
||||
* rooted above another site would otherwise serve the inner site's `__workers/`
|
||||
* as ordinary content under `/inner/__workers/...`.
|
||||
*/
|
||||
const WORKERS_FOLDER = '__workers';
|
||||
const isWorkersSourcePath = (urlPath: string): boolean =>
|
||||
urlPath
|
||||
.split('/')
|
||||
.some((segment) => segment.toLowerCase() === WORKERS_FOLDER);
|
||||
|
||||
const SUBDOMAIN_404 = `<div style="font-size: 20px;
|
||||
text-align: center;
|
||||
height: calc(100vh);
|
||||
@@ -493,9 +511,14 @@ export const createPuterSiteMiddleware = (
|
||||
// Subdomain hosting bypasses ACL by design: anything the owner placed
|
||||
// under the registered root_dir is treated as public. Path traversal
|
||||
// is blocked above by `pathPosix.normalize` anchoring at `/`.
|
||||
let entry = isConfigRequest
|
||||
? null
|
||||
: await layers.stores.fsEntry.getEntryByPath(filePath);
|
||||
// `__workers/` is carved out of that bypass: worker source is the one
|
||||
// thing under a site root that is NOT public. Treated exactly like the
|
||||
// config file — no lookup at all, so the folder→index.html fallback
|
||||
// below can't resolve into it either.
|
||||
let entry =
|
||||
isConfigRequest || isWorkersSourcePath(resolvedUrlPath)
|
||||
? null
|
||||
: await layers.stores.fsEntry.getEntryByPath(filePath);
|
||||
if (entry?.isDir) {
|
||||
// Folder request → fall back to <folder>/index.html, the same
|
||||
// way `/` is rewritten to `/index.html` at the site root above.
|
||||
@@ -513,7 +536,13 @@ export const createPuterSiteMiddleware = (
|
||||
let statusOverride: number | undefined;
|
||||
if (!entry || entry.isDir) {
|
||||
const errorTarget = resolveErrorTarget(siteConfig, 404, rootPath);
|
||||
if (errorTarget) {
|
||||
// A config pointing `errors.404.file` into `__workers/` would
|
||||
// publish worker source through the error page — refuse it the
|
||||
// same way a nonexistent target is refused.
|
||||
if (
|
||||
errorTarget &&
|
||||
!isWorkersSourcePath(errorTarget.absPath.slice(rootPath.length))
|
||||
) {
|
||||
const candidate = await layers.stores.fsEntry.getEntryByPath(
|
||||
errorTarget.absPath,
|
||||
);
|
||||
|
||||
@@ -116,6 +116,20 @@ async function checkMemory(key, limit, windowMs) {
|
||||
|
||||
// -- Redis backend ---------------------------------------------------
|
||||
|
||||
// ioredis MULTI/EXEC reports per-command failures inside the exec() result
|
||||
// (as `[err, res]` pairs) rather than throwing, so a failed command's result
|
||||
// reads as `undefined` — and `Number(undefined)` is NaN, which every
|
||||
// comparison below treats as "under the limit". Pull results through this so
|
||||
// a command failure surfaces like a thrown one instead of silently admitting.
|
||||
function multiResult(results, i) {
|
||||
const entry = results[i];
|
||||
if (Array.isArray(entry)) {
|
||||
if (entry[0]) throw entry[0];
|
||||
return entry[1];
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
|
||||
async function checkRedis(
|
||||
/** @type {import('ioredis').Cluster} */
|
||||
redis,
|
||||
@@ -144,9 +158,7 @@ async function checkRedis(
|
||||
.pexpire(redisKey, windowMs)
|
||||
.exec();
|
||||
|
||||
const count = Number(
|
||||
Array.isArray(results[2]) ? results[2][1] : results[2],
|
||||
);
|
||||
const count = Number(multiResult(results, 2));
|
||||
if (count > limit) {
|
||||
await redis.zrem(redisKey, member);
|
||||
return false;
|
||||
@@ -216,7 +228,7 @@ async function acquireMemoryConcurrent(key, limit) {
|
||||
};
|
||||
}
|
||||
|
||||
async function acquireRedisConcurrent(redis, key, limit) {
|
||||
async function acquireRedisConcurrent(redis, key, limit, retried = false) {
|
||||
const redisKey = `concurrent:${key}`;
|
||||
const member = `${Date.now()}-${crypto.randomUUID()}`;
|
||||
// One sorted-set member per held slot, scored by acquire time — the same
|
||||
@@ -231,20 +243,32 @@ async function acquireRedisConcurrent(redis, key, limit) {
|
||||
// nobody is actually using. Here the sweep below drops each slot on its own
|
||||
// age, so a leak drains on schedule no matter how hard anyone retries.
|
||||
const now = Date.now();
|
||||
const results = await redis
|
||||
.multi()
|
||||
// Slots older than the orphan window belonged to a process that died
|
||||
// before releasing; drop them before counting.
|
||||
.zremrangebyscore(redisKey, 0, now - ORPHAN_SAFETY_TTL_MS)
|
||||
.zadd(redisKey, now, member)
|
||||
.zcard(redisKey)
|
||||
// Key-level TTL is only garbage collection for a bucket that goes
|
||||
// quiet — the per-member sweep above is what bounds a live one.
|
||||
.expire(redisKey, ORPHAN_SAFETY_TTL_SEC)
|
||||
.exec();
|
||||
const count = Number(
|
||||
Array.isArray(results[2]) ? results[2][1] : results[2],
|
||||
);
|
||||
let count;
|
||||
try {
|
||||
const results = await redis
|
||||
.multi()
|
||||
// Slots older than the orphan window belonged to a process that
|
||||
// died before releasing; drop them before counting.
|
||||
.zremrangebyscore(redisKey, 0, now - ORPHAN_SAFETY_TTL_MS)
|
||||
.zadd(redisKey, now, member)
|
||||
.zcard(redisKey)
|
||||
// Key-level TTL is only garbage collection for a bucket that goes
|
||||
// quiet — the per-member sweep above is what bounds a live one.
|
||||
.expire(redisKey, ORPHAN_SAFETY_TTL_SEC)
|
||||
.exec();
|
||||
count = Number(multiResult(results, 2));
|
||||
} catch (err) {
|
||||
// Keys left behind by the INCR-counter version of this backend are
|
||||
// plain strings, so every zset command above fails WRONGTYPE — while
|
||||
// the EXPIRE at the end still succeeds, meaning steady traffic keeps
|
||||
// refreshing the stale key and it never ages out on its own. Drop the
|
||||
// legacy key and count against a clean one.
|
||||
if (!retried && /WRONGTYPE/.test(err?.message ?? '')) {
|
||||
await redis.del(redisKey);
|
||||
return acquireRedisConcurrent(redis, key, limit, true);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
if (count > limit) {
|
||||
await redis.zrem(redisKey, member);
|
||||
return { ok: false };
|
||||
|
||||
@@ -959,6 +959,28 @@ describe('acquireConcurrent — orphan recovery (redis)', () => {
|
||||
|
||||
expect((await acquireConcurrent(key, 1)).ok).toBe(true);
|
||||
});
|
||||
|
||||
it('heals a legacy string counter key and enforces the limit again', async () => {
|
||||
// The pre-zset implementation stored these buckets as INCR string
|
||||
// counters. Against such a key every zset command fails WRONGTYPE
|
||||
// inside the MULTI while the trailing EXPIRE succeeds — so traffic
|
||||
// kept the stale key alive indefinitely, and the undefined zcard
|
||||
// result (NaN) admitted every caller unbounded.
|
||||
// The file-level afterEach resets wiring to memory after every test,
|
||||
// so re-point the default at this suite's redis for this test.
|
||||
configureRateLimit({ default: 'redis', redis });
|
||||
const key = 'legacy-counter';
|
||||
await redis.set(`concurrent:${key}`, '7');
|
||||
|
||||
const first = await acquireConcurrent(key, 1);
|
||||
expect(first.ok).toBe(true);
|
||||
// The key was rebuilt as a zset and the cap is live again.
|
||||
expect(await redis.type(`concurrent:${key}`)).toBe('zset');
|
||||
expect((await acquireConcurrent(key, 1)).ok).toBe(false);
|
||||
|
||||
await first.release();
|
||||
expect((await acquireConcurrent(key, 1)).ok).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ── concurrency: subscription-based limits ──────────────────────────
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../../types';
|
||||
import { isHttpError } from '../HttpError';
|
||||
import { requireSubscriptionGate } from './subscription';
|
||||
|
||||
// The decision itself is covered in `services/metering/enforcement.test.ts`;
|
||||
// what matters here is that the gate resolves it and hands the outcome to
|
||||
// `next()` rather than leaving the request hanging or throwing async.
|
||||
|
||||
const runGate = (
|
||||
gate: ReturnType<typeof requireSubscriptionGate>,
|
||||
plan: string,
|
||||
): Promise<unknown> =>
|
||||
new Promise((resolve) => {
|
||||
gate(
|
||||
{ actor: { user: { uuid: 'u-1' } } } as unknown as Request,
|
||||
{} as Response,
|
||||
resolve as (err?: unknown) => void,
|
||||
);
|
||||
});
|
||||
|
||||
const metering = (id: string) => ({
|
||||
getActorSubscription: vi.fn().mockResolvedValue({ id }),
|
||||
});
|
||||
|
||||
const config = {} as IConfig;
|
||||
|
||||
describe('requireSubscriptionGate', () => {
|
||||
it('calls next() with nothing for a subscriber', async () => {
|
||||
const got = await runGate(
|
||||
requireSubscriptionGate(metering('business'), config, true),
|
||||
'business',
|
||||
);
|
||||
expect(got).toBeUndefined();
|
||||
});
|
||||
|
||||
it('calls next(err) with a 402 for a free account', async () => {
|
||||
const got = await runGate(
|
||||
requireSubscriptionGate(metering('user_free'), config, true),
|
||||
'user_free',
|
||||
);
|
||||
expect(isHttpError(got)).toBe(true);
|
||||
expect(got).toMatchObject({
|
||||
statusCode: 402,
|
||||
legacyCode: 'subscription_required',
|
||||
});
|
||||
});
|
||||
|
||||
it('passes a rejected lookup to next() instead of leaving it unhandled', async () => {
|
||||
const broken = {
|
||||
getActorSubscription: vi
|
||||
.fn()
|
||||
.mockRejectedValue(new Error('store down')),
|
||||
};
|
||||
const got = await runGate(
|
||||
requireSubscriptionGate(broken, config, true),
|
||||
'irrelevant',
|
||||
);
|
||||
expect(got).toBeInstanceOf(Error);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { RequestHandler } from 'express';
|
||||
import {
|
||||
assertActorHasSubscription,
|
||||
type SubscriptionMetering,
|
||||
type SubscriptionRequirement,
|
||||
} from '../../../services/metering/enforcement.js';
|
||||
import type { IConfig } from '../../../types';
|
||||
import '../expressAugmentation';
|
||||
|
||||
/**
|
||||
* Reject a caller whose plan doesn't include this surface, for routes that opt
|
||||
* in with `requireSubscription`. Off unless a route asks for it — no surface is
|
||||
* subscriber-only by default.
|
||||
*
|
||||
* `true` accepts any non-free plan; an array of policy ids accepts only those.
|
||||
* Either way the decision is `assertActorHasSubscription`, which the driver
|
||||
* dispatch path calls directly — `/drivers/call` is one shared route, so a
|
||||
* per-driver requirement can't ride on the route chain.
|
||||
*/
|
||||
export const requireSubscriptionGate = (
|
||||
metering: SubscriptionMetering | undefined,
|
||||
config: IConfig,
|
||||
requirement: SubscriptionRequirement,
|
||||
): RequestHandler => {
|
||||
return (req, _res, next) => {
|
||||
assertActorHasSubscription(
|
||||
metering,
|
||||
req.actor,
|
||||
requirement,
|
||||
config,
|
||||
).then(
|
||||
() => next(),
|
||||
(err) => next(err),
|
||||
);
|
||||
};
|
||||
};
|
||||
@@ -67,8 +67,9 @@ export type RoutePath = string | RegExp | Array<string | RegExp>;
|
||||
* middleware chain in this order:
|
||||
*
|
||||
* subdomain → requireAuth (+ suspended) → emailConfirmed →
|
||||
* requireUserActor → adminOnly → allowedAppIds → rateLimit →
|
||||
* requireCredits → concurrent → caller `middleware: []` → handler
|
||||
* requireUserActor → adminOnly → allowedAppIds → phoneVerified →
|
||||
* cardVerified → requireSubscription → rateLimit → requireCredits →
|
||||
* concurrent → caller `middleware: []` → handler
|
||||
*
|
||||
* `requireUserActor`, `adminOnly`, and `allowedAppIds` all imply `requireAuth`;
|
||||
* the materializer dedupes so only one auth gate ends up in the chain.
|
||||
@@ -191,6 +192,24 @@ export interface RouteOptions {
|
||||
*/
|
||||
requireVerified?: boolean;
|
||||
|
||||
/**
|
||||
* Reject unless the user has a phone number verified on file (verified at
|
||||
* some point, not merely never asked to). 403
|
||||
* `phone_verification_required`. Implies `requireAuth`; independent of the
|
||||
* default-on pending-verification gate, which only turns away accounts the
|
||||
* abuse harness flagged.
|
||||
*
|
||||
* Opt-in per route, for surfaces worth the friction of an SMS round trip.
|
||||
*/
|
||||
requirePhoneVerified?: boolean;
|
||||
|
||||
/**
|
||||
* Reject unless the user has a card verified on file. 403
|
||||
* `card_verification_required`. Same shape as `requirePhoneVerified` — the
|
||||
* opt-in, "must have actually done it" counterpart of the pending gate.
|
||||
*/
|
||||
requireCardVerified?: boolean;
|
||||
|
||||
/**
|
||||
* Per-route JSON body parsing override. By default the global parser
|
||||
* handles every `application/json` request with a 50mb limit and stashes
|
||||
@@ -328,6 +347,24 @@ export interface RouteOptions {
|
||||
*/
|
||||
requireCredits?: boolean;
|
||||
|
||||
/**
|
||||
* Reject a caller whose plan doesn't include this route with 402
|
||||
* `subscription_required`.
|
||||
*
|
||||
* `true` accepts any plan that isn't one of the free ones, so a plan
|
||||
* registered by an extension counts without being named here. An array of
|
||||
* `SubscriptionPolicy.id`s accepts only those, for a feature that belongs
|
||||
* to specific plans (`requireSubscription: ['business', 'pro']`).
|
||||
*
|
||||
* Opt-in: nothing is subscriber-only unless it says so. Implies
|
||||
* `requireAuth` — there is no plan to read off an anonymous caller. The
|
||||
* answer comes from the metering service's per-actor subscription cache, so
|
||||
* the check costs a map lookup on a warm actor. Distinct from
|
||||
* `requireCredits`, which asks whether an account has budget left rather
|
||||
* than which plan it is on; a route may want both.
|
||||
*/
|
||||
requireSubscription?: boolean | string[];
|
||||
|
||||
// Reserved — wire as the corresponding features/services land:
|
||||
// bodyFiles?: string[]; // multer-style multipart fields
|
||||
// responseTimeout?: number;
|
||||
@@ -393,7 +430,16 @@ export type AuthRequired<O extends RouteOptions> = O extends {
|
||||
? true
|
||||
: O extends { noUserSession: true }
|
||||
? true
|
||||
: false;
|
||||
: O extends { requirePhoneVerified: true }
|
||||
? true
|
||||
: O extends { requireCardVerified: true }
|
||||
? true
|
||||
: O extends {
|
||||
requireSubscription:
|
||||
true | readonly string[] | string[];
|
||||
}
|
||||
? true
|
||||
: false;
|
||||
|
||||
/** Express `Request` with `actor` narrowed based on the route's options. */
|
||||
export type TypedRequest<O extends RouteOptions> = Omit<Request, 'actor'> & {
|
||||
|
||||
@@ -88,55 +88,22 @@ const implicit_user_app_permissions = [
|
||||
},
|
||||
];
|
||||
|
||||
// Pre-v2 each entry below carried a `policy: { 'rate-limit': { max, period } }`
|
||||
// block attached by a `policyPerm('temp.kv')` / `policyPerm('user.es')` helper,
|
||||
// and a `driverPolicies` table held the actual `{max, period}` values. Nothing
|
||||
// ever read those policy blocks in v2 — rate limiting is now declared
|
||||
// directly on each driver via `@Driver({ rateLimit })` / `readonly rateLimit`
|
||||
// (see KVStoreDriver, AppDriver, SubdomainDriver, NotificationDriver), with
|
||||
// per-subscription overrides via `bySubscription`. The grants themselves
|
||||
// stay so the permission scan still emits a path for each group/permission;
|
||||
// only the dead policy plumbing was removed.
|
||||
const hardcoded_user_group_permissions = {
|
||||
system: {
|
||||
'ca342a5e-b13d-4dee-9048-58b11a57cc55': {
|
||||
driver: {},
|
||||
service: {},
|
||||
feature: {},
|
||||
'local-terminal:access': {},
|
||||
},
|
||||
'b7220104-7905-4985-b996-649fdcdb3c8f': {
|
||||
driver: {},
|
||||
service: {},
|
||||
'service:hello-world:ii:hello-world': {},
|
||||
'service:puter-kvstore:ii:puter-kvstore': {},
|
||||
'driver:puter-kvstore': {},
|
||||
'service:puter-notifications:ii:crud-q': {},
|
||||
'service:puter-apps:ii:crud-q': {},
|
||||
'service:puter-subdomains:ii:crud-q': {},
|
||||
'service:apps:ii:crud-q': {},
|
||||
'service:es\\Cnotification:ii:crud-q': {},
|
||||
'service:es\\Capp:ii:crud-q': {},
|
||||
'service:app:ii:crud-q': {},
|
||||
'service:es\\Csubdomain:ii:crud-q': {},
|
||||
},
|
||||
'78b1b1dd-c959-44d2-b02c-8735671f9997': {
|
||||
driver: {},
|
||||
service: {},
|
||||
'service:hello-world:ii:hello-world': {},
|
||||
'service:puter-kvstore:ii:puter-kvstore': {},
|
||||
'driver:puter-kvstore': {},
|
||||
'service:es\\Cnotification:ii:crud-q': {},
|
||||
'service:es\\Capp:ii:crud-q': {},
|
||||
'service:app:ii:crud-q': {},
|
||||
'service:es\\Csubdomain:ii:crud-q': {},
|
||||
'service:apps:ii:crud-q': {},
|
||||
},
|
||||
},
|
||||
// Permissions every user actor holds, regardless of group membership.
|
||||
//
|
||||
// Roots only: a grant subsumes everything beneath it, so `service` already
|
||||
// answers `service:puter-kvstore:ii:puter-kvstore`. Listing descendants adds
|
||||
// entries no check can ever need.
|
||||
//
|
||||
// A floor, not a ceiling — anything that depends on *who* the user is belongs
|
||||
// in the ACL as a `user_to_group_permissions` / `user_to_user_permissions`
|
||||
// row. Adding a root here grants it to every user, temp accounts included.
|
||||
const default_user_permissions = {
|
||||
driver: {},
|
||||
service: {},
|
||||
};
|
||||
|
||||
module.exports = {
|
||||
implicit_user_app_permissions,
|
||||
default_implicit_user_app_permissions,
|
||||
hardcoded_user_group_permissions,
|
||||
default_user_permissions,
|
||||
};
|
||||
@@ -22,6 +22,6 @@ import { toMicroCents } from '../../services/metering/utils.js';
|
||||
|
||||
export const REGISTERED_USER_FREE = {
|
||||
id: DEFAULT_FREE_SUBSCRIPTION,
|
||||
monthUsageAllowance: toMicroCents(0.25),
|
||||
monthUsageAllowance: toMicroCents(0.5),
|
||||
monthlyStorageAllowance: 100 * 1024 * 1024, // 100MiB
|
||||
};
|
||||
@@ -62,6 +62,7 @@ const FULL_PROVIDER_CONFIG = {
|
||||
'azure-openai': { apiKey: 'k', apiURL: 'https://azure.test/openai/v1' },
|
||||
'openai-completion': { apiKey: 'k' },
|
||||
gemini: { apiKey: 'k' },
|
||||
meta: { apiKey: 'k' },
|
||||
groq: { apiKey: 'k' },
|
||||
deepseek: { apiKey: 'k' },
|
||||
mistral: { apiKey: 'k' },
|
||||
@@ -73,6 +74,7 @@ const FULL_PROVIDER_CONFIG = {
|
||||
'together-ai': { apiKey: 'k' },
|
||||
openrouter: { apiKey: 'k', apiBaseUrl: 'https://openrouter.test' },
|
||||
infron: { apiKey: 'k' },
|
||||
byteplus: { apiKey: 'k' },
|
||||
neuralwatt: { apiKey: 'k' },
|
||||
// Suppress auto-discovery of a developer's local Ollama.
|
||||
ollama: { enabled: false },
|
||||
@@ -148,10 +150,13 @@ const completeFake = (args: Record<string, unknown>) =>
|
||||
} as never),
|
||||
);
|
||||
|
||||
const errorFor = async (thrown: unknown): Promise<HttpError> => {
|
||||
const errorFor = async (
|
||||
thrown: unknown,
|
||||
args: Record<string, unknown> = {},
|
||||
): Promise<HttpError> => {
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockRejectedValue(thrown);
|
||||
try {
|
||||
await completeFake({});
|
||||
await completeFake(args);
|
||||
} catch (e) {
|
||||
return e as HttpError;
|
||||
}
|
||||
@@ -170,6 +175,7 @@ describe('ChatCompletionDriver provider registration', () => {
|
||||
'azure-openai',
|
||||
'openai-completion',
|
||||
'gemini',
|
||||
'meta',
|
||||
'groq',
|
||||
'deepseek',
|
||||
'mistral',
|
||||
@@ -181,6 +187,7 @@ describe('ChatCompletionDriver provider registration', () => {
|
||||
'together-ai',
|
||||
'openrouter',
|
||||
'infron',
|
||||
'byteplus',
|
||||
'neuralwatt',
|
||||
'fake-chat',
|
||||
]) {
|
||||
@@ -257,6 +264,32 @@ describe('ChatCompletionDriver exhausted-chain classification', () => {
|
||||
expect(err.message).toBe('AI provider rate limit exceeded');
|
||||
});
|
||||
|
||||
it('mutes the alarm when every rate-limited attempt was on a free model', async () => {
|
||||
// `fake` is priced at zero throughout: an upstream throttle there is
|
||||
// expected and costs nobody anything, so the caller still gets the
|
||||
// 429 but nothing is recorded.
|
||||
const err = await errorFor(
|
||||
Object.assign(new Error('slow down'), { status: 429 }),
|
||||
);
|
||||
expect(err.noAlarm).toBe(true);
|
||||
});
|
||||
|
||||
it('keeps the alarm when a paid model is the one being rate limited', async () => {
|
||||
const err = await errorFor(
|
||||
Object.assign(new Error('slow down'), { status: 429 }),
|
||||
{ model: 'costly' },
|
||||
);
|
||||
expect(err).toMatchObject({ legacyCode: 'upstream_rate_limited' });
|
||||
expect(err.noAlarm).toBe(false);
|
||||
});
|
||||
|
||||
it('leaves non-rate-limit failures on a free model alarming as usual', async () => {
|
||||
const err = await errorFor(
|
||||
Object.assign(new Error('invalid api key'), { status: 401 }),
|
||||
);
|
||||
expect(err.noAlarm).toBeFalsy();
|
||||
});
|
||||
|
||||
it('classifies a rate limit reported only in the message text', async () => {
|
||||
const err = await errorFor(new Error('Quota exceeded for this key'));
|
||||
expect(err.statusCode).toBe(429);
|
||||
@@ -411,9 +444,9 @@ describe('ChatCompletionDriver cross-provider fallback', () => {
|
||||
Object.assign(new Error('azure down'), { status: 503 }),
|
||||
);
|
||||
const openai = vi.spyOn(OpenAiChatProvider.prototype, 'complete');
|
||||
vi.spyOn(server.services.metering, 'hasEnoughCredits')
|
||||
.mockResolvedValueOnce(true) // pre-flight
|
||||
.mockResolvedValue(false); // drained by a parallel request
|
||||
vi.spyOn(server.services.metering, 'getRemainingUsage')
|
||||
.mockResolvedValueOnce(1_000_000) // pre-flight
|
||||
.mockResolvedValue(0); // drained by a parallel request
|
||||
|
||||
await expect(completeShared()).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
|
||||
@@ -0,0 +1,508 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* What a chat completion is charged when the usual path doesn't complete.
|
||||
*
|
||||
* Providers meter from the usage they hand to `chatStream.end`, which is the
|
||||
* last thing a stream does — so every way a stream can stop early is a way a
|
||||
* completion the upstream billed us for reaches the account as free. These
|
||||
* tests pin the driver's backstop: output that was produced gets charged, and
|
||||
* output that was never produced doesn't.
|
||||
*/
|
||||
import type { Readable } from 'node:stream';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
|
||||
import type { UsageInput } from '../../services/metering/types.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { withTestActor } from '../integrationTestUtil.js';
|
||||
import { ChatCompletionDriver } from './ChatCompletionDriver.js';
|
||||
import { FakeChatProvider } from './providers/FakeChatProvider.js';
|
||||
import type { IChatCompleteResult } from './types.js';
|
||||
import type { AIChatStream } from './utils/Streaming.js';
|
||||
|
||||
let server: PuterServer;
|
||||
|
||||
const PRICED_MODEL = {
|
||||
id: 'priced',
|
||||
aliases: [],
|
||||
costs_currency: 'usd-cents',
|
||||
costs: { input_tokens: 1000, output_tokens: 2000 },
|
||||
max_tokens: 8192,
|
||||
};
|
||||
|
||||
const makeDriver = async () => {
|
||||
const d = new ChatCompletionDriver(
|
||||
{ providers: { ollama: { enabled: false } } } as never,
|
||||
server.clients,
|
||||
server.stores,
|
||||
server.services,
|
||||
);
|
||||
d.onServerStart();
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const m = await d.models();
|
||||
if (m.length > 0) return d;
|
||||
await new Promise((r) => setTimeout(r, 5));
|
||||
}
|
||||
throw new Error('ChatCompletionDriver model map never populated in test');
|
||||
};
|
||||
|
||||
const drain = async (stream: Readable): Promise<void> => {
|
||||
for await (const _chunk of stream as AsyncIterable<Buffer>) {
|
||||
void _chunk;
|
||||
}
|
||||
// The driver meters in the `finally` of the fire-and-forget stream pump,
|
||||
// which can land a tick after the last byte.
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
};
|
||||
|
||||
/** Every usage entry the driver recorded, flattened across batches. */
|
||||
const meteredUsages = (
|
||||
spy: ReturnType<typeof vi.spyOn>,
|
||||
): UsageInput[] =>
|
||||
spy.mock.calls.flatMap((call) => (call[1] as UsageInput[]) ?? []);
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
let driver: ChatCompletionDriver;
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.spyOn(FakeChatProvider.prototype, 'models').mockResolvedValue([
|
||||
PRICED_MODEL,
|
||||
] as never);
|
||||
driver = await makeDriver();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const streamOf = (
|
||||
init: (args: { chatStream: AIChatStream }) => Promise<void>,
|
||||
): IChatCompleteResult =>
|
||||
({
|
||||
init_chat_stream: init,
|
||||
stream: true,
|
||||
}) as unknown as IChatCompleteResult;
|
||||
|
||||
const startStream = async (driverUnderTest: ChatCompletionDriver) =>
|
||||
(await withTestActor(() =>
|
||||
driverUnderTest.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'write me something' }],
|
||||
stream: true,
|
||||
}),
|
||||
)) as unknown as { stream: Readable };
|
||||
|
||||
describe('ChatCompletionDriver streaming metering backstop', () => {
|
||||
it('charges an estimate when a stream dies after producing output', async () => {
|
||||
const metered = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
const message = chatStream.message();
|
||||
const block = message.contentBlock({ type: 'text' });
|
||||
block.addText('x'.repeat(4000));
|
||||
throw new Error('upstream died mid-response');
|
||||
}) as never,
|
||||
);
|
||||
|
||||
const result = await startStream(driver);
|
||||
await drain(result.stream);
|
||||
|
||||
const estimated = meteredUsages(metered).filter((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
);
|
||||
expect(estimated.length).toBe(2);
|
||||
const output = estimated.find((u) =>
|
||||
u.usageType.endsWith('estimated_output_tokens'),
|
||||
)!;
|
||||
// 4000 characters at 4 chars/token, priced at 2000 ucents/token.
|
||||
expect(output.usageAmount).toBe(1000);
|
||||
expect(output.costOverride).toBe(2_000_000);
|
||||
});
|
||||
|
||||
it('charges an estimate when a stream completes without a usage report', async () => {
|
||||
const metered = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
const message = chatStream.message();
|
||||
const block = message.contentBlock({ type: 'text' });
|
||||
block.addText('y'.repeat(2000));
|
||||
// Provider sent no usage chunk — `end` with nothing to report.
|
||||
chatStream.end(undefined as never);
|
||||
}) as never,
|
||||
);
|
||||
|
||||
const result = await startStream(driver);
|
||||
await drain(result.stream);
|
||||
|
||||
const estimated = meteredUsages(metered).filter((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
);
|
||||
expect(estimated.length).toBe(2);
|
||||
});
|
||||
|
||||
it('does not charge when the stream failed before producing anything', async () => {
|
||||
const metered = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async () => {
|
||||
throw new Error('upstream refused the request');
|
||||
}) as never,
|
||||
);
|
||||
|
||||
const result = await startStream(driver);
|
||||
await drain(result.stream);
|
||||
|
||||
expect(
|
||||
meteredUsages(metered).filter((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
// The provider metered, then died before `chatStream.end` — the one
|
||||
// path where the backstop used to charge a second, estimated time on
|
||||
// top of the real usage.
|
||||
it('does not double-charge a stream whose provider metered before it threw', async () => {
|
||||
const metered = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
const message = chatStream.message();
|
||||
const block = message.contentBlock({ type: 'text' });
|
||||
block.addText('w'.repeat(4000));
|
||||
// Real usage was metered here (providers report the moment
|
||||
// they meter)...
|
||||
chatStream.reportUsage({ input_tokens: 10, output_tokens: 1000 });
|
||||
// ...then the stream died before reaching chatStream.end —
|
||||
// e.g. a malformed tool-call payload failing to parse.
|
||||
throw new Error('malformed tool-call payload');
|
||||
}) as never,
|
||||
);
|
||||
|
||||
const result = await startStream(driver);
|
||||
await drain(result.stream);
|
||||
|
||||
expect(
|
||||
meteredUsages(metered).filter((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
// `end({})` meters nothing — an empty usage object must not pass for a
|
||||
// usage report, or a stream full of output goes out billed at zero.
|
||||
it('charges the estimate when the usage report is empty', async () => {
|
||||
const metered = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
const message = chatStream.message();
|
||||
const block = message.contentBlock({ type: 'text' });
|
||||
block.addText('v'.repeat(2000));
|
||||
chatStream.end({} as never);
|
||||
}) as never,
|
||||
);
|
||||
|
||||
const result = await startStream(driver);
|
||||
await drain(result.stream);
|
||||
|
||||
expect(
|
||||
meteredUsages(metered).filter((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
),
|
||||
).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('leaves a provider-reported stream alone', async () => {
|
||||
const metered = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
const message = chatStream.message();
|
||||
const block = message.contentBlock({ type: 'text' });
|
||||
block.addText('z'.repeat(2000));
|
||||
chatStream.end({ input_tokens: 10, output_tokens: 500 });
|
||||
}) as never,
|
||||
);
|
||||
|
||||
const result = await startStream(driver);
|
||||
await drain(result.stream);
|
||||
|
||||
expect(
|
||||
meteredUsages(metered).filter((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
const freeUser = () => ({
|
||||
user: {
|
||||
uuid: `chat-gate-${Math.random().toString(36).slice(2)}`,
|
||||
username: 'chat-gate-user',
|
||||
email: '[email protected]',
|
||||
},
|
||||
});
|
||||
|
||||
// The gate against the real MeteringService, no mocks: a free account whose
|
||||
// month has already outrun its allowance must not reach a provider again.
|
||||
describe('ChatCompletionDriver credit gate against real metering', () => {
|
||||
|
||||
it('rejects a free account that has already spent its allowance', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const metering = server.services.metering;
|
||||
const allowance = (await metering.getActorSubscription(actor))
|
||||
.monthUsageAllowance;
|
||||
|
||||
await metering.incrementUsage(
|
||||
actor,
|
||||
'test:prior-spend',
|
||||
1,
|
||||
allowance + 1,
|
||||
);
|
||||
expect(await metering.getRemainingUsage(actor)).toBe(0);
|
||||
|
||||
const completeSpy = vi.spyOn(FakeChatProvider.prototype, 'complete');
|
||||
|
||||
await expect(
|
||||
withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'one more' }],
|
||||
}),
|
||||
actor,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
expect(completeSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// The incident this exists for: usage is recorded when a completion
|
||||
// finishes, so a second request that starts while the first is still
|
||||
// running used to read a balance that had nothing in flight subtracted
|
||||
// from it, and was told it could spend the whole thing too. Concurrency,
|
||||
// not budget, was what bounded the spend.
|
||||
it('does not let a second request spend a balance the first already has in flight', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const metering = server.services.metering;
|
||||
// Spent down to where one completion's worst case is the whole of
|
||||
// what's left — the shape of an expensive model against a small
|
||||
// allowance, which is when parallel requests overshoot.
|
||||
const allowance = (await metering.getActorSubscription(actor))
|
||||
.monthUsageAllowance;
|
||||
await metering.incrementUsage(
|
||||
actor,
|
||||
'test:prior-spend',
|
||||
1,
|
||||
Math.floor(allowance * 0.9),
|
||||
);
|
||||
|
||||
let releaseFirst: (v: unknown) => void = () => {};
|
||||
const firstInFlight = new Promise((r) => {
|
||||
releaseFirst = r;
|
||||
});
|
||||
const completeSpy = vi
|
||||
.spyOn(FakeChatProvider.prototype, 'complete')
|
||||
.mockImplementationOnce(
|
||||
async () =>
|
||||
firstInFlight.then(() => ({
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
},
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
finish_reason: 'stop',
|
||||
})) as never,
|
||||
);
|
||||
|
||||
const first = withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'first request' }],
|
||||
}),
|
||||
actor,
|
||||
);
|
||||
// Let the first request clear the gate and take its hold.
|
||||
await vi.waitFor(() => expect(completeSpy).toHaveBeenCalledTimes(1));
|
||||
|
||||
await expect(
|
||||
withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'second request' }],
|
||||
}),
|
||||
actor,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
// The second request never reached a provider.
|
||||
expect(completeSpy).toHaveBeenCalledTimes(1);
|
||||
|
||||
releaseFirst(undefined);
|
||||
await first;
|
||||
|
||||
// And the hold is given back, so the account can spend again.
|
||||
expect(
|
||||
await server.services.metering.getRemainingUsage(actor),
|
||||
).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('caps output to the balance left, so one call cannot run away with the month', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const metering = server.services.metering;
|
||||
const allowance = (await metering.getActorSubscription(actor))
|
||||
.monthUsageAllowance;
|
||||
|
||||
// Nine tenths spent — a tenth of the allowance is left to bound the
|
||||
// next completion's output.
|
||||
await metering.incrementUsage(
|
||||
actor,
|
||||
'test:prior-spend',
|
||||
1,
|
||||
Math.floor(allowance * 0.9),
|
||||
);
|
||||
const remaining = await metering.getRemainingUsage(actor);
|
||||
|
||||
const completeSpy = vi
|
||||
.spyOn(FakeChatProvider.prototype, 'complete')
|
||||
.mockResolvedValueOnce({
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
},
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
finish_reason: 'stop',
|
||||
} as never);
|
||||
|
||||
await withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'go' }],
|
||||
max_tokens: 100_000,
|
||||
}),
|
||||
actor,
|
||||
);
|
||||
|
||||
const passed = completeSpy.mock.calls[0]![0] as { max_tokens?: number };
|
||||
// 2000 ucents per output token — the cap has to fit what's left.
|
||||
expect(passed.max_tokens).toBeDefined();
|
||||
expect(passed.max_tokens! * 2000).toBeLessThanOrEqual(remaining);
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver credit gate on multimodal prompts', () => {
|
||||
it('prices attachments into the pre-flight affordability check', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const metering = server.services.metering;
|
||||
const allowance = (await metering.getActorSubscription(actor))
|
||||
.monthUsageAllowance;
|
||||
|
||||
// Leave half of what the frame below estimates to: a plain-text
|
||||
// prompt is still affordable, the same prompt carrying the frame is
|
||||
// not. (~150KB of base64 payload ≈ 1000 tokens at 1000 ucents each.)
|
||||
await metering.incrementUsage(
|
||||
actor,
|
||||
'test:prior-spend',
|
||||
1,
|
||||
allowance - 500_000,
|
||||
);
|
||||
|
||||
const completeSpy = vi
|
||||
.spyOn(FakeChatProvider.prototype, 'complete')
|
||||
.mockResolvedValue({
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
},
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
finish_reason: 'stop',
|
||||
} as never);
|
||||
|
||||
// Text alone clears the gate...
|
||||
await withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'describe' }],
|
||||
max_tokens: 10,
|
||||
}),
|
||||
actor,
|
||||
);
|
||||
expect(completeSpy).toHaveBeenCalledTimes(1);
|
||||
|
||||
// ...the same prompt carrying a frame that used to price as ~nothing
|
||||
// does not.
|
||||
await expect(
|
||||
withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'priced',
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: 'describe' },
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: `data:image/jpeg;base64,${'A'.repeat(200_000)}`,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
max_tokens: 10,
|
||||
}),
|
||||
actor,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
expect(completeSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -128,6 +128,10 @@ const OPENROUTER_CATALOG = [
|
||||
'deepseek/deepseek-v4-pro',
|
||||
'deepseek-ai/deepseek-v4-pro',
|
||||
'google/gemini-2.5-flash',
|
||||
// OpenRouter really does carry Muse Spark under Meta's own id, which is
|
||||
// also the alias the Meta provider publishes — the two have to land in
|
||||
// one bucket rather than becoming separate models.
|
||||
'meta/muse-spark-1.2',
|
||||
].map((id) => ({
|
||||
id,
|
||||
name: `${id} (via OpenRouter)`,
|
||||
@@ -158,6 +162,7 @@ beforeAll(async () => {
|
||||
providers: {
|
||||
gemini: { apiKey: 'test-key' },
|
||||
deepseek: { apiKey: 'test-key' },
|
||||
meta: { apiKey: 'test-key' },
|
||||
infron: { apiKey: 'test-key' },
|
||||
openrouter: { apiKey: 'test-key' },
|
||||
ollama: { enabled: false },
|
||||
@@ -250,6 +255,26 @@ describe('ChatCompletionDriver gemini routing', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver muse spark routing', () => {
|
||||
it('serves Muse Spark from Meta, with OpenRouter only as fallback', async () => {
|
||||
const attempts = await attemptsFor('muse-spark-1.2');
|
||||
|
||||
expect(attempts[0]).toMatchObject({
|
||||
provider: 'meta',
|
||||
model: 'muse-spark-1.2',
|
||||
});
|
||||
expect(attempts[1]).toMatchObject({
|
||||
provider: 'openrouter',
|
||||
model: 'openrouter:meta/muse-spark-1.2',
|
||||
});
|
||||
});
|
||||
|
||||
it('routes the vendor-qualified alias to Meta too', async () => {
|
||||
const attempts = await attemptsFor('meta/muse-spark-1.2');
|
||||
expect(attempts[0].provider).toBe('meta');
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver duplicate-model fallback', () => {
|
||||
// deepseek-v4-pro is served directly by DeepSeek, by Infron, and twice by
|
||||
// OpenRouter (two upstream orgs) — four routes in one bucket.
|
||||
@@ -293,6 +318,73 @@ describe('ChatCompletionDriver duplicate-model fallback', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// Each attempt in a fallback chain is a whole completion at that model's
|
||||
// prices, so each one has to clear the credit gate on its own. The chain used
|
||||
// to re-check with a nominal 1-microcent amount, which any account with a
|
||||
// fraction of a credit left passed — three attempts could cost three times
|
||||
// what the balance allowed.
|
||||
describe('ChatCompletionDriver credit gate across the fallback chain', () => {
|
||||
it('runs the full gate once per attempt, not a nominal re-check', async () => {
|
||||
// Observed, not stubbed — the counter that shows each attempt did a
|
||||
// real balance read of its own.
|
||||
const remaining = vi.spyOn(
|
||||
server.services.metering,
|
||||
'getRemainingUsage',
|
||||
);
|
||||
|
||||
const attempts = await attemptsFor('deepseek-v4-pro');
|
||||
|
||||
expect(attempts).toHaveLength(3);
|
||||
// The balance is read for each attempt because each one's
|
||||
// affordability and output cap are decided at that model's prices
|
||||
// against what's actually left.
|
||||
expect(remaining.mock.calls.length).toBeGreaterThanOrEqual(
|
||||
attempts.length,
|
||||
);
|
||||
remaining.mockRestore();
|
||||
});
|
||||
|
||||
it('aborts the chain when the balance runs out mid-fallback', async () => {
|
||||
const actor = {
|
||||
user: {
|
||||
uuid: `routing-gate-${Math.random().toString(36).slice(2)}`,
|
||||
username: 'routing-gate-user',
|
||||
email: '[email protected]',
|
||||
},
|
||||
} as never;
|
||||
const metering = server.services.metering;
|
||||
|
||||
// The first attempt fails upstream, and while it does, a "parallel
|
||||
// request" spends the rest of the month's allowance — real usage
|
||||
// rows, not a stubbed balance — so the next attempt's gate reads an
|
||||
// account with nothing left.
|
||||
createMock.mockImplementation(async () => {
|
||||
const { remaining } = await metering.getAllowedUsage(actor);
|
||||
await metering.incrementUsage(
|
||||
actor,
|
||||
'test:parallel-spend',
|
||||
1,
|
||||
remaining,
|
||||
);
|
||||
throw new Error('upstream down');
|
||||
});
|
||||
|
||||
await expect(
|
||||
withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'deepseek-v4-pro',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
actor,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver unhealthy-route skipping', () => {
|
||||
it('skips a route marked by an earlier failure and serves the next one', async () => {
|
||||
markRouteUnhealthy('deepseek', 'deepseek-v4-pro');
|
||||
|
||||
@@ -481,8 +481,8 @@ describe('ChatCompletionDriver.complete validation event routing', () => {
|
||||
|
||||
describe('ChatCompletionDriver.complete credit gate and max_tokens cap', () => {
|
||||
it('throws 402 `insufficient_funds` when the actor has no remaining credits', async () => {
|
||||
vi.spyOn(server.services.metering, 'hasEnoughCredits').mockResolvedValue(
|
||||
false,
|
||||
vi.spyOn(server.services.metering, 'getRemainingUsage').mockResolvedValue(
|
||||
0,
|
||||
);
|
||||
|
||||
await expect(
|
||||
@@ -562,11 +562,8 @@ describe('ChatCompletionDriver.complete credit gate and max_tokens cap', () => {
|
||||
]);
|
||||
const d = await makeDriver();
|
||||
|
||||
// Pass the cheap pre-flight gate but leave a balance too small to
|
||||
// Pass the cheap pre-flight check but leave a balance too small to
|
||||
// afford a single 2000-microcent output token.
|
||||
vi.spyOn(server.services.metering, 'hasEnoughCredits').mockResolvedValue(
|
||||
true,
|
||||
);
|
||||
vi.spyOn(server.services.metering, 'getRemainingUsage').mockResolvedValue(
|
||||
100,
|
||||
);
|
||||
@@ -610,10 +607,6 @@ describe('ChatCompletionDriver.complete credit gate and max_tokens cap', () => {
|
||||
] as never);
|
||||
const d = await makeDriver();
|
||||
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'hasEnoughCredits',
|
||||
).mockResolvedValue(true);
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'getRemainingUsage',
|
||||
@@ -658,9 +651,6 @@ describe('ChatCompletionDriver.complete credit gate and max_tokens cap', () => {
|
||||
]);
|
||||
const d = await makeDriver();
|
||||
// Plenty of credits so the credit gate doesn't intercept first.
|
||||
vi.spyOn(server.services.metering, 'hasEnoughCredits').mockResolvedValue(
|
||||
true,
|
||||
);
|
||||
vi.spyOn(server.services.metering, 'getRemainingUsage').mockResolvedValue(
|
||||
1_000_000,
|
||||
);
|
||||
@@ -768,19 +758,20 @@ describe('ChatCompletionDriver.complete fallback and error envelope', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('re-checks `hasEnoughCredits` between fallback attempts so a parallel request that drains the wallet aborts the chain', async () => {
|
||||
// The primary provider throws; the fallback loop checks credits
|
||||
// before its next upstream hit. We force `false` on the second
|
||||
// check to verify the 402 short-circuit, even though no actual
|
||||
// fallback model is wired (the loop bails on the credit gate
|
||||
// before `#findFallback` decides there's nowhere to go).
|
||||
it('re-reads the balance between fallback attempts so a parallel request that drains the wallet aborts the chain', async () => {
|
||||
// The primary provider throws; the fallback loop runs the full gate
|
||||
// (one balance read per attempt) before its next upstream hit. We
|
||||
// force an empty balance on the second read to verify the 402
|
||||
// short-circuit, even though no actual fallback model is wired (the
|
||||
// loop bails on the credit gate before `#findFallback` decides
|
||||
// there's nowhere to go).
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockRejectedValueOnce(
|
||||
new Error('boom'),
|
||||
);
|
||||
const credits = vi
|
||||
.spyOn(server.services.metering, 'hasEnoughCredits')
|
||||
.mockResolvedValueOnce(true) // pre-flight
|
||||
.mockResolvedValueOnce(false); // mid-fallback re-check
|
||||
const remaining = vi
|
||||
.spyOn(server.services.metering, 'getRemainingUsage')
|
||||
.mockResolvedValueOnce(1_000_000) // pre-flight
|
||||
.mockResolvedValueOnce(0); // drained mid-fallback
|
||||
|
||||
// No second provider serves `fake`, so `#findFallback` returns
|
||||
// null and the loop exits before reaching the credit re-check.
|
||||
@@ -795,7 +786,7 @@ describe('ChatCompletionDriver.complete fallback and error envelope', () => {
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 500 });
|
||||
expect(credits.mock.calls.length).toBeGreaterThanOrEqual(1);
|
||||
expect(remaining.mock.calls.length).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -20,24 +20,29 @@
|
||||
import crypto from 'node:crypto';
|
||||
import { PassThrough } from 'node:stream';
|
||||
import { EventMap } from '../../clients/event/types.js';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { Context } from '../../core/context.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
|
||||
import {
|
||||
DEFAULT_FREE_SUBSCRIPTION,
|
||||
DEFAULT_TEMP_SUBSCRIPTION,
|
||||
} from '../../services/metering/consts.js';
|
||||
import type { CreditHold } from '../../services/metering/types.js';
|
||||
import { NO_CREDIT_HOLD } from '../../services/metering/types.js';
|
||||
import type { DriverStreamResult } from '../meta.js';
|
||||
import { PuterDriver } from '../types.js';
|
||||
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../util/aiLimits.js';
|
||||
import { AlibabaProvider } from './providers/alibaba/AlibabaProvider.js';
|
||||
import { AzureChatProvider } from './providers/azure/AzureChatProvider.js';
|
||||
import { AzureResponsesProvider } from './providers/azure/AzureResponsesProvider.js';
|
||||
import { BytePlusProvider } from './providers/byteplus/BytePlusProvider.js';
|
||||
import { ClaudeProvider } from './providers/claude/ClaudeProvider.js';
|
||||
import { DeepSeekProvider } from './providers/deepseek/DeepSeekProvider.js';
|
||||
import { FakeChatProvider } from './providers/FakeChatProvider.js';
|
||||
import { GeminiChatProvider } from './providers/gemini/GeminiChatProvider.js';
|
||||
import { GroqAIProvider } from './providers/groq/GroqAIProvider.js';
|
||||
import { InfronProvider } from './providers/infron/InfronProvider.js';
|
||||
import { MetaProvider } from './providers/meta/MetaProvider.js';
|
||||
import { MiniMaxProvider } from './providers/minimax/MiniMaxProvider.js';
|
||||
import { MistralAIProvider } from './providers/mistral/MistralAiProvider.js';
|
||||
import { MoonshotProvider } from './providers/moonshot/MoonshotProvider.js';
|
||||
@@ -57,7 +62,6 @@ import type {
|
||||
} from './types.js';
|
||||
import { normalize_tools_object } from './utils/FunctionCalling.js';
|
||||
import {
|
||||
extract_text,
|
||||
normalize_messages,
|
||||
normalize_single_message,
|
||||
} from './utils/Messages.js';
|
||||
@@ -66,14 +70,35 @@ import {
|
||||
isIdentityKey,
|
||||
normalizeModelKey,
|
||||
} from './utils/modelRouting.js';
|
||||
import { costKeys, isFreeModel } from './utils/pricing.js';
|
||||
import {
|
||||
isRouteUnhealthy,
|
||||
markRouteUnhealthy,
|
||||
} from './utils/providerHealth.js';
|
||||
import { AIChatStream } from './utils/Streaming.js';
|
||||
import {
|
||||
estimateOutputTokens,
|
||||
estimatePromptTokens,
|
||||
} from './utils/usageEstimate.js';
|
||||
|
||||
const MAX_ATTEMPTS = 3; // the first attempt plus two fallbacks
|
||||
|
||||
/**
|
||||
* How often a streaming completion renews its credit hold. Holds default to a
|
||||
* 10-minute TTL; a long generation (a reasoning model with a large
|
||||
* `max_tokens`) can stream past that, and a hold that expires mid-stream
|
||||
* reopens the overspend window it exists to close.
|
||||
*/
|
||||
const HOLD_RENEW_INTERVAL_MS = 5 * 60 * 1000;
|
||||
|
||||
/**
|
||||
* A moderation refusal is a completion that was produced and charged, then
|
||||
* withheld — not a route failure. Retrying it on a fallback provider would bill
|
||||
* the account again for another completion the user will never see.
|
||||
*/
|
||||
const isModerationRefusal = (e: unknown): boolean =>
|
||||
isHttpError(e) && e.code === 'moderation_flagged';
|
||||
|
||||
type ProviderAttempt = {
|
||||
model: string;
|
||||
provider: string;
|
||||
@@ -157,13 +182,17 @@ const routeId = (provider: string, modelId: string) => `${provider}:${modelId}`;
|
||||
*
|
||||
* Per-class rules (see also alarm gate in server.ts):
|
||||
*
|
||||
* - All rate-limited → 429 `upstream_rate_limited` (paged: forced alert)
|
||||
* - All rate-limited → 429 `upstream_rate_limited` (alerted, unless every attempt
|
||||
* was on a free model — see `allModelsFree`)
|
||||
* - All auth failures → 500 `upstream_auth_failed` (paged: our config)
|
||||
* - All upstream 5xx → 400 `upstream_provider_unavailable` (no page)
|
||||
* - All upstream 4xx (other) → 400 `upstream_bad_request` (no page)
|
||||
* - Mixed → 400 `upstream_failed` (no page)
|
||||
*/
|
||||
const classifyAttempts = (attempts: ProviderAttempt[]): HttpError => {
|
||||
const classifyAttempts = (
|
||||
attempts: ProviderAttempt[],
|
||||
{ allModelsFree = false } = {},
|
||||
): HttpError => {
|
||||
const fields = { attempts };
|
||||
if (attempts.length === 0) {
|
||||
return new HttpError(500, 'No providers attempted', {
|
||||
@@ -176,6 +205,11 @@ const classifyAttempts = (attempts: ProviderAttempt[]): HttpError => {
|
||||
return new HttpError(429, 'AI provider rate limit exceeded', {
|
||||
legacyCode: 'upstream_rate_limited',
|
||||
fields,
|
||||
// A free model getting throttled upstream is the deal we took
|
||||
// when we picked it up for nothing: there's no billing at stake
|
||||
// and nothing to act on, and the volume tracks traffic. The
|
||||
// caller still gets the 429; we just don't record it.
|
||||
noAlarm: allModelsFree,
|
||||
});
|
||||
}
|
||||
if (attempts.every(isAuthFailure)) {
|
||||
@@ -334,6 +368,15 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
normalize_tools_object(args.tools);
|
||||
}
|
||||
|
||||
// Both estimated once, before any attempt: providers rewrite
|
||||
// `args.messages` in place (tool_use blocks move out of `content`), so
|
||||
// an estimate taken after a failed attempt would undercount the same
|
||||
// prompt — and the gate writes each attempt's output cap into
|
||||
// `args.max_tokens`, so the user's requested value has to be kept
|
||||
// apart from what the previous attempt was capped to.
|
||||
const promptTokenEstimate = estimatePromptTokens(args.messages ?? []);
|
||||
const requestedMaxTokens = args.max_tokens;
|
||||
|
||||
const completionId = crypto
|
||||
.randomUUID()
|
||||
.replaceAll('-', '')
|
||||
@@ -376,89 +419,20 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
}
|
||||
}
|
||||
|
||||
// -- Credit / subscription gates (metering) --------------------
|
||||
// Cheap pre-flight: reject when the user can't afford even the
|
||||
// approximate input cost, keep subscriber-only models gated, and
|
||||
// cap `max_tokens` so output can't exceed remaining credits.
|
||||
// Skipped for blocked requests since fake-chat is free and the
|
||||
// user shouldn't see a billing error in place of the abuse page.
|
||||
// Skipped for blocked requests since fake-chat is free and the user
|
||||
// shouldn't see a billing error in place of the abuse page.
|
||||
//
|
||||
// The gate hands back a hold on what this attempt could cost, which
|
||||
// stands in for its usage until the real numbers land. It is released
|
||||
// on every way out of this method — including the streaming path,
|
||||
// where "done" is the stream draining rather than this method
|
||||
// returning.
|
||||
let hold: CreditHold = NO_CREDIT_HOLD;
|
||||
if (!blocked) {
|
||||
const metering = this.services.metering;
|
||||
const inputCostKey =
|
||||
(model.input_cost_key as string | undefined) ?? 'input_tokens';
|
||||
const outputCostKey =
|
||||
(model.output_cost_key as string | undefined) ??
|
||||
'output_tokens';
|
||||
const inputTokenCost = Number(model.costs?.[inputCostKey] ?? 0);
|
||||
const outputTokenCost = Number(model.costs?.[outputCostKey] ?? 0);
|
||||
const text = extract_text(args.messages ?? []);
|
||||
// Rough estimator from v1 — avg of char/4 and word*(4/3), halved.
|
||||
// See https://help.openai.com/en/articles/4936856
|
||||
const approximateTokenCount = Math.floor(
|
||||
(text.length / 4 + text.split(/\s+/).length * (4 / 3)) / 2,
|
||||
);
|
||||
const approximateInputCost = approximateTokenCount * inputTokenCost;
|
||||
const minimumCredits = Number(model.minimumCredits || 1);
|
||||
|
||||
const usageAllowed = await metering.hasEnoughCredits(
|
||||
actor,
|
||||
Math.max(approximateInputCost, minimumCredits),
|
||||
);
|
||||
if (!usageAllowed) {
|
||||
throw new HttpError(402, 'No usage left for request.', {
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
}
|
||||
|
||||
if (model.subscriberOnly) {
|
||||
const subscription = await metering.getActorSubscription(actor);
|
||||
const isDefaultPolicy =
|
||||
subscription.id === DEFAULT_FREE_SUBSCRIPTION ||
|
||||
subscription.id === DEFAULT_TEMP_SUBSCRIPTION;
|
||||
if (isDefaultPolicy) {
|
||||
throw new HttpError(
|
||||
403,
|
||||
`The model ${model.id} is only available to subscribers. Please subscribe to access this model.`,
|
||||
{ legacyCode: 'permission_denied' },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (outputTokenCost > 0) {
|
||||
const remainingCredits =
|
||||
await metering.getRemainingUsage(actor);
|
||||
const maxAllowedOutputUcents =
|
||||
remainingCredits - approximateInputCost;
|
||||
const maxAllowedOutputTokens =
|
||||
maxAllowedOutputUcents / outputTokenCost;
|
||||
// A provider may not know a model's output ceiling. Drop the
|
||||
// term rather than let a missing value drive the cap: `null`
|
||||
// coerces to 0, so the subtraction goes negative instead of
|
||||
// NaN and the user is told they're out of credits.
|
||||
const modelOutputCeiling =
|
||||
Number.isFinite(model.max_tokens) && model.max_tokens > 0
|
||||
? model.max_tokens - approximateTokenCount
|
||||
: Number.POSITIVE_INFINITY;
|
||||
const cap = Math.floor(
|
||||
Math.min(
|
||||
args.max_tokens ?? Number.POSITIVE_INFINITY,
|
||||
maxAllowedOutputTokens,
|
||||
modelOutputCeiling,
|
||||
),
|
||||
);
|
||||
// `cap` is the credit-bounded ceiling on output tokens. When
|
||||
// it drops below 1 the user can't afford even a single output
|
||||
// token, so reject the request. Crucially we must NOT leave
|
||||
// `max_tokens` unset here: an undefined max_tokens lets the
|
||||
// provider run to the model's full output limit (e.g. 128k for
|
||||
// Claude), billing far past the user's remaining balance.
|
||||
if (cap < 1) {
|
||||
throw new HttpError(402, 'No usage left for request.', {
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
}
|
||||
args.max_tokens = cap;
|
||||
}
|
||||
hold = await this.#applyCreditGate(actor, model, args, {
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
});
|
||||
}
|
||||
|
||||
// First attempt
|
||||
@@ -473,18 +447,18 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
|
||||
const attempts: ProviderAttempt[] = [];
|
||||
let res: IChatCompleteResult | undefined;
|
||||
// Tracked across the chain so the classifier can tell a chain that
|
||||
// only ever touched free models from one that cost the user something.
|
||||
let allModelsFree = true;
|
||||
|
||||
// A failed route is remembered briefly so the next request skips it
|
||||
// rather than paying its timeout again.
|
||||
const recordFailure = (
|
||||
modelId: string,
|
||||
providerId: string,
|
||||
err: unknown,
|
||||
) => {
|
||||
const attempt = toAttempt(modelId, providerId, err);
|
||||
const recordFailure = (failed: IChatModel, err: unknown) => {
|
||||
const attempt = toAttempt(failed.id, failed.provider!, err);
|
||||
attempts.push(attempt);
|
||||
if (!isFreeModel(failed)) allModelsFree = false;
|
||||
if (isRouteLevelFailure(attempt)) {
|
||||
markRouteUnhealthy(providerId, modelId);
|
||||
markRouteUnhealthy(failed.provider!, failed.id);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -495,7 +469,15 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
provider: model.provider,
|
||||
});
|
||||
} catch (e) {
|
||||
recordFailure(model.id, model.provider!, e);
|
||||
// This attempt is over and cost whatever it cost; the next one
|
||||
// takes a hold of its own.
|
||||
await hold.release();
|
||||
hold = NO_CREDIT_HOLD;
|
||||
|
||||
// A withheld completion was still a completion — charged, final,
|
||||
// not a route failure worth another (billed) attempt elsewhere.
|
||||
if (isModerationRefusal(e)) throw e;
|
||||
recordFailure(model, e);
|
||||
|
||||
// Fallback loop — the bucket holds every provider that serves this
|
||||
// model, ranked by `compareModelPreference`, so each miss walks one
|
||||
@@ -511,14 +493,18 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
const fbProvider = this.#providers[fallback.provider!];
|
||||
if (!fbProvider) break;
|
||||
|
||||
// Credits can be exhausted mid-fallback by parallel requests;
|
||||
// re-check before another upstream hit. Same bail as the
|
||||
// pre-flight above.
|
||||
const fallbackUsageAllowed =
|
||||
await this.services.metering.hasEnoughCredits(actor, 1);
|
||||
if (!fallbackUsageAllowed) {
|
||||
throw new HttpError(402, 'No usage left for request.', {
|
||||
legacyCode: 'insufficient_funds',
|
||||
// Every attempt is a whole completion the account pays for, so
|
||||
// each one goes through the full gate again rather than a
|
||||
// token "do they have anything left" check: the balance may
|
||||
// have been spent by a parallel request, and the fallback is
|
||||
// a different model at a different price, whose output has to
|
||||
// be capped against what is actually left.
|
||||
// The previous attempt released its hold when it failed, so
|
||||
// this one starts from nothing held.
|
||||
if (!blocked) {
|
||||
hold = await this.#applyCreditGate(actor, fallback, args, {
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -533,14 +519,18 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
model = fallback;
|
||||
lastError = null;
|
||||
} catch (fbErr) {
|
||||
await hold.release();
|
||||
hold = NO_CREDIT_HOLD;
|
||||
if (isModerationRefusal(fbErr)) throw fbErr;
|
||||
lastError = fbErr as Error;
|
||||
recordFailure(fallback.id, fallback.provider!, fbErr);
|
||||
recordFailure(fallback, fbErr);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!res) {
|
||||
throw classifyAttempts(attempts);
|
||||
await hold.release();
|
||||
throw classifyAttempts(attempts, { allModelsFree });
|
||||
}
|
||||
|
||||
const username = actor.user?.username;
|
||||
@@ -588,6 +578,13 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
return originalEnd(enrichedUsage!);
|
||||
};
|
||||
|
||||
// The hold lives for the whole stream, which can outlast its TTL —
|
||||
// keep pushing the deadline out until the pump is done.
|
||||
const renewHold = setInterval(() => {
|
||||
void hold.extend?.();
|
||||
}, HOLD_RENEW_INTERVAL_MS);
|
||||
renewHold.unref?.();
|
||||
|
||||
// Fire-and-forget — the stream writes happen async while the
|
||||
// response is being piped to the client.
|
||||
(async () => {
|
||||
@@ -602,6 +599,26 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
);
|
||||
passthrough.end();
|
||||
} finally {
|
||||
clearInterval(renewHold);
|
||||
// Providers report usage the moment they meter it (see
|
||||
// `AIChatStream.reportUsage`); a stream that never got
|
||||
// there was never charged for.
|
||||
if (!blocked && !chatStream.reportedUsage) {
|
||||
this.#meterUnreportedStream({
|
||||
actor,
|
||||
chatStream,
|
||||
model,
|
||||
promptTokenEstimate,
|
||||
completionId,
|
||||
username,
|
||||
intendedProvider,
|
||||
});
|
||||
}
|
||||
// Held until the generation is actually over: for a
|
||||
// stream, the provider returns as soon as it has a
|
||||
// populator, and everything the account pays for happens
|
||||
// after that.
|
||||
await hold.release();
|
||||
if (cleanup) await cleanup();
|
||||
}
|
||||
})();
|
||||
@@ -615,6 +632,10 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
return streamResult as unknown as IChatCompleteResult;
|
||||
}
|
||||
|
||||
// The provider recorded this completion's usage before returning it,
|
||||
// so the hold has served its purpose.
|
||||
await hold.release();
|
||||
|
||||
// -- Post-completion audit event ------------------------------
|
||||
// Only for non-streaming results (streaming emits from the
|
||||
// `chatStream.end` wrapper above). Extensions like prompt_block /
|
||||
@@ -676,10 +697,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
outputMicroCents: number;
|
||||
totalMicroCents: number;
|
||||
} | null {
|
||||
const inputKey =
|
||||
(model.input_cost_key as string | undefined) ?? 'input_tokens';
|
||||
const outputKey =
|
||||
(model.output_cost_key as string | undefined) ?? 'output_tokens';
|
||||
const { inputKey, outputKey } = costKeys(model);
|
||||
|
||||
const costs = model.costs;
|
||||
if (!costs) return null;
|
||||
@@ -763,6 +781,199 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The credit and subscription gate for one upstream attempt.
|
||||
*
|
||||
* Runs before every attempt, not once per request: each attempt is a whole
|
||||
* completion the account pays for, at that model's prices, against whatever
|
||||
* balance is left by the time it starts.
|
||||
*
|
||||
* Rejects when the account can't afford the approximate input cost, keeps
|
||||
* subscriber-only models gated, and tightens `args.max_tokens` so the
|
||||
* output this attempt can produce is bounded by the remaining balance.
|
||||
*
|
||||
* Returns a hold on what the attempt can cost at worst, so requests this
|
||||
* account is running in parallel see the spend before it is recorded. The
|
||||
* caller releases it once the attempt is done.
|
||||
*/
|
||||
async #applyCreditGate(
|
||||
actor: Actor,
|
||||
model: IChatModel,
|
||||
args: ICompleteArguments,
|
||||
estimates: {
|
||||
/**
|
||||
* Prompt tokens, estimated once before any attempt — counts
|
||||
* attachments as well as text, and predates any in-place message
|
||||
* rewriting a previous attempt's provider did.
|
||||
*/
|
||||
promptTokenEstimate: number;
|
||||
/**
|
||||
* What the user asked for, kept apart from `args.max_tokens`, which
|
||||
* carries the previous attempt's cap: a cheap fallback must not
|
||||
* inherit the ceiling computed at an expensive model's price.
|
||||
*/
|
||||
requestedMaxTokens: number | undefined;
|
||||
},
|
||||
): Promise<CreditHold> {
|
||||
const metering = this.services.metering;
|
||||
const { promptTokenEstimate, requestedMaxTokens } = estimates;
|
||||
const { inputKey, outputKey } = costKeys(model);
|
||||
// `|| 0` also catches NaN from a malformed cost table.
|
||||
const inputTokenCost = Number(model.costs?.[inputKey] ?? 0) || 0;
|
||||
const outputTokenCost = Number(model.costs?.[outputKey] ?? 0) || 0;
|
||||
const approximateInputCost = promptTokenEstimate * inputTokenCost;
|
||||
const minimumCredits = Number(model.minimumCredits || 1);
|
||||
|
||||
// One balance read serves the whole gate: the affordability check
|
||||
// here and the output cap below.
|
||||
const remainingCredits = await metering.getRemainingUsage(actor);
|
||||
if (remainingCredits < Math.max(approximateInputCost, minimumCredits)) {
|
||||
throw new HttpError(402, 'No usage left for request.', {
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
}
|
||||
|
||||
if (model.subscriberOnly) {
|
||||
const subscription = await metering.getActorSubscription(actor);
|
||||
const isDefaultPolicy =
|
||||
subscription.id === DEFAULT_FREE_SUBSCRIPTION ||
|
||||
subscription.id === DEFAULT_TEMP_SUBSCRIPTION;
|
||||
if (isDefaultPolicy) {
|
||||
throw new HttpError(
|
||||
403,
|
||||
`The model ${model.id} is only available to subscribers. Please subscribe to access this model.`,
|
||||
{ legacyCode: 'permission_denied' },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (outputTokenCost > 0) {
|
||||
const maxAllowedOutputUcents =
|
||||
remainingCredits - approximateInputCost;
|
||||
const maxAllowedOutputTokens =
|
||||
maxAllowedOutputUcents / outputTokenCost;
|
||||
// A provider may not know a model's output ceiling. Drop the term
|
||||
// rather than let a missing value drive the cap: `null` coerces to
|
||||
// 0, so the subtraction goes negative instead of NaN and the user
|
||||
// is told they're out of credits.
|
||||
const modelOutputCeiling =
|
||||
Number.isFinite(model.max_tokens) && model.max_tokens > 0
|
||||
? model.max_tokens - promptTokenEstimate
|
||||
: Number.POSITIVE_INFINITY;
|
||||
const cap = Math.floor(
|
||||
Math.min(
|
||||
requestedMaxTokens ?? Number.POSITIVE_INFINITY,
|
||||
maxAllowedOutputTokens,
|
||||
modelOutputCeiling,
|
||||
),
|
||||
);
|
||||
// `cap` is the credit-bounded ceiling on output tokens. When it
|
||||
// drops below 1 the user can't afford even a single output token,
|
||||
// so reject the request. Crucially we must NOT leave `max_tokens`
|
||||
// unset here: an undefined max_tokens lets the provider run to the
|
||||
// model's full output limit (e.g. 128k for Claude), billing far
|
||||
// past the user's remaining balance.
|
||||
if (cap < 1) {
|
||||
throw new HttpError(402, 'No usage left for request.', {
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
}
|
||||
args.max_tokens = cap;
|
||||
} else {
|
||||
// No output price, nothing to bound — but a previous attempt may
|
||||
// have written its cap here; give this one the user's own value.
|
||||
args.max_tokens = requestedMaxTokens;
|
||||
}
|
||||
|
||||
// What this attempt can cost at worst: the prompt, plus output run to
|
||||
// the cap just set. Capped output is what makes the number finite —
|
||||
// for a model with no output price the output term is zero and the
|
||||
// prompt estimate stands alone.
|
||||
const worstCaseCost =
|
||||
approximateInputCost + (args.max_tokens ?? 0) * outputTokenCost;
|
||||
return this.services.metering.reserveCredits(
|
||||
actor,
|
||||
Math.max(worstCaseCost, minimumCredits),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Charge a stream that produced output but never reported usage.
|
||||
*
|
||||
* Providers meter from the usage they hand to `chatStream.end`, at the very
|
||||
* end of the stream — so anything that stops the stream short of that point
|
||||
* (an upstream error mid-response, a malformed tool-call payload, a
|
||||
* provider that never sends a usage chunk) leaves a completion the upstream
|
||||
* has already billed us for and the account has paid nothing for. This is
|
||||
* the backstop: what the stream actually emitted, priced off the model's
|
||||
* own cost table.
|
||||
*
|
||||
* Only when there was output. A stream that failed before producing
|
||||
* anything cost the user nothing, and charging an estimated prompt to
|
||||
* someone whose request we failed to serve is worse than the leak.
|
||||
*
|
||||
* Recorded under `estimated_*` usage keys so the numbers stay separable
|
||||
* from provider-reported ones in the usage breakdown.
|
||||
*/
|
||||
#meterUnreportedStream(params: {
|
||||
actor: Actor;
|
||||
chatStream: AIChatStream;
|
||||
model: IChatModel;
|
||||
/** Estimated before any provider rewrote `args.messages` in place. */
|
||||
promptTokenEstimate: number;
|
||||
completionId: string;
|
||||
username?: string;
|
||||
intendedProvider: string;
|
||||
}): void {
|
||||
const {
|
||||
actor,
|
||||
chatStream,
|
||||
model,
|
||||
promptTokenEstimate,
|
||||
completionId,
|
||||
username,
|
||||
intendedProvider,
|
||||
} = params;
|
||||
|
||||
const outputTokens = estimateOutputTokens(chatStream.outputChars ?? 0);
|
||||
if (outputTokens <= 0) return;
|
||||
|
||||
const { inputKey, outputKey } = costKeys(model);
|
||||
const inputTokens = promptTokenEstimate;
|
||||
const usage = {
|
||||
[inputKey]: inputTokens,
|
||||
[outputKey]: outputTokens,
|
||||
};
|
||||
|
||||
const cost = this.#computeCost(usage, model);
|
||||
this.services.metering.utilRecordUsageObject(
|
||||
{
|
||||
[`estimated_${inputKey}`]: inputTokens,
|
||||
[`estimated_${outputKey}`]: outputTokens,
|
||||
},
|
||||
actor,
|
||||
`${model.provider}:${model.id}`,
|
||||
{
|
||||
// Undefined when the model has no cost table: the entry is
|
||||
// recorded unpriced rather than free.
|
||||
[`estimated_${inputKey}`]: cost?.inputMicroCents,
|
||||
[`estimated_${outputKey}`]: cost?.outputMicroCents,
|
||||
},
|
||||
);
|
||||
|
||||
console.warn(
|
||||
`[ai-chat] stream ended without usage; charged an estimate (${completionId}, ${model.provider}:${model.id}, ~${inputTokens} in / ~${outputTokens} out)`,
|
||||
);
|
||||
|
||||
this.#emitCostCalculated({
|
||||
completionId,
|
||||
username,
|
||||
usage,
|
||||
model,
|
||||
intendedProvider,
|
||||
});
|
||||
}
|
||||
|
||||
// Add `usd_cents` to the usage object. Skips if the provider already
|
||||
// set an authoritative value (e.g. OpenRouter's `usage.cost`).
|
||||
// Sets `null` when cost data is unavailable for the model.
|
||||
@@ -795,10 +1006,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
params;
|
||||
|
||||
const cost = this.#computeCost(usage, model);
|
||||
const inputKey =
|
||||
(model.input_cost_key as string | undefined) ?? 'input_tokens';
|
||||
const outputKey =
|
||||
(model.output_cost_key as string | undefined) ?? 'output_tokens';
|
||||
const { inputKey, outputKey } = costKeys(model);
|
||||
const inputTokens = cost?.inputTokens ?? 0;
|
||||
const outputTokens = cost?.outputTokens ?? 0;
|
||||
const inputMicroCents = cost?.inputMicroCents ?? 0;
|
||||
@@ -926,6 +1134,23 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
});
|
||||
}
|
||||
|
||||
const meta = providers['meta'];
|
||||
const metaKey = readKey(meta);
|
||||
if (metaKey) {
|
||||
this.#providers['meta'] = new MetaProvider(
|
||||
metering,
|
||||
{
|
||||
fsEntry: this.stores.fsEntry,
|
||||
s3Object: this.stores.s3Object,
|
||||
},
|
||||
this.services.fs,
|
||||
{
|
||||
apiKey: metaKey,
|
||||
apiBaseUrl: meta?.apiBaseUrl as string | undefined,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
const groqKey = readKey(providers['groq']);
|
||||
if (groqKey) {
|
||||
this.#providers['groq'] = new GroqAIProvider(
|
||||
@@ -1002,14 +1227,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
);
|
||||
}
|
||||
|
||||
const togetherKey = readKey(providers['together-ai']);
|
||||
if (togetherKey) {
|
||||
this.#providers['together-ai'] = new TogetherAIProvider(
|
||||
{ apiKey: togetherKey },
|
||||
metering,
|
||||
);
|
||||
}
|
||||
|
||||
// Ollama — auto-discover local instance unless `enabled: false`.
|
||||
const ollama = providers['ollama'];
|
||||
if (ollama?.enabled !== false) {
|
||||
@@ -1021,6 +1238,18 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
);
|
||||
}
|
||||
|
||||
const infron = providers['infron'];
|
||||
const infronKey = readKey(infron);
|
||||
if (infronKey) {
|
||||
this.#providers['infron'] = new InfronProvider(
|
||||
{
|
||||
apiKey: infronKey,
|
||||
apiBaseUrl: infron?.apiBaseUrl as string | undefined,
|
||||
},
|
||||
metering,
|
||||
);
|
||||
}
|
||||
|
||||
const openrouter = providers['openrouter'];
|
||||
const openrouterKey = readKey(openrouter);
|
||||
if (openrouterKey) {
|
||||
@@ -1033,13 +1262,21 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
);
|
||||
}
|
||||
|
||||
const infron = providers['infron'];
|
||||
const infronKey = readKey(infron);
|
||||
if (infronKey) {
|
||||
this.#providers['infron'] = new InfronProvider(
|
||||
const togetherKey = readKey(providers['together-ai']);
|
||||
if (togetherKey) {
|
||||
this.#providers['together-ai'] = new TogetherAIProvider(
|
||||
{ apiKey: togetherKey },
|
||||
metering,
|
||||
);
|
||||
}
|
||||
|
||||
const byteplus = providers['byteplus'];
|
||||
const byteplusKey = readKey(byteplus);
|
||||
if (byteplusKey) {
|
||||
this.#providers['byteplus'] = new BytePlusProvider(
|
||||
{
|
||||
apiKey: infronKey,
|
||||
apiBaseUrl: infron?.apiBaseUrl as string | undefined,
|
||||
apiKey: byteplusKey,
|
||||
apiBaseUrl: byteplus?.apiBaseUrl as string | undefined,
|
||||
},
|
||||
metering,
|
||||
);
|
||||
|
||||
@@ -157,6 +157,93 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
},
|
||||
},
|
||||
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen3.6-flash',
|
||||
id: 'qwen3.6-flash',
|
||||
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-04-27',
|
||||
name: 'Qwen3.6 Flash',
|
||||
aliases: ['qwen/qwen3.6-flash'],
|
||||
context: 1_000_000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 19,
|
||||
completion_tokens: 113,
|
||||
},
|
||||
max_tokens: 65_536,
|
||||
},
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen3.7-plus',
|
||||
id: 'qwen3.7-plus',
|
||||
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-06-02',
|
||||
name: 'Qwen3.7 Plus',
|
||||
aliases: ['qwen/qwen3.7-plus'],
|
||||
context: 1_000_000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
completion_tokens: 300,
|
||||
cached_tokens: 5,
|
||||
},
|
||||
max_tokens: 65_536,
|
||||
},
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen3.7-max',
|
||||
id: 'qwen3.7-max',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-05-21',
|
||||
name: 'Qwen3.7 Max',
|
||||
aliases: ['qwen/qwen3.7-max'],
|
||||
context: 1_000_000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
completion_tokens: 750,
|
||||
cached_tokens: 50,
|
||||
},
|
||||
max_tokens: 65_536,
|
||||
},
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen3.8-max',
|
||||
id: 'qwen3.8-max',
|
||||
modalities: {
|
||||
input: ['text', 'image', 'video', 'pdf'],
|
||||
output: ['text'],
|
||||
},
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-08-03',
|
||||
name: 'Qwen3.8 Max',
|
||||
aliases: ['qwen/qwen3.8-max'],
|
||||
context: 1_000_000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
completion_tokens: 600,
|
||||
cached_tokens: 25,
|
||||
},
|
||||
max_tokens: 131_072,
|
||||
},
|
||||
|
||||
// -- Turbo / Flash tier -----------------------------------------
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen-turbo',
|
||||
@@ -270,29 +357,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
cached_tokens: 0,
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen3-coder-30b-a3b-instruct',
|
||||
id: 'qwen3-coder-30b-a3b-instruct',
|
||||
name: 'Qwen3-Coder 30B-A3B Instruct',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
knowledge: '2025-04',
|
||||
release_date: '2025-04',
|
||||
aliases: ['qwen/qwen3-coder-30b-a3b-instruct'],
|
||||
context: 262_144,
|
||||
max_tokens: 65_536,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 45,
|
||||
completion_tokens: 225,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
},
|
||||
|
||||
// -- Reasoning models -------------------------------------------
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwq-plus',
|
||||
@@ -666,28 +730,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
cached_tokens: 0,
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen3-vl-30b-a3b',
|
||||
id: 'qwen3-vl-30b-a3b',
|
||||
name: 'Qwen3-VL 30B-A3B',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
knowledge: '2025-04',
|
||||
release_date: '2025-04',
|
||||
aliases: ['qwen/qwen3-vl-30b-a3b'],
|
||||
context: 131_072,
|
||||
max_tokens: 32_768,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
completion_tokens: 80,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'alibaba:qwen/qvq-max',
|
||||
id: 'qvq-max',
|
||||
|
||||
@@ -186,7 +186,7 @@ describe('AzureChatProvider model catalog', () => {
|
||||
expect(ids).toContain('openai/gpt-4o');
|
||||
expect(ids).toContain('x-ai/grok-4-20-non-reasoning');
|
||||
// Codex is Responses-only and must not be advertised here.
|
||||
expect(ids).not.toContain('gpt-5-codex');
|
||||
expect(ids).not.toContain('gpt-5.3-codex');
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
baseURL: config.apiURL,
|
||||
});
|
||||
}
|
||||
checkModeration(_text: string): { flagged: boolean; categories: string[] } {
|
||||
checkModeration(_text: string) {
|
||||
throw new Error('Method not implemented.');
|
||||
}
|
||||
|
||||
@@ -180,7 +180,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
// })
|
||||
|
||||
const userIdentifier =
|
||||
actor.user?.id + actor.app?.uid ? `:${actor?.app?.uid}` : '';
|
||||
`${actor.user?.id}` + actor.app?.uid ? `:${actor?.app?.uid}` : '';
|
||||
|
||||
// Resolve any `puter_path` content parts into inline base64 data URLs.
|
||||
// Chat Completions doesn't support file uploads, so this is the only
|
||||
|
||||
+3
-3
@@ -20,7 +20,7 @@
|
||||
/**
|
||||
* Integration test for the Azure AI Foundry Responses provider.
|
||||
*
|
||||
* Hits the real Azure endpoint with `gpt-5-codex` — a Responses-API-only
|
||||
* Hits the real Azure endpoint with `gpt-5.3-codex` — a Responses-API-only
|
||||
* (Codex) model that the Chat Completions endpoint rejects. This verifies
|
||||
* the completions/responses split actually routes Codex correctly. Codex is
|
||||
* a reasoning model, so we give it a generous `max_tokens` and low reasoning
|
||||
@@ -47,7 +47,7 @@ describe.skipIf(skipUnlessEnv(KEY_ENV) || skipUnlessEnv(URL_ENV))(
|
||||
'AzureResponsesProvider (integration)',
|
||||
() => {
|
||||
it(
|
||||
'returns a non-empty completion from gpt-5-codex',
|
||||
'returns a non-empty completion from gpt-5.3-codex',
|
||||
{ timeout: INTEGRATION_TEST_TIMEOUT_MS },
|
||||
async () => {
|
||||
const provider = new AzureResponsesProvider(
|
||||
@@ -65,7 +65,7 @@ describe.skipIf(skipUnlessEnv(KEY_ENV) || skipUnlessEnv(URL_ENV))(
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [
|
||||
{ role: 'user', content: 'Say hi in one word.' },
|
||||
],
|
||||
|
||||
@@ -156,15 +156,15 @@ describe('AzureResponsesProvider construction', () => {
|
||||
// -- Model catalog ---------------------------------------------------
|
||||
|
||||
describe('AzureResponsesProvider model catalog', () => {
|
||||
it('returns gpt-5-codex as the default model', () => {
|
||||
expect(makeProvider().getDefaultModel()).toBe('gpt-5-codex');
|
||||
it('returns gpt-5.3-codex as the default model', () => {
|
||||
expect(makeProvider().getDefaultModel()).toBe('gpt-5.3-codex');
|
||||
});
|
||||
|
||||
it('models() exposes only the responses_api_only slice of the Azure catalog', () => {
|
||||
const ids = makeProvider()
|
||||
.models()
|
||||
.map((m) => m.id);
|
||||
expect(ids).toContain('gpt-5-codex');
|
||||
expect(ids).toContain('gpt-5.3-codex');
|
||||
// Chat-Completions models belong to the sibling provider.
|
||||
expect(ids).not.toContain('gpt-4o');
|
||||
});
|
||||
@@ -173,15 +173,15 @@ describe('AzureResponsesProvider model catalog', () => {
|
||||
const ids = makeProvider()
|
||||
.models({ no_restrictions: true })
|
||||
.map((m) => m.id);
|
||||
expect(ids).toContain('gpt-5-codex');
|
||||
expect(ids).toContain('gpt-5.3-codex');
|
||||
expect(ids).toContain('gpt-4o');
|
||||
expect(ids).toHaveLength(AZURE_MODELS.length);
|
||||
});
|
||||
|
||||
it('list() flattens responses-only ids and their aliases', () => {
|
||||
const ids = makeProvider().list();
|
||||
expect(ids).toContain('gpt-5-codex');
|
||||
expect(ids).toContain('openai/gpt-5-codex');
|
||||
expect(ids).toContain('gpt-5.3-codex');
|
||||
expect(ids).toContain('openai/gpt-5.3-codex');
|
||||
expect(ids).not.toContain('gpt-4o');
|
||||
});
|
||||
});
|
||||
@@ -194,7 +194,7 @@ describe('AzureResponsesProvider.complete argument validation', () => {
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: 'hello' as unknown as never,
|
||||
}),
|
||||
),
|
||||
@@ -212,7 +212,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hello' }],
|
||||
max_tokens: 256,
|
||||
temperature: 0.3,
|
||||
@@ -220,7 +220,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
);
|
||||
|
||||
const [args] = responsesCreateMock.mock.calls[0]!;
|
||||
expect(args.model).toBe('gpt-5-codex');
|
||||
expect(args.model).toBe('gpt-5.3-codex');
|
||||
expect(args.input).toEqual([{ role: 'user', content: 'hello' }]);
|
||||
expect(args.max_output_tokens).toBe(256);
|
||||
expect(args.temperature).toBe(0.3);
|
||||
@@ -233,12 +233,12 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'openai/gpt-5-codex',
|
||||
model: 'openai/gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(responsesCreateMock.mock.calls[0]![0].model).toBe('gpt-5-codex');
|
||||
expect(responsesCreateMock.mock.calls[0]![0].model).toBe('gpt-5.3-codex');
|
||||
});
|
||||
|
||||
it('falls back to the default model for an unknown id', async () => {
|
||||
@@ -252,7 +252,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
expect(responsesCreateMock.mock.calls[0]![0].model).toBe('gpt-5-codex');
|
||||
expect(responsesCreateMock.mock.calls[0]![0].model).toBe('gpt-5.3-codex');
|
||||
});
|
||||
|
||||
it('flattens chat-style function tools into the Responses shape', async () => {
|
||||
@@ -261,7 +261,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
tools: [
|
||||
{
|
||||
@@ -299,7 +299,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
@@ -338,7 +338,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
tool_choice: 'auto',
|
||||
parallel_tool_calls: false,
|
||||
@@ -380,7 +380,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
compaction: { trigger_tokens: 120_000 },
|
||||
} as never),
|
||||
@@ -398,7 +398,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
compaction: true,
|
||||
context_management: raw,
|
||||
@@ -416,7 +416,7 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
responsesCreateMock.mockResolvedValueOnce(okResponse);
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
reasoning: { effort: 'high' },
|
||||
verbosity: 'high',
|
||||
@@ -460,7 +460,7 @@ describe('AzureResponsesProvider usage accounting', () => {
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
@@ -475,11 +475,11 @@ describe('AzureResponsesProvider usage accounting', () => {
|
||||
cached_tokens: 10,
|
||||
});
|
||||
|
||||
const codex = AZURE_MODELS.find((m) => m.id === 'gpt-5-codex')!;
|
||||
const codex = AZURE_MODELS.find((m) => m.id === 'gpt-5.3-codex')!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(actor).toBe(SYSTEM_ACTOR);
|
||||
expect(prefix).toBe('azure-openai:gpt-5-codex');
|
||||
expect(prefix).toBe('azure-openai:gpt-5.3-codex');
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 90,
|
||||
completion_tokens: 50,
|
||||
@@ -502,7 +502,7 @@ describe('AzureResponsesProvider usage accounting', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
@@ -539,7 +539,7 @@ describe('AzureResponsesProvider.complete streaming', () => {
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'say hi' }],
|
||||
stream: true,
|
||||
}),
|
||||
@@ -563,7 +563,7 @@ describe('AzureResponsesProvider.complete streaming', () => {
|
||||
completion_tokens: 2,
|
||||
cached_tokens: 1,
|
||||
});
|
||||
expect(recordSpy.mock.calls[0]![2]).toBe('azure-openai:gpt-5-codex');
|
||||
expect(recordSpy.mock.calls[0]![2]).toBe('azure-openai:gpt-5.3-codex');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -617,7 +617,7 @@ describe('AzureResponsesProvider.complete error mapping', () => {
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5-codex',
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'boom' }],
|
||||
}),
|
||||
),
|
||||
|
||||
@@ -49,7 +49,7 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
/** @type {import('openai').OpenAI} */
|
||||
#openAi: OpenAI;
|
||||
|
||||
#defaultModel = 'gpt-5-codex';
|
||||
#defaultModel = 'gpt-5.3-codex';
|
||||
|
||||
#meteringService: MeteringService;
|
||||
|
||||
|
||||
@@ -170,30 +170,6 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5-codex (same list price as gpt-5).
|
||||
puterId: 'azure:openai/gpt-5-codex',
|
||||
id: 'gpt-5-codex',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2024-09-30',
|
||||
release_date: '2025-08-07',
|
||||
aliases: ['openai/gpt-5-codex'],
|
||||
// Codex models are Responses-API only on Azure, same as OpenAI.
|
||||
responses_api_only: true,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
cached_tokens: 13,
|
||||
completion_tokens: 1000,
|
||||
},
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5-nano.
|
||||
puterId: 'azure:openai/gpt-5-nano',
|
||||
@@ -260,29 +236,6 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
context: 128_000,
|
||||
max_tokens: 16384,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5.1-codex-mini.
|
||||
puterId: 'azure:openai/gpt-5.1-codex-mini',
|
||||
id: 'gpt-5.1-codex-mini',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2024-09-30',
|
||||
release_date: '2025-11-13',
|
||||
aliases: ['openai/gpt-5.1-codex-mini'],
|
||||
responses_api_only: true,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
cached_tokens: 3,
|
||||
completion_tokens: 200,
|
||||
},
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5.1.
|
||||
puterId: 'azure:openai/gpt-5.1',
|
||||
@@ -305,29 +258,6 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5.1-codex.
|
||||
puterId: 'azure:openai/gpt-5.1-codex',
|
||||
id: 'gpt-5.1-codex',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2024-09-30',
|
||||
release_date: '2025-11-13',
|
||||
aliases: ['openai/gpt-5.1-codex'],
|
||||
responses_api_only: true,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
cached_tokens: 13,
|
||||
completion_tokens: 1000,
|
||||
},
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5.2.
|
||||
puterId: 'azure:openai/gpt-5.2',
|
||||
@@ -350,29 +280,6 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5.2-codex.
|
||||
puterId: 'azure:openai/gpt-5.2-codex',
|
||||
id: 'gpt-5.2-codex',
|
||||
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2025-08-31',
|
||||
release_date: '2025-12-11',
|
||||
aliases: ['openai/gpt-5.2-codex'],
|
||||
responses_api_only: true,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 175,
|
||||
cached_tokens: 18,
|
||||
completion_tokens: 1400,
|
||||
},
|
||||
context: 128_000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
// Costs mirror openai gpt-5.4-nano.
|
||||
puterId: 'azure:openai/gpt-5.4-nano',
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Integration test for the BytePlus ModelArk provider.
|
||||
*
|
||||
* Uses `seed-1-6-flash-250715` with `thinking: disabled` passed through
|
||||
* `custom`. Ark's seed models default to deep reasoning and route those
|
||||
* tokens to a `reasoning_content` field, leaving `content` empty under
|
||||
* tight budgets. Disabling thinking forces a plain text response so the
|
||||
* usual `message.content` assertion works. Skipped when
|
||||
* `PUTER_TEST_AI_BYTEPLUS_API_KEY` is unset.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
makeMeteringStub,
|
||||
optionalEnv,
|
||||
skipUnlessEnv,
|
||||
withTestActor,
|
||||
} from '../../../integrationTestUtil.js';
|
||||
import { BytePlusProvider } from './BytePlusProvider.js';
|
||||
|
||||
const ENV_VAR = 'PUTER_TEST_AI_BYTEPLUS_API_KEY';
|
||||
|
||||
describe.skipIf(skipUnlessEnv(ENV_VAR))('BytePlusProvider (integration)', () => {
|
||||
it('returns a non-empty completion from seed-1-6-flash-250715', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
|
||||
const provider = new BytePlusProvider(
|
||||
{ apiKey: optionalEnv(ENV_VAR)! },
|
||||
makeMeteringStub(),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-flash-250715',
|
||||
messages: [{ role: 'user', content: 'Say hi in one word.' }],
|
||||
max_tokens: 16,
|
||||
custom: { thinking: { type: 'disabled' } },
|
||||
}),
|
||||
);
|
||||
|
||||
const text = (result as { message?: { content?: string } }).message
|
||||
?.content;
|
||||
expect(typeof text === 'string' && text.length > 0).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,815 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Offline unit tests for BytePlusProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs BytePlusProvider directly against the live
|
||||
* wired `MeteringService` so the recording side is exercised
|
||||
* end-to-end. The OpenAI SDK is mocked at the module boundary —
|
||||
* ModelArk is OpenAI-compatible so the provider talks to it through
|
||||
* the same client — so the provider never reaches the network. The
|
||||
* companion integration test (BytePlusProvider.integration.test.ts)
|
||||
* exercises the real ModelArk endpoint.
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
type MockInstance,
|
||||
} from 'vitest';
|
||||
|
||||
import { SYSTEM_ACTOR } from '../../../../core/actor.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import { PuterServer } from '../../../../server.js';
|
||||
import { setupTestServer } from '../../../../testUtil.js';
|
||||
import { withTestActor } from '../../../integrationTestUtil.js';
|
||||
import { AIChatStream } from '../../utils/Streaming.js';
|
||||
import { BytePlusProvider } from './BytePlusProvider.js';
|
||||
import { BYTEPLUS_MODELS } from './models.js';
|
||||
|
||||
// -- OpenAI SDK mock -------------------------------------------------
|
||||
//
|
||||
// `vi.hoisted` lets us share spies between the (hoisted) factory and
|
||||
// the test body so each test can stub `chat.completions.create` with
|
||||
// the response shape it cares about. ModelArk uses the OpenAI wire
|
||||
// shape so the provider talks to it via the OpenAI SDK.
|
||||
|
||||
const { createMock, openAICtor } = vi.hoisted(() => {
|
||||
const createMock = vi.fn();
|
||||
const openAICtor = vi.fn();
|
||||
return { createMock, openAICtor };
|
||||
});
|
||||
|
||||
vi.mock('openai', () => {
|
||||
const OpenAICtor = vi.fn().mockImplementation(function (
|
||||
this: Record<string, unknown>,
|
||||
opts: unknown,
|
||||
) {
|
||||
openAICtor(opts);
|
||||
this.chat = { completions: { create: createMock } };
|
||||
});
|
||||
// Some providers (e.g. OllamaChatProvider) import the default export
|
||||
// and access `.OpenAI` on it, so expose the same constructor under
|
||||
// both shapes — the test server boots every provider, not just
|
||||
// BytePlus.
|
||||
return { OpenAI: OpenAICtor, default: { OpenAI: OpenAICtor } };
|
||||
});
|
||||
|
||||
// -- Test harness ----------------------------------------------------
|
||||
|
||||
let server: PuterServer;
|
||||
let recordSpy: MockInstance<MeteringService['utilRecordUsageObject']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
const makeProvider = (
|
||||
config: { apiKey?: string; apiBaseUrl?: string } = {},
|
||||
) => {
|
||||
const provider = new BytePlusProvider(
|
||||
{
|
||||
apiKey: config.apiKey ?? 'test-key',
|
||||
...(config.apiBaseUrl ? { apiBaseUrl: config.apiBaseUrl } : {}),
|
||||
},
|
||||
server.services.metering,
|
||||
);
|
||||
return { provider };
|
||||
};
|
||||
|
||||
const asAsyncIterable = <T>(items: T[]): AsyncIterable<T> => ({
|
||||
async *[Symbol.asyncIterator]() {
|
||||
for (const item of items) {
|
||||
yield item;
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
const makeCapturingChatStream = () => {
|
||||
const chunks: string[] = [];
|
||||
const sink = new Writable({
|
||||
write(chunk, _enc, cb) {
|
||||
chunks.push(chunk.toString('utf8'));
|
||||
cb();
|
||||
},
|
||||
});
|
||||
const chatStream = new AIChatStream({ stream: sink });
|
||||
return {
|
||||
chatStream,
|
||||
events: () =>
|
||||
chunks
|
||||
.join('')
|
||||
.split('\n')
|
||||
.filter(Boolean)
|
||||
.map((line) => JSON.parse(line)),
|
||||
};
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
createMock.mockReset();
|
||||
openAICtor.mockReset();
|
||||
// Spy on the live MeteringService — we don't replace the impl
|
||||
// (that would skip the recording side we want covered) but we
|
||||
// capture the calls the provider makes so per-test assertions
|
||||
// can verify metering shape.
|
||||
recordSpy = vi.spyOn(server.services.metering, 'utilRecordUsageObject');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
// -- Construction ----------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider construction', () => {
|
||||
it('points the OpenAI SDK at the ModelArk base URL with the configured key', () => {
|
||||
makeProvider();
|
||||
expect(openAICtor).toHaveBeenCalledTimes(1);
|
||||
expect(openAICtor).toHaveBeenCalledWith({
|
||||
apiKey: 'test-key',
|
||||
baseURL: 'https://ark.ap-southeast.bytepluses.com/api/v3',
|
||||
});
|
||||
});
|
||||
|
||||
it('honours a custom apiBaseUrl override (e.g. the eu-west region)', () => {
|
||||
makeProvider({
|
||||
apiBaseUrl: 'https://ark.eu-west.bytepluses.com/api/v3',
|
||||
});
|
||||
expect(openAICtor).toHaveBeenCalledWith({
|
||||
apiKey: 'test-key',
|
||||
baseURL: 'https://ark.eu-west.bytepluses.com/api/v3',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// -- Model catalog ---------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider model catalog', () => {
|
||||
it('returns seed-2-0-lite-260428 as the default', () => {
|
||||
const { provider } = makeProvider();
|
||||
expect(provider.getDefaultModel()).toBe('seed-2-0-lite-260428');
|
||||
});
|
||||
|
||||
it('exposes the static BYTEPLUS_MODELS list verbatim from models()', () => {
|
||||
const { provider } = makeProvider();
|
||||
expect(provider.models()).toBe(BYTEPLUS_MODELS);
|
||||
});
|
||||
|
||||
it('list() flattens canonical ids and aliases', () => {
|
||||
const { provider } = makeProvider();
|
||||
const names = provider.list();
|
||||
for (const m of BYTEPLUS_MODELS) {
|
||||
expect(names).toContain(m.id);
|
||||
for (const a of m.aliases ?? []) {
|
||||
expect(names).toContain(a);
|
||||
}
|
||||
}
|
||||
// Sanity: undated series aliases resolve alongside canonical ids.
|
||||
expect(names).toContain('seed-1-6');
|
||||
expect(names).toContain('byteplus/seed-1-6');
|
||||
expect(names).toContain('byteplus/seed-1-6-250915');
|
||||
});
|
||||
|
||||
it('never claims the bare deepseek-v4 names owned by the DeepSeek provider', () => {
|
||||
const { provider } = makeProvider();
|
||||
const names = provider.list();
|
||||
expect(names).not.toContain('deepseek-v4-pro');
|
||||
expect(names).not.toContain('deepseek-v4-flash');
|
||||
expect(names).toContain('byteplus/deepseek-v4-pro');
|
||||
expect(names).toContain('byteplus/deepseek-v4-flash');
|
||||
});
|
||||
});
|
||||
|
||||
// -- Request shape ---------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider.complete request shape', () => {
|
||||
const baseCompletion = {
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'hi', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
};
|
||||
|
||||
it('forwards model, messages, and bare-bones request without optional knobs', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hello' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.model).toBe('seed-1-6-250915');
|
||||
expect(args.messages).toEqual([{ role: 'user', content: 'hello' }]);
|
||||
// Optional generation knobs should be absent unless supplied.
|
||||
expect('max_tokens' in args).toBe(false);
|
||||
expect('temperature' in args).toBe(false);
|
||||
expect('top_p' in args).toBe(false);
|
||||
expect('tools' in args).toBe(false);
|
||||
expect('tool_choice' in args).toBe(false);
|
||||
expect('thinking' in args).toBe(false);
|
||||
});
|
||||
|
||||
it('forwards max_tokens, temperature, top_p, tools, and tool_choice when supplied', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
const tools = [
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
description: 'find a thing',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: { q: { type: 'string' } },
|
||||
required: ['q'],
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
max_tokens: 256,
|
||||
temperature: 0.4,
|
||||
top_p: 0.9,
|
||||
tools,
|
||||
tool_choice: 'auto',
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.max_tokens).toBe(256);
|
||||
expect(args.temperature).toBe(0.4);
|
||||
expect(args.top_p).toBe(0.9);
|
||||
expect(args.tools).toBe(tools);
|
||||
expect(args.tool_choice).toBe('auto');
|
||||
});
|
||||
|
||||
it('forwards Ark-specific custom params (thinking, stop, response_format)', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
custom: {
|
||||
thinking: { type: 'disabled' },
|
||||
stop: ['\n\n'],
|
||||
response_format: { type: 'json_object' },
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.thinking).toEqual({ type: 'disabled' });
|
||||
expect(args.stop).toEqual(['\n\n']);
|
||||
expect(args.response_format).toEqual({ type: 'json_object' });
|
||||
});
|
||||
|
||||
it('strips Anthropic-style cache_control from messages before sending', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: 'hi',
|
||||
cache_control: { type: 'ephemeral' },
|
||||
} as unknown as { role: string; content: string },
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect('cache_control' in args.messages[0]).toBe(false);
|
||||
});
|
||||
|
||||
it('only sets stream_options.include_usage when streaming', async () => {
|
||||
const { provider } = makeProvider();
|
||||
// Non-stream path.
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
stream: false,
|
||||
}),
|
||||
);
|
||||
const [nonStreamArgs] = createMock.mock.calls[0]!;
|
||||
expect(nonStreamArgs.stream).toBe(false);
|
||||
expect('stream_options' in nonStreamArgs).toBe(false);
|
||||
|
||||
// Stream path.
|
||||
createMock.mockReturnValueOnce(asAsyncIterable([]));
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
stream: true,
|
||||
}),
|
||||
);
|
||||
const [streamArgs] = createMock.mock.calls[1]!;
|
||||
expect(streamArgs.stream).toBe(true);
|
||||
expect(streamArgs.stream_options).toEqual({ include_usage: true });
|
||||
});
|
||||
});
|
||||
|
||||
// -- Model resolution ------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider model resolution', () => {
|
||||
const baseCompletion = {
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
};
|
||||
|
||||
it('resolves an exact canonical id', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'glm-4-7-251222',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(createMock.mock.calls[0]![0].model).toBe('glm-4-7-251222');
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.any(Object),
|
||||
expect.anything(),
|
||||
'byteplus:glm-4-7-251222',
|
||||
expect.any(Object),
|
||||
);
|
||||
});
|
||||
|
||||
it('resolves an undated series alias to the newest snapshot (alias rewriting)', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
// The wire model should be the canonical dated id, not the alias.
|
||||
expect(createMock.mock.calls[0]![0].model).toBe('seed-1-6-250915');
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.any(Object),
|
||||
expect.anything(),
|
||||
'byteplus:seed-1-6-250915',
|
||||
expect.any(Object),
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to the default model when given an unknown id', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'totally-not-a-real-model',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(createMock.mock.calls[0]![0].model).toBe(
|
||||
'seed-2-0-lite-260428',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// -- Non-stream completion + reasoning_content normalisation ---------
|
||||
|
||||
describe('BytePlusProvider.complete non-stream output', () => {
|
||||
it('returns the first choice and runs the metered usage calculator', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'hi there', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
prompt_tokens_details: { cached_tokens: 10 },
|
||||
},
|
||||
});
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(result).toMatchObject({
|
||||
message: { content: 'hi there', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
});
|
||||
expect((result as { usage: unknown }).usage).toEqual({
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
cached_tokens: 10,
|
||||
});
|
||||
|
||||
// Cost overrides scale per-token usage by the per-token cents from
|
||||
// the model's costs table, so derive expectations from
|
||||
// BYTEPLUS_MODELS directly to avoid hardcoded float-precision drift.
|
||||
const seed16 = BYTEPLUS_MODELS.find(
|
||||
(m) => m.id === 'seed-1-6-250915',
|
||||
)!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
cached_tokens: 10,
|
||||
});
|
||||
expect(actor).toBe(SYSTEM_ACTOR);
|
||||
expect(prefix).toBe('byteplus:seed-1-6-250915');
|
||||
expect(overrides.prompt_tokens).toBeCloseTo(
|
||||
100 * Number(seed16.costs.prompt_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides.completion_tokens).toBeCloseTo(
|
||||
50 * Number(seed16.costs.completion_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides.cached_tokens).toBeCloseTo(
|
||||
10 * Number(seed16.costs.cached_tokens ?? 0),
|
||||
5,
|
||||
);
|
||||
});
|
||||
|
||||
it('preserves OpenAI-shaped tool_calls on the assistant response', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: null,
|
||||
tool_calls: [
|
||||
{
|
||||
id: 'call_1',
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
arguments: '{"q":"puter"}',
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
finish_reason: 'tool_calls',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
});
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'do a tool call' }],
|
||||
tools: [
|
||||
{
|
||||
type: 'function',
|
||||
function: { name: 'lookup', parameters: {} },
|
||||
},
|
||||
],
|
||||
}),
|
||||
)) as { message: { tool_calls?: unknown[] }; finish_reason: string };
|
||||
|
||||
expect(result.finish_reason).toBe('tool_calls');
|
||||
expect(result.message.tool_calls).toEqual([
|
||||
{
|
||||
id: 'call_1',
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
arguments: '{"q":"puter"}',
|
||||
},
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('renames Ark `reasoning_content` to `reasoning` on the message', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: 'final answer',
|
||||
reasoning_content: 'thinking out loud',
|
||||
},
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
});
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
)) as { message: Record<string, unknown> };
|
||||
|
||||
expect(result.message.reasoning).toBe('thinking out loud');
|
||||
expect('reasoning_content' in result.message).toBe(false);
|
||||
});
|
||||
|
||||
it('does not overwrite an existing `reasoning` field if both are present', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: 'final',
|
||||
reasoning: 'original',
|
||||
reasoning_content: 'should-be-dropped',
|
||||
},
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
});
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
)) as { message: Record<string, unknown> };
|
||||
|
||||
expect(result.message.reasoning).toBe('original');
|
||||
expect('reasoning_content' in result.message).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// -- Streaming deltas ------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider.complete streaming', () => {
|
||||
it('streams text deltas through to text events and meters final usage', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{ choices: [{ delta: { content: 'hel' } }] },
|
||||
{ choices: [{ delta: { content: 'lo' } }] },
|
||||
{
|
||||
choices: [{ delta: {} }],
|
||||
usage: {
|
||||
prompt_tokens: 4,
|
||||
completion_tokens: 2,
|
||||
prompt_tokens_details: { cached_tokens: 1 },
|
||||
},
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'say hi' }],
|
||||
stream: true,
|
||||
}),
|
||||
);
|
||||
expect((result as { stream: boolean }).stream).toBe(true);
|
||||
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const events = harness.events();
|
||||
const textEvents = events.filter((e) => e.type === 'text');
|
||||
expect(textEvents.map((e) => e.text)).toEqual(['hel', 'lo']);
|
||||
|
||||
const usageEvent = events.find((e) => e.type === 'usage');
|
||||
expect(usageEvent?.usage).toEqual({
|
||||
prompt_tokens: 4,
|
||||
completion_tokens: 2,
|
||||
cached_tokens: 1,
|
||||
});
|
||||
|
||||
const seed16 = BYTEPLUS_MODELS.find(
|
||||
(m) => m.id === 'seed-1-6-250915',
|
||||
)!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('byteplus:seed-1-6-250915');
|
||||
expect(overrides.prompt_tokens).toBeCloseTo(
|
||||
4 * Number(seed16.costs.prompt_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides.completion_tokens).toBeCloseTo(
|
||||
2 * Number(seed16.costs.completion_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides.cached_tokens).toBeCloseTo(
|
||||
1 * Number(seed16.costs.cached_tokens ?? 0),
|
||||
5,
|
||||
);
|
||||
});
|
||||
|
||||
it('streams reasoning_content deltas as reasoning events', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{
|
||||
choices: [
|
||||
{ delta: { reasoning_content: 'pondering...' } },
|
||||
],
|
||||
},
|
||||
{ choices: [{ delta: { content: 'answer' } }] },
|
||||
{
|
||||
choices: [{ delta: {} }],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'think hard' }],
|
||||
stream: true,
|
||||
}),
|
||||
);
|
||||
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const events = harness.events();
|
||||
const reasoningEvents = events.filter((e) => e.type === 'reasoning');
|
||||
expect(reasoningEvents.map((e) => e.reasoning)).toEqual([
|
||||
'pondering...',
|
||||
]);
|
||||
const textEvents = events.filter((e) => e.type === 'text');
|
||||
expect(textEvents.map((e) => e.text)).toEqual(['answer']);
|
||||
});
|
||||
|
||||
it('builds a tool_use block from streamed function-call deltas', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{
|
||||
choices: [
|
||||
{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{
|
||||
index: 0,
|
||||
id: 'call_1',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
arguments: '{"q":',
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
choices: [
|
||||
{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{
|
||||
index: 0,
|
||||
function: { arguments: '"puter"}' },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
choices: [{ delta: {} }],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'do tool call' }],
|
||||
tools: [
|
||||
{
|
||||
type: 'function',
|
||||
function: { name: 'lookup', parameters: {} },
|
||||
},
|
||||
],
|
||||
stream: true,
|
||||
}),
|
||||
);
|
||||
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const events = harness.events();
|
||||
const toolEvent = events.find((e) => e.type === 'tool_use');
|
||||
expect(toolEvent).toBeDefined();
|
||||
expect(toolEvent?.id).toBe('call_1');
|
||||
expect(toolEvent?.name).toBe('lookup');
|
||||
expect(toolEvent?.input).toEqual({ q: 'puter' });
|
||||
});
|
||||
});
|
||||
|
||||
// -- Error mapping ---------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider.complete error mapping', () => {
|
||||
it('rethrows errors raised by the OpenAI client unchanged', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const apiError = new Error('ModelArk exploded');
|
||||
createMock.mockRejectedValueOnce(apiError);
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'seed-1-6-250915',
|
||||
messages: [{ role: 'user', content: 'boom' }],
|
||||
}),
|
||||
),
|
||||
).rejects.toBe(apiError);
|
||||
|
||||
// No metering should be recorded on a failed call.
|
||||
expect(recordSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// -- Moderation ------------------------------------------------------
|
||||
|
||||
describe('BytePlusProvider.checkModeration', () => {
|
||||
it('throws — BytePlus provider does not implement moderation', () => {
|
||||
const { provider } = makeProvider();
|
||||
expect(() => provider.checkModeration('anything')).toThrow(
|
||||
/not implemented/i,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,180 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type { IChatProvider, ICompleteArguments } from '../../types.js';
|
||||
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
|
||||
import { BYTEPLUS_MODELS } from './models.js';
|
||||
|
||||
type BytePlusConfig = {
|
||||
apiKey: string;
|
||||
apiBaseUrl?: string;
|
||||
};
|
||||
|
||||
type BytePlusCustomParams = {
|
||||
response_format?: unknown;
|
||||
stop?: string[];
|
||||
// Ark-specific toggle for deep reasoning; the seed models reason by
|
||||
// default and route those tokens to `reasoning_content`.
|
||||
thinking?: {
|
||||
type?: 'enabled' | 'disabled' | 'auto';
|
||||
};
|
||||
};
|
||||
|
||||
const asRecord = (value: unknown): Record<string, unknown> =>
|
||||
value && typeof value === 'object' && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: {};
|
||||
|
||||
/**
|
||||
* BytePlus ModelArk provider — an OpenAI-compatible endpoint serving
|
||||
* ByteDance's Seed models plus hosted third-party models (GLM, DeepSeek,
|
||||
* GPT-OSS). https://docs.byteplus.com/en/docs/ModelArk/1330626
|
||||
*/
|
||||
export class BytePlusProvider implements IChatProvider {
|
||||
#openai: OpenAI;
|
||||
|
||||
#meteringService: MeteringService;
|
||||
|
||||
#defaultModel = 'seed-2-0-lite-260428';
|
||||
|
||||
constructor(config: BytePlusConfig, meteringService: MeteringService) {
|
||||
this.#openai = new OpenAI({
|
||||
apiKey: config.apiKey,
|
||||
baseURL:
|
||||
config.apiBaseUrl ??
|
||||
'https://ark.ap-southeast.bytepluses.com/api/v3',
|
||||
});
|
||||
this.#meteringService = meteringService;
|
||||
}
|
||||
|
||||
getDefaultModel() {
|
||||
return this.#defaultModel;
|
||||
}
|
||||
|
||||
models() {
|
||||
return BYTEPLUS_MODELS;
|
||||
}
|
||||
|
||||
list() {
|
||||
const modelIds: string[] = [];
|
||||
for (const model of this.models()) {
|
||||
modelIds.push(model.id);
|
||||
if (model.aliases) {
|
||||
modelIds.push(...model.aliases);
|
||||
}
|
||||
}
|
||||
return modelIds;
|
||||
}
|
||||
|
||||
async complete(
|
||||
params: ICompleteArguments,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
const {
|
||||
custom,
|
||||
max_tokens,
|
||||
stream,
|
||||
temperature,
|
||||
tools,
|
||||
tool_choice,
|
||||
top_p,
|
||||
} = params;
|
||||
let { messages, model } = params;
|
||||
const actor = Context.get('actor');
|
||||
const availableModels = this.models();
|
||||
const modelUsed =
|
||||
availableModels.find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) || availableModels.find((m) => m.id === this.getDefaultModel())!;
|
||||
|
||||
messages = await OpenAIUtil.process_input_messages(messages);
|
||||
messages = messages.map((message) => {
|
||||
delete message.cache_control;
|
||||
return message;
|
||||
});
|
||||
|
||||
const customParams = asRecord(custom) as BytePlusCustomParams;
|
||||
|
||||
const completionParams: ChatCompletionCreateParams = {
|
||||
messages,
|
||||
model: modelUsed.id,
|
||||
...(tools ? { tools } : {}),
|
||||
...(tool_choice !== undefined ? { tool_choice } : {}),
|
||||
...(max_tokens !== undefined ? { max_tokens } : {}),
|
||||
...(temperature !== undefined ? { temperature } : {}),
|
||||
...(top_p !== undefined ? { top_p } : {}),
|
||||
...(customParams.response_format
|
||||
? { response_format: customParams.response_format }
|
||||
: {}),
|
||||
...(customParams.stop ? { stop: customParams.stop } : {}),
|
||||
...(customParams.thinking
|
||||
? { thinking: customParams.thinking }
|
||||
: {}),
|
||||
stream: !!stream,
|
||||
...(stream
|
||||
? {
|
||||
stream_options: { include_usage: true },
|
||||
}
|
||||
: {}),
|
||||
} as ChatCompletionCreateParams;
|
||||
|
||||
const completion =
|
||||
await this.#openai.chat.completions.create(completionParams);
|
||||
|
||||
const result = await OpenAIUtil.handle_completion_output({
|
||||
usage_calculator: ({ usage }) => {
|
||||
const trackedUsage = usage
|
||||
? OpenAIUtil.extractMeteredUsage(usage)
|
||||
: {
|
||||
prompt_tokens: 0,
|
||||
completion_tokens: 0,
|
||||
cached_tokens: 0,
|
||||
};
|
||||
const costsOverride = Object.fromEntries(
|
||||
Object.entries(trackedUsage).map(([key, value]) => {
|
||||
return [key, value * Number(modelUsed.costs[key] ?? 0)];
|
||||
}),
|
||||
);
|
||||
this.#meteringService.utilRecordUsageObject(
|
||||
trackedUsage,
|
||||
actor!,
|
||||
`byteplus:${modelUsed.id}`,
|
||||
costsOverride,
|
||||
);
|
||||
return trackedUsage;
|
||||
},
|
||||
stream,
|
||||
completion,
|
||||
});
|
||||
|
||||
// Ark's deep-reasoning models return `reasoning_content` (DeepSeek
|
||||
// wire convention); expose it under `reasoning` like other providers.
|
||||
OpenAIUtil.normalizeReasoningContent(result);
|
||||
return result;
|
||||
}
|
||||
|
||||
checkModeration(
|
||||
_text: string,
|
||||
): ReturnType<IChatProvider['checkModeration']> {
|
||||
throw new Error('Method not implemented.');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { IChatModel } from '../../types.js';
|
||||
import { usdPerMToken } from '../../utils/pricing.js';
|
||||
|
||||
const K = 1_000;
|
||||
|
||||
const bytePlusModel = (
|
||||
id: string,
|
||||
name: string,
|
||||
context: number,
|
||||
maxTokens: number,
|
||||
costs: IChatModel['costs'],
|
||||
opts: {
|
||||
input?: string[];
|
||||
extraAliases?: string[];
|
||||
openWeights?: boolean;
|
||||
} = {},
|
||||
): IChatModel => ({
|
||||
puterId: `byteplus:byteplus/${id}`,
|
||||
id,
|
||||
name,
|
||||
aliases: [`byteplus/${id}`, ...(opts.extraAliases ?? [])],
|
||||
modalities: { input: opts.input ?? ['text'], output: ['text'] },
|
||||
open_weights: opts.openWeights ?? false,
|
||||
tool_call: true,
|
||||
context,
|
||||
max_tokens: maxTokens,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs,
|
||||
});
|
||||
|
||||
// An undated series alias (e.g. `seed-1-6`) points at the newest snapshot of
|
||||
// that series, mirroring how other providers alias their rolling names.
|
||||
const series = (alias: string) => [alias, `byteplus/${alias}`];
|
||||
|
||||
const VISION = ['text', 'image', 'video'];
|
||||
|
||||
// Hardcoded from https://docs.byteplus.com/en/docs/ModelArk/1330310 (catalog)
|
||||
// and https://docs.byteplus.com/en/docs/ModelArk/1544106 (pricing).
|
||||
//
|
||||
// Some models bill a higher rate for prompts above 128K tokens; the costs
|
||||
// below are the base tier. seed-2-0-lite/mini-260428 also accept audio input
|
||||
// at a much higher per-token rate ModelArk folds into prompt_tokens, so audio
|
||||
// is deliberately left out of `modalities` to keep metering honest.
|
||||
export const BYTEPLUS_MODELS: IChatModel[] = [
|
||||
bytePlusModel(
|
||||
'dola-seed-2-1-turbo-260628',
|
||||
'Dola Seed 2.1 Turbo',
|
||||
256 * K,
|
||||
256 * K,
|
||||
usdPerMToken(0.5, 2.5, 0.1),
|
||||
{ input: VISION, extraAliases: series('dola-seed-2-1-turbo') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-2-0-lite-260428',
|
||||
'Seed 2.0 Lite',
|
||||
256 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.25, 2, 0.05),
|
||||
{ input: VISION, extraAliases: series('seed-2-0-lite') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-2-0-lite-260228',
|
||||
'Seed 2.0 Lite',
|
||||
256 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.25, 2, 0.05),
|
||||
{ input: VISION },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-2-0-mini-260428',
|
||||
'Seed 2.0 Mini',
|
||||
256 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.1, 0.4, 0.02),
|
||||
{ input: VISION, extraAliases: series('seed-2-0-mini') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-2-0-mini-260215',
|
||||
'Seed 2.0 Mini',
|
||||
256 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.1, 0.4, 0.02),
|
||||
{ input: VISION },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-2-0-pro-260328',
|
||||
'Seed 2.0 Pro',
|
||||
256 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.5, 3, 0.1),
|
||||
{ input: VISION, extraAliases: series('seed-2-0-pro') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-2-0-code-preview-260328',
|
||||
'Seed 2.0 Code Preview',
|
||||
256 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.5, 3, 0.1),
|
||||
{ input: VISION, extraAliases: series('seed-2-0-code-preview') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-1-8-251228',
|
||||
'Seed 1.8',
|
||||
256 * K,
|
||||
64 * K,
|
||||
usdPerMToken(0.25, 2, 0.05),
|
||||
{ input: VISION, extraAliases: series('seed-1-8') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-1-6-250915',
|
||||
'Seed 1.6',
|
||||
256 * K,
|
||||
32 * K,
|
||||
usdPerMToken(0.25, 2, 0.05),
|
||||
{ input: VISION, extraAliases: series('seed-1-6') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'seed-1-6-flash-250715',
|
||||
'Seed 1.6 Flash',
|
||||
256 * K,
|
||||
32 * K,
|
||||
usdPerMToken(0.075, 0.3, 0.015),
|
||||
{ input: VISION, extraAliases: series('seed-1-6-flash') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'glm-5-2-260617',
|
||||
'GLM-5.2',
|
||||
1_024 * K,
|
||||
128 * K,
|
||||
usdPerMToken(1.4, 4.4, 0.26),
|
||||
{ extraAliases: series('glm-5-2') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'glm-4-7-251222',
|
||||
'GLM-4.7',
|
||||
200 * K,
|
||||
128 * K,
|
||||
usdPerMToken(0.6, 2.2, 0.11),
|
||||
{ extraAliases: series('glm-4-7') },
|
||||
),
|
||||
// The bare `deepseek-v4-*` names belong to the first-party DeepSeek
|
||||
// provider, so these only carry the byteplus-prefixed aliases.
|
||||
bytePlusModel(
|
||||
'deepseek-v4-pro-260425',
|
||||
'DeepSeek V4 Pro',
|
||||
1_024 * K,
|
||||
384 * K,
|
||||
usdPerMToken(1.74, 3.48, 0.145),
|
||||
{ extraAliases: ['byteplus/deepseek-v4-pro'] },
|
||||
),
|
||||
bytePlusModel(
|
||||
'deepseek-v4-flash-260425',
|
||||
'DeepSeek V4 Flash',
|
||||
1_024 * K,
|
||||
384 * K,
|
||||
usdPerMToken(0.14, 0.28, 0.028),
|
||||
{ extraAliases: ['byteplus/deepseek-v4-flash'] },
|
||||
),
|
||||
bytePlusModel(
|
||||
'deepseek-v3-2-251201',
|
||||
'DeepSeek V3.2',
|
||||
128 * K,
|
||||
32 * K,
|
||||
usdPerMToken(0.28, 0.42, 0.056),
|
||||
{ extraAliases: series('deepseek-v3-2') },
|
||||
),
|
||||
bytePlusModel(
|
||||
'gpt-oss-120b-250805',
|
||||
'GPT-OSS 120B',
|
||||
128 * K,
|
||||
64 * K,
|
||||
usdPerMToken(0.1, 0.5),
|
||||
{ extraAliases: series('gpt-oss-120b'), openWeights: true },
|
||||
),
|
||||
];
|
||||
@@ -536,7 +536,10 @@ export class ClaudeProvider implements IChatProvider {
|
||||
}
|
||||
}
|
||||
}
|
||||
chatStream.end(usageSum);
|
||||
// Metered before `end`: handing usage to `end` is what tells
|
||||
// the driver this completion has been charged for, so anything
|
||||
// that throws between the two would leave it charged to
|
||||
// nobody.
|
||||
const costsOverrideFromModel =
|
||||
this.#buildCostsOverrideFromModel(usageSum, modelUsed);
|
||||
this.#meteringService.utilRecordUsageObject(
|
||||
@@ -545,6 +548,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
`claude:${modelUsed.id}`,
|
||||
costsOverrideFromModel,
|
||||
);
|
||||
chatStream.end(usageSum);
|
||||
};
|
||||
|
||||
return {
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
/**
|
||||
* Integration test for the Gemini chat provider.
|
||||
*
|
||||
* Hits the real Google Gemini API with `gemini-2.0-flash-lite` (one
|
||||
* Hits the real Google Gemini API with `gemini-3.7-flash` (one
|
||||
* of the cheapest variants). Skipped when `PUTER_TEST_AI_GEMINI_API_KEY`
|
||||
* is unset.
|
||||
*/
|
||||
@@ -40,14 +40,14 @@ const ENV_VAR = 'PUTER_TEST_AI_GEMINI_API_KEY';
|
||||
describe.skipIf(skipUnlessEnv(ENV_VAR))(
|
||||
'GeminiChatProvider (integration)',
|
||||
() => {
|
||||
it('returns a non-empty completion from gemini-2.0-flash-lite', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
|
||||
it('returns a non-empty completion from gemini-3.7-flash', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
|
||||
const provider = new GeminiChatProvider(makeMeteringStub(), {
|
||||
apiKey: optionalEnv(ENV_VAR)!,
|
||||
});
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.0-flash-lite',
|
||||
model: 'gemini-3.7-flash',
|
||||
messages: [
|
||||
{ role: 'user', content: 'Say hi in one word.' },
|
||||
],
|
||||
|
||||
@@ -400,46 +400,51 @@ describe('GeminiChatProvider.complete non-stream output', () => {
|
||||
});
|
||||
|
||||
it('bills cached tokens at the input rate when the model prices no cache read', async () => {
|
||||
// gemini-2.0-flash-lite's catalogue entry has no cached_tokens rate.
|
||||
// Cached tokens are subtracted out of prompt_tokens, so pricing them
|
||||
// at zero bills them nowhere.
|
||||
// Every shipped entry currently prices cache reads, so drop the rate
|
||||
// off one for this call. Cached tokens are subtracted out of
|
||||
// prompt_tokens, so pricing them at zero bills them nowhere.
|
||||
const lite = GEMINI_MODELS.find(
|
||||
(m) => m.id === 'gemini-2.0-flash-lite',
|
||||
(m) => m.id === 'gemini-3.1-flash-lite',
|
||||
)!;
|
||||
expect(lite.costs.cached_tokens).toBeUndefined();
|
||||
const cachedRate = lite.costs.cached_tokens;
|
||||
delete lite.costs.cached_tokens;
|
||||
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'cached', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
try {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'cached', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 3000,
|
||||
completion_tokens: 40,
|
||||
prompt_tokens_details: { cached_tokens: 2900 },
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 3000,
|
||||
completion_tokens: 40,
|
||||
prompt_tokens_details: { cached_tokens: 2900 },
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.0-flash-lite',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-3.1-flash-lite',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const [, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
const inputRate = Number(lite.costs.prompt_tokens);
|
||||
expect(overrides).toMatchObject({
|
||||
prompt_tokens: (3000 - 2900) * inputRate,
|
||||
completion_tokens: 40 * Number(lite.costs.completion_tokens),
|
||||
cached_tokens: 2900 * inputRate,
|
||||
});
|
||||
expect(
|
||||
(overrides as Record<string, number>).cached_tokens,
|
||||
).toBeGreaterThan(0);
|
||||
const [, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
const inputRate = Number(lite.costs.prompt_tokens);
|
||||
expect(overrides).toMatchObject({
|
||||
prompt_tokens: (3000 - 2900) * inputRate,
|
||||
completion_tokens: 40 * Number(lite.costs.completion_tokens),
|
||||
cached_tokens: 2900 * inputRate,
|
||||
});
|
||||
expect(
|
||||
(overrides as Record<string, number>).cached_tokens,
|
||||
).toBeGreaterThan(0);
|
||||
} finally {
|
||||
lite.costs.cached_tokens = cachedRate;
|
||||
}
|
||||
});
|
||||
|
||||
it('zeroes cached_tokens when prompt_tokens_details is missing', async () => {
|
||||
@@ -643,9 +648,9 @@ describe('GeminiChatProvider.complete grounding request metering', () => {
|
||||
// generation; without its own rate the fee fell through to the input
|
||||
// token rate, which is several orders of magnitude below list.
|
||||
const lite = GEMINI_MODELS.find(
|
||||
(m) => m.id === 'gemini-2.0-flash-lite',
|
||||
(m) => m.id === 'gemini-3.1-flash-lite',
|
||||
)!;
|
||||
expect(lite.costs.grounding_requests).toBe(3_500_000);
|
||||
expect(lite.costs.grounding_requests).toBe(1_400_000);
|
||||
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
@@ -666,7 +671,7 @@ describe('GeminiChatProvider.complete grounding request metering', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemini-2.0-flash-lite',
|
||||
model: 'gemini-3.1-flash-lite',
|
||||
messages: [{ role: 'user', content: 'search for foo' }],
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -50,57 +50,85 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'google:google/gemini-2.0-flash',
|
||||
id: 'gemini-2.0-flash',
|
||||
puterId: 'google:google/gemini-3.5-flash-lite',
|
||||
id: 'gemini-3.5-flash-lite',
|
||||
modalities: {
|
||||
input: ['text', 'image', 'audio', 'video', 'pdf'],
|
||||
output: ['text'],
|
||||
},
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2024-06',
|
||||
release_date: '2024-12-11',
|
||||
name: 'Gemini 2.0 Flash',
|
||||
aliases: ['google/gemini-2.0-flash'],
|
||||
context: 131072,
|
||||
knowledge: '2025-01',
|
||||
release_date: '2026-07-21',
|
||||
name: 'Gemini 3.5 Flash-Lite',
|
||||
aliases: ['google/gemini-3.5-flash-lite'],
|
||||
context: 1_048_576,
|
||||
max_tokens: 65_536,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
completion_tokens: 40,
|
||||
prompt_tokens: 30,
|
||||
completion_tokens: 250,
|
||||
thinking_tokens: 250,
|
||||
cached_tokens: 3,
|
||||
// Gemini 2.x grounding is $35 / 1,000 requests
|
||||
grounding_requests: 3_500_000,
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
puterId: 'google:google/gemini-2.0-flash-lite',
|
||||
id: 'gemini-2.0-flash-lite',
|
||||
puterId: 'google:google/gemini-3.6-flash',
|
||||
id: 'gemini-3.6-flash',
|
||||
modalities: {
|
||||
input: ['text', 'image', 'audio', 'video', 'pdf'],
|
||||
output: ['text'],
|
||||
},
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2024-06',
|
||||
release_date: '2024-12-11',
|
||||
name: 'Gemini 2.0 Flash-Lite',
|
||||
aliases: ['google/gemini-2.0-flash-lite'],
|
||||
knowledge: '2025-01',
|
||||
release_date: '2026-07-21',
|
||||
name: 'Gemini 3.6 Flash',
|
||||
aliases: ['google/gemini-3.6-flash'],
|
||||
context: 1_048_576,
|
||||
max_tokens: 65_536,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 8,
|
||||
completion_tokens: 30,
|
||||
// Gemini 2.x grounding is $35 / 1,000 requests
|
||||
grounding_requests: 3_500_000,
|
||||
prompt_tokens: 150,
|
||||
completion_tokens: 750,
|
||||
thinking_tokens: 750,
|
||||
cached_tokens: 15,
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
},
|
||||
{
|
||||
puterId: 'google:google/gemini-3.7-flash',
|
||||
id: 'gemini-3.7-flash',
|
||||
modalities: {
|
||||
input: ['text', 'image', 'audio', 'video', 'pdf'],
|
||||
output: ['text'],
|
||||
},
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2025-01',
|
||||
release_date: '2026-08-13',
|
||||
name: 'Gemini 3.7 Flash',
|
||||
aliases: ['google/gemini-3.7-flash'],
|
||||
context: 1_048_576,
|
||||
max_tokens: 65_536,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 75,
|
||||
completion_tokens: 375,
|
||||
thinking_tokens: 375,
|
||||
cached_tokens: 8,
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
puterId: 'google:google/gemini-2.5-flash',
|
||||
@@ -273,4 +301,32 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
{
|
||||
puterId: 'google:google/gemini-3.7-flash',
|
||||
id: 'gemini-3.7-flash',
|
||||
modalities: {
|
||||
input: ['text', 'image', 'video', 'audio', 'pdf'],
|
||||
output: ['text'],
|
||||
},
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
knowledge: '2026-03',
|
||||
release_date: '2026-08-13',
|
||||
name: 'Gemini 3.7 Flash',
|
||||
aliases: ['google/gemini-3.7-flash'],
|
||||
context: 1_048_576,
|
||||
max_tokens: 65_536,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 75,
|
||||
completion_tokens: 375,
|
||||
thinking_tokens: 375,
|
||||
cached_tokens: 7.5,
|
||||
// Gemini 3.x grounding is $14 / 1,000 requests
|
||||
grounding_requests: 1_400_000,
|
||||
},
|
||||
},
|
||||
];
|
||||
@@ -305,16 +305,16 @@ describe('GroqAIProvider model resolution', () => {
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gemma2-9b-it',
|
||||
model: 'llama-3.3-70b-versatile',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(createMock.mock.calls[0]![0].model).toBe('gemma2-9b-it');
|
||||
expect(createMock.mock.calls[0]![0].model).toBe('llama-3.3-70b-versatile');
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.any(Object),
|
||||
expect.anything(),
|
||||
'groq:gemma2-9b-it',
|
||||
'groq:llama-3.3-70b-versatile',
|
||||
expect.any(Object),
|
||||
);
|
||||
});
|
||||
|
||||
@@ -19,116 +19,17 @@
|
||||
|
||||
import type { IChatModel } from '../../types.js';
|
||||
|
||||
// Hardcoded from https://models.dev/api.json
|
||||
// Hardcoded from https://models.dev/api.json (Groq chat/text models)
|
||||
export const GROQ_MODELS: IChatModel[] = [
|
||||
{
|
||||
id: 'gemma2-9b-it',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
knowledge: '2024-06',
|
||||
release_date: '2024-06-27',
|
||||
name: 'Gemma 2 9B 8k',
|
||||
context: 8192,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
completion_tokens: 20,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
id: 'gemma-7b-it',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Gemma 7B 8k Instruct',
|
||||
context: 8192,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 7,
|
||||
completion_tokens: 7,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
id: 'llama3-groq-70b-8192-tool-use-preview',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3 Groq 70B Tool Use Preview 8k',
|
||||
context: 8192,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 89,
|
||||
completion_tokens: 89,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
id: 'llama3-groq-8b-8192-tool-use-preview',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3 Groq 8B Tool Use Preview 8k',
|
||||
context: 8192,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 19,
|
||||
completion_tokens: 19,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
id: 'llama-3.1-70b-versatile',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3.1 70B Versatile 128k',
|
||||
context: 128000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 59,
|
||||
completion_tokens: 79,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
id: 'llama-3.1-70b-specdec',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3.1 8B Instant 128k',
|
||||
context: 128000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 59,
|
||||
completion_tokens: 99,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:llama-3.1-8b-instant',
|
||||
id: 'llama-3.1-8b-instant',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
knowledge: '2023-12',
|
||||
release_date: '2024-07-23',
|
||||
name: 'Llama 3.1 8B Instant 128k',
|
||||
name: 'Llama 3.1 8B Instant',
|
||||
aliases: ['llama-3.1-8b-instant'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
@@ -142,28 +43,182 @@ export const GROQ_MODELS: IChatModel[] = [
|
||||
max_tokens: 131072,
|
||||
},
|
||||
{
|
||||
id: 'meta-llama/llama-guard-4-12b',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
puterId: 'groq:llama-3.3-70b-versatile',
|
||||
id: 'llama-3.3-70b-versatile',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2025-04-05',
|
||||
name: 'Llama Guard 4 12B',
|
||||
tool_call: true,
|
||||
release_date: '2024-12-06',
|
||||
name: 'Llama 3.3 70B Versatile',
|
||||
aliases: ['llama-3.3-70b-versatile'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
completion_tokens: 20,
|
||||
prompt_tokens: 59,
|
||||
completion_tokens: 79,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 1024,
|
||||
max_tokens: 32768,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:openai/gpt-oss-120b',
|
||||
id: 'openai/gpt-oss-120b',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
release_date: '2025-08-05',
|
||||
name: 'GPT OSS 120B',
|
||||
aliases: ['openai/gpt-oss-120b'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
completion_tokens: 60,
|
||||
cached_tokens: 8,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:openai/gpt-oss-20b',
|
||||
id: 'openai/gpt-oss-20b',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
release_date: '2025-08-05',
|
||||
name: 'GPT OSS 20B',
|
||||
aliases: ['openai/gpt-oss-20b'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 8,
|
||||
completion_tokens: 30,
|
||||
cached_tokens: 4,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:openai/gpt-oss-safeguard-20b',
|
||||
id: 'openai/gpt-oss-safeguard-20b',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
release_date: '2025-10-29',
|
||||
name: 'GPT OSS Safeguard 20B',
|
||||
aliases: ['openai/gpt-oss-safeguard-20b'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 8,
|
||||
completion_tokens: 30,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 65536,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:groq/compound',
|
||||
id: 'groq/compound',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2025-09-04',
|
||||
name: 'Groq Compound',
|
||||
aliases: ['groq/compound'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 0,
|
||||
completion_tokens: 0,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:groq/compound-mini',
|
||||
id: 'groq/compound-mini',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2025-09-04',
|
||||
name: 'Groq Compound Mini',
|
||||
aliases: ['groq/compound-mini'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 0,
|
||||
completion_tokens: 0,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:qwen/qwen3.6-27b',
|
||||
id: 'qwen/qwen3.6-27b',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
release_date: '2026-04-22',
|
||||
name: 'Qwen3.6 27B',
|
||||
aliases: ['qwen/qwen3.6-27b'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60,
|
||||
completion_tokens: 300,
|
||||
cached_tokens: 30,
|
||||
},
|
||||
max_tokens: 16384,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:meta-llama/llama-prompt-guard-2-22m',
|
||||
id: 'meta-llama/llama-prompt-guard-2-22m',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2025-05-29',
|
||||
name: 'Llama Prompt Guard 2 22M',
|
||||
aliases: ['meta-llama/llama-prompt-guard-2-22m'],
|
||||
context: 512,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 3,
|
||||
completion_tokens: 3,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 512,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:meta-llama/llama-prompt-guard-2-86m',
|
||||
id: 'meta-llama/llama-prompt-guard-2-86m',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Prompt Guard 2 86M',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2025-05-29',
|
||||
name: 'Llama Prompt Guard 2 86M',
|
||||
aliases: ['meta-llama/llama-prompt-guard-2-86m'],
|
||||
context: 512,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
@@ -177,133 +232,36 @@ export const GROQ_MODELS: IChatModel[] = [
|
||||
max_tokens: 512,
|
||||
},
|
||||
{
|
||||
id: 'llama-3.2-1b-preview',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3.2 1B (Preview) 8k',
|
||||
context: 128000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 4,
|
||||
completion_tokens: 4,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
id: 'llama-3.2-3b-preview',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3.2 3B (Preview) 8k',
|
||||
context: 128000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 6,
|
||||
completion_tokens: 6,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
id: 'llama-3.2-11b-vision-preview',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3.2 11B Vision 8k (Preview)',
|
||||
context: 8000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 18,
|
||||
completion_tokens: 18,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8000,
|
||||
},
|
||||
{
|
||||
id: 'llama-3.2-90b-vision-preview',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Llama 3.2 90B Vision 8k (Preview)',
|
||||
context: 8000,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 90,
|
||||
completion_tokens: 90,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8000,
|
||||
},
|
||||
{
|
||||
id: 'llama3-70b-8192',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
knowledge: '2023-03',
|
||||
release_date: '2024-04-18',
|
||||
name: 'Llama 3 70B 8k',
|
||||
context: 8192,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 59,
|
||||
completion_tokens: 79,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
id: 'llama3-8b-8192',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: true,
|
||||
knowledge: '2023-03',
|
||||
release_date: '2024-04-18',
|
||||
name: 'Llama 3 8B 8k',
|
||||
context: 8192,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 5,
|
||||
completion_tokens: 8,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
},
|
||||
{
|
||||
id: 'mixtral-8x7b-32768',
|
||||
// Not present in models.dev/api.json (as of 2026-02-11)
|
||||
name: 'Mixtral 8x7B Instruct 32k',
|
||||
context: 32768,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 24,
|
||||
completion_tokens: 24,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 32768,
|
||||
},
|
||||
{
|
||||
id: 'llama-guard-3-8b',
|
||||
puterId: 'groq:allam-2-7b',
|
||||
id: 'allam-2-7b',
|
||||
modalities: { input: ['text'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2024-07-23',
|
||||
name: 'Llama Guard 3 8B 8k',
|
||||
context: 8192,
|
||||
release_date: '2025-01-23',
|
||||
name: 'ALLaM 2 7B',
|
||||
aliases: ['allam-2-7b'],
|
||||
context: 4096,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 0,
|
||||
completion_tokens: 0,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 4096,
|
||||
},
|
||||
{
|
||||
puterId: 'groq:meta-llama/llama-guard-4-12b',
|
||||
id: 'meta-llama/llama-guard-4-12b',
|
||||
modalities: { input: ['text', 'image'], output: ['text'] },
|
||||
open_weights: true,
|
||||
tool_call: false,
|
||||
release_date: '2025-04-05',
|
||||
name: 'Llama Guard 4 12B',
|
||||
aliases: ['meta-llama/llama-guard-4-12b'],
|
||||
context: 131072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
@@ -313,6 +271,6 @@ export const GROQ_MODELS: IChatModel[] = [
|
||||
completion_tokens: 20,
|
||||
cached_tokens: 0,
|
||||
},
|
||||
max_tokens: 8192,
|
||||
max_tokens: 1024,
|
||||
},
|
||||
];
|
||||
@@ -0,0 +1,77 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Integration test for the Meta Model API (Muse Spark) provider.
|
||||
*
|
||||
* Muse Spark always reasons, and those tokens come out of the same output
|
||||
* budget, so a tight `max_tokens` comes back as `content: null` with
|
||||
* `finish_reason: 'length'` — hence the generous cap on a one-word prompt.
|
||||
* Skipped when `PUTER_TEST_AI_META_API_KEY` is unset.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { FSService } from '../../../../services/fs/FSService.js';
|
||||
import type { FSEntryStore } from '../../../../stores/fs/FSEntryStore.js';
|
||||
import type { S3ObjectStore } from '../../../../stores/fs/S3ObjectStore.js';
|
||||
import {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
makeMeteringStub,
|
||||
optionalEnv,
|
||||
skipUnlessEnv,
|
||||
withTestActor,
|
||||
} from '../../../integrationTestUtil.js';
|
||||
import { MetaProvider } from './MetaProvider.js';
|
||||
|
||||
const ENV_VAR = 'PUTER_TEST_AI_META_API_KEY';
|
||||
|
||||
// Only `puter_path` content parts reach the filesystem, and this test sends
|
||||
// plain text, so the stores stay untouched.
|
||||
const UNUSED_STORES = {} as { fsEntry: FSEntryStore; s3Object: S3ObjectStore };
|
||||
const UNUSED_FS_SERVICE = {} as FSService;
|
||||
|
||||
describe.skipIf(skipUnlessEnv(ENV_VAR))('MetaProvider (integration)', () => {
|
||||
it(
|
||||
'returns a non-empty completion from muse-spark-1.2',
|
||||
{ timeout: INTEGRATION_TEST_TIMEOUT_MS },
|
||||
async () => {
|
||||
const provider = new MetaProvider(
|
||||
makeMeteringStub(),
|
||||
UNUSED_STORES,
|
||||
UNUSED_FS_SERVICE,
|
||||
{ apiKey: optionalEnv(ENV_VAR)! },
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'muse-spark-1.2',
|
||||
messages: [
|
||||
{ role: 'user', content: 'Say hi in one word.' },
|
||||
],
|
||||
max_tokens: 2048,
|
||||
reasoning_effort: 'low',
|
||||
}),
|
||||
);
|
||||
|
||||
const text = (result as { message?: { content?: string } }).message
|
||||
?.content;
|
||||
expect(typeof text === 'string' && text.length > 0).toBe(true);
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,744 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Offline unit tests for MetaProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs MetaProvider against the live wired `MeteringService` so the
|
||||
* recording side is exercised end-to-end. The OpenAI SDK is mocked at the
|
||||
* module boundary — Meta's Model API is OpenAI-compatible, so the provider
|
||||
* reaches it through the same client — meaning nothing leaves the process. The
|
||||
* companion integration test (MetaProvider.integration.test.ts) exercises the
|
||||
* real api.meta.ai endpoint.
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
type MockInstance,
|
||||
} from 'vitest';
|
||||
|
||||
import type { Actor } from '../../../../core/actor.js';
|
||||
import { SYSTEM_ACTOR } from '../../../../core/actor.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import { PuterServer } from '../../../../server.js';
|
||||
import { setupTestServer } from '../../../../testUtil.js';
|
||||
import { withTestActor } from '../../../integrationTestUtil.js';
|
||||
import { AIChatStream } from '../../utils/Streaming.js';
|
||||
import { MetaProvider } from './MetaProvider.js';
|
||||
import { META_MODELS } from './models.js';
|
||||
|
||||
// -- OpenAI SDK mock ----------------------------------------------
|
||||
//
|
||||
// `vi.hoisted` shares spies between the hoisted factory and the test body so
|
||||
// each test can stub `chat.completions.create` with the response shape it
|
||||
// cares about.
|
||||
|
||||
const { createMock, openAICtor } = vi.hoisted(() => {
|
||||
const createMock = vi.fn();
|
||||
const openAICtor = vi.fn();
|
||||
return { createMock, openAICtor };
|
||||
});
|
||||
|
||||
vi.mock('openai', () => {
|
||||
const OpenAICtor = vi.fn().mockImplementation(function (
|
||||
this: Record<string, unknown>,
|
||||
opts: unknown,
|
||||
) {
|
||||
openAICtor(opts);
|
||||
this.chat = { completions: { create: createMock } };
|
||||
});
|
||||
// The test server boots every provider, and some (e.g. OllamaChatProvider)
|
||||
// import the default export and read `.OpenAI` off it — expose both shapes.
|
||||
return { OpenAI: OpenAICtor, default: { OpenAI: OpenAICtor } };
|
||||
});
|
||||
|
||||
// -- Test harness -------------------------------------------------
|
||||
|
||||
let server: PuterServer;
|
||||
let recordSpy: MockInstance<MeteringService['utilRecordUsageObject']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
const makeProvider = (config: { apiKey?: string; apiBaseUrl?: string } = {}) =>
|
||||
new MetaProvider(
|
||||
server.services.metering,
|
||||
{
|
||||
fsEntry: server.stores.fsEntry,
|
||||
s3Object: server.stores.s3Object,
|
||||
},
|
||||
server.services.fs,
|
||||
{
|
||||
apiKey: config.apiKey ?? 'test-key',
|
||||
...(config.apiBaseUrl ? { apiBaseUrl: config.apiBaseUrl } : {}),
|
||||
},
|
||||
);
|
||||
|
||||
const asAsyncIterable = <T>(items: T[]): AsyncIterable<T> => ({
|
||||
async *[Symbol.asyncIterator]() {
|
||||
for (const item of items) {
|
||||
yield item;
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
const makeCapturingChatStream = () => {
|
||||
const chunks: string[] = [];
|
||||
const sink = new Writable({
|
||||
write(chunk, _enc, cb) {
|
||||
chunks.push(chunk.toString('utf8'));
|
||||
cb();
|
||||
},
|
||||
});
|
||||
const chatStream = new AIChatStream({ stream: sink });
|
||||
return {
|
||||
chatStream,
|
||||
events: () =>
|
||||
chunks
|
||||
.join('')
|
||||
.split('\n')
|
||||
.filter(Boolean)
|
||||
.map((line) => JSON.parse(line)),
|
||||
};
|
||||
};
|
||||
|
||||
const OK_COMPLETION = {
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'hi', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
};
|
||||
|
||||
const complete = (
|
||||
provider: MetaProvider,
|
||||
params: Record<string, unknown> = {},
|
||||
actor?: Actor,
|
||||
) =>
|
||||
withTestActor(
|
||||
() =>
|
||||
provider.complete({
|
||||
model: 'muse-spark-1.2',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
...params,
|
||||
} as never),
|
||||
actor,
|
||||
);
|
||||
|
||||
beforeEach(() => {
|
||||
createMock.mockReset();
|
||||
openAICtor.mockReset();
|
||||
// Spy on the live MeteringService without replacing the impl, so the
|
||||
// recording side still runs while per-test assertions see the calls.
|
||||
recordSpy = vi.spyOn(server.services.metering, 'utilRecordUsageObject');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
// -- Construction -------------------------------------------------
|
||||
|
||||
describe('MetaProvider construction', () => {
|
||||
it('points the OpenAI SDK at the Meta Model API with the configured key', () => {
|
||||
makeProvider();
|
||||
expect(openAICtor).toHaveBeenCalledTimes(1);
|
||||
expect(openAICtor).toHaveBeenCalledWith({
|
||||
apiKey: 'test-key',
|
||||
baseURL: 'https://api.meta.ai/v1',
|
||||
});
|
||||
});
|
||||
|
||||
it('honours a custom apiBaseUrl override', () => {
|
||||
makeProvider({ apiBaseUrl: 'https://staging.meta.test/v1' });
|
||||
expect(openAICtor).toHaveBeenCalledWith({
|
||||
apiKey: 'test-key',
|
||||
baseURL: 'https://staging.meta.test/v1',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// -- Model catalog ------------------------------------------------
|
||||
|
||||
describe('MetaProvider model catalog', () => {
|
||||
it('returns muse-spark-1.2 as the default', () => {
|
||||
expect(makeProvider().getDefaultModel()).toBe('muse-spark-1.2');
|
||||
});
|
||||
|
||||
it('exposes the static META_MODELS list verbatim from models()', () => {
|
||||
expect(makeProvider().models()).toBe(META_MODELS);
|
||||
});
|
||||
|
||||
it('list() flattens canonical ids and aliases', () => {
|
||||
const names = makeProvider().list();
|
||||
for (const m of META_MODELS) {
|
||||
expect(names).toContain(m.id);
|
||||
for (const a of m.aliases ?? []) {
|
||||
expect(names).toContain(a);
|
||||
}
|
||||
}
|
||||
expect(names).toContain('meta/muse-spark-1.2');
|
||||
});
|
||||
|
||||
it('leaves the contributor tier out of the catalog entirely', () => {
|
||||
// It is the cheapest input rate Meta sells, so any name pointing at it
|
||||
// would win bucket routing — and Meta trains on what it serves. It is
|
||||
// also gated behind a separate enrolment, so a standard-tier key gets
|
||||
// `model_not_found` for it.
|
||||
const names = new Set(makeProvider().list());
|
||||
for (const name of names) {
|
||||
expect(name).not.toContain('contributor');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// -- Request shape ------------------------------------------------
|
||||
|
||||
describe('MetaProvider.complete request shape', () => {
|
||||
it('sends model and messages without optional knobs', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider());
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.model).toBe('muse-spark-1.2');
|
||||
expect(args.messages).toEqual([{ role: 'user', content: 'hi' }]);
|
||||
for (const key of [
|
||||
'max_completion_tokens',
|
||||
'temperature',
|
||||
'top_p',
|
||||
'tools',
|
||||
'tool_choice',
|
||||
'reasoning_effort',
|
||||
'prompt_cache_key',
|
||||
'prompt_cache_retention',
|
||||
'response_format',
|
||||
'seed',
|
||||
]) {
|
||||
expect(key in args).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('caps output with max_completion_tokens, not the legacy max_tokens', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), { max_tokens: 512 });
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.max_completion_tokens).toBe(512);
|
||||
expect('max_tokens' in args).toBe(false);
|
||||
});
|
||||
|
||||
it('forwards temperature, top_p, tools, and tool_choice when supplied', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
const tools = [
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: { q: { type: 'string' } },
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
await complete(makeProvider(), {
|
||||
temperature: 0.4,
|
||||
top_p: 0.9,
|
||||
tools,
|
||||
tool_choice: 'auto',
|
||||
});
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.temperature).toBe(0.4);
|
||||
expect(args.top_p).toBe(0.9);
|
||||
expect(args.tools).toBe(tools);
|
||||
expect(args.tool_choice).toBe('auto');
|
||||
});
|
||||
|
||||
it('forwards reasoning_effort, including the Meta-only tiers', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), { reasoning_effort: 'xhigh' });
|
||||
expect(createMock.mock.calls[0]![0].reasoning_effort).toBe('xhigh');
|
||||
});
|
||||
|
||||
it('falls back to reasoning.effort when reasoning_effort is absent', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), { reasoning: { effort: 'low' } });
|
||||
expect(createMock.mock.calls[0]![0].reasoning_effort).toBe('low');
|
||||
});
|
||||
|
||||
it('drops reasoning_effort: none, which Muse Spark rejects with a 400', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), { reasoning_effort: 'none' });
|
||||
expect('reasoning_effort' in createMock.mock.calls[0]![0]).toBe(false);
|
||||
});
|
||||
|
||||
it("translates the hyphenated in-memory cache retention to Meta's enum", async () => {
|
||||
const provider = makeProvider();
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(provider, {
|
||||
prompt_cache_key: 'cache-abc',
|
||||
prompt_cache_retention: 'in-memory',
|
||||
});
|
||||
|
||||
// Meta rejects the hyphenated spelling outright: "unknown variant
|
||||
// `in-memory`, expected `in_memory` or `24h`".
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.prompt_cache_key).toBe('cache-abc');
|
||||
expect(args.prompt_cache_retention).toBe('in_memory');
|
||||
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(provider, { prompt_cache_retention: '24h' });
|
||||
expect(createMock.mock.calls[1]![0].prompt_cache_retention).toBe('24h');
|
||||
});
|
||||
|
||||
it('forwards Muse-Spark-specific custom params', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), {
|
||||
custom: {
|
||||
seed: 7,
|
||||
response_format: { type: 'json_object' },
|
||||
frequency_penalty: 0.5,
|
||||
presence_penalty: -0.5,
|
||||
},
|
||||
});
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.seed).toBe(7);
|
||||
expect(args.response_format).toEqual({ type: 'json_object' });
|
||||
expect(args.frequency_penalty).toBe(0.5);
|
||||
expect(args.presence_penalty).toBe(-0.5);
|
||||
});
|
||||
|
||||
it('strips Anthropic-style cache_control from messages before sending', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), {
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: 'hi',
|
||||
cache_control: { type: 'ephemeral' },
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(
|
||||
'cache_control' in createMock.mock.calls[0]![0].messages[0],
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('derives safety_identifier from the actor and truncates it to 64 chars', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
const userActor: Actor = {
|
||||
user: { id: 42, uuid: 'u42', username: 'alice' },
|
||||
app: { id: 7, uid: 'a'.repeat(80) },
|
||||
};
|
||||
|
||||
await complete(makeProvider(), {}, userActor);
|
||||
|
||||
const identifier = createMock.mock.calls[0]![0].safety_identifier;
|
||||
expect(identifier.startsWith('puter-42-a')).toBe(true);
|
||||
expect(identifier.length).toBe(64);
|
||||
});
|
||||
|
||||
it('prefers an explicit custom.safety_identifier over the actor-derived one', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
const userActor: Actor = { user: { id: 42, uuid: 'u42' } };
|
||||
await complete(
|
||||
makeProvider(),
|
||||
{ custom: { safety_identifier: 'caller-supplied' } },
|
||||
userActor,
|
||||
);
|
||||
expect(createMock.mock.calls[0]![0].safety_identifier).toBe(
|
||||
'caller-supplied',
|
||||
);
|
||||
});
|
||||
|
||||
it('omits safety_identifier for the system actor (no user.id)', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider());
|
||||
expect('safety_identifier' in createMock.mock.calls[0]![0]).toBe(false);
|
||||
});
|
||||
|
||||
it('only sets stream_options.include_usage when streaming', async () => {
|
||||
const provider = makeProvider();
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(provider, { stream: false });
|
||||
|
||||
const [nonStreamArgs] = createMock.mock.calls[0]!;
|
||||
expect(nonStreamArgs.stream).toBe(false);
|
||||
expect('stream_options' in nonStreamArgs).toBe(false);
|
||||
|
||||
createMock.mockReturnValueOnce(asAsyncIterable([]));
|
||||
await complete(provider, { stream: true });
|
||||
|
||||
const [streamArgs] = createMock.mock.calls[1]!;
|
||||
expect(streamArgs.stream).toBe(true);
|
||||
expect(streamArgs.stream_options).toEqual({ include_usage: true });
|
||||
});
|
||||
});
|
||||
|
||||
// -- Model resolution ---------------------------------------------
|
||||
|
||||
describe('MetaProvider model resolution', () => {
|
||||
it('resolves an alias to its canonical id', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), { model: 'meta/muse-spark-1.1' });
|
||||
|
||||
expect(createMock.mock.calls[0]![0].model).toBe('muse-spark-1.1');
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.any(Object),
|
||||
expect.anything(),
|
||||
'meta:muse-spark-1.1',
|
||||
expect.any(Object),
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to the default model when given an unknown id', async () => {
|
||||
createMock.mockResolvedValueOnce(OK_COMPLETION);
|
||||
await complete(makeProvider(), { model: 'totally-not-a-real-model' });
|
||||
|
||||
expect(createMock.mock.calls[0]![0].model).toBe('muse-spark-1.2');
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.any(Object),
|
||||
expect.anything(),
|
||||
'meta:muse-spark-1.2',
|
||||
expect.any(Object),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// -- Non-stream output --------------------------------------------
|
||||
|
||||
describe('MetaProvider.complete non-stream output', () => {
|
||||
it('bills cache reads separately from the remaining prompt tokens', async () => {
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'hi there', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
prompt_tokens_details: { cached_tokens: 40 },
|
||||
},
|
||||
});
|
||||
|
||||
const result = await complete(makeProvider());
|
||||
|
||||
expect(result).toMatchObject({
|
||||
message: { content: 'hi there', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
});
|
||||
// Meta reports cache reads inside prompt_tokens; only the uncached
|
||||
// remainder is charged at the input rate.
|
||||
expect((result as { usage: unknown }).usage).toEqual({
|
||||
prompt_tokens: 60,
|
||||
completion_tokens: 50,
|
||||
cached_tokens: 40,
|
||||
});
|
||||
|
||||
const model = META_MODELS.find((m) => m.id === 'muse-spark-1.2')!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 60,
|
||||
completion_tokens: 50,
|
||||
cached_tokens: 40,
|
||||
});
|
||||
expect(actor).toBe(SYSTEM_ACTOR);
|
||||
expect(prefix).toBe('meta:muse-spark-1.2');
|
||||
expect(overrides!.prompt_tokens).toBeCloseTo(
|
||||
60 * Number(model.costs.prompt_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides!.completion_tokens).toBeCloseTo(
|
||||
50 * Number(model.costs.completion_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides!.cached_tokens).toBeCloseTo(
|
||||
40 * Number(model.costs.cached_tokens),
|
||||
5,
|
||||
);
|
||||
});
|
||||
|
||||
it('zeroes cached_tokens when prompt_tokens_details is missing', async () => {
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 7, completion_tokens: 3 },
|
||||
});
|
||||
|
||||
await complete(makeProvider());
|
||||
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 7,
|
||||
completion_tokens: 3,
|
||||
cached_tokens: 0,
|
||||
});
|
||||
expect(overrides).toMatchObject({ cached_tokens: 0 });
|
||||
});
|
||||
|
||||
it('preserves OpenAI-shaped tool_calls on the assistant response', async () => {
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: null,
|
||||
tool_calls: [
|
||||
{
|
||||
id: 'call_1',
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
arguments: '{"q":"puter"}',
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
finish_reason: 'tool_calls',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
});
|
||||
|
||||
const result = (await complete(makeProvider(), {
|
||||
tools: [
|
||||
{
|
||||
type: 'function',
|
||||
function: { name: 'lookup', parameters: {} },
|
||||
},
|
||||
],
|
||||
})) as { message: { tool_calls?: unknown[] }; finish_reason: string };
|
||||
|
||||
expect(result.finish_reason).toBe('tool_calls');
|
||||
expect(result.message.tool_calls).toEqual([
|
||||
{
|
||||
id: 'call_1',
|
||||
type: 'function',
|
||||
function: { name: 'lookup', arguments: '{"q":"puter"}' },
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('leaves reasoning token counts out of the metered usage', async () => {
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'done', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 8,
|
||||
completion_tokens: 100,
|
||||
completion_tokens_details: { reasoning_tokens: 90 },
|
||||
},
|
||||
});
|
||||
|
||||
await complete(makeProvider());
|
||||
|
||||
// Muse Spark always reasons, and counts those tokens inside
|
||||
// `completion_tokens` — billing them again would double-charge.
|
||||
const [usage] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 8,
|
||||
completion_tokens: 100,
|
||||
cached_tokens: 0,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// -- Streaming ----------------------------------------------------
|
||||
|
||||
describe('MetaProvider.complete streaming', () => {
|
||||
it('streams text deltas and meters the final usage chunk', async () => {
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{ choices: [{ delta: { content: 'hel' } }] },
|
||||
{ choices: [{ delta: { content: 'lo' } }] },
|
||||
{
|
||||
choices: [{ delta: {} }],
|
||||
usage: {
|
||||
prompt_tokens: 10,
|
||||
completion_tokens: 2,
|
||||
prompt_tokens_details: { cached_tokens: 4 },
|
||||
},
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await complete(makeProvider(), { stream: true });
|
||||
expect((result as { stream: boolean }).stream).toBe(true);
|
||||
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const events = harness.events();
|
||||
expect(
|
||||
events.filter((e) => e.type === 'text').map((e) => e.text),
|
||||
).toEqual(['hel', 'lo']);
|
||||
expect(events.find((e) => e.type === 'usage')?.usage).toEqual({
|
||||
prompt_tokens: 6,
|
||||
completion_tokens: 2,
|
||||
cached_tokens: 4,
|
||||
});
|
||||
|
||||
const model = META_MODELS.find((m) => m.id === 'muse-spark-1.2')!;
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('meta:muse-spark-1.2');
|
||||
expect(overrides!.prompt_tokens).toBeCloseTo(
|
||||
6 * Number(model.costs.prompt_tokens),
|
||||
5,
|
||||
);
|
||||
expect(overrides!.cached_tokens).toBeCloseTo(
|
||||
4 * Number(model.costs.cached_tokens),
|
||||
5,
|
||||
);
|
||||
});
|
||||
|
||||
it('builds a tool_use event from streamed function-call deltas', async () => {
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{
|
||||
choices: [
|
||||
{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{
|
||||
index: 0,
|
||||
id: 'call_1',
|
||||
function: {
|
||||
name: 'lookup',
|
||||
arguments: '{"q":',
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
choices: [
|
||||
{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{
|
||||
index: 0,
|
||||
function: { arguments: '"puter"}' },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
choices: [{ delta: {} }],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await complete(makeProvider(), {
|
||||
stream: true,
|
||||
tools: [
|
||||
{
|
||||
type: 'function',
|
||||
function: { name: 'lookup', parameters: {} },
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const toolEvent = harness.events().find((e) => e.type === 'tool_use');
|
||||
expect(toolEvent).toBeDefined();
|
||||
expect(toolEvent?.id).toBe('call_1');
|
||||
expect(toolEvent?.name).toBe('lookup');
|
||||
expect(toolEvent?.input).toEqual({ q: 'puter' });
|
||||
});
|
||||
});
|
||||
|
||||
// -- Error mapping ------------------------------------------------
|
||||
|
||||
describe('MetaProvider.complete error mapping', () => {
|
||||
it('rethrows errors raised by the OpenAI client unchanged', async () => {
|
||||
const apiError = new Error('Model API exploded');
|
||||
createMock.mockRejectedValueOnce(apiError);
|
||||
|
||||
await expect(complete(makeProvider())).rejects.toBe(apiError);
|
||||
// A failed call must not be metered.
|
||||
expect(recordSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects a missing messages payload with a 400', async () => {
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
makeProvider().complete({
|
||||
model: 'muse-spark-1.2',
|
||||
} as never),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(createMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// -- Moderation ---------------------------------------------------
|
||||
|
||||
describe('MetaProvider.checkModeration', () => {
|
||||
it('throws — the Meta provider does not implement moderation', () => {
|
||||
expect(() => makeProvider().checkModeration('anything')).toThrow(
|
||||
/not implemented/i,
|
||||
);
|
||||
});
|
||||
});
|
||||
Loaded 100 of 403 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user