mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-21 11:29:29 -10:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
15aeb2edfb |
No files matched your search
@@ -145,14 +145,13 @@ AI drivers live in [src/backend/drivers/ai/](src/backend/drivers/ai/): `interfac
|
||||
|
||||
Layout: SDK modules in [src/puter-js/src/modules/](src/puter-js/src/modules/) (one file or directory per area — `FileSystem/`, `kv/`, `ai/`, …), shared plumbing in [src/puter-js/src/lib/](src/puter-js/src/lib/), generated (gitignored) type declarations in `src/puter-js/types/`, API tests in [src/puter-js/tests/api/](src/puter-js/tests/api/), UI e2e tests in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/), developer docs in [src/docs/](src/docs/).
|
||||
|
||||
Every SDK change carries all six of these, or the PR is incomplete. [doc/contributing-apis.md](doc/contributing-apis.md) has the steps for each.
|
||||
Every SDK change carries all five of these, or the PR is incomplete. [doc/contributing-apis.md](doc/contributing-apis.md) has the steps for each.
|
||||
|
||||
1. **Backward compatibility**, unless a maintainer explicitly signs off on a break ([rules](doc/contributing-apis.md#maintaining-an-existing-api)).
|
||||
2. **Tests** in the cross-platform API suite, or the Playwright harness for desktop-rendered `puter.ui.*` ([steps](doc/contributing-apis.md#6-tests)).
|
||||
3. **Docs** in [src/docs/src/](src/docs/src/) that match the implementation exactly ([steps](doc/contributing-apis.md#5-docs), [writing rules](#developer-docs)).
|
||||
4. **Types**, per the rules below.
|
||||
5. **Error handling**: `{ message, code }` with stable `snake_case` codes, as in [modules/kv/](src/puter-js/src/modules/kv/) ([steps](doc/contributing-apis.md#3-puterjs)).
|
||||
6. **Changelog** entry for anything a developer would notice ([rules](#changelog)).
|
||||
|
||||
### Types
|
||||
|
||||
@@ -225,7 +224,7 @@ These apply to reference pages and recipes alike.
|
||||
|
||||
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
|
||||
- **Document Puter as it runs on puter.com.** The docs are for developers building on production Puter. Leave out behavior that only happens on a self-hosted or differently configured server, such as a provider that isn't set up there.
|
||||
- **Describe the current state.** Docs aren't a changelog; [changelog.md](src/docs/src/changelog.md) is the one file that describes changes ([rules](#changelog)). Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
|
||||
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
|
||||
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
|
||||
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
|
||||
- **State the rule, then its consequence, in short sentences.** No metaphor in place of a rule. Name a concrete action ("you can create any files and folders inside it") over an abstract permission ("yours to organize freely"). Cut a sentence that only repeats the next one.
|
||||
@@ -251,22 +250,6 @@ Recipes are tested, copy-pasteable patterns for a task, one layer above the per-
|
||||
- **Tradeoffs.** Justify a choice with one scenario the reader can picture right away, in about two short paragraphs, or fold each point into the section where it matters. The description states the benefit only.
|
||||
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit the recipe needs goes in the section it applies to, and other facts belong on the linked reference page.
|
||||
|
||||
### Changelog
|
||||
|
||||
[src/docs/src/changelog.md](src/docs/src/changelog.md) lists the changes a developer building on puter.js would notice, newest first. A PR that makes one adds its entry in the same PR, whether the change is in puter.js, the backend, or the docs. [src/docs/src/changelog.js](src/docs/src/changelog.js) builds it into a standalone `/changelog/` page, not a docs page, into Markdown at `https://docs.puter.com/changelog.md`, and into an RSS feed at `https://docs.puter.com/changelog/rss.xml`. The docs sidebar links to it only through the "What's new" button. It stays out of the sidebar's page list, search, the sitemap, and `llms.txt`, so don't add it to [sidebar.js](src/docs/src/sidebar.js). The build fails on a `##` heading that isn't a date, a day listed twice, or days out of order.
|
||||
|
||||
- **What gets an entry.** A change gets an entry only if a developer whose app relies on puter.js would change their code, or handle something new, because of it. The entry says what they can do now or what their code sees differently. That's usually:
|
||||
- new, changed, or removed methods, options, return fields, events, and error codes
|
||||
- an AI model that's removed, or a model name that points to a different model
|
||||
- bug fixes with an effect a developer can see
|
||||
- a `puter.*` call that behaves differently because of a change in the Puter desktop or GUI
|
||||
- **Billing, limits, and quotas get no entry by default.** [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) already publishes every number, and a short changelog is one developers read. The PR's author adds an entry for one only when they decide developers need to hear about it, such as a lowered limit that makes working calls fail. An agent doesn't add one unless the author asks.
|
||||
- **What doesn't get an entry.** Adding a model or provider, a model's price or billing unit, docs changes such as a new recipe or a moved page, refactors, internal fixes, GUI-only changes, tests, APIs only Puter's own apps can call, and a new check that rejects input no working app sends and returns an existing error such as `bad_request`.
|
||||
- **One heading per day.** Entries go under a `## Month D, YYYY` heading, such as `## November 12, 2026`, for the UTC day the PR merges. Use the day you open the PR, and move the entry if it merges on a later day. If that day's heading already exists, add your bullet under it. A day never gets two headings, and a day with no changes gets none. When two PRs conflict on the same day's heading, keep one heading and both bullets.
|
||||
- **One bullet per change.** Start with a past-tense verb (Added, Changed, Fixed, Removed, Renamed, Deprecated, Raised, Lowered) and keep it to one or two sentences. Link the page that has the details instead of copying options or limits into the entry. A removal or rename says what to use instead.
|
||||
- **Same writing rules as the other pages** ([above](#writing-docs)), so link every API mention and skip em dashes.
|
||||
- **Write each entry as of its day.** A later change gets its own entry, and earlier entries don't mention it, so write "Added `puter.ai.convo()`", not "Added `puter.ai.convo()`, now `puter.ai.realtime()`". Leave past entries as written, except to fix a link after a page moves.
|
||||
|
||||
---
|
||||
|
||||
## GUI
|
||||
|
||||
+4
-18
@@ -66,25 +66,12 @@ To participate, you must:
|
||||
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see _Proof of Reproduction_ below).
|
||||
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
|
||||
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
|
||||
4. **Identify your testing**: Test only from accounts you declared, with the identifying header on every request (see _Identify Your Testing_ below).
|
||||
5. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
|
||||
6. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
|
||||
7. **Check Eligibility and Known Non-Issues**: Reports that fall outside "Eligibility" or match an item in "Known Non-Issues" above are not eligible and will be closed.
|
||||
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
|
||||
5. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
|
||||
6. **Check Eligibility and Known Non-Issues**: Reports that fall outside "Eligibility" or match an item in "Known Non-Issues" above are not eligible and will be closed.
|
||||
|
||||
Reports that do not meet these guidelines may not be eligible for a reward.
|
||||
|
||||
## Identify Your Testing
|
||||
|
||||
Testing on `puter.com` and `api.puter.com` is authorized only when we can tell it is yours while it happens. We can't tell unidentified testing from an attack, so we treat it as one: the accounts and addresses involved may be blocked without notice, and findings from it are not eligible for a reward.
|
||||
|
||||
- **Declare your test accounts first.** Before testing, email **[[email protected]](mailto:[email protected])** with the subject `Bug bounty test accounts`, the handle you will use, and the usernames of the accounts you will test with. You don't need to wait for a reply. Tell us when you add an account.
|
||||
- **Use only accounts you created for testing.** Do not use accounts that belong to anyone else, with or without their permission, and do not use bought, borrowed, aged, or compromised accounts or credentials. To test an interaction between two users, create both accounts and declare both.
|
||||
- **Send the identifying header on every request.** Add `X-Bug-Bounty: <your handle>` to all requests your testing makes, from scripts, proxies, and scanners as well as the browser. Use the handle you declared.
|
||||
- **Test from addresses you control and keep them stable.** Do not rotate IP addresses, and do not route testing through residential proxies, anonymity networks, or other people's devices.
|
||||
- **Keep automated traffic small.** A few requests per second at most, and stop if you see errors or slow responses that your traffic may be causing.
|
||||
|
||||
Testing your own self-hosted Puter instance needs none of this, and is the best place for anything noisy or destructive.
|
||||
|
||||
## Proof of Reproduction
|
||||
|
||||
Reports **MUST** demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
|
||||
@@ -110,7 +97,6 @@ Include:
|
||||
- Steps to reproduce
|
||||
- Potential impact
|
||||
- Suggested remediation (if available)
|
||||
- The handle and test accounts you declared (see _Identify Your Testing_)
|
||||
- Paypal id where we can send the bounty (if eligible)
|
||||
|
||||
Send the full report in your first email. We don't reply to messages that only ask whether we run a bounty or where to report.
|
||||
@@ -139,4 +125,4 @@ At this time, **payments will only be made via PayPal**. To be eligible to recei
|
||||
|
||||
## Legal Safe Harbor
|
||||
|
||||
If you make a good-faith effort to comply with this policy, we will consider your research to be authorized. Testing that isn't identified as described in _Identify Your Testing_ is not authorized and is not covered by this safe harbor. If you inadvertently access data outside your own account, stop immediately and include details in your report so we can investigate and remediate.
|
||||
If you make a good-faith effort to comply with this policy, we will consider your research to be authorized. If you inadvertently access data outside your own account, stop immediately and include details in your report so we can investigate and remediate.
|
||||
@@ -16,7 +16,6 @@ Check your diff against these sections of AGENTS.md:
|
||||
- [Working rules of thumb](AGENTS.md#working-rules-of-thumb): run it end to end, test new behavior and bug fixes, understand every line you commit (AI assistance is fine), leave touched files a bit better, and publish limit changes.
|
||||
- [Reuse and abstraction](AGENTS.md#reuse-and-abstraction) and [hot paths](AGENTS.md#hot-paths).
|
||||
- [Security & privacy](AGENTS.md#security--privacy): expose nothing the caller doesn't need.
|
||||
- [Changelog](AGENTS.md#changelog): add an entry for anything a developer building on puter.js would notice.
|
||||
|
||||
## Opening a PR
|
||||
|
||||
|
||||
+3
-30
@@ -400,30 +400,18 @@
|
||||
"pagerduty": {
|
||||
"enabled": false,
|
||||
"routingKey": "",
|
||||
// REST API key (read-only is enough). Lets an alarm that calls
|
||||
// link to the incident it opened.
|
||||
"apiToken": "",
|
||||
// Lowest severity that reaches PagerDuty. Default "warning",
|
||||
// which keeps `info` out of the paging system entirely.
|
||||
"minSeverity": "warning"
|
||||
},
|
||||
|
||||
// Slack — the destination for everything that shouldn't page.
|
||||
// Slack incoming webhook — the low-noise destination for everything
|
||||
// that shouldn't page.
|
||||
"slack": {
|
||||
"enabled": false,
|
||||
// Bot token with chat:write. Routes alarms to `channels` and
|
||||
// lets a call link to its message. Without one, `webhookUrl`
|
||||
// posts everything to the webhook's own channel.
|
||||
"botToken": "",
|
||||
"webhookUrl": "",
|
||||
// Default channel: alarms with no channel key, or a key not
|
||||
// listed below. Required with `botToken`.
|
||||
// Optional; defaults to the channel the webhook was created for.
|
||||
"channel": "#alerts",
|
||||
// Alarm channel keys mapped to channels (see doc/alarms.md).
|
||||
"channels": {
|
||||
// "billing": "#alerts-billing"
|
||||
},
|
||||
// With botToken, needs the chat:write.customize scope.
|
||||
"username": "puter-alarms",
|
||||
// Severity window posted to Slack. The ceiling defaults to
|
||||
// "info" when PagerDuty is configured — what pages belongs in
|
||||
@@ -435,21 +423,6 @@
|
||||
// occurrence always posts; the next post that gets through
|
||||
// reports how many occurrences piled up. 0 disables throttling.
|
||||
"repeatThrottleMs": 900000
|
||||
},
|
||||
|
||||
// Alarms that phone someone. An external monitor polls
|
||||
// GET /alarms/calls?group=<group> with `secret` in the
|
||||
// x-puter-internal-auth header; it answers 503 while an alarm in
|
||||
// the group is calling.
|
||||
"calls": {
|
||||
"secret": "",
|
||||
// Alarm ids, or prefixes ending in `*`, mapped to a call group.
|
||||
"rules": {
|
||||
// "metering_usage_dropped:*": "billing"
|
||||
},
|
||||
// A call stays active until its alarm has been quiet this long.
|
||||
// Minimum 5 minutes.
|
||||
"ttlMs": 1800000
|
||||
}
|
||||
},
|
||||
|
||||
|
||||
+23
-91
@@ -2,7 +2,7 @@
|
||||
|
||||
`clients.alarm` is the one way code reports that something is wrong. It
|
||||
de-dupes by alarm id, counts occurrences, and routes each alarm to alert
|
||||
transports by **severity**, then to a chat channel by its **channel key**.
|
||||
transports by **severity**.
|
||||
|
||||
```ts
|
||||
this.clients.alarm.create(
|
||||
@@ -10,27 +10,25 @@ this.clients.alarm.create(
|
||||
'Metering write failed', // what a human reads
|
||||
{ userUuid, appId, error }, // context fields
|
||||
'info', // severity
|
||||
{ channel: 'billing' }, // who owns it
|
||||
);
|
||||
```
|
||||
|
||||
## Severity is the routing decision
|
||||
|
||||
| Severity | Meaning | Goes to |
|
||||
| ---------- | ------------------------------------------------------------ | ----------- |
|
||||
| `critical` | An unhandled server error. Someone gets woken up. | Pager |
|
||||
| `error` | Same urgency as critical; prefer one of the other two. | Pager |
|
||||
| `warning` | For on-call to work through today. Nobody is woken up. | Pager (low) |
|
||||
| `info` | For the area that owns it, in that area's chat channel. | Chat |
|
||||
| Severity | Meaning | Goes to |
|
||||
| ---------- | ------------------------------------------------------ | ----------- |
|
||||
| `critical` | An unhandled server error. Someone gets woken up. | Pager |
|
||||
| `error` | Same urgency as critical; prefer one of the other two. | Pager |
|
||||
| `warning` | Worth a look today. Nobody is paged. | Pager (low) |
|
||||
| `info` | A record of something expected-but-notable. | Chat |
|
||||
|
||||
Each transport declares the severity window it accepts, so the value a call
|
||||
site passes is what decides where the alarm lands. The two windows don't
|
||||
overlap by default: anything that pages lives in the paging system, and chat
|
||||
is the record of what didn't. The bar for `critical` is deliberately high: an
|
||||
unhandled 5xx out of the HTTP error handler is the main thing that still
|
||||
pages. `warning` is on-call's low-urgency queue, mostly abuse signals (signup
|
||||
farms, usage spikes) someone should look at today. Everything else that isn't
|
||||
urgent is `info` with a channel key, so the people who own that area see it.
|
||||
is the record of what didn't. The bar for `critical` is
|
||||
deliberately high: an unhandled 5xx out of the HTTP error handler is the main
|
||||
thing that still pages. Anything a human can look at tomorrow is `warning`,
|
||||
and anything that's just worth recording is `info`.
|
||||
|
||||
Omitting the severity takes `pager.defaultSeverity` (itself `critical`), so
|
||||
pass one explicitly unless you really mean "page someone".
|
||||
@@ -38,17 +36,15 @@ pass one explicitly unless you really mean "page someone".
|
||||
### Choosing one
|
||||
|
||||
- Did the server fail to do its job in a way nobody expected? → `critical`
|
||||
- Is a user or the fleet doing something on-call should look at today
|
||||
(tripping a signup or usage abuse heuristic, spending far past its
|
||||
credits)? → `warning`
|
||||
- Is a background job, dependency, or one area's pipeline degraded? → `info`,
|
||||
with that area's channel key
|
||||
- Is a background job, rate, or dependency degraded, or has an account spent
|
||||
far past its credits? → `warning`
|
||||
- Is this a user doing something notable (tripping an abuse heuristic)?
|
||||
→ `info`
|
||||
- Did one of *our own* limits reject a caller — a rate limit, a concurrency
|
||||
cap, a quota? → don't alarm at all. The limit doing its job is not an
|
||||
event; the 429 is the whole signal, and alarming on it only produces noise
|
||||
proportional to traffic. An *upstream provider* rate-limiting us is the
|
||||
opposite case and still alarms (`upstream_rate_limited`, `info` in
|
||||
`ai_providers`) — that one
|
||||
opposite case and still alarms (`upstream_rate_limited`, `info`) — that one
|
||||
is not something we chose. The exception is a free model: nothing is billed,
|
||||
nothing is actionable, and the throttling is the price of the model, so the
|
||||
AI chat driver marks those `noAlarm` and only paid models still record.
|
||||
@@ -59,30 +55,9 @@ call site *already knows* are expected and traffic-proportional — reach for it
|
||||
there, not as a way to quiet an alarm you haven't diagnosed
|
||||
(`severityOverrides` below is the knob for that).
|
||||
|
||||
An extension whose signals are all one tier and channel can fix both in its
|
||||
own local alarm helper instead of repeating them at every call site.
|
||||
|
||||
## Channel keys
|
||||
|
||||
`{ channel: '<key>' }` names the area that owns an alarm. Config maps each key
|
||||
to a chat channel (`pager.slack.channels`); an alarm with no key, or a key the
|
||||
map doesn't list, goes to the default channel (`pager.slack.channel`). A
|
||||
thrown `HttpError` sets its key with `alarmChannel`. Keys only matter for what
|
||||
reaches chat; the pager ignores them.
|
||||
|
||||
The keys in use:
|
||||
|
||||
| Key | Area |
|
||||
| --------------- | ---------------------------------------------------------- |
|
||||
| `billing` | Purchases, subscriptions, top-ups, metering correctness |
|
||||
| `ai_providers` | Upstream AI provider failures: credits, auth, rate limits |
|
||||
| `content_abuse` | Hosted site and app abuse verdicts, and the scanner itself |
|
||||
| `signup` | The signup pipeline and the checks it depends on |
|
||||
| `email` | Sending and mailbox infrastructure |
|
||||
| `platform` | Jobs, events, KV, and other shared backend machinery |
|
||||
|
||||
Reuse one before adding a new key; a key nobody maps lands in the default
|
||||
channel.
|
||||
An extension whose signals are all one tier can default its own local
|
||||
`raiseAlarm` helper to that tier instead of repeating it at every call site —
|
||||
see [extensions/cronMonitor](../../../extensions/cronMonitor/index.js).
|
||||
|
||||
## One incident per occurrence, unless you say otherwise
|
||||
|
||||
@@ -112,28 +87,6 @@ body, so it has to be safe to show the caller.
|
||||
Reach for it anywhere else only when the id is that specific — otherwise a
|
||||
per-request alarm can flood the pager.
|
||||
|
||||
## Phone calls
|
||||
|
||||
Some alarms should phone someone, whatever their severity and wherever else
|
||||
they go. `pager.calls.rules`
|
||||
maps alarm ids, or prefixes ending in `*`, to a **call group**; the exact id
|
||||
beats a pattern and the longest prefix wins. While an alarm in a group keeps
|
||||
firing, it is recorded in the system KV store until it has been quiet for
|
||||
`ttlMs` (default 30 minutes, minimum 5).
|
||||
|
||||
An external uptime monitor polls `GET /alarms/calls?group=<group>` on the
|
||||
`api` subdomain, sending `pager.calls.secret` in `x-puter-internal-auth`. The
|
||||
route answers 200 while nothing in the group is calling and 503 while
|
||||
something is, or when it couldn't read the whole listing, so the monitor's own
|
||||
escalation policy places the call; leave out `group` to watch every group.
|
||||
The body is short plain text, one line per call, with a link to where the
|
||||
alarm landed: the PagerDuty incident when `pagerduty.apiToken` is set, or the
|
||||
chat message when Slack posts with a `botToken`. A call with no link shows the
|
||||
alarm's short id.
|
||||
|
||||
A call resolves by going quiet, not by a clear: acknowledge it in the monitor,
|
||||
and the route goes back to 200 once the alarm stops firing for `ttlMs`.
|
||||
|
||||
## Configuration
|
||||
|
||||
Everything lives under `pager` in config (see
|
||||
@@ -145,42 +98,21 @@ alarms to the console.
|
||||
"pager": {
|
||||
"defaultSeverity": "critical",
|
||||
"severityOverrides": { "cronMonitor:*": "info" },
|
||||
"pagerduty": {
|
||||
"enabled": true,
|
||||
"routingKey": "…",
|
||||
"apiToken": "…",
|
||||
"minSeverity": "warning",
|
||||
},
|
||||
"pagerduty": { "enabled": true, "routingKey": "…", "minSeverity": "warning" },
|
||||
"slack": {
|
||||
"enabled": true,
|
||||
"botToken": "xoxb-…",
|
||||
"webhookUrl": "…",
|
||||
"channel": "#alerts",
|
||||
"channels": { "billing": "#alerts-billing", "ai_providers": "#alerts-ai" },
|
||||
"minSeverity": "info",
|
||||
"maxSeverity": "info",
|
||||
"repeatThrottleMs": 900000,
|
||||
},
|
||||
"calls": {
|
||||
"secret": "…",
|
||||
"rules": { "metering_usage_dropped:*": "billing" },
|
||||
"ttlMs": 1800000,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Slack posts through the Web API when `botToken` is set (scope `chat:write`;
|
||||
invite the bot to each channel, or add `chat:write.public`). `channel` is then
|
||||
required. Channels can be names or IDs; IDs survive a rename. `username` needs
|
||||
the `chat:write.customize` scope with a bot token, so leave it out otherwise.
|
||||
With only a `webhookUrl`, every alarm lands in the webhook's own channel and
|
||||
calls carry no chat link.
|
||||
|
||||
PagerDuty's `apiToken` is a REST API key; read-only is enough. It's only used
|
||||
to find the incident an alarm that calls just opened.
|
||||
|
||||
Slack's `maxSeverity` defaults to `info` whenever PagerDuty is configured, and
|
||||
to `critical` when it isn't — a node with only Slack still sees everything.
|
||||
Raise it to have chat mirror the paging tiers as well.
|
||||
to `critical` when it isn't — a node with only a webhook still sees
|
||||
everything. Raise it to have chat mirror the paging tiers as well.
|
||||
|
||||
### Retiering without a deploy
|
||||
|
||||
|
||||
@@ -64,7 +64,6 @@ Type the method in JSDoc on the implementation, following the [puter.js type rul
|
||||
|
||||
- Add the method page at [src/docs/src/](../src/docs/src/)`<Area>/<method>.md` — frontmatter (`title`, `description`, `platforms`), syntax, parameters, return value, and at least one runnable example — and update the area overview (`<Area>.md`). Copy the structure of an existing page.
|
||||
- The docs are the contract users code against: signatures, defaults, and return shapes match the implementation exactly.
|
||||
- Add a bullet to [src/docs/src/changelog.md](../src/docs/src/changelog.md) under the day the PR merges, per the [changelog rules](../AGENTS.md#changelog).
|
||||
|
||||
### 6. Tests
|
||||
|
||||
@@ -102,7 +101,6 @@ The old surface keeps working. Mark it `@deprecated` in the type declarations, n
|
||||
- [ ] puter.js method matches sibling conventions; errors are `{ message, code }` with stable codes
|
||||
- [ ] Types updated and matching runtime behavior
|
||||
- [ ] Docs page + area overview updated, with a runnable example
|
||||
- [ ] Changelog entry added
|
||||
- [ ] Tests: backend + three-platform SDK suite (+ e2e for desktop-rendered UI)
|
||||
- [ ] Security pass on the diff
|
||||
- [ ] You ran it end to end
|
||||
@@ -17,8 +17,7 @@ destination with no opener.
|
||||
The embed follows the OS's light or dark setting. A host with a fixed theme adds
|
||||
`theme=dark` or `theme=light` to the fragment (`#user=<uuid>&theme=dark`), since
|
||||
Chromium doesn't pass an iframe's `color-scheme` on to the page inside it.
|
||||
Apps opened from the embed carry `utm_medium=referral&utm_campaign=app_browser_embed&utm_content=app_icon`,
|
||||
with `utm_source` set to the framing page's host (`puter.com` when no allowed page frames it);
|
||||
Apps opened from the embed carry `utm_source=puter.com&utm_medium=referral&utm_campaign=app_browser_embed`;
|
||||
the desktop and dashboard send `utm_campaign=desktop` and `utm_campaign=dashboard`.
|
||||
|
||||
## Session and framing
|
||||
@@ -226,12 +225,6 @@ pickers) for `puter.ui.requestUpgrade(details)`, and for the Upgrade button of
|
||||
the usage limit dialog a refused call shows there. Elsewhere both keep their
|
||||
behavior: inside an app `requestUpgrade()` asks the desktop, and on any other
|
||||
page the usage limit dialog opens the dashboard's upgrade page in a new tab.
|
||||
It frames it too for the Upgrade button `<puter-user-menu>` shows unless
|
||||
`hide-upgrade` is set, with only `user` in the fragment. On a page that can't frame
|
||||
the GUI, that button opens the dashboard's upgrade page in a new tab. Signed
|
||||
out, the button signs in first, and the upgrade then shows for that account,
|
||||
through the consent dialog on a page that can't frame it, since the click has
|
||||
expired by then.
|
||||
|
||||
The frame covers the page with no box or close button, and the GUI page is
|
||||
transparent (`html.popup-frame-transparent`) so the dialog's own scrim covers
|
||||
@@ -264,9 +257,7 @@ it shows still rejects but opens no usage limit dialog over it.
|
||||
When the frame can't show the upgrade (refused, never ready, `unavailable`, or
|
||||
no user to name), puter.js opens the dashboard's upgrade page in a new tab with
|
||||
what was refused, through a consent dialog when the click has expired or the
|
||||
tab is blocked, and `requestUpgrade()` resolves `{ upgraded: false }`. The
|
||||
tab's URL always carries `upgrade`, empty when nothing was refused, which is
|
||||
what the dashboard opens its upgrade flow for.
|
||||
tab is blocked, and `requestUpgrade()` resolves `{ upgraded: false }`.
|
||||
|
||||
## Top-up
|
||||
|
||||
@@ -280,11 +271,6 @@ shows only when whoami reports `subscribed: true` and no `team` paying for the
|
||||
account. On a page that can't frame the GUI, the item opens the dashboard with
|
||||
`?topup=` in a new tab instead.
|
||||
|
||||
The Low Balance dialog puter.js shows on a website when a call is refused for
|
||||
credit offers the same top-up to the same accounts: **Add Credits** is its main
|
||||
button, with **Upgrade Plan** beside it. Everyone else gets **Upgrade Now**
|
||||
alone.
|
||||
|
||||
The GUI reports `unavailable` unless its session belongs to `user`, the account
|
||||
has no verification or password change waiting, and the deployment defines
|
||||
`window.UITopUp` with `UITopUp.embeddable === true`. Otherwise it runs:
|
||||
|
||||
+1
-22
@@ -326,7 +326,7 @@ Used for password resets, email confirmation, and notifications. Without it thos
|
||||
|
||||
To require email confirmation before login, also set `"strict_email_verification_required": true`.
|
||||
|
||||
Account and notification mail is sent on a named stream: `auth` (password, address and 2FA changes, new sign-in and exposed-password notices), `signup` (address confirmation; a `puter.signup.validate` listener may name another: lowercase letters, digits, `:`, `_` and `-`, up to 64 characters, starting with a letter or digit) and `notify` (share, app feedback and contact form mail). Add `"streams"` inside `"email"` to configure one, for instance to let your mail provider route or report each kind of mail separately:
|
||||
Account and notification mail is sent on a named stream: `auth` (password, address and 2FA changes), `signup` (address confirmation; a `puter.signup.validate` listener may name another: lowercase letters, digits, `:`, `_` and `-`, up to 64 characters, starting with a letter or digit) and `notify` (share, app feedback and contact form mail). Add `"streams"` inside `"email"` to configure one, for instance to let your mail provider route or report each kind of mail separately:
|
||||
|
||||
```json
|
||||
"streams": {
|
||||
@@ -503,27 +503,6 @@ set as an account's email.
|
||||
"min_pass_length": 12
|
||||
```
|
||||
|
||||
### Sign-in protection
|
||||
|
||||
A sign-in to an account with no activity for `dormantSignInDays` days (default `180`, `0` turns it off) counts as dormant: an unconfirmed email on file has to be confirmed again when `"email"` is configured (a code goes to it), and the owner gets a "new sign-in" email. Team seats and guest accounts never count as dormant. The owner also gets that email when an account that has signed in before is signed in to from a browser it hasn't seen, at most once an hour.
|
||||
|
||||
```json
|
||||
"dormantSignInDays": 180
|
||||
```
|
||||
|
||||
To check passwords against known breach data at sign-in, turn on the Pwned Passwords range lookup. Only the first five hex characters of the password's SHA-1 hash leave the server, the check runs after the sign-in answers, and a failing or slow API never blocks a sign-in. A hit emails the owner (at most once per password and once in 30 days).
|
||||
|
||||
```json
|
||||
"pwnedPasswords": { "enabled": true, "timeoutMs": 2000 }
|
||||
```
|
||||
|
||||
A proxy in front of Puter can flag requests with headers. `requestSignalHeaders` maps each header to the signal it raises, and a header counts only when it holds the `requestSignalValue` secret, so clients can't send one themselves (the proxy should also strip these headers from client requests). Without `requestSignalValue`, signals are off and the server logs a warning at startup. Signals only ever count against the caller. Extensions see them on signed-in requests as the `puter.request.flagged` event, and the signal `credential_exposed` on `POST /login` reports the password as exposed once it verifies.
|
||||
|
||||
```json
|
||||
"requestSignalHeaders": { "x-proxy-flag-bot": "bot", "x-proxy-flag-leaked": "credential_exposed" },
|
||||
"requestSignalValue": "<random shared secret>"
|
||||
```
|
||||
|
||||
### Contact-form recipient
|
||||
|
||||
Where the in-app contact form posts. Defaults to `support@puter.com`.
|
||||
|
||||
@@ -215,27 +215,4 @@ describe('installedApps extension — handleInstalledApps', () => {
|
||||
|
||||
expect(Array.isArray(captured.body)).toBe(true);
|
||||
});
|
||||
|
||||
it('reads fractional and huge page or limit values as whole numbers', async () => {
|
||||
const user = await seedUser();
|
||||
for (let i = 0; i < 3; i++) {
|
||||
const app = await seedApp(user.id as number);
|
||||
await grantInstalled(app!.id as number, user.id as number);
|
||||
}
|
||||
const list = async (query: Record<string, string>) => {
|
||||
const { res, captured } = makeRes();
|
||||
await runWithContext(
|
||||
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
|
||||
() => handleInstalledApps(makeReq(query), res),
|
||||
);
|
||||
return captured.body as unknown[];
|
||||
};
|
||||
|
||||
expect(await list({ limit: '0.5' })).toHaveLength(1);
|
||||
expect(await list({ limit: '2.9' })).toHaveLength(2);
|
||||
expect(await list({ page: '2.5', limit: '2' })).toHaveLength(1);
|
||||
for (const page of ['Infinity', '1e400']) {
|
||||
expect(await list({ page })).toHaveLength(3);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -30,14 +30,8 @@ export const handleInstalledApps = async (
|
||||
);
|
||||
}
|
||||
|
||||
const requestedPage = Math.floor(Number(req.query.page));
|
||||
// Far past any real list; keeps the offset an exact integer.
|
||||
const page = Number.isSafeInteger(requestedPage)
|
||||
? Math.min(Math.max(requestedPage, 1), 1_000_000)
|
||||
: 1;
|
||||
const limit = Math.floor(
|
||||
Math.min(Math.max(Number(req.query.limit) || 100, 1), 100),
|
||||
);
|
||||
const page = Math.max(Number(req.query.page) || 1, 1);
|
||||
const limit = Math.min(Math.max(Number(req.query.limit) || 100, 1), 100);
|
||||
const offset = (page - 1) * limit;
|
||||
const installedApps = await stores.permission.listAppsGrantedByUser(
|
||||
actor.user.id!,
|
||||
|
||||
@@ -349,54 +349,6 @@ describe('thumbnails extension — handleThumbnailRead', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('reuses a URL signed this hour and signs afresh under a new signing key', async () => {
|
||||
// Short-lived credentials are fetched again on every signing, so a
|
||||
// swapped secret shows whether a read was signed or served from cache.
|
||||
const creds = { accessKeyId: 'AKIDFIRST', secretAccessKey: 'one' };
|
||||
const client = new S3Client({
|
||||
region: 'us-west-2',
|
||||
endpoint: 'http://127.0.0.1:4566',
|
||||
forcePathStyle: true,
|
||||
credentials: async () => ({
|
||||
...creds,
|
||||
expiration: new Date(Date.now() + 60_000),
|
||||
}),
|
||||
});
|
||||
const keys = [mintedKey(), mintedKey(), mintedKey()];
|
||||
const read = async (key: string) => {
|
||||
const entry: Record<string, unknown> = {
|
||||
thumbnail: `s3://${BUCKET}/${key}`,
|
||||
};
|
||||
await handleThumbnailRead(entry, {
|
||||
s3,
|
||||
s3Presign: client,
|
||||
bucketName: BUCKET,
|
||||
bucketEndpoint: BUCKET_ENDPOINT,
|
||||
fsEntry: server.stores.fsEntry,
|
||||
});
|
||||
return entry.thumbnail as string;
|
||||
};
|
||||
const listing = () => Promise.all(keys.map(read));
|
||||
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
try {
|
||||
vi.setSystemTime(Date.UTC(2026, 9, 26, 10, 30));
|
||||
// A cold client's first listing signs its entries concurrently.
|
||||
const first = await listing();
|
||||
creds.secretAccessKey = 'two';
|
||||
expect(await listing()).toEqual(first);
|
||||
|
||||
creds.accessKeyId = 'AKIDSECOND';
|
||||
const rotated = await read(keys[0]);
|
||||
expect(rotated).not.toBe(first[0]);
|
||||
expect(
|
||||
new URL(rotated).searchParams.get('X-Amz-Credential'),
|
||||
).toMatch(/^AKIDSECOND\//);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
// Thumbnails stored before keys were bound to entries keep displaying.
|
||||
it.each([
|
||||
['an entry-bound key', () => mintedKey()],
|
||||
@@ -1196,27 +1148,4 @@ describe('thumbnails extension — signed batch upload through /fs', () => {
|
||||
});
|
||||
expect(await objectExists(s3, copyKey)).toBe(true);
|
||||
});
|
||||
|
||||
it('deletes the thumbnail object an overwrite drops', async () => {
|
||||
const { actor, username } = await makeActor();
|
||||
const s3 = server.clients.s3.get();
|
||||
const path = `/${username}/Documents/edited.png`;
|
||||
const { entry } = await upload(actor, path, (s) => s.thumbnailUrl);
|
||||
const key = entry.thumbnail!.slice(`s3://${BUCKET}/`.length);
|
||||
expect(await objectExists(s3, key)).toBe(true);
|
||||
|
||||
// An app saving over the file sends no thumbnail.
|
||||
await server.services.fs.write(actor.user!.id!, {
|
||||
fileMetadata: { path, size: 4, overwrite: true },
|
||||
fileContent: 'efgh',
|
||||
} as never);
|
||||
|
||||
expect(
|
||||
(await server.stores.fsEntry.getEntryByPath(path))?.thumbnail,
|
||||
).toBeNull();
|
||||
// `thumbnail.discarded` is fire-and-forget.
|
||||
await vi.waitFor(async () => {
|
||||
expect(await objectExists(s3, key)).toBe(false);
|
||||
});
|
||||
});
|
||||
});
|
||||
+10
-60
@@ -9,7 +9,6 @@ import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
|
||||
import { extension } from '@heyputer/backend/src/extensions';
|
||||
import type { FSEntryStore } from '@heyputer/backend/src/stores/fs/FSEntryStore';
|
||||
import { isMissingObjectError } from '@heyputer/backend/src/stores/fs/S3ObjectStore';
|
||||
import { BoundedTtlMap } from '@heyputer/backend/src/util/boundedTtlMap.js';
|
||||
import crypto from 'node:crypto';
|
||||
import type { Readable } from 'node:stream';
|
||||
import sharp from 'sharp';
|
||||
@@ -295,44 +294,6 @@ export const handleThumbnailUploadPrepare = async (
|
||||
}
|
||||
};
|
||||
|
||||
const SIGNING_PERIOD_MS = 3_600_000;
|
||||
const SIGNED_URL_CACHE_ENTRIES = 5_000;
|
||||
// Read URLs signed this hour, per signing client. A URL is the same all hour,
|
||||
// so a repeat listing reuses it; the key names the signing key too, so a
|
||||
// credential rotation signs afresh.
|
||||
const signedReadUrls = new WeakMap<S3Client, BoundedTtlMap<string, string>>();
|
||||
|
||||
const signThumbnailRead = async (
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
): Promise<string> => {
|
||||
// Registered before any await, so a listing's concurrent signings share it.
|
||||
let cache = signedReadUrls.get(client);
|
||||
if (!cache) {
|
||||
cache = new BoundedTtlMap({
|
||||
maxEntries: SIGNED_URL_CACHE_ENTRIES,
|
||||
ttlMs: SIGNING_PERIOD_MS,
|
||||
});
|
||||
signedReadUrls.set(client, cache);
|
||||
}
|
||||
const period = Math.floor(Date.now() / SIGNING_PERIOD_MS);
|
||||
const { accessKeyId } = await client.config.credentials();
|
||||
const cacheKey = `${accessKeyId}\0${period}\0${bucket}\0${key}`;
|
||||
const cached = cache.get(cacheKey);
|
||||
if (cached) return cached;
|
||||
const url = await getSignedUrl(
|
||||
client,
|
||||
new GetObjectCommand({ Bucket: bucket, Key: key }),
|
||||
{
|
||||
expiresIn: 604800,
|
||||
signingDate: new Date(period * SIGNING_PERIOD_MS),
|
||||
},
|
||||
);
|
||||
cache.set(cacheKey, url);
|
||||
return url;
|
||||
};
|
||||
|
||||
export const handleThumbnailRead = async (
|
||||
entry: Record<string, unknown>,
|
||||
deps: {
|
||||
@@ -346,6 +307,9 @@ export const handleThumbnailRead = async (
|
||||
const thumb = entry.thumbnail;
|
||||
if (typeof thumb !== 'string' || !thumb) return;
|
||||
const presignClient = deps.s3Presign;
|
||||
const signingDate = new Date(
|
||||
Math.floor(Date.now() / 3_600_000) * 3_600_000,
|
||||
);
|
||||
|
||||
if (isStoragePointer(thumb, deps.bucketEndpoint)) {
|
||||
const key = resolveThumbnailKey(thumb);
|
||||
@@ -355,10 +319,10 @@ export const handleThumbnailRead = async (
|
||||
entry.thumbnail = null;
|
||||
return;
|
||||
}
|
||||
entry.thumbnail = await signThumbnailRead(
|
||||
entry.thumbnail = await getSignedUrl(
|
||||
presignClient,
|
||||
deps.bucketName,
|
||||
key,
|
||||
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
|
||||
{ expiresIn: 604800, signingDate },
|
||||
);
|
||||
} else if (thumb.startsWith('data')) {
|
||||
// Move an inline thumbnail into storage. The row only takes the new
|
||||
@@ -385,7 +349,7 @@ export const handleThumbnailRead = async (
|
||||
migrated = await deps.fsEntry.updateEntryThumbnailByUuid(
|
||||
uuid,
|
||||
`s3://${deps.bucketName}/${key}`,
|
||||
{ thumbnail: thumb },
|
||||
thumb,
|
||||
);
|
||||
} catch (err) {
|
||||
// The row may or may not point at the object now, so keep it.
|
||||
@@ -409,10 +373,10 @@ export const handleThumbnailRead = async (
|
||||
return;
|
||||
}
|
||||
|
||||
entry.thumbnail = await signThumbnailRead(
|
||||
entry.thumbnail = await getSignedUrl(
|
||||
presignClient,
|
||||
deps.bucketName,
|
||||
key,
|
||||
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
|
||||
{ expiresIn: 604800, signingDate },
|
||||
);
|
||||
}
|
||||
};
|
||||
@@ -570,17 +534,3 @@ extension.on('fs.remove.node', async (_key, payload) => {
|
||||
bucketEndpoint: extensionBucketEndpoint,
|
||||
});
|
||||
});
|
||||
|
||||
// -- thumbnail.discarded ---------------------------------------------
|
||||
// An overwrite or a newer thumbnail replaced this one; delete its object.
|
||||
|
||||
extension.on('thumbnail.discarded', async (_key, { uuid, url }) => {
|
||||
await handleFsRemoveNodeThumbnail(
|
||||
{ target: { uuid, thumbnail: url } },
|
||||
{
|
||||
s3: getClient(),
|
||||
bucketName: thumbnailBucketName,
|
||||
bucketEndpoint: extensionBucketEndpoint,
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -20,32 +20,13 @@ const state = vi.hoisted(() => ({
|
||||
writes: [] as Array<{
|
||||
userId: number;
|
||||
path: string;
|
||||
size: number;
|
||||
homeRegion?: string;
|
||||
content?: unknown;
|
||||
}>,
|
||||
feedback: [] as Array<{ to: string; from: string | null; raw: Buffer }>,
|
||||
takeFeedback: true,
|
||||
validated: [] as Array<ValidateEvent>,
|
||||
/** What the `email.ingress.validate` listener does, or null for none. */
|
||||
onValidate: null as null | ((event: ValidateEvent) => unknown),
|
||||
}));
|
||||
|
||||
type ValidateEvent = {
|
||||
to: string;
|
||||
envelopeFrom: string | null;
|
||||
head: Buffer;
|
||||
size: number;
|
||||
readMessage: () => Promise<Buffer>;
|
||||
allow: boolean;
|
||||
message: string | null;
|
||||
senderAuth: {
|
||||
result: 'pass' | 'fail' | 'temperror';
|
||||
fromDomains: string[];
|
||||
signingDomains: string[];
|
||||
} | null;
|
||||
};
|
||||
|
||||
vi.mock('@heyputer/backend/src/extensions', () => ({
|
||||
extension: {
|
||||
get config() {
|
||||
@@ -64,39 +45,16 @@ vi.mock('@heyputer/backend/src/extensions', () => ({
|
||||
event: {
|
||||
async emitAndWait(
|
||||
key: string,
|
||||
data: unknown,
|
||||
meta: { listener_failed?: boolean },
|
||||
opts: { revertOnFailure?: readonly string[] } = {},
|
||||
) {
|
||||
if (key === 'email.ingress.validate') {
|
||||
state.validated.push(data as ValidateEvent);
|
||||
const fields = data as Record<string, unknown>;
|
||||
const before = (opts.revertOnFailure ?? []).map(
|
||||
(name) => [name, fields[name]] as const,
|
||||
);
|
||||
try {
|
||||
await state.onValidate?.(
|
||||
data as ValidateEvent,
|
||||
);
|
||||
} catch {
|
||||
// What the event client does with a
|
||||
// listener that throws.
|
||||
for (const [name, value] of before) {
|
||||
fields[name] = value;
|
||||
}
|
||||
meta.listener_failed = true;
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (key !== 'email.ingress.feedback') return;
|
||||
const feedback = data as {
|
||||
data: {
|
||||
to: string;
|
||||
from: string | null;
|
||||
raw: Buffer;
|
||||
handled: boolean;
|
||||
};
|
||||
state.feedback.push(feedback);
|
||||
if (state.takeFeedback) feedback.handled = true;
|
||||
},
|
||||
) {
|
||||
if (key !== 'email.ingress.feedback') return;
|
||||
state.feedback.push(data);
|
||||
if (state.takeFeedback) data.handled = true;
|
||||
},
|
||||
},
|
||||
};
|
||||
@@ -117,7 +75,7 @@ vi.mock('@heyputer/backend/src/extensions', () => ({
|
||||
fileMetadata,
|
||||
fileContent,
|
||||
}: {
|
||||
fileMetadata: { path: string; size: number };
|
||||
fileMetadata: { path: string };
|
||||
fileContent: unknown;
|
||||
},
|
||||
_uploadTracker?: unknown,
|
||||
@@ -127,7 +85,6 @@ vi.mock('@heyputer/backend/src/extensions', () => ({
|
||||
state.writes.push({
|
||||
userId,
|
||||
path: fileMetadata.path,
|
||||
size: fileMetadata.size,
|
||||
homeRegion,
|
||||
content: fileContent,
|
||||
});
|
||||
@@ -157,19 +114,12 @@ const temp = (id: number, username: string) => ({
|
||||
|
||||
const ingress = async (
|
||||
to: string,
|
||||
over: {
|
||||
secret?: string;
|
||||
contentLength?: string | null;
|
||||
body?: string;
|
||||
query?: Record<string, string>;
|
||||
} = {},
|
||||
over: { secret?: string; contentLength?: string | null } = {},
|
||||
) => {
|
||||
const route = state.routes.find((r) => r.path === '/email/ingress');
|
||||
if (!route) throw new Error('route not registered');
|
||||
|
||||
const body = Buffer.from(
|
||||
over.body ?? 'From: [email protected]\r\n\r\nhello\r\n',
|
||||
);
|
||||
const body = Buffer.from('From: [email protected]\r\n\r\nhello\r\n');
|
||||
const req = Readable.from([body], { objectMode: false }) as unknown as {
|
||||
headers: Record<string, string>;
|
||||
query: Record<string, string>;
|
||||
@@ -187,12 +137,10 @@ const ingress = async (
|
||||
to,
|
||||
from: '[email protected]',
|
||||
subject: 'hi',
|
||||
...over.query,
|
||||
};
|
||||
|
||||
let status = 200;
|
||||
let ended = false;
|
||||
let responseBody: string | undefined;
|
||||
// `end` takes (chunk, encoding) and no callback, the way the compression
|
||||
// middleware leaves it, and emits `finish` — which is what the refusal
|
||||
// paths hang their drain/destroy on. A stub that accepts a callback
|
||||
@@ -205,7 +153,6 @@ const ingress = async (
|
||||
},
|
||||
end(chunk?: unknown, encoding?: BufferEncoding) {
|
||||
if (chunk) Buffer.byteLength(chunk as string, encoding);
|
||||
if (typeof chunk === 'string') responseBody = chunk;
|
||||
ended = true;
|
||||
emitter.emit('finish');
|
||||
},
|
||||
@@ -219,30 +166,16 @@ const ingress = async (
|
||||
return {
|
||||
status,
|
||||
ended,
|
||||
body: responseBody,
|
||||
drained: (req as unknown as Readable).readableFlowing,
|
||||
destroyed: (req as unknown as Readable).destroyed,
|
||||
};
|
||||
};
|
||||
|
||||
/** The bytes the nth write stored, whether handed over whole or as a stream. */
|
||||
const stored = async (n: number): Promise<string> => {
|
||||
const content = state.writes[n].content;
|
||||
if (Buffer.isBuffer(content)) return content.toString('latin1');
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of content as Readable) {
|
||||
chunks.push(Buffer.from(chunk));
|
||||
}
|
||||
return Buffer.concat(chunks).toString('latin1');
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
state.users.clear();
|
||||
state.writes.length = 0;
|
||||
state.feedback.length = 0;
|
||||
state.takeFeedback = true;
|
||||
state.validated.length = 0;
|
||||
state.onValidate = null;
|
||||
});
|
||||
|
||||
describe('temporary accounts cannot receive', () => {
|
||||
@@ -420,207 +353,3 @@ describe('feedback addresses', () => {
|
||||
expect(state.feedback).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the sender check', () => {
|
||||
// Two forged copies of the header only the ingress may write: one with
|
||||
// whitespace before its colon, one folded over two lines.
|
||||
const FORGED = [
|
||||
'From: [email protected]',
|
||||
'X-Puter-Sender-Auth: pass; from=puter.com; signed-by=puter.com',
|
||||
'Subject: hi',
|
||||
'x-puter-sender-auth \t: pass',
|
||||
' ; from=puter.com',
|
||||
'',
|
||||
'hello',
|
||||
'',
|
||||
].join('\r\n');
|
||||
const CLEAN = 'From: [email protected]\r\nSubject: hi\r\n\r\nhello\r\n';
|
||||
|
||||
test('a copy of its header the sender wrote is stripped, checked or not', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
|
||||
const { status } = await ingress('[email protected]', { body: FORGED });
|
||||
expect(status).toBe(200);
|
||||
const bytes = await stored(0);
|
||||
expect(bytes).toBe(CLEAN);
|
||||
expect(state.writes[0].size).toBe(Buffer.byteLength(bytes, 'latin1'));
|
||||
});
|
||||
|
||||
test("a listener's outcome is the one copy the stored message carries", async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
state.onValidate = (event) => {
|
||||
event.senderAuth = {
|
||||
result: 'fail',
|
||||
fromDomains: ['puter.com'],
|
||||
signingDomains: [],
|
||||
};
|
||||
};
|
||||
|
||||
await ingress('[email protected]', { body: FORGED });
|
||||
const bytes = await stored(0);
|
||||
expect(bytes).toBe(
|
||||
`X-Puter-Sender-Auth: fail; from=puter.com; signed-by=none\r\n${CLEAN}`,
|
||||
);
|
||||
expect(state.writes[0].size).toBe(Buffer.byteLength(bytes, 'latin1'));
|
||||
});
|
||||
|
||||
test('a listener reading the whole message gets it as received, and the copy filed is built from it', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
let seen = '';
|
||||
state.onValidate = async (event) => {
|
||||
const [first, second] = await Promise.all([
|
||||
event.readMessage(),
|
||||
event.readMessage(),
|
||||
]);
|
||||
expect(second).toBe(first);
|
||||
seen = first.toString('latin1');
|
||||
event.senderAuth = {
|
||||
result: 'pass',
|
||||
fromDomains: ['puter.com'],
|
||||
signingDomains: ['puter.com'],
|
||||
};
|
||||
};
|
||||
|
||||
await ingress('[email protected]', { body: FORGED });
|
||||
expect(seen).toBe(FORGED);
|
||||
expect(Buffer.isBuffer(state.writes[0].content)).toBe(true);
|
||||
expect(await stored(0)).toBe(
|
||||
`X-Puter-Sender-Auth: pass; from=puter.com; signed-by=puter.com\r\n${CLEAN}`,
|
||||
);
|
||||
});
|
||||
|
||||
test('the listener sees the envelope, the head as received and the size', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
|
||||
await ingress('[email protected]', {
|
||||
body: FORGED,
|
||||
query: { mailfrom: '[email protected]' },
|
||||
});
|
||||
const [event] = state.validated;
|
||||
expect(event.to).toBe('[email protected]');
|
||||
expect(event.envelopeFrom).toBe('[email protected]');
|
||||
expect(event.head.toString('latin1')).toBe(
|
||||
FORGED.slice(0, FORGED.indexOf('\r\n\r\n') + 4),
|
||||
);
|
||||
expect(event.size).toBe(Buffer.byteLength(FORGED));
|
||||
});
|
||||
|
||||
test('mail with nothing to strip or record streams through untouched', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
|
||||
await ingress('[email protected]', { body: CLEAN });
|
||||
expect(Buffer.isBuffer(state.writes[0].content)).toBe(false);
|
||||
expect(await stored(0)).toBe(CLEAN);
|
||||
expect(state.writes[0].size).toBe(Buffer.byteLength(CLEAN));
|
||||
});
|
||||
|
||||
test('a listener that fails is no opinion: the message is filed, unmarked, and that is logged', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
state.onValidate = (event) => {
|
||||
// Whatever it set before failing is undone.
|
||||
event.allow = false;
|
||||
event.senderAuth = {
|
||||
result: 'pass',
|
||||
fromDomains: ['puter.com'],
|
||||
signingDomains: ['puter.com'],
|
||||
};
|
||||
throw new Error('verifier down');
|
||||
};
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
try {
|
||||
const { status } = await ingress('[email protected]', {
|
||||
body: FORGED,
|
||||
});
|
||||
expect(status).toBe(200);
|
||||
expect(await stored(0)).toBe(CLEAN);
|
||||
expect(warn).toHaveBeenCalledTimes(1);
|
||||
expect(String(warn.mock.calls[0][0])).toContain(
|
||||
'email.ingress.validate',
|
||||
);
|
||||
} finally {
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
test('a forged copy a lenient reader still reads as the header is stripped too', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
const body = [
|
||||
' X-Puter-Sender-Auth: pass; from=puter.com',
|
||||
'From: [email protected]',
|
||||
'X-Puter-Sender-Auth',
|
||||
' : pass; from=puter.com',
|
||||
'X-PUTER-SENDER-AUTH',
|
||||
': pass; from=puter.com',
|
||||
'Subject: hi',
|
||||
'',
|
||||
'hello',
|
||||
'',
|
||||
].join('\r\n');
|
||||
|
||||
await ingress('[email protected]', { body });
|
||||
expect(await stored(0)).toBe(CLEAN);
|
||||
});
|
||||
|
||||
test('a refusal is a 422 carrying one line of its reason, and nothing is filed', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
state.onValidate = (event) => {
|
||||
event.allow = false;
|
||||
event.message =
|
||||
'Unauthenticated mail from puter.com is not accepted\r\nX-Injected: 1';
|
||||
};
|
||||
|
||||
const { status, body, drained } = await ingress('[email protected]', {
|
||||
body: FORGED,
|
||||
});
|
||||
expect(status).toBe(422);
|
||||
expect(body).toBe(
|
||||
'Unauthenticated mail from puter.com is not accepted X-Injected: 1',
|
||||
);
|
||||
expect(drained).toBe(true);
|
||||
expect(state.writes).toEqual([]);
|
||||
});
|
||||
|
||||
test('a refusal with no reason still says why it failed', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
state.onValidate = (event) => {
|
||||
event.allow = false;
|
||||
};
|
||||
|
||||
const { status, body } = await ingress('[email protected]');
|
||||
expect(status).toBe(422);
|
||||
expect(body).toBe('Message refused');
|
||||
});
|
||||
|
||||
test('a header block past the cap is refused with 413', async () => {
|
||||
state.users.set('bob', permanent(2, 'bob'));
|
||||
|
||||
const { status, destroyed } = await ingress('[email protected]', {
|
||||
body: `From: [email protected]\r\nX-Pad: ${'a'.repeat(300 * 1024)}\r\n\r\nhello`,
|
||||
});
|
||||
expect(status).toBe(413);
|
||||
expect(destroyed).toBe(true);
|
||||
expect(state.validated).toEqual([]);
|
||||
expect(state.writes).toEqual([]);
|
||||
});
|
||||
|
||||
test('feedback nobody took is held to the same header cap', async () => {
|
||||
state.takeFeedback = false;
|
||||
state.users.set('abuse', permanent(3, 'abuse'));
|
||||
|
||||
const { status } = await ingress('[email protected]', {
|
||||
body: `From: [email protected]\r\nX-Pad: ${'a'.repeat(300 * 1024)}\r\n\r\nhello`,
|
||||
});
|
||||
expect(status).toBe(413);
|
||||
expect(state.validated).toEqual([]);
|
||||
expect(state.writes).toEqual([]);
|
||||
});
|
||||
|
||||
test('feedback nobody took is stripped like any other mail', async () => {
|
||||
state.takeFeedback = false;
|
||||
state.users.set('abuse', permanent(3, 'abuse'));
|
||||
|
||||
await ingress('[email protected]', { body: FORGED });
|
||||
expect(await stored(0)).toBe(CLEAN);
|
||||
expect(state.validated[0].size).toBe(Buffer.byteLength(FORGED));
|
||||
});
|
||||
});
|
||||
@@ -1,22 +1,13 @@
|
||||
import { extension } from '@heyputer/backend/src/extensions';
|
||||
import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
|
||||
import type { Request } from 'express';
|
||||
import type { Readable } from 'node:stream';
|
||||
import {
|
||||
MAX_MESSAGE_BYTES,
|
||||
type SenderAuthOutcome,
|
||||
isPuterEmailAddress,
|
||||
isTempUser,
|
||||
puterEmailUsername,
|
||||
stampSenderAuth,
|
||||
storeInboxMessage,
|
||||
} from '@heyputer/backend/src/services/email/mailbox.js';
|
||||
import {
|
||||
MAX_HEADER_BYTES,
|
||||
headBlockOf,
|
||||
readHeadBlock,
|
||||
replaceHead,
|
||||
} from '@heyputer/backend/src/services/email/rawMessage.js';
|
||||
|
||||
// copied form peer secrets checker
|
||||
const COMPARE_KEY = randomBytes(32);
|
||||
@@ -53,16 +44,6 @@ const isFeedbackAddress = (to: string): boolean =>
|
||||
isPuterEmailAddress(to) &&
|
||||
FEEDBACK_ADDRESSES.has(puterEmailUsername(to).toLowerCase());
|
||||
|
||||
/**
|
||||
* A listener's refusal reason as one short line of printable ASCII: the relay
|
||||
* quotes it in an SMTP reply.
|
||||
*/
|
||||
const refusalText = (reason: string | null): string =>
|
||||
(reason ?? '')
|
||||
.replace(/[^\x20-\x7e]+/g, ' ')
|
||||
.trim()
|
||||
.slice(0, 200) || 'Message refused';
|
||||
|
||||
const readBody = async (req: Request): Promise<Buffer> => {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of req) chunks.push(chunk as Buffer);
|
||||
@@ -93,15 +74,11 @@ if (!INGRESS_SECRET) {
|
||||
// leaves the connection reusable. `close` is for bodies we do not
|
||||
// want to read at all: an unauthenticated caller's, or one whose
|
||||
// declared length is missing or over the cap.
|
||||
const refuse = (
|
||||
status: number,
|
||||
how: 'drain' | 'close',
|
||||
body?: string,
|
||||
) => {
|
||||
const refuse = (status: number, how: 'drain' | 'close') => {
|
||||
res.once('finish', () =>
|
||||
how === 'drain' ? req.resume() : req.destroy(),
|
||||
);
|
||||
res.status(status).end(body);
|
||||
res.status(status).end();
|
||||
};
|
||||
|
||||
const presented = req.query.SECRET;
|
||||
@@ -170,78 +147,14 @@ if (!INGRESS_SECRET) {
|
||||
return;
|
||||
}
|
||||
|
||||
let head: Buffer;
|
||||
if (Buffer.isBuffer(content)) {
|
||||
head = headBlockOf(content);
|
||||
if (head.length > MAX_HEADER_BYTES) {
|
||||
refuse(413, 'drain');
|
||||
return;
|
||||
}
|
||||
} else {
|
||||
const block = await readHeadBlock(req, MAX_HEADER_BYTES);
|
||||
if (!block) {
|
||||
refuse(413, 'close');
|
||||
return;
|
||||
}
|
||||
head = block.head;
|
||||
// A message that is all header block was read whole.
|
||||
if (block.spent) content = head;
|
||||
}
|
||||
|
||||
let reading: Promise<Buffer> | null = Buffer.isBuffer(content)
|
||||
? Promise.resolve(content)
|
||||
: null;
|
||||
const check = {
|
||||
to,
|
||||
envelopeFrom:
|
||||
typeof req.query.mailfrom === 'string' && req.query.mailfrom
|
||||
? req.query.mailfrom
|
||||
: null,
|
||||
head,
|
||||
size: Buffer.isBuffer(content) ? content.length : size,
|
||||
readMessage: () => (reading ??= readBody(req)),
|
||||
allow: true,
|
||||
message: null as string | null,
|
||||
senderAuth: null as SenderAuthOutcome | null,
|
||||
};
|
||||
const meta: { listener_failed?: boolean } = {};
|
||||
// A listener that throws is no opinion: what it set is undone and
|
||||
// the message is filed.
|
||||
await extension
|
||||
.import('client')
|
||||
.event.emitAndWait('email.ingress.validate', check, meta, {
|
||||
revertOnFailure: ['allow', 'message', 'senderAuth'],
|
||||
});
|
||||
if (meta.listener_failed) {
|
||||
console.warn(
|
||||
'[useremailrx] an email.ingress.validate listener failed; filing the message without its check',
|
||||
);
|
||||
}
|
||||
if (check.allow === false) {
|
||||
refuse(422, 'drain', refusalText(check.message));
|
||||
return;
|
||||
}
|
||||
|
||||
// The sender check's header is ours alone: any copy the sender
|
||||
// wrote goes, whether or not a check ran.
|
||||
const newHead = stampSenderAuth(head, check.senderAuth);
|
||||
|
||||
const whole = reading ? await reading : null;
|
||||
let stored: Buffer | Request | Readable = whole ?? req;
|
||||
let storedSize = check.size;
|
||||
if (newHead !== head) {
|
||||
stored = whole
|
||||
? Buffer.concat([newHead, whole.subarray(head.length)])
|
||||
: replaceHead(req, head.length, newHead);
|
||||
storedSize = check.size - head.length + newHead.length;
|
||||
}
|
||||
|
||||
await storeInboxMessage(services.fs, user, {
|
||||
subject: req.query.subject,
|
||||
content: stored,
|
||||
size: storedSize,
|
||||
content,
|
||||
size: Buffer.isBuffer(content) ? content.length : size,
|
||||
});
|
||||
|
||||
const { SECRET: _secret, ..._loggableQuery } = req.query;
|
||||
// console.log('got email: ', puterUser, loggableQuery, size);
|
||||
res.end();
|
||||
},
|
||||
);
|
||||
|
||||
@@ -597,57 +597,6 @@ describe('whoami extension — handleWhoami', () => {
|
||||
).toBe('fp_NESTED');
|
||||
});
|
||||
|
||||
describe('an actor token auth built from the account row', () => {
|
||||
const answerFor = async (
|
||||
password: string | null,
|
||||
email: string | null,
|
||||
) => {
|
||||
const slug = Math.random().toString(36).slice(2, 8);
|
||||
const row = await server.stores.user.create({
|
||||
username: `wrow_${slug}`,
|
||||
uuid: uuidv4(),
|
||||
password,
|
||||
email,
|
||||
});
|
||||
const actor = server.services.auth.buildPasswordActor(row);
|
||||
const getById = vi.spyOn(server.stores.user, 'getById');
|
||||
const { res, captured } = makeRes();
|
||||
try {
|
||||
await runWithContext({ actor }, () =>
|
||||
handleWhoami(makeReq(), res),
|
||||
);
|
||||
return {
|
||||
body: captured.body as Record<string, unknown>,
|
||||
reads: getById.mock.calls.length,
|
||||
};
|
||||
} finally {
|
||||
getById.mockRestore();
|
||||
}
|
||||
};
|
||||
|
||||
it('answers from that row without reading it again', async () => {
|
||||
const { body, reads } = await answerFor(
|
||||
'hashedpw',
|
||||
'[email protected]',
|
||||
);
|
||||
expect(reads).toBe(0);
|
||||
expect(body.username).toMatch(/^wrow_/);
|
||||
expect(body.oidc_only).toBe(false);
|
||||
expect(body.is_temp).toBe(false);
|
||||
expect(body).not.toHaveProperty('password');
|
||||
});
|
||||
|
||||
it('still tells a passwordless account and a temp account apart', async () => {
|
||||
const oidc = await answerFor(null, '[email protected]');
|
||||
expect(oidc.body.oidc_only).toBe(true);
|
||||
expect(oidc.body.is_temp).toBe(false);
|
||||
|
||||
const temp = await answerFor(null, null);
|
||||
expect(temp.body.oidc_only).toBe(true);
|
||||
expect(temp.body.is_temp).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it('marks the user as oidc_only when password is null', async () => {
|
||||
const slug = Math.random().toString(36).slice(2, 8);
|
||||
const oidcUser = await server.stores.user.create({
|
||||
|
||||
@@ -4,7 +4,6 @@ import {
|
||||
buildUserDetails,
|
||||
scrubSensitive,
|
||||
} from '@heyputer/backend/src/util/userDetails.js';
|
||||
import type { UserRow } from '@heyputer/backend/src/stores/user/UserStore.js';
|
||||
import type { Request, Response } from 'express';
|
||||
import TimeAgo from 'javascript-time-ago';
|
||||
import localeEn from 'javascript-time-ago/locale/en';
|
||||
@@ -48,12 +47,7 @@ export const handleWhoami = async (
|
||||
};
|
||||
// Started beside the row read: the actor carries the row's uuid.
|
||||
const emailCheck = isUser ? null : canReadEmail(actor.user.uuid);
|
||||
// Token auth loaded this request's row already; `hasPassword` marks an
|
||||
// actor built from it.
|
||||
const user =
|
||||
actor.user.hasPassword === undefined
|
||||
? await stores.user.getById(actor.user.id)
|
||||
: (actor.user as UserRow & { hasPassword: boolean });
|
||||
const user = await stores.user.getById(actor.user.id);
|
||||
if (!user) {
|
||||
res.status(404).json({ error: 'User not found' });
|
||||
return;
|
||||
|
||||
Generated
+92
-89
@@ -16,7 +16,7 @@
|
||||
"@aws-sdk/s3-request-presigner": "^3.1028.0",
|
||||
"dedent": "^1.5.3",
|
||||
"javascript-time-ago": "^2.5.11",
|
||||
"libphonenumber-js": "1.13.16",
|
||||
"libphonenumber-js": "1.13.15",
|
||||
"miniflare": "^4.20260617.1",
|
||||
"open": "^10.1.0",
|
||||
"sharp": "^0.35.5"
|
||||
@@ -3709,22 +3709,25 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@napi-rs/wasm-runtime": {
|
||||
"version": "1.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.6.tgz",
|
||||
"integrity": "sha512-ZLv/JdUfkvOy9eCnnBaGfiO+XimbjebAeO+MRQqD/B+FR1tnRN0tpKSJHRbE8sFfS6aqsXZ67TQjfwfsxULVbg==",
|
||||
"version": "1.2.3",
|
||||
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.2.3.tgz",
|
||||
"integrity": "sha512-UMduMbqO5s5zF2NkNacMT/yK5Y5QiKvWr2+50bzIIxFDwVJ2h49b+oyjaCGPhJxd2/gC2x39EHv/gHVuu36x2Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"@tybys/wasm-util": "^0.10.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": "^20.19.0 || ^22.13.0 || >=23.5.0"
|
||||
},
|
||||
"funding": {
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/Brooooooklyn"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@emnapi/core": "^1.7.1",
|
||||
"@emnapi/runtime": "^1.7.1"
|
||||
"@emnapi/core": "^1.7.1 || ^2.0.0-alpha.4",
|
||||
"@emnapi/runtime": "^1.7.1 || ^2.0.0-alpha.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@noble/hashes": {
|
||||
@@ -5307,9 +5310,9 @@
|
||||
"license": "BSD-3-Clause"
|
||||
},
|
||||
"node_modules/@rolldown/binding-android-arm64": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.1.8.tgz",
|
||||
"integrity": "sha512-yBy+BGoNYRh7qdsPTT8fj9nSrwEB/2LkIY9s0s9/PCsTNbrmJi0G0jS3PKJQ5NEbjnjbzkHqT7SN1ubuAykZ9w==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.1.5.tgz",
|
||||
"integrity": "sha512-lZg8fqIv2v7FF237bwMgzGZEJvGL79/s5knJ/i6FmsGF4XXlzccZ4jb+TrFIxtSSxFtIpdsgrPZeMk1I9AFcyQ==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -5324,9 +5327,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-darwin-arm64": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.1.8.tgz",
|
||||
"integrity": "sha512-U/K7P+QWIVDhPKZwky0hZU3tSDZ8CW9BQp96ebyc8A9R5WumAgJxnNAm/85HmYNBmdoCpliRrL90nk61lBE/UQ==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.1.5.tgz",
|
||||
"integrity": "sha512-51Bnx9pNiMRKSUNtBfySkNJ9vMU9Hh3I1ozDd6gyPPYzaXCfnptUcEZxXGYFn+ul2dtcMUiqGR1Yai2K10uoTw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -5341,9 +5344,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-darwin-x64": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.1.8.tgz",
|
||||
"integrity": "sha512-JBU0CYfyaKsTCJeTGVutZIY97nK+NqOXV534xiSiJI4Q60ZbxY8ouSb108pK9VRZXgIRg2g06dj5l8dHMXCtuw==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.1.5.tgz",
|
||||
"integrity": "sha512-Tm+gbfC0aHu1tBA/JvKQh32S0K6YgCHkiAF4/W6xX0K0RmNuc94VeK419dJoE65R5aRxmo+noZQSWrAMF6yb6g==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -5358,9 +5361,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-freebsd-x64": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.1.8.tgz",
|
||||
"integrity": "sha512-pZGUfncYS01m4QVGsZn686UDTAVvG+McN/l7y6NXYU7o1AG+yelH0aQmwD2ahNT7fJUQ9OOLNxKI4AUXTV+stw==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.1.5.tgz",
|
||||
"integrity": "sha512-JMzDKCCXq93YccG5gz3hvOs1oXRKAf0XYpfOS88e+wZrC8Iugj6j68867vrYZkvpDDpKn/KoKORThmchMpF6TA==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -5375,9 +5378,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.1.8.tgz",
|
||||
"integrity": "sha512-VXquzD6435ouqkDS6WaB5pMfMKNR6X/hW5d5VcImYrPKU26QGo8pa27+WBpT+qVCxLLOuNekevcIClz4fFY/Qg==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.1.5.tgz",
|
||||
"integrity": "sha512-uML21j2K5TfPGutKxub+M+nLjZIrWjXQ5Grx4lCe/nimTj9B4L63zHpjXLl4y0L3mcm2htEQIb06oCG/szerNw==",
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
@@ -5392,9 +5395,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-arm64-gnu": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.1.8.tgz",
|
||||
"integrity": "sha512-vv/iS1diZtqRuezB1SX01YOhn7zoR1So38V6JlpzSaPi84vkBTuNj9fOwo8PlEYewLu2UnSYfGJTWt82wE2zkQ==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.1.5.tgz",
|
||||
"integrity": "sha512-navSiuTMogvnQoZoM/v+l3ZWo50/NTwSHSzheABx/RCnmUPaKwq9qSo4Br2OYRs21+Fz8uFqITZM3H4opOB0/Q==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -5412,9 +5415,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-arm64-musl": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.1.8.tgz",
|
||||
"integrity": "sha512-aZnX74L/zcKgZ4LwoAfRLkysIBzU2bpNszR7y8UCkABiTlaUsAM90NAfsV/Wn6kDU9TwMWQsxiqDL7q4pzVv4Q==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.1.5.tgz",
|
||||
"integrity": "sha512-lAryqH7IteztmCXQXk0etKj4wBQ7Gx5S6LjKhsgp9zb8I5bsuvU/2llH1hDQcjsFeqIsovMVN339/8pUDDBXxA==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -5432,9 +5435,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.1.8.tgz",
|
||||
"integrity": "sha512-jv4AgWy8oNcXF4bqjlgpSB4mPzbitcA9DYusT3HKXRli/5+W2uBQCMrcC9v6IvyFgOTIcbck2gklp74wnogGYA==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.1.5.tgz",
|
||||
"integrity": "sha512-fsK/sNBnxzBlL4O1JNrZakVQxPspqpED5dLtNsZS9oOKmtSpdNIzxH2kkol5HYTWJN47sE20ztMJPxfZ89qGOg==",
|
||||
"cpu": [
|
||||
"ppc64"
|
||||
],
|
||||
@@ -5452,9 +5455,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-s390x-gnu": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.1.8.tgz",
|
||||
"integrity": "sha512-xsB9p1k2D7WBPInrjl2SrDbJcOA9OK58JRarNxy0EW7Eir2NFkpt/CqB0TSl5hOhZbMbaOz27F0MGFh8vnY8Lw==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.1.5.tgz",
|
||||
"integrity": "sha512-gLYb4BIadlfTOYT5gO503n8zQjXflgzpD0FcyKh0Mzx3rqCZKnHoJWV9xe1KXUJ5lx2JfcSHr/mhzS0PC/McAA==",
|
||||
"cpu": [
|
||||
"s390x"
|
||||
],
|
||||
@@ -5472,9 +5475,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-x64-gnu": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.1.8.tgz",
|
||||
"integrity": "sha512-TDdWNZPFqSwp103hVxTWQZi6iwOrwB+BJzw+Xsc0DjKrRI7HSfc2CxhafZi0cSa3Xb+Dkf88rvPaeRWgkvY8rA==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.1.5.tgz",
|
||||
"integrity": "sha512-FjcpEKUyJygHgs1o50VYNvkt5+7Le/VEdYt0AkRpkL33MnyQfwr8l5mXwMmfmTbyMPr5vJLC+8/Gd9gXnwU1QQ==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -5492,9 +5495,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-x64-musl": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.1.8.tgz",
|
||||
"integrity": "sha512-eh4qd+nhy+hBVGb/g+1c8iXMny985/kpyp5bcwfCiHd+G6+pbZAHhWfP7bDSQfe8rbKkU1BDAAu/yc3tSqKv+A==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.1.5.tgz",
|
||||
"integrity": "sha512-Me+PfPI2TMeOQk0gYWfLQZtTktrmzbr8cDboqX83XKc7UrgAi55gF+2dUkWdxd19n55Essp2yeca+O9N5rBxHg==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -5512,9 +5515,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-openharmony-arm64": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.1.8.tgz",
|
||||
"integrity": "sha512-Krg6+XkBZYLQLhTMwM9nLVCoawzwOBqsMP0ID2GsFMjmFCFg/lot8bUP1Dsup8hAfSPzbEekupVtAW3mJ1mydA==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.1.5.tgz",
|
||||
"integrity": "sha512-yc5WrLzXks6zCQfn9Oxr8pORKyl/pF+QjHmW/Qx3qu0oyrrNC+y2JLTU1E2rcWYAmzlnqngWXHQjy51VzW70Vw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -5529,9 +5532,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-wasm32-wasi": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.1.8.tgz",
|
||||
"integrity": "sha512-e9eQU3d7fcxLVgChl2ptJsPqBg+4R46n8u8PNm5iovRGFyEfsZdSpyHoVzwGr6pqJzByzgkh/PgQXYHriPr6Eg==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.1.5.tgz",
|
||||
"integrity": "sha512-VbQGPX2b4r48TAMIM2cjgluIM1HYutm4pcTEJsle7iEP7sB1dFqtPLBVbdLAZCxy1txCcPxf4QFf4v8uvltPqA==",
|
||||
"cpu": [
|
||||
"wasm32"
|
||||
],
|
||||
@@ -5541,7 +5544,7 @@
|
||||
"dependencies": {
|
||||
"@emnapi/core": "1.11.1",
|
||||
"@emnapi/runtime": "1.11.1",
|
||||
"@napi-rs/wasm-runtime": "1.1.6"
|
||||
"@napi-rs/wasm-runtime": "^1.1.6"
|
||||
},
|
||||
"engines": {
|
||||
"node": "^20.19.0 || >=22.12.0"
|
||||
@@ -5559,9 +5562,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-win32-arm64-msvc": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.1.8.tgz",
|
||||
"integrity": "sha512-s+I5YiLU2lUp4lw4WLPDjImKWRlGDhOZoXiNsDHyjXOJrt4X9NZGGP6cCC8Cu+DxSd4j0L6m1mImtr5qlGDx/g==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.1.5.tgz",
|
||||
"integrity": "sha512-gHv82k63z4qpV5+Q1y/12KrK0ltWBukVDI8nZcbT7Tt/ZlOIVwppazneq0F93oDxTo3IgAMEDIoQh3E2n6mVsw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -5576,9 +5579,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-win32-x64-msvc": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.1.8.tgz",
|
||||
"integrity": "sha512-nOq87ZXHflxsdEWeNTkD3KjqV5+QxvM/nRYyZlbnlpkxdlUvnifC+oM2hVxQ02SFztT7ReqoqMyvcM4T9T/BQg==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.1.5.tgz",
|
||||
"integrity": "sha512-tTZuDBPw85tEN5PQi1pnEBzDy0Z49HtScLAbD5t6hyeU92A95pRWaSMw1GZZi/RwgSgUIl0xrSlXIT/9QzvYSA==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -5593,9 +5596,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/pluginutils": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.1.1.tgz",
|
||||
"integrity": "sha512-glFIJDvLNDtJhT9qrMFWLPbvS+skxgt4L0K8Ei2YtnhPysAJnsI7nyxSJxKj4ADeKX0OLjC4ntRkwpLOoS3rwA==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.1.tgz",
|
||||
"integrity": "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
@@ -5842,9 +5845,9 @@
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@tybys/wasm-util": {
|
||||
"version": "0.10.4",
|
||||
"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.4.tgz",
|
||||
"integrity": "sha512-W3c4gRigFS0T/Ma4qIYF3GDAc5AQdHb1yL5znJT1Zv1YaD9Kitx656wBjvr19qbiosmZT8lWDM5BEMynUqX65A==",
|
||||
"version": "0.10.3",
|
||||
"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.3.tgz",
|
||||
"integrity": "sha512-F3fo1MYrRJYL3zER0OUOmkutjr1Vp23m7OsSgp7nq4SP6OqX6C/56XFIPAl5bt3zaBRjmW7SGz3u/6LwFpYcOg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
@@ -11501,9 +11504,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/libphonenumber-js": {
|
||||
"version": "1.13.16",
|
||||
"resolved": "https://registry.npmjs.org/libphonenumber-js/-/libphonenumber-js-1.13.16.tgz",
|
||||
"integrity": "sha512-tIOCM9yx2Go6d5/ZANe0K+G4AZek1Ymx81HxQpf19Sri4zMP95eQbKdnUGHAvFQ4nx0PeLhFqvAfisTADXj3tQ==",
|
||||
"version": "1.13.15",
|
||||
"resolved": "https://registry.npmjs.org/libphonenumber-js/-/libphonenumber-js-1.13.15.tgz",
|
||||
"integrity": "sha512-AHdhEQeDFEJ4GKDAQ9AwxChy+v2kUILUB3TGQRpkXMkWePjXTU/WQEzWVQoIx/Gz8fE73Wr5SEuZCy2vk8jrMQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/light-my-request": {
|
||||
@@ -14234,9 +14237,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/postal-mime": {
|
||||
"version": "4.2.1",
|
||||
"resolved": "https://registry.npmjs.org/postal-mime/-/postal-mime-4.2.1.tgz",
|
||||
"integrity": "sha512-uFQnu6TPJe5IGzj3ZIjzAYCk7qnVYM3KFvo0JpRFftJKtxJcUSMgTaWsIXDakTKfN71fUULhjNNVbNonim5f0Q==",
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/postal-mime/-/postal-mime-4.2.0.tgz",
|
||||
"integrity": "sha512-QG5Rh8wY/zjwFF1lKWcBbwWcAWR5jWodmMPL4gMik43XDLebtWoAAPyG0lrE2kYYZ5IHTASrepGuGO0YIwNKAg==",
|
||||
"license": "MIT-0",
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
@@ -14915,9 +14918,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/rolldown": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.1.8.tgz",
|
||||
"integrity": "sha512-YvKOji41VFBeXSo2yPDwZDI+0k4MY6zpRDDHjVdfeiVJB2KvOfGuXAKlLtrS2cgMNd9kCBxM9Z+gmuc0H84UOw==",
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.1.5.tgz",
|
||||
"integrity": "sha512-t9z29cJjXf/vxQ8dyhCSpt6H6aSwHTk8cT5I3iy6SMXuFpk5mB6PL6XfC8PCwrPTx93udwKUm9HRteAlTGBLiA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
@@ -14931,21 +14934,21 @@
|
||||
"node": "^20.19.0 || >=22.12.0"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@rolldown/binding-android-arm64": "1.1.8",
|
||||
"@rolldown/binding-darwin-arm64": "1.1.8",
|
||||
"@rolldown/binding-darwin-x64": "1.1.8",
|
||||
"@rolldown/binding-freebsd-x64": "1.1.8",
|
||||
"@rolldown/binding-linux-arm-gnueabihf": "1.1.8",
|
||||
"@rolldown/binding-linux-arm64-gnu": "1.1.8",
|
||||
"@rolldown/binding-linux-arm64-musl": "1.1.8",
|
||||
"@rolldown/binding-linux-ppc64-gnu": "1.1.8",
|
||||
"@rolldown/binding-linux-s390x-gnu": "1.1.8",
|
||||
"@rolldown/binding-linux-x64-gnu": "1.1.8",
|
||||
"@rolldown/binding-linux-x64-musl": "1.1.8",
|
||||
"@rolldown/binding-openharmony-arm64": "1.1.8",
|
||||
"@rolldown/binding-wasm32-wasi": "1.1.8",
|
||||
"@rolldown/binding-win32-arm64-msvc": "1.1.8",
|
||||
"@rolldown/binding-win32-x64-msvc": "1.1.8"
|
||||
"@rolldown/binding-android-arm64": "1.1.5",
|
||||
"@rolldown/binding-darwin-arm64": "1.1.5",
|
||||
"@rolldown/binding-darwin-x64": "1.1.5",
|
||||
"@rolldown/binding-freebsd-x64": "1.1.5",
|
||||
"@rolldown/binding-linux-arm-gnueabihf": "1.1.5",
|
||||
"@rolldown/binding-linux-arm64-gnu": "1.1.5",
|
||||
"@rolldown/binding-linux-arm64-musl": "1.1.5",
|
||||
"@rolldown/binding-linux-ppc64-gnu": "1.1.5",
|
||||
"@rolldown/binding-linux-s390x-gnu": "1.1.5",
|
||||
"@rolldown/binding-linux-x64-gnu": "1.1.5",
|
||||
"@rolldown/binding-linux-x64-musl": "1.1.5",
|
||||
"@rolldown/binding-openharmony-arm64": "1.1.5",
|
||||
"@rolldown/binding-wasm32-wasi": "1.1.5",
|
||||
"@rolldown/binding-win32-arm64-msvc": "1.1.5",
|
||||
"@rolldown/binding-win32-x64-msvc": "1.1.5"
|
||||
}
|
||||
},
|
||||
"node_modules/router": {
|
||||
@@ -16745,16 +16748,16 @@
|
||||
}
|
||||
},
|
||||
"node_modules/vite": {
|
||||
"version": "8.1.7",
|
||||
"resolved": "https://registry.npmjs.org/vite/-/vite-8.1.7.tgz",
|
||||
"integrity": "sha512-VUfShUOzJPHXz1o6YpAy92kkbuke3SWfEnHzFKg6oIc+tfXHgSiIz36RhKR4UP0FvJaY8Ku/NPFjMRodAccWWg==",
|
||||
"version": "8.1.6",
|
||||
"resolved": "https://registry.npmjs.org/vite/-/vite-8.1.6.tgz",
|
||||
"integrity": "sha512-aRAHIxx5SW5zc1sMqFwJU/t5qaFfXb5Hj0J7W0VXihu/X99+346tz6080tO1T1xVIJORX/b3zVTXGFqUXh35uw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"lightningcss": "^1.32.0",
|
||||
"picomatch": "^4.0.5",
|
||||
"postcss": "^8.5.17",
|
||||
"rolldown": "~1.1.7",
|
||||
"rolldown": "~1.1.5",
|
||||
"tinyglobby": "^0.2.17"
|
||||
},
|
||||
"bin": {
|
||||
@@ -18301,7 +18304,7 @@
|
||||
"openai": "^6.34.0",
|
||||
"otpauth": "^9.5.2",
|
||||
"pg": "^8.21.0",
|
||||
"postal-mime": "^4.2.1",
|
||||
"postal-mime": "^4.2.0",
|
||||
"replicate": "^1.0.0",
|
||||
"sharp": "^0.35.5",
|
||||
"smtp-server": "^3.19.24",
|
||||
@@ -18329,7 +18332,7 @@
|
||||
"@types/ws": "^8.18.2",
|
||||
"pgmock": "^1.0.3",
|
||||
"typescript": "^5.9.3",
|
||||
"vite": "^8.1.7",
|
||||
"vite": "^8.1.6",
|
||||
"vitest": "^4.0.14"
|
||||
}
|
||||
},
|
||||
@@ -19130,7 +19133,7 @@
|
||||
"engine.io-client": "~6.6.1",
|
||||
"open": "^10.2.0",
|
||||
"path-browserify": "1.0.1",
|
||||
"postal-mime": "^4.2.1",
|
||||
"postal-mime": "^4.2.0",
|
||||
"socket.io-client": "^4.8.4",
|
||||
"ws": "^8.23.0"
|
||||
},
|
||||
|
||||
+1
-1
@@ -79,7 +79,7 @@
|
||||
"@aws-sdk/s3-request-presigner": "^3.1028.0",
|
||||
"dedent": "^1.5.3",
|
||||
"javascript-time-ago": "^2.5.11",
|
||||
"libphonenumber-js": "1.13.16",
|
||||
"libphonenumber-js": "1.13.15",
|
||||
"miniflare": "^4.20260617.1",
|
||||
"open": "^10.1.0",
|
||||
"sharp": "^0.35.5"
|
||||
|
||||
@@ -5,7 +5,7 @@ import {
|
||||
MAX_TRACKED_ALARMS,
|
||||
} from './AlarmClient';
|
||||
import type { IConfig, IPagerConfig } from '../../types';
|
||||
import type { AlarmCall, AlertPayload } from './types';
|
||||
import type { AlertPayload } from './types';
|
||||
|
||||
const pdEvent = vi.hoisted(() => vi.fn(async () => ({})));
|
||||
vi.mock('@pagerduty/pdjs', () => ({ event: pdEvent }));
|
||||
@@ -210,10 +210,9 @@ describe('AlarmClient transport registration', () => {
|
||||
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
|
||||
expect(pdEvent).not.toHaveBeenCalled();
|
||||
|
||||
client.create('slow:thing', 'look at this today', {}, 'warning');
|
||||
client.create('loud:thing', 'paging', {}, 'critical');
|
||||
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
|
||||
// The pager has them; chat doesn't repeat them.
|
||||
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(1));
|
||||
// The pager has it; chat doesn't repeat it.
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
vi.unstubAllGlobals();
|
||||
@@ -300,22 +299,6 @@ describe('AlarmClient transport registration', () => {
|
||||
expect(keys[1]).toBe(keys[0]);
|
||||
});
|
||||
|
||||
it('refuses a Slack bot token without a default channel', async () => {
|
||||
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
|
||||
const client = makeClient({
|
||||
slack: { enabled: true, botToken: 'xoxb-test' },
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
client.create('boom', 'nowhere to send this', {}, 'info');
|
||||
await Promise.resolve();
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('skips transports that are enabled but not configured', async () => {
|
||||
const client = makeClient({
|
||||
pagerduty: { enabled: true },
|
||||
@@ -707,223 +690,3 @@ describe('AlarmClient repeat reporting', () => {
|
||||
expect(seen).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AlarmClient channel keys', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('carries the channel key on the payload', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('billing:stuck', 'stuck', {}, 'info', {
|
||||
channel: 'billing',
|
||||
});
|
||||
client.create('plain', 'no key', {}, 'info');
|
||||
|
||||
expect(seen[0].channel).toBe('billing');
|
||||
expect(seen[1]).not.toHaveProperty('channel');
|
||||
});
|
||||
|
||||
it('keeps the channel of the first occurrence', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('a', 'first', {}, 'info', { channel: 'billing' });
|
||||
client.create('a', 'second', {}, 'info', { channel: 'ai_providers' });
|
||||
|
||||
expect(seen.map((alert) => alert.channel)).toEqual([
|
||||
'billing',
|
||||
'billing',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AlarmClient calls', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const callClient = (pager: IPagerConfig = {}) => {
|
||||
const client = makeClient({
|
||||
...pager,
|
||||
calls: { rules: { 'billing:*': 'billing' }, ...pager.calls },
|
||||
});
|
||||
const calls: AlarmCall[] = [];
|
||||
client.addCallHandler(async (call) => {
|
||||
calls.push(call);
|
||||
});
|
||||
return { client, calls };
|
||||
};
|
||||
|
||||
it('hands a matching alarm to call handlers with the links transports returned', async () => {
|
||||
const { client, calls } = callClient();
|
||||
client.addAlertHandler(async () => ({ url: 'https://chat/msg/1' }));
|
||||
client.addAlertHandler(async () => {});
|
||||
|
||||
client.create('billing:stuck:tx1', 'top-up stuck', {}, 'info');
|
||||
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(1));
|
||||
expect(calls[0].group).toBe('billing');
|
||||
expect(calls[0].alert.callGroup).toBe('billing');
|
||||
expect(calls[0].links).toEqual(['https://chat/msg/1']);
|
||||
});
|
||||
|
||||
it('leaves alarms no rule names alone', async () => {
|
||||
const { client, calls } = callClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('ai:outage', 'provider down', {}, 'info');
|
||||
await Promise.resolve();
|
||||
|
||||
expect(calls).toHaveLength(0);
|
||||
expect(seen[0]).not.toHaveProperty('callGroup');
|
||||
});
|
||||
|
||||
it('does not resolve call groups before a call handler is registered', () => {
|
||||
const client = makeClient({
|
||||
calls: { rules: { 'billing:*': 'billing' } },
|
||||
});
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
|
||||
expect(seen[0]).not.toHaveProperty('callGroup');
|
||||
});
|
||||
|
||||
it('never calls for a muted alarm', async () => {
|
||||
const { client, calls } = callClient({
|
||||
severityOverrides: { 'billing:*': 'mute' },
|
||||
});
|
||||
capture(client);
|
||||
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
await Promise.resolve();
|
||||
|
||||
expect(calls).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('takes links only from transports that took the alarm', async () => {
|
||||
const { client, calls } = callClient();
|
||||
client.addAlertHandler(
|
||||
async () => ({ url: 'https://pager/incident/1' }),
|
||||
{
|
||||
minSeverity: 'warning',
|
||||
},
|
||||
);
|
||||
client.addAlertHandler(async () => ({ url: 'https://chat/msg/2' }), {
|
||||
maxSeverity: 'info',
|
||||
});
|
||||
|
||||
client.create('billing:outage', 'billing down', {}, 'critical');
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(2));
|
||||
const linksById = Object.fromEntries(
|
||||
calls.map((call) => [call.alert.id, call.links]),
|
||||
);
|
||||
expect(linksById).toEqual({
|
||||
'billing:outage': ['https://pager/incident/1'],
|
||||
'billing:stuck': ['https://chat/msg/2'],
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps the last links when a repeat is not delivered again', async () => {
|
||||
const { client, calls } = callClient();
|
||||
let posts = 0;
|
||||
client.addAlertHandler(async () => {
|
||||
posts++;
|
||||
// A throttled repeat returns nothing.
|
||||
return posts === 1 ? { url: 'https://chat/msg/3' } : undefined;
|
||||
});
|
||||
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(2));
|
||||
expect(calls[1].links).toEqual(['https://chat/msg/3']);
|
||||
});
|
||||
|
||||
it('still calls when a transport fails', async () => {
|
||||
const { client, calls } = callClient();
|
||||
client.addAlertHandler(async () => {
|
||||
throw new Error('chat is down');
|
||||
});
|
||||
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(1));
|
||||
expect(calls[0].links).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps an open call refreshed by repeats too frequent to report', async () => {
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
try {
|
||||
const { client, calls } = callClient();
|
||||
for (let i = 0; i < 10; i++) {
|
||||
client.create('billing:hot', 'hot', {}, 'info');
|
||||
}
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(10));
|
||||
|
||||
// Unreported, and the call went out under a minute ago.
|
||||
client.create('billing:hot', 'hot', {}, 'info');
|
||||
await Promise.resolve();
|
||||
expect(calls).toHaveLength(10);
|
||||
|
||||
vi.setSystemTime(Date.now() + 61_000);
|
||||
client.create('billing:hot', 'hot', {}, 'info');
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(11));
|
||||
expect(calls[10].group).toBe('billing');
|
||||
expect(calls[10].alert.repeatCount).toBe(12);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('raises the call without links when a transport never answers', async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const { client, calls } = callClient();
|
||||
client.addAlertHandler(() => new Promise(() => {}));
|
||||
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info');
|
||||
await vi.advanceTimersByTimeAsync(14_000);
|
||||
expect(calls).toHaveLength(0);
|
||||
await vi.advanceTimersByTimeAsync(1_000);
|
||||
expect(calls).toHaveLength(1);
|
||||
expect(calls[0].links).toEqual([]);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps a failing call handler from reaching the caller', async () => {
|
||||
const { client, calls } = callClient();
|
||||
const error = vi.mocked(console.error);
|
||||
client.addCallHandler(async () => {
|
||||
throw new Error('store is down');
|
||||
});
|
||||
|
||||
expect(() =>
|
||||
client.create('billing:stuck', 'top-up stuck', {}, 'info'),
|
||||
).not.toThrow();
|
||||
await vi.waitFor(() => expect(calls).toHaveLength(1));
|
||||
await vi.waitFor(() =>
|
||||
expect(error).toHaveBeenCalledWith(
|
||||
expect.stringContaining('call handler failed'),
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -17,39 +17,32 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { event as pdEvent } from '@pagerduty/pdjs';
|
||||
import { inspect } from 'node:util';
|
||||
import { createHash } from 'node:crypto';
|
||||
import type { IConfig, PagerSeverity, SeverityRule } from '../../types';
|
||||
import { PuterClient } from '../types';
|
||||
import {
|
||||
meetsMinSeverity,
|
||||
resolveCallGroup,
|
||||
resolveSeverityOverride,
|
||||
withinMaxSeverity,
|
||||
} from './severity';
|
||||
import { createPagerDutyAlertHandler } from './pagerduty';
|
||||
import { createSlackAlertHandler } from './slack';
|
||||
import type {
|
||||
Alarm,
|
||||
AlarmCall,
|
||||
AlarmFields,
|
||||
AlarmOptions,
|
||||
AlertDelivery,
|
||||
AlertHandler,
|
||||
AlertPayload,
|
||||
CallHandler,
|
||||
KnownErrorRule,
|
||||
} from './types';
|
||||
|
||||
export type {
|
||||
Alarm,
|
||||
AlarmCall,
|
||||
AlarmFields,
|
||||
AlarmOptions,
|
||||
AlertDelivery,
|
||||
AlertHandler,
|
||||
AlertPayload,
|
||||
CallHandler,
|
||||
KnownErrorRule,
|
||||
} from './types';
|
||||
export type { PagerSeverity } from '../../types';
|
||||
@@ -95,10 +88,6 @@ const REPEAT_REPORT_INTERVAL = 500;
|
||||
|
||||
const shouldReportRepeat = (count: number): boolean =>
|
||||
count <= REPEAT_REPORT_BURST || count % REPEAT_REPORT_INTERVAL === 0;
|
||||
/** How often an unreported repeat re-sends its alarm's open call. */
|
||||
const CALL_REFRESH_MS = 60 * 1000;
|
||||
/** Longest a call waits on transports for links before it goes out without. */
|
||||
const CALL_LINK_WAIT_MS = 15 * 1000;
|
||||
/** Keeps `info` alarms out of the paging system unless config says otherwise. */
|
||||
const DEFAULT_PAGERDUTY_MIN_SEVERITY: PagerSeverity = 'warning';
|
||||
/** Slack's ceiling once a pager exists: chat gets what doesn't page. */
|
||||
@@ -203,17 +192,15 @@ function cleanFields(fields: AlarmFields): Record<string, string> {
|
||||
* Manages system alarms and routes them to alert transports by severity.
|
||||
*
|
||||
* Severity is the routing decision: each transport declares the severity window
|
||||
* it accepts, so `critical` pages on-call while `info` only reaches chat, where
|
||||
* the alarm's `channel` key picks the channel. Config can retier or mute any
|
||||
* alarm id after the fact (`pager.severityOverrides` in {@link IConfig}), and
|
||||
* can mark ids that phone someone (`pager.calls`).
|
||||
* it accepts, so `critical` pages on-call while `info` only reaches the chat
|
||||
* channel. Config can retier or mute any alarm id after the fact — see
|
||||
* `pager.severityOverrides` in {@link IConfig}.
|
||||
*/
|
||||
export class AlarmClient extends PuterClient {
|
||||
/** Ordered by last occurrence: a repeat moves its alarm to the back. */
|
||||
private alarms = new Map<string, Alarm>();
|
||||
private aliases = new Map<string, Alarm>();
|
||||
private alertHandlers: RegisteredHandler[] = [];
|
||||
private callHandlers: CallHandler[] = [];
|
||||
private knownErrors: KnownErrorRule[] = [];
|
||||
private draining = false;
|
||||
private drainLogged = false;
|
||||
@@ -248,15 +235,29 @@ export class AlarmClient extends PuterClient {
|
||||
return false;
|
||||
}
|
||||
|
||||
const serverId = this.config.serverId;
|
||||
const minSeverity =
|
||||
pagerDutyConf.minSeverity ?? DEFAULT_PAGERDUTY_MIN_SEVERITY;
|
||||
|
||||
this.addAlertHandler(
|
||||
createPagerDutyAlertHandler({
|
||||
routingKey,
|
||||
apiToken: pagerDutyConf.apiToken,
|
||||
serverId: this.config.serverId,
|
||||
}),
|
||||
async (alert) => {
|
||||
await pdEvent({
|
||||
data: {
|
||||
routing_key: routingKey,
|
||||
event_action: 'trigger',
|
||||
dedup_key: alert.dedupKey,
|
||||
payload: {
|
||||
summary: alert.message,
|
||||
source: alert.source,
|
||||
severity: alert.severity,
|
||||
custom_details: {
|
||||
...alert.custom,
|
||||
server_id: serverId,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
},
|
||||
{ name: 'pagerduty', minSeverity },
|
||||
);
|
||||
|
||||
@@ -270,14 +271,8 @@ export class AlarmClient extends PuterClient {
|
||||
const slackConf = this.config.pager?.slack;
|
||||
if (!slackConf?.enabled) return;
|
||||
|
||||
if (!slackConf.botToken && !slackConf.webhookUrl) {
|
||||
console.warn(
|
||||
'[alarm] Slack enabled but no botToken or webhookUrl configured',
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (slackConf.botToken && !slackConf.channel) {
|
||||
console.warn('[alarm] Slack botToken needs a default channel');
|
||||
if (!slackConf.webhookUrl) {
|
||||
console.warn('[alarm] Slack enabled but no webhookUrl configured');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -321,8 +316,7 @@ export class AlarmClient extends PuterClient {
|
||||
* afterwards, so the value here is the starting point, not the last word.
|
||||
*
|
||||
* Pass `{ dedup: true }` when repeats of this id are one recurring fault
|
||||
* that should collapse into a single incident, and `channel` to post it to
|
||||
* the chat channel of the area that owns it — see {@link AlarmOptions}.
|
||||
* that should collapse into a single incident — see {@link AlarmOptions}.
|
||||
*/
|
||||
create(
|
||||
id: string,
|
||||
@@ -357,7 +351,6 @@ export class AlarmClient extends PuterClient {
|
||||
fields,
|
||||
severity,
|
||||
dedup: opts.dedup,
|
||||
channel: opts.channel,
|
||||
started: Date.now(),
|
||||
// `recordOccurrence` below stamps the first occurrence; seeding one
|
||||
// here too would report every alarm as one occurrence ahead.
|
||||
@@ -410,15 +403,6 @@ export class AlarmClient extends PuterClient {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Register a handler for alarms `pager.calls.rules` says phone someone. It
|
||||
* runs once the transports that took the alarm settle, with the links they
|
||||
* returned. Call groups aren't resolved until one is registered.
|
||||
*/
|
||||
addCallHandler(handler: CallHandler): void {
|
||||
this.callHandlers.push(handler);
|
||||
}
|
||||
|
||||
/** Add rules that can suppress or adjust severity of known errors. */
|
||||
setKnownErrors(rules: KnownErrorRule[]): void {
|
||||
this.knownErrors = rules;
|
||||
@@ -522,11 +506,7 @@ export class AlarmClient extends PuterClient {
|
||||
private handleRepeat(alarm: Alarm): void {
|
||||
this.applyKnownErrors(alarm);
|
||||
|
||||
if (!shouldReportRepeat(alarm.count)) {
|
||||
// Too frequent to report, but still firing: keep its call open.
|
||||
if (alarm.call && !alarm.noAlert) this.refreshCall(alarm);
|
||||
return;
|
||||
}
|
||||
if (!shouldReportRepeat(alarm.count)) return;
|
||||
|
||||
console.warn(
|
||||
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.count})`,
|
||||
@@ -578,14 +558,9 @@ export class AlarmClient extends PuterClient {
|
||||
}
|
||||
alarm.severity = resolved;
|
||||
|
||||
const callGroup =
|
||||
this.callHandlers.length > 0
|
||||
? resolveCallGroup(alarm.id, this.config.pager?.calls?.rules)
|
||||
: undefined;
|
||||
|
||||
// Build nothing for an alarm nothing takes — the payload (cleaned
|
||||
// fields plus the stack, twice) is the expensive part.
|
||||
if (!callGroup && !this.hasHandlerFor(resolved)) return;
|
||||
// Build nothing for a severity no transport accepts — the payload
|
||||
// (cleaned fields plus the stack, twice) is the expensive part.
|
||||
if (!this.hasHandlerFor(resolved)) return;
|
||||
|
||||
const fieldsClean = cleanFields(alarm.fields);
|
||||
const repeatCount = alarm.count;
|
||||
@@ -609,8 +584,6 @@ export class AlarmClient extends PuterClient {
|
||||
trace: alarm.error?.stack,
|
||||
repeatCount,
|
||||
isRepeat: repeatCount > 1,
|
||||
...(alarm.channel ? { channel: alarm.channel } : {}),
|
||||
...(callGroup ? { callGroup } : {}),
|
||||
custom: {
|
||||
fields: fieldsClean,
|
||||
trace: alarm.error?.stack,
|
||||
@@ -618,86 +591,15 @@ export class AlarmClient extends PuterClient {
|
||||
},
|
||||
};
|
||||
|
||||
const deliveries: Promise<AlertDelivery | undefined>[] = [];
|
||||
for (const registered of this.alertHandlers) {
|
||||
if (!meetsMinSeverity(resolved, registered.minSeverity)) continue;
|
||||
if (!withinMaxSeverity(resolved, registered.maxSeverity)) continue;
|
||||
deliveries.push(deliver(registered, payload));
|
||||
for (const { name, minSeverity, maxSeverity, handler } of this
|
||||
.alertHandlers) {
|
||||
if (!meetsMinSeverity(resolved, minSeverity)) continue;
|
||||
if (!withinMaxSeverity(resolved, maxSeverity)) continue;
|
||||
handler(payload).catch((err) => {
|
||||
console.error(
|
||||
`[alarm] ${name} alert handler failed: ${err?.message}`,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
if (callGroup)
|
||||
void this.raiseCall(alarm, payload, callGroup, deliveries);
|
||||
}
|
||||
|
||||
/** A throttled repeat returns no link, so the last call's links carry over. */
|
||||
private async raiseCall(
|
||||
alarm: Alarm,
|
||||
alert: AlertPayload,
|
||||
group: string,
|
||||
deliveries: Promise<AlertDelivery | undefined>[],
|
||||
): Promise<void> {
|
||||
const links: string[] = [];
|
||||
const collected = deliveries.map(async (delivery) => {
|
||||
const url = (await delivery)?.url;
|
||||
if (url) links.push(url);
|
||||
});
|
||||
let timer: NodeJS.Timeout | undefined;
|
||||
await Promise.race([
|
||||
Promise.all(collected),
|
||||
new Promise<void>((resolve) => {
|
||||
timer = setTimeout(resolve, CALL_LINK_WAIT_MS);
|
||||
timer.unref?.();
|
||||
}),
|
||||
]);
|
||||
clearTimeout(timer);
|
||||
|
||||
const call: AlarmCall = {
|
||||
group,
|
||||
alert,
|
||||
links: links.length > 0 ? [...links] : (alarm.call?.links ?? []),
|
||||
};
|
||||
alarm.call = call;
|
||||
await this.notifyCallHandlers(alarm, call);
|
||||
}
|
||||
|
||||
private refreshCall(alarm: Alarm): void {
|
||||
const { call, callAt = 0 } = alarm;
|
||||
if (!call || Date.now() - callAt < CALL_REFRESH_MS) return;
|
||||
void this.notifyCallHandlers(alarm, {
|
||||
...call,
|
||||
alert: { ...call.alert, repeatCount: alarm.count, isRepeat: true },
|
||||
});
|
||||
}
|
||||
|
||||
private async notifyCallHandlers(
|
||||
alarm: Alarm,
|
||||
call: AlarmCall,
|
||||
): Promise<void> {
|
||||
alarm.callAt = Date.now();
|
||||
await Promise.all(
|
||||
this.callHandlers.map(async (handler) => {
|
||||
try {
|
||||
await handler(call);
|
||||
} catch (err) {
|
||||
console.error(
|
||||
`[alarm] call handler failed for ${displayId(alarm)}: ${(err as Error)?.message}`,
|
||||
);
|
||||
}
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Run one transport; a failure is logged and never reaches the others. */
|
||||
async function deliver(
|
||||
{ name, handler }: RegisteredHandler,
|
||||
payload: AlertPayload,
|
||||
): Promise<AlertDelivery | undefined> {
|
||||
try {
|
||||
return (await handler(payload)) as AlertDelivery | undefined;
|
||||
} catch (err) {
|
||||
console.error(
|
||||
`[alarm] ${name} alert handler failed: ${(err as Error)?.message}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,138 +0,0 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { createPagerDutyAlertHandler } from './pagerduty';
|
||||
import type { AlertPayload } from './types';
|
||||
|
||||
const pdEvent = vi.hoisted(() => vi.fn(async () => ({})));
|
||||
vi.mock('@pagerduty/pdjs', () => ({ event: pdEvent }));
|
||||
|
||||
const alert = (over: Partial<AlertPayload> = {}): AlertPayload => ({
|
||||
id: 'billing:stuck',
|
||||
dedupKey: 'billing:stuck#1.1',
|
||||
shortId: 'amber-delta-fox',
|
||||
message: 'Top-up stuck',
|
||||
source: 'alarm',
|
||||
severity: 'warning',
|
||||
fields: {},
|
||||
repeatCount: 1,
|
||||
isRepeat: false,
|
||||
custom: { repeat_count: 1 },
|
||||
...over,
|
||||
});
|
||||
|
||||
const incidents = (htmlUrls: string[]) => ({
|
||||
ok: true,
|
||||
status: 200,
|
||||
json: async () => ({ incidents: htmlUrls.map((u) => ({ html_url: u })) }),
|
||||
});
|
||||
|
||||
describe('createPagerDutyAlertHandler', () => {
|
||||
let fetchMock: ReturnType<typeof vi.fn>;
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
pdEvent.mockClear();
|
||||
fetchMock = vi.fn(async () =>
|
||||
incidents(['https://acme.pagerduty.com/incidents/Q1']),
|
||||
);
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.unstubAllGlobals();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('triggers an event keyed by the dedup key', async () => {
|
||||
const handler = createPagerDutyAlertHandler({
|
||||
routingKey: 'rk',
|
||||
serverId: 'node-1',
|
||||
});
|
||||
expect(await handler(alert())).toBeUndefined();
|
||||
|
||||
expect(pdEvent).toHaveBeenCalledWith({
|
||||
data: {
|
||||
routing_key: 'rk',
|
||||
event_action: 'trigger',
|
||||
dedup_key: 'billing:stuck#1.1',
|
||||
payload: {
|
||||
summary: 'Top-up stuck',
|
||||
source: 'alarm',
|
||||
severity: 'warning',
|
||||
custom_details: { repeat_count: 1, server_id: 'node-1' },
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('links an alarm that calls to the incident it opened', async () => {
|
||||
const handler = createPagerDutyAlertHandler({
|
||||
routingKey: 'rk',
|
||||
apiToken: 'pd-token',
|
||||
});
|
||||
const delivery = handler(alert({ callGroup: 'billing' }));
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
|
||||
expect(await delivery).toEqual({
|
||||
url: 'https://acme.pagerduty.com/incidents/Q1',
|
||||
});
|
||||
const [url, init] = fetchMock.mock.calls[0];
|
||||
const query = new URL(url).searchParams;
|
||||
expect(query.get('incident_key')).toBe('billing:stuck#1.1');
|
||||
expect(query.getAll('statuses[]')).toEqual([
|
||||
'triggered',
|
||||
'acknowledged',
|
||||
]);
|
||||
expect(init.headers.Authorization).toBe('Token token=pd-token');
|
||||
});
|
||||
|
||||
it('waits for PagerDuty to open the incident', async () => {
|
||||
fetchMock.mockResolvedValueOnce(incidents([]));
|
||||
const handler = createPagerDutyAlertHandler({
|
||||
routingKey: 'rk',
|
||||
apiToken: 'pd-token',
|
||||
});
|
||||
const delivery = handler(alert({ callGroup: 'billing' }));
|
||||
await vi.advanceTimersByTimeAsync(3000);
|
||||
|
||||
expect(await delivery).toEqual({
|
||||
url: 'https://acme.pagerduty.com/incidents/Q1',
|
||||
});
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('looks nothing up without an API token', async () => {
|
||||
const handler = createPagerDutyAlertHandler({ routingKey: 'rk' });
|
||||
expect(await handler(alert({ callGroup: 'billing' }))).toBeUndefined();
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('keeps the event when the lookup fails', async () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
fetchMock.mockResolvedValueOnce({ ok: false, status: 401 });
|
||||
const handler = createPagerDutyAlertHandler({
|
||||
routingKey: 'rk',
|
||||
apiToken: 'bad-token',
|
||||
});
|
||||
const delivery = handler(alert({ callGroup: 'billing' }));
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
|
||||
expect(await delivery).toBeUndefined();
|
||||
expect(pdEvent).toHaveBeenCalledTimes(1);
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('401'));
|
||||
});
|
||||
|
||||
it('gives up when no incident appears', async () => {
|
||||
fetchMock.mockResolvedValue(incidents([]));
|
||||
const handler = createPagerDutyAlertHandler({
|
||||
routingKey: 'rk',
|
||||
apiToken: 'pd-token',
|
||||
});
|
||||
const delivery = handler(alert({ callGroup: 'billing' }));
|
||||
await vi.advanceTimersByTimeAsync(7000);
|
||||
|
||||
expect(await delivery).toBeUndefined();
|
||||
expect(fetchMock).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
});
|
||||
@@ -1,115 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { event as pdEvent } from '@pagerduty/pdjs';
|
||||
import type { AlertHandler } from './types';
|
||||
|
||||
const PAGERDUTY_API = 'https://api.pagerduty.com';
|
||||
const REQUEST_TIMEOUT_MS = 5000;
|
||||
/** PagerDuty opens the incident a moment after it accepts the event. */
|
||||
const INCIDENT_LOOKUP_DELAYS_MS = [1000, 2000, 4000];
|
||||
|
||||
/**
|
||||
* Trigger a PagerDuty event per alert. An alert that calls also looks up the
|
||||
* incident it opened, with a REST `apiToken`, so the call can link to it.
|
||||
*/
|
||||
export function createPagerDutyAlertHandler(conf: {
|
||||
routingKey: string;
|
||||
apiToken?: string;
|
||||
serverId?: string;
|
||||
}): AlertHandler {
|
||||
return async (alert) => {
|
||||
await pdEvent({
|
||||
data: {
|
||||
routing_key: conf.routingKey,
|
||||
event_action: 'trigger',
|
||||
dedup_key: alert.dedupKey,
|
||||
payload: {
|
||||
summary: alert.message,
|
||||
source: alert.source,
|
||||
severity: alert.severity,
|
||||
custom_details: {
|
||||
...alert.custom,
|
||||
server_id: conf.serverId,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
if (!alert.callGroup || !conf.apiToken) return;
|
||||
|
||||
const url = await findIncidentUrl(conf.apiToken, alert.dedupKey);
|
||||
return url ? { url } : undefined;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The open incident for a dedup key, retried while PagerDuty creates it. The
|
||||
* event is already sent, so a failed lookup only costs the call its link.
|
||||
*/
|
||||
async function findIncidentUrl(
|
||||
apiToken: string,
|
||||
dedupKey: string,
|
||||
): Promise<string | undefined> {
|
||||
for (const delay of INCIDENT_LOOKUP_DELAYS_MS) {
|
||||
await new Promise((resolve) => setTimeout(resolve, delay));
|
||||
try {
|
||||
const url = await openIncidentUrl(apiToken, dedupKey);
|
||||
if (url) return url;
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`[alarm] PagerDuty incident lookup failed: ${(err as Error)?.message}`,
|
||||
);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
async function openIncidentUrl(
|
||||
apiToken: string,
|
||||
dedupKey: string,
|
||||
): Promise<string | undefined> {
|
||||
const query = new URLSearchParams({
|
||||
incident_key: dedupKey,
|
||||
sort_by: 'created_at:desc',
|
||||
});
|
||||
query.append('statuses[]', 'triggered');
|
||||
query.append('statuses[]', 'acknowledged');
|
||||
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
try {
|
||||
const res = await fetch(`${PAGERDUTY_API}/incidents?${query}`, {
|
||||
headers: {
|
||||
Authorization: `Token token=${apiToken}`,
|
||||
Accept: 'application/vnd.pagerduty+json;version=2',
|
||||
},
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (!res.ok) {
|
||||
throw new Error(`PagerDuty incidents returned ${res.status}`);
|
||||
}
|
||||
const body = (await res.json()) as {
|
||||
incidents?: Array<{ html_url?: string }>;
|
||||
};
|
||||
return body.incidents?.[0]?.html_url;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,5 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
meetsMinSeverity,
|
||||
resolveCallGroup,
|
||||
resolveSeverityOverride,
|
||||
} from './severity';
|
||||
import { meetsMinSeverity, resolveSeverityOverride } from './severity';
|
||||
|
||||
describe('meetsMinSeverity', () => {
|
||||
it('accepts everything at or above the floor', () => {
|
||||
@@ -78,30 +74,3 @@ describe('resolveSeverityOverride', () => {
|
||||
warn.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveCallGroup', () => {
|
||||
it('picks the group of the longest matching rule', () => {
|
||||
const rules = {
|
||||
'appstore_billing_error:*': 'billing',
|
||||
'appstore_billing_error:topup_stuck:*': 'billing-urgent',
|
||||
};
|
||||
expect(
|
||||
resolveCallGroup('appstore_billing_error:topup_stuck:tx1', rules),
|
||||
).toBe('billing-urgent');
|
||||
expect(
|
||||
resolveCallGroup('appstore_billing_error:refund:tx1', rules),
|
||||
).toBe('billing');
|
||||
expect(resolveCallGroup('http_500:GET:/x', rules)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ignores a group that cannot be used as a key', () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
expect(resolveCallGroup('a:b', { 'a:*': 'on call' })).toBeUndefined();
|
||||
expect(warn).toHaveBeenCalled();
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it('does not read inherited keys as rules', () => {
|
||||
expect(resolveCallGroup('constructor', {})).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -54,20 +54,22 @@ export function withinMaxSeverity(
|
||||
}
|
||||
|
||||
/**
|
||||
* The rule keyed by an alarm id, or by a prefix pattern ending in `*`
|
||||
* (`cronMonitor:*`). Exact ids win over patterns; among patterns the longest
|
||||
* prefix wins, so a specific rule can carve an exception out of a broad one.
|
||||
* Look up the operator override for an alarm id. Exact ids win over prefix
|
||||
* patterns (`cronMonitor:*`); among patterns the longest prefix wins, so a
|
||||
* specific rule can carve an exception out of a broad one.
|
||||
*/
|
||||
export function matchIdRule<T>(
|
||||
export function resolveSeverityOverride(
|
||||
id: string,
|
||||
rules: Record<string, T> | undefined,
|
||||
): T | undefined {
|
||||
if (!rules) return undefined;
|
||||
if (Object.hasOwn(rules, id)) return rules[id];
|
||||
overrides: Record<string, SeverityRule> | undefined,
|
||||
): SeverityRule | undefined {
|
||||
if (!overrides) return undefined;
|
||||
|
||||
const exact = overrides[id];
|
||||
if (exact !== undefined) return validRule(id, exact);
|
||||
|
||||
let bestLength = -1;
|
||||
let best: T | undefined;
|
||||
for (const [pattern, rule] of Object.entries(rules)) {
|
||||
let best: SeverityRule | undefined;
|
||||
for (const [pattern, rule] of Object.entries(overrides)) {
|
||||
if (!pattern.endsWith('*')) continue;
|
||||
const prefix = pattern.slice(0, -1);
|
||||
if (!id.startsWith(prefix)) continue;
|
||||
@@ -75,35 +77,7 @@ export function matchIdRule<T>(
|
||||
bestLength = prefix.length;
|
||||
best = rule;
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
/** The operator override for an alarm id; see {@link matchIdRule}. */
|
||||
export function resolveSeverityOverride(
|
||||
id: string,
|
||||
overrides: Record<string, SeverityRule> | undefined,
|
||||
): SeverityRule | undefined {
|
||||
const rule = matchIdRule(id, overrides);
|
||||
return rule === undefined ? undefined : validRule(id, rule);
|
||||
}
|
||||
|
||||
/** Call groups end up in storage keys and query strings. */
|
||||
export const CALL_GROUP_PATTERN = /^[A-Za-z0-9_-]{1,64}$/;
|
||||
|
||||
/** The call group config assigns an alarm id; see {@link matchIdRule}. */
|
||||
export function resolveCallGroup(
|
||||
id: string,
|
||||
rules: Record<string, string> | undefined,
|
||||
): string | undefined {
|
||||
const group = matchIdRule(id, rules);
|
||||
if (group === undefined) return undefined;
|
||||
if (typeof group === 'string' && CALL_GROUP_PATTERN.test(group))
|
||||
return group;
|
||||
console.warn(
|
||||
`[alarm] ignoring invalid call group "${group}" for ${id} ` +
|
||||
'(expected letters, digits, _ or -, up to 64)',
|
||||
);
|
||||
return undefined;
|
||||
return best === undefined ? undefined : validRule(id, best);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,9 +1,5 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
buildSlackMessage,
|
||||
createSlackAlertHandler,
|
||||
resolveSlackChannel,
|
||||
} from './slack';
|
||||
import { buildSlackMessage, createSlackAlertHandler } from './slack';
|
||||
import type { AlertPayload } from './types';
|
||||
|
||||
const alert = (over: Partial<AlertPayload> = {}): AlertPayload => ({
|
||||
@@ -78,12 +74,6 @@ describe('buildSlackMessage', () => {
|
||||
expect(msg.attachments[0].fields[0].value.endsWith('…')).toBe(true);
|
||||
});
|
||||
|
||||
it('says when the alarm is calling someone', () => {
|
||||
const msg = buildSlackMessage(alert({ callGroup: 'billing' }));
|
||||
expect(msg.text).toContain('calling `billing`');
|
||||
expect(buildSlackMessage(alert()).text).not.toContain('calling');
|
||||
});
|
||||
|
||||
it('omits channel and username when unset', () => {
|
||||
const msg = buildSlackMessage(alert());
|
||||
expect(msg.channel).toBeUndefined();
|
||||
@@ -214,115 +204,3 @@ describe('createSlackAlertHandler', () => {
|
||||
expect(fetchMock).toHaveBeenCalledTimes(5002);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSlackChannel', () => {
|
||||
const conf = {
|
||||
channel: '#alerts-oncall',
|
||||
channels: { billing: '#alerts-billing', empty: '' },
|
||||
};
|
||||
|
||||
it('maps a listed key to its channel', () => {
|
||||
expect(resolveSlackChannel(conf, 'billing')).toBe('#alerts-billing');
|
||||
});
|
||||
|
||||
it('falls back to the default channel', () => {
|
||||
expect(resolveSlackChannel(conf, undefined)).toBe('#alerts-oncall');
|
||||
expect(resolveSlackChannel(conf, 'unlisted')).toBe('#alerts-oncall');
|
||||
expect(resolveSlackChannel(conf, 'empty')).toBe('#alerts-oncall');
|
||||
expect(resolveSlackChannel(conf, 'constructor')).toBe('#alerts-oncall');
|
||||
});
|
||||
});
|
||||
|
||||
describe('createSlackAlertHandler with a bot token', () => {
|
||||
let fetchMock: ReturnType<typeof vi.fn>;
|
||||
const apiResponse = (body: Record<string, unknown>) => ({
|
||||
ok: true,
|
||||
status: 200,
|
||||
json: async () => body,
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
fetchMock = vi.fn(async (url: string) =>
|
||||
url.includes('chat.getPermalink')
|
||||
? apiResponse({
|
||||
ok: true,
|
||||
permalink: 'https://team.slack.com/archives/C1/p12',
|
||||
})
|
||||
: apiResponse({ ok: true, channel: 'C1', ts: '1.2' }),
|
||||
);
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
const conf = {
|
||||
botToken: 'xoxb-test',
|
||||
channel: '#alerts-oncall',
|
||||
channels: { billing: '#alerts-billing' },
|
||||
};
|
||||
|
||||
it('posts to the channel the key maps to', async () => {
|
||||
const handler = createSlackAlertHandler(conf);
|
||||
const delivery = await handler(alert({ channel: 'billing' }));
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
const [url, init] = fetchMock.mock.calls[0];
|
||||
expect(url).toBe('https://slack.com/api/chat.postMessage');
|
||||
expect(init.headers.Authorization).toBe('Bearer xoxb-test');
|
||||
expect(JSON.parse(init.body).channel).toBe('#alerts-billing');
|
||||
expect(delivery).toEqual({});
|
||||
});
|
||||
|
||||
it('links the message of an alarm that calls', async () => {
|
||||
const handler = createSlackAlertHandler(conf);
|
||||
const delivery = await handler(alert({ callGroup: 'billing' }));
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
const [url, init] = fetchMock.mock.calls[1];
|
||||
expect(url).toBe(
|
||||
'https://slack.com/api/chat.getPermalink?channel=C1&message_ts=1.2',
|
||||
);
|
||||
expect(init.headers.Authorization).toBe('Bearer xoxb-test');
|
||||
expect(delivery).toEqual({
|
||||
url: 'https://team.slack.com/archives/C1/p12',
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps the post when only the permalink lookup fails', async () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
fetchMock
|
||||
.mockResolvedValueOnce(
|
||||
apiResponse({ ok: true, channel: 'C1', ts: '1.2' }),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
apiResponse({ ok: false, error: 'ratelimited' }),
|
||||
);
|
||||
const handler = createSlackAlertHandler({
|
||||
...conf,
|
||||
repeatThrottleMs: 60_000,
|
||||
});
|
||||
|
||||
expect(await handler(alert({ callGroup: 'billing' }))).toEqual({});
|
||||
// Still throttled: the message went out, so a repeat doesn't repost.
|
||||
await handler(alert({ callGroup: 'billing' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it('treats an ok:false answer as a failed post', async () => {
|
||||
fetchMock.mockResolvedValueOnce(
|
||||
apiResponse({ ok: false, error: 'channel_not_found' }),
|
||||
);
|
||||
const handler = createSlackAlertHandler({
|
||||
...conf,
|
||||
repeatThrottleMs: 60_000,
|
||||
});
|
||||
|
||||
await expect(handler(alert())).rejects.toThrow('channel_not_found');
|
||||
// The failure released the throttle slot.
|
||||
await handler(alert());
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
@@ -19,10 +19,9 @@
|
||||
|
||||
import type { ISlackAlertConfig, PagerSeverity } from '../../types';
|
||||
import { BoundedTtlMap } from '../../util/boundedTtlMap';
|
||||
import type { AlertDelivery, AlertHandler, AlertPayload } from './types';
|
||||
import type { AlertHandler, AlertPayload } from './types';
|
||||
|
||||
const REQUEST_TIMEOUT_MS = 5000;
|
||||
const SLACK_API = 'https://slack.com/api';
|
||||
export const DEFAULT_REPEAT_THROTTLE_MS = 15 * 60 * 1000;
|
||||
|
||||
/** Beyond this many tracked alarm ids, throttle state is pruned. */
|
||||
@@ -67,9 +66,9 @@ function truncate(value: string, max: number): string {
|
||||
}
|
||||
|
||||
/**
|
||||
* Render an alert as a message body (webhook or `chat.postMessage`):
|
||||
* severity-coloured attachment, the alarm's fields as a table, and the stack
|
||||
* (when there is one) as a code block.
|
||||
* Render an alert as an incoming-webhook payload: severity-coloured attachment,
|
||||
* the alarm's fields as a table, and the stack (when there is one) as a code
|
||||
* block. Split out from the handler so the formatting is testable on its own.
|
||||
*/
|
||||
export function buildSlackMessage(
|
||||
alert: AlertPayload,
|
||||
@@ -77,10 +76,7 @@ export function buildSlackMessage(
|
||||
): SlackMessage {
|
||||
const style = STYLE[alert.severity] ?? STYLE.info;
|
||||
const repeat = alert.isRepeat ? ` (x${alert.repeatCount})` : '';
|
||||
const call = alert.callGroup
|
||||
? ` :telephone_receiver: calling \`${alert.callGroup}\``
|
||||
: '';
|
||||
const headline = `${style.emoji} *[${alert.severity.toUpperCase()}]* ${alert.message}${repeat}${call}`;
|
||||
const headline = `${style.emoji} *[${alert.severity.toUpperCase()}]* ${alert.message}${repeat}`;
|
||||
|
||||
const fields: SlackField[] = [];
|
||||
for (const [key, value] of Object.entries(alert.fields)) {
|
||||
@@ -117,141 +113,51 @@ export function buildSlackMessage(
|
||||
};
|
||||
}
|
||||
|
||||
/** The channel an alert posts to: its key's mapping, else the default. */
|
||||
export function resolveSlackChannel(
|
||||
conf: ISlackAlertConfig,
|
||||
key: string | undefined,
|
||||
): string | undefined {
|
||||
const mapped =
|
||||
key && conf.channels && Object.hasOwn(conf.channels, key)
|
||||
? conf.channels[key]
|
||||
: undefined;
|
||||
return mapped || conf.channel;
|
||||
}
|
||||
|
||||
/**
|
||||
* Post alerts to Slack: through the Web API with a bot token, which can reach
|
||||
* any channel and link to the message, or else to an incoming webhook. Repeats
|
||||
* of the same alarm id are throttled so a hot loop doesn't flood the channel —
|
||||
* the occurrence count on the next post that gets through tells the reader what
|
||||
* they missed.
|
||||
* Post alerts to a Slack incoming webhook. Repeats of the same alarm id are
|
||||
* throttled so a hot loop doesn't flood the channel — the occurrence count on
|
||||
* the next post that gets through tells the reader what they missed.
|
||||
*/
|
||||
export function createSlackAlertHandler(
|
||||
conf: ISlackAlertConfig,
|
||||
opts: { serverId?: string } = {},
|
||||
): AlertHandler {
|
||||
const webhookUrl = conf.webhookUrl as string;
|
||||
const throttleMs = conf.repeatThrottleMs ?? DEFAULT_REPEAT_THROTTLE_MS;
|
||||
// A zero or negative throttle expires every entry on arrival.
|
||||
const lastPosted = new BoundedTtlMap<string, true>({
|
||||
maxEntries: MAX_THROTTLE_ENTRIES,
|
||||
ttlMs: throttleMs,
|
||||
});
|
||||
const post = conf.botToken
|
||||
? postWithBot(conf.botToken)
|
||||
: postToWebhook(conf.webhookUrl as string);
|
||||
|
||||
return async (alert) => {
|
||||
if (!lastPosted.shouldEmit(alert.id)) return;
|
||||
|
||||
const message = buildSlackMessage(alert, {
|
||||
channel: resolveSlackChannel(conf, alert.channel),
|
||||
username: conf.username,
|
||||
serverId: opts.serverId,
|
||||
});
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
try {
|
||||
return await post(message, alert);
|
||||
} catch (err) {
|
||||
// A rejected post shouldn't leave the id marked as delivered.
|
||||
lastPosted.delete(alert.id);
|
||||
throw err;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
type SlackPost = (
|
||||
message: SlackMessage,
|
||||
alert: AlertPayload,
|
||||
) => Promise<AlertDelivery | void>;
|
||||
|
||||
function postToWebhook(webhookUrl: string): SlackPost {
|
||||
return (message) =>
|
||||
slackFetch(
|
||||
webhookUrl,
|
||||
{
|
||||
const res = await fetch(webhookUrl, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(message),
|
||||
},
|
||||
async (res) => {
|
||||
if (!res.ok)
|
||||
throw new Error(`Slack webhook returned ${res.status}`);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/** Only an alert that calls asks for its permalink; that's a second request. */
|
||||
function postWithBot(token: string): SlackPost {
|
||||
const auth = { Authorization: `Bearer ${token}` };
|
||||
return async (message, alert) => {
|
||||
const posted = await slackApi<{ channel: string; ts: string }>(
|
||||
'chat.postMessage',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
...auth,
|
||||
'Content-Type': 'application/json; charset=utf-8',
|
||||
},
|
||||
body: JSON.stringify(message),
|
||||
},
|
||||
);
|
||||
if (!alert.callGroup) return {};
|
||||
|
||||
const query = new URLSearchParams({
|
||||
channel: posted.channel,
|
||||
message_ts: posted.ts,
|
||||
});
|
||||
// The message is posted either way; a missing link must not repost it.
|
||||
try {
|
||||
const link = await slackApi<{ permalink: string }>(
|
||||
`chat.getPermalink?${query}`,
|
||||
{ headers: auth },
|
||||
);
|
||||
return { url: link.permalink };
|
||||
body: JSON.stringify(
|
||||
buildSlackMessage(alert, {
|
||||
channel: conf.channel,
|
||||
username: conf.username,
|
||||
serverId: opts.serverId,
|
||||
}),
|
||||
),
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (!res.ok) {
|
||||
// A rejected post shouldn't leave the id marked as delivered.
|
||||
lastPosted.delete(alert.id);
|
||||
throw new Error(`Slack webhook returned ${res.status}`);
|
||||
}
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`[alarm] Slack permalink lookup failed: ${(err as Error)?.message}`,
|
||||
);
|
||||
return {};
|
||||
lastPosted.delete(alert.id);
|
||||
throw err;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/** Web API methods answer 200 with `ok: false` on failure. */
|
||||
function slackApi<T>(method: string, init: RequestInit): Promise<T> {
|
||||
return slackFetch(`${SLACK_API}/${method}`, init, async (res) => {
|
||||
if (!res.ok) throw new Error(`Slack ${method} returned ${res.status}`);
|
||||
const body = (await res.json()) as {
|
||||
ok?: boolean;
|
||||
error?: string;
|
||||
} & T;
|
||||
if (!body.ok) throw new Error(`Slack ${method} failed: ${body.error}`);
|
||||
return body;
|
||||
});
|
||||
}
|
||||
|
||||
/** The timeout covers `read` as well as the response headers. */
|
||||
async function slackFetch<T>(
|
||||
url: string,
|
||||
init: RequestInit,
|
||||
read: (res: Response) => Promise<T>,
|
||||
): Promise<T> {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
try {
|
||||
return await read(
|
||||
await fetch(url, { ...init, signal: controller.signal }),
|
||||
);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
@@ -39,12 +39,6 @@ export interface AlarmOptions {
|
||||
* alarm raises its own incident per occurrence.
|
||||
*/
|
||||
dedup?: boolean;
|
||||
/**
|
||||
* Chat channel key naming the area that owns this alarm (`billing`,
|
||||
* `ai_providers`). Config maps keys to channels; an unmapped or absent key
|
||||
* posts to the default channel. Fixed by the alarm's first occurrence.
|
||||
*/
|
||||
channel?: string;
|
||||
}
|
||||
|
||||
export interface Alarm extends AlarmOptions {
|
||||
@@ -70,10 +64,6 @@ export interface Alarm extends AlarmOptions {
|
||||
noAlert?: boolean;
|
||||
/** Set once a config mute has been logged, so it's reported only once. */
|
||||
muteLogged?: boolean;
|
||||
/** The last call raised, so repeats too frequent to report keep it open. */
|
||||
call?: AlarmCall;
|
||||
/** When `call` was last handed to the call handlers, epoch ms. */
|
||||
callAt?: number;
|
||||
}
|
||||
|
||||
export interface AlertPayload {
|
||||
@@ -96,33 +86,11 @@ export interface AlertPayload {
|
||||
repeatCount: number;
|
||||
/** False for the first occurrence of an alarm id. */
|
||||
isRepeat: boolean;
|
||||
/** Chat channel key from {@link AlarmOptions.channel}. */
|
||||
channel?: string;
|
||||
/** Call group config assigned this alarm, when it phones someone. */
|
||||
callGroup?: string;
|
||||
/** PagerDuty `custom_details` payload. */
|
||||
custom?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** What a transport reports about an alert it delivered. */
|
||||
export interface AlertDelivery {
|
||||
/** Link to the delivered alert, such as the chat message it became. */
|
||||
url?: string;
|
||||
}
|
||||
|
||||
export type AlertHandler = (
|
||||
alert: AlertPayload,
|
||||
) => Promise<AlertDelivery | void>;
|
||||
|
||||
/** An alarm that config says phones someone, once its transports settled. */
|
||||
export interface AlarmCall {
|
||||
group: string;
|
||||
alert: AlertPayload;
|
||||
/** Where a responder can read the alarm; empty when no transport linked. */
|
||||
links: string[];
|
||||
}
|
||||
|
||||
export type CallHandler = (call: AlarmCall) => Promise<void>;
|
||||
export type AlertHandler = (alert: AlertPayload) => Promise<void>;
|
||||
|
||||
export interface KnownErrorRule {
|
||||
match: {
|
||||
|
||||
@@ -192,18 +192,6 @@ describe('AbstractDatabaseClient — replica-aware reads', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('refuses a batch with an array parameter before any read', async () => {
|
||||
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
|
||||
await expect(
|
||||
client.tryHardReadAll([
|
||||
{ query: 'SELECT 1', params: ['a'] },
|
||||
{ query: 'SELECT 2', params: [['a', 'b']] },
|
||||
]),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(client.reads).toEqual([]);
|
||||
expect(client.preads).toEqual([]);
|
||||
});
|
||||
|
||||
it('issues one read per query when there is no replica', async () => {
|
||||
const client = new FixtureDatabaseClient([], [{ id: 2 }], false);
|
||||
expect(await client.tryHardReadAll(reads)).toEqual([[], []]);
|
||||
|
||||
@@ -22,7 +22,6 @@ import { isAbsolute, resolve as resolvePath } from 'path';
|
||||
import type { IConfig } from '../../types';
|
||||
import { Span } from '../../util/span.js';
|
||||
import { PuterClient } from '../types';
|
||||
import { assertBindableParams } from './bindableParams.js';
|
||||
import { compareMigrationFilenames } from './migrationFilenames.js';
|
||||
|
||||
export interface WriteResult {
|
||||
@@ -180,9 +179,6 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
async tryHardReadAll(
|
||||
reads: { query: string; params?: unknown[] }[],
|
||||
): Promise<Record<string, unknown>[][]> {
|
||||
// A failed replica read counts as a miss below, so refuse up front.
|
||||
for (const { query, params } of reads)
|
||||
assertBindableParams(query, params);
|
||||
if (!this.hasReadReplica()) {
|
||||
return Promise.all(
|
||||
reads.map(({ query, params = [] }) => this.read(query, params)),
|
||||
|
||||
@@ -615,61 +615,6 @@ describe('MySQLDatabaseClient — query interface', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — non-scalar parameters', () => {
|
||||
// The driver formats in the client: an array becomes a value list and an
|
||||
// object becomes `col = val` pairs, so neither may reach it.
|
||||
it.each([
|
||||
['an array', ['a', 'b']],
|
||||
['an object', { username: 1 }],
|
||||
['a function', () => 1],
|
||||
])('refuses %s with a 400 before any query is sent', async (_, value) => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
const sql = 'SELECT * FROM `user` WHERE `username` = ?';
|
||||
for (const call of [
|
||||
() => client.read(sql, [value]),
|
||||
() => client.pread(sql, [value]),
|
||||
() => client.tryHardRead(sql, [value]),
|
||||
() => client.write('DELETE FROM `user` WHERE `uuid` = ?', [value]),
|
||||
() =>
|
||||
client.batchWrite([
|
||||
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
|
||||
{
|
||||
statement: 'DELETE FROM `user` WHERE `uuid` = ?',
|
||||
values: [value],
|
||||
},
|
||||
]),
|
||||
]) {
|
||||
await expect(call()).rejects.toMatchObject({ statusCode: 400 });
|
||||
}
|
||||
for (const pool of createdPools) {
|
||||
expect(pool.calls).toEqual([]);
|
||||
expect(pool.connections).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('binds scalars, null, dates and buffers', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
pool.respond = () => rowSets([]);
|
||||
const values = [
|
||||
'a',
|
||||
1,
|
||||
10n,
|
||||
true,
|
||||
null,
|
||||
undefined,
|
||||
new Date(0),
|
||||
Buffer.from('x'),
|
||||
];
|
||||
|
||||
await client.read(
|
||||
`SELECT 1 WHERE ${values.map(() => '?').join(' AND ')}`,
|
||||
values,
|
||||
);
|
||||
expect(pool.calls[0].values).toEqual(values);
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — batchWrite transactions', () => {
|
||||
it('runs every statement on one connection inside a transaction', async () => {
|
||||
const client = await startClient();
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
import { metrics } from '@opentelemetry/api';
|
||||
import { createPool, ExecuteValues, type Pool } from 'mysql2';
|
||||
import { Span } from '../../util/span.js';
|
||||
import { assertBindableParams } from './bindableParams.js';
|
||||
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
|
||||
import { SQLBatcher } from './SQLBatcher.js';
|
||||
import { isRetriableError } from './retriableErrors.js';
|
||||
@@ -135,7 +134,6 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
assertBindableParams(query, params);
|
||||
let result;
|
||||
try {
|
||||
result = await this.dbReplica.execute(query, params);
|
||||
@@ -162,7 +160,6 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
assertBindableParams(query, params);
|
||||
const result = await this.dbPrimaryRead.execute(query, params);
|
||||
if (!result) return [];
|
||||
return (result[0] as Record<string, unknown>[]) ?? [];
|
||||
@@ -173,7 +170,6 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<WriteResult> {
|
||||
assertBindableParams(query, params);
|
||||
const result = await this.db.execute(query, params);
|
||||
const header = result[0] as {
|
||||
insertId?: number;
|
||||
@@ -196,8 +192,6 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
entries: { statement: string; values: unknown[] }[],
|
||||
): Promise<void> {
|
||||
if (entries.length === 0) return;
|
||||
for (const { statement, values } of entries)
|
||||
assertBindableParams(statement, values);
|
||||
// Bypass the SQLBatcher: it coalesces queries from unrelated callers
|
||||
// into a single multi-statement string, which is incompatible with
|
||||
// wrapping a transaction around just *our* statements. Acquire a
|
||||
@@ -229,7 +223,6 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
query: string,
|
||||
params: unknown[],
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
assertBindableParams(query, params);
|
||||
const result = await this.dbReplica.execute(query, params);
|
||||
if (!result) return [];
|
||||
return (result[0] as Record<string, unknown>[]) ?? [];
|
||||
|
||||
@@ -325,24 +325,6 @@ describe('PostgresDatabaseClient', () => {
|
||||
).resolves.toEqual([{ created_at: 3 }]);
|
||||
});
|
||||
|
||||
it('refuses an array or object parameter with a 400', async () => {
|
||||
const pool = new RecordingPool();
|
||||
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
|
||||
await client.onServerStart();
|
||||
const before = pool.calls.length;
|
||||
const query = 'SELECT `name` FROM `apps` WHERE `uid` = ?';
|
||||
|
||||
for (const value of [['a', 'b'], { uid: 'a' }]) {
|
||||
await expect(client.read(query, [value])).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
});
|
||||
await expect(client.write(query, [value])).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
});
|
||||
}
|
||||
expect(pool.calls.length).toBe(before);
|
||||
});
|
||||
|
||||
it('runs batch writes in order and commits', async () => {
|
||||
const conn = new RecordingPoolClient();
|
||||
const client = new PostgresDatabaseClient(
|
||||
|
||||
@@ -25,7 +25,6 @@ import {
|
||||
type BatchEntry,
|
||||
type WriteResult,
|
||||
} from './DatabaseClient';
|
||||
import { assertBindableParams } from './bindableParams.js';
|
||||
import { preparePostgresSql } from './preparePostgresSql.js';
|
||||
import { splitPostgresStatements } from './splitPostgresStatements.js';
|
||||
import type { IConfig } from '../../types';
|
||||
@@ -284,7 +283,6 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<QueryResult> {
|
||||
assertBindableParams(query, params);
|
||||
let text = this.preparedSql.get(query);
|
||||
if (text === undefined) {
|
||||
text = preparePostgresSql(query).text;
|
||||
|
||||
@@ -378,7 +378,6 @@ export class SQLBatcher {
|
||||
await Promise.all(workers);
|
||||
|
||||
let anySucceeded = false;
|
||||
let anyUnreachable = false;
|
||||
let failureCount = 0;
|
||||
for (let i = 0; i < batch.length; i++) {
|
||||
const b = batch[i];
|
||||
@@ -388,7 +387,6 @@ export class SQLBatcher {
|
||||
b.resolve(r.value);
|
||||
} else {
|
||||
failureCount++;
|
||||
if (isRetriableError(r.error)) anyUnreachable = true;
|
||||
b.reject(r.error);
|
||||
}
|
||||
}
|
||||
@@ -396,11 +394,11 @@ export class SQLBatcher {
|
||||
fallbackItemFailuresCounter.add(failureCount, this.#metricAttrs);
|
||||
}
|
||||
|
||||
// Only escalate the breaker when the database itself looks unhealthy:
|
||||
// nothing got through and something failed to reach it. An error the
|
||||
// server answered with (bad SQL, duplicate key) means it is up.
|
||||
// Only escalate the breaker when the database itself looks unhealthy
|
||||
// (no item got through). Row-level errors like duplicate-key are
|
||||
// application concerns, not DB outages, and shouldn't trip it.
|
||||
this.#lastFailureAt = Date.now();
|
||||
if (anySucceeded || !anyUnreachable) {
|
||||
if (anySucceeded) {
|
||||
this.#consecutiveFailures = 0;
|
||||
} else {
|
||||
this.#consecutiveFailures++;
|
||||
|
||||
@@ -686,33 +686,6 @@ describe('SQLBatcher', () => {
|
||||
expect(getConnection).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('keeps the breaker closed when every read fails with an error the server answered', async () => {
|
||||
const conn = makeConnection(() => {
|
||||
// The message quotes the statement, which mentions a lost connection.
|
||||
throw makeError(
|
||||
'ER_PARSE_ERROR',
|
||||
"syntax error near 'Connection lost'",
|
||||
);
|
||||
});
|
||||
const { pool } = makePool(conn);
|
||||
const batcher = new SQLBatcher(pool, {
|
||||
maxTimeInQueue: 5,
|
||||
failureThreshold: 1,
|
||||
cooldownMs: 60_000,
|
||||
readOnly: true,
|
||||
});
|
||||
|
||||
const results = await Promise.allSettled([
|
||||
batcher.query('SELECT bad', []),
|
||||
batcher.query('SELECT worse', []),
|
||||
]);
|
||||
expect(results.map((r) => r.status)).toEqual(['rejected', 'rejected']);
|
||||
conn.query.mockImplementation(
|
||||
makeConnection(happyBatch).query.getMockImplementation()!,
|
||||
);
|
||||
await expect(batcher.query('SELECT a', [])).resolves.toBeTruthy();
|
||||
});
|
||||
|
||||
it('never retries deterministic row-level errors and does not escalate the breaker', async () => {
|
||||
let fallbackAttempts = 0;
|
||||
const conn = makeConnection((sql) => {
|
||||
|
||||
@@ -1211,42 +1211,6 @@ describe('SqliteDatabaseClient — query interface', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses an array or object parameter with a 400', async () => {
|
||||
await client.write('INSERT INTO `widget` (`name`) VALUES (?)', ['a']);
|
||||
for (const value of [['a'], { name: 'a' }]) {
|
||||
await expect(
|
||||
client.read('SELECT * FROM `widget` WHERE `name` = ?', [
|
||||
value,
|
||||
]),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
await expect(
|
||||
client.write('DELETE FROM `widget` WHERE `name` = ?', [value]),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
await expect(
|
||||
client.batchWrite([
|
||||
{
|
||||
statement: 'DELETE FROM `widget` WHERE `name` = ?',
|
||||
values: [value],
|
||||
},
|
||||
]),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
}
|
||||
expect(await client.read('SELECT `name` FROM `widget`')).toEqual([
|
||||
{ name: 'a' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('refuses a non-finite or unsafe number parameter with a 400', async () => {
|
||||
for (const value of [NaN, Infinity, -Infinity, 2 ** 53, 1e21]) {
|
||||
await expect(
|
||||
client.read('SELECT * FROM `widget` WHERE `name` = ?', [value]),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
}
|
||||
await expect(
|
||||
client.read('SELECT ? AS n', [Number.MAX_SAFE_INTEGER]),
|
||||
).resolves.toEqual([{ n: Number.MAX_SAFE_INTEGER }]);
|
||||
});
|
||||
|
||||
it('closes the database on shutdown', async () => {
|
||||
const closable = await bootClient();
|
||||
closable.onServerShutdown();
|
||||
|
||||
@@ -23,7 +23,6 @@ import { createContext, runInContext } from 'vm';
|
||||
import type { IConfig } from '../../types';
|
||||
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
|
||||
import { Span } from '../../util/span.js';
|
||||
import { assertBindableParams } from './bindableParams.js';
|
||||
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
|
||||
|
||||
const MIGRATIONS_DIR = resolve(__dirname, './migrations/sqlite');
|
||||
@@ -196,7 +195,7 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
params: unknown[] = [],
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
return this.prepare(query).all(
|
||||
...this.transformParams(query, params),
|
||||
...this.transformParams(params),
|
||||
) as Record<string, unknown>[];
|
||||
}
|
||||
|
||||
@@ -213,9 +212,7 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<WriteResult> {
|
||||
const info = this.prepare(query).run(
|
||||
...this.transformParams(query, params),
|
||||
);
|
||||
const info = this.prepare(query).run(...this.transformParams(params));
|
||||
|
||||
return {
|
||||
insertId: info.lastInsertRowid,
|
||||
@@ -235,9 +232,7 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
): Promise<void> {
|
||||
this.db.transaction(() => {
|
||||
for (const { statement, values } of entries) {
|
||||
this.prepare(statement).run(
|
||||
...this.transformParams(statement, values),
|
||||
);
|
||||
this.prepare(statement).run(...this.transformParams(values));
|
||||
}
|
||||
})();
|
||||
}
|
||||
@@ -259,8 +254,7 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
return query.replace(/now\(\)/gi, "datetime('now')");
|
||||
}
|
||||
|
||||
private transformParams(query: string, params: unknown[]): unknown[] {
|
||||
assertBindableParams(query, params);
|
||||
private transformParams(params: unknown[]): unknown[] {
|
||||
return params.map((p) => {
|
||||
if (typeof p === 'boolean') return p ? 1 : 0;
|
||||
return p;
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
|
||||
const isBindable = (value: unknown): boolean => {
|
||||
if (value === null || value === undefined) return true;
|
||||
switch (typeof value) {
|
||||
case 'number':
|
||||
// mysql writes NaN and Infinity into the SQL as bare words, and
|
||||
// integers past 2^53 lose precision or overflow BIGINT.
|
||||
return Math.abs(value as number) <= Number.MAX_SAFE_INTEGER;
|
||||
case 'string':
|
||||
case 'bigint':
|
||||
case 'boolean':
|
||||
return true;
|
||||
case 'object':
|
||||
// Buffer is a Uint8Array.
|
||||
return value instanceof Date || value instanceof Uint8Array;
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Refuses a bound value that isn't a scalar. mysql formats queries in the
|
||||
* client and expands an array into a value list and an object into `col = val`
|
||||
* pairs, so a request value of the wrong shape would change the statement
|
||||
* instead of failing to match. `IN` lists bind one `?` per item.
|
||||
*/
|
||||
export const assertBindableParams = (
|
||||
statement: string,
|
||||
params: readonly unknown[] | undefined,
|
||||
): void => {
|
||||
if (!params) return;
|
||||
for (const value of params) {
|
||||
if (isBindable(value)) continue;
|
||||
console.warn(
|
||||
`[db] refused a ${Array.isArray(value) ? 'array' : typeof value} parameter:`,
|
||||
statement.slice(0, 200),
|
||||
);
|
||||
throw new HttpError(400, 'Invalid parameter.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
};
|
||||
@@ -89,8 +89,6 @@ const errorCode = (error: unknown): string | undefined =>
|
||||
export const isRetriableError = (error: unknown): boolean => {
|
||||
const code = errorCode(error);
|
||||
if (code && RETRIABLE_ERROR_CODES.has(code)) return true;
|
||||
// The server answered; its message can quote text such as "Connection lost".
|
||||
if (code?.startsWith('ER_')) return false;
|
||||
|
||||
const msg = String((error as Error)?.message ?? '');
|
||||
return RETRIABLE_ERROR_MESSAGES.some((m) => msg.includes(m));
|
||||
|
||||
@@ -1215,87 +1215,3 @@ describe('DDBClient — throttle logging', () => {
|
||||
expect(ddbWarnings()).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — the item a refusal returns', () => {
|
||||
let server: Server;
|
||||
let endpoint: string;
|
||||
let sent: Record<string, unknown>[];
|
||||
|
||||
beforeAll(async () => {
|
||||
server = createServer((req, res) => {
|
||||
let body = '';
|
||||
req.on('data', (chunk) => (body += chunk));
|
||||
req.on('end', () => {
|
||||
sent.push(JSON.parse(body));
|
||||
res.writeHead(400, {
|
||||
'content-type': 'application/x-amz-json-1.0',
|
||||
});
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
__type: 'com.amazonaws.dynamodb.v20120810#ConditionalCheckFailedException',
|
||||
message: 'The conditional request failed',
|
||||
Item: { pk: { S: 'held' }, ttl: { N: '42' } },
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', resolve),
|
||||
);
|
||||
endpoint = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
sent = [];
|
||||
});
|
||||
|
||||
const stubClient = () =>
|
||||
new DDBClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: {
|
||||
aws: {
|
||||
access_key: 'a',
|
||||
secret_key: 'b',
|
||||
region: 'eu-central-1',
|
||||
},
|
||||
endpoint,
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
const condition = { expression: 'attribute_not_exists(pk)' };
|
||||
|
||||
it.each([
|
||||
[
|
||||
'put',
|
||||
(o: object) => stubClient().put(TABLE, { pk: 'held', sk: 'a' }, o),
|
||||
],
|
||||
[
|
||||
'del',
|
||||
(o: object) => stubClient().del(TABLE, { pk: 'held', sk: 'a' }, o),
|
||||
],
|
||||
])(
|
||||
'%s asks for the refusing item only when told to, and hands it back undecoded',
|
||||
async (_label, write) => {
|
||||
await expect(
|
||||
write({ condition, returnOldOnRefusal: true }),
|
||||
).rejects.toMatchObject({
|
||||
name: 'ConditionalCheckFailedException',
|
||||
Item: { pk: { S: 'held' }, ttl: { N: '42' } },
|
||||
});
|
||||
expect(sent[0]).toMatchObject({
|
||||
ReturnValuesOnConditionCheckFailure: 'ALL_OLD',
|
||||
});
|
||||
|
||||
await expect(write({ condition })).rejects.toMatchObject({
|
||||
name: 'ConditionalCheckFailedException',
|
||||
});
|
||||
expect(sent[1]).not.toHaveProperty(
|
||||
'ReturnValuesOnConditionCheckFailure',
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -262,8 +262,6 @@ export class DDBClient extends PuterClient {
|
||||
names?: Record<string, string>;
|
||||
values?: Record<string, unknown>;
|
||||
};
|
||||
/** A refusal carries the refusing item, undecoded, as `Item`. */
|
||||
returnOldOnRefusal?: boolean;
|
||||
},
|
||||
) {
|
||||
const condition = opts?.condition;
|
||||
@@ -276,12 +274,6 @@ export class DDBClient extends PuterClient {
|
||||
TableName: table,
|
||||
Item: itemToWrite,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
...(opts?.returnOldOnRefusal
|
||||
? {
|
||||
ReturnValuesOnConditionCheckFailure:
|
||||
'ALL_OLD' as const,
|
||||
}
|
||||
: {}),
|
||||
...(condition
|
||||
? {
|
||||
ConditionExpression: condition.expression,
|
||||
@@ -501,8 +493,6 @@ export class DDBClient extends PuterClient {
|
||||
names?: Record<string, string>;
|
||||
values?: Record<string, unknown>;
|
||||
};
|
||||
/** A refusal carries the refusing item, undecoded, as `Item`. */
|
||||
returnOldOnRefusal?: boolean;
|
||||
},
|
||||
) {
|
||||
const command = new DeleteCommand({
|
||||
@@ -512,9 +502,6 @@ export class DDBClient extends PuterClient {
|
||||
// ALL_OLD makes the delete an atomic claim: exactly one caller
|
||||
// gets the attributes back.
|
||||
...(opts?.returnOld ? { ReturnValues: 'ALL_OLD' as const } : {}),
|
||||
...(opts?.returnOldOnRefusal
|
||||
? { ReturnValuesOnConditionCheckFailure: 'ALL_OLD' as const }
|
||||
: {}),
|
||||
...(opts?.condition
|
||||
? {
|
||||
ConditionExpression: opts.condition.expression,
|
||||
|
||||
@@ -319,51 +319,6 @@ [email protected] immediately.
|
||||
</p>
|
||||
`,
|
||||
},
|
||||
login_locked_notification: {
|
||||
subject: 'Sign-in attempts to your Puter account were blocked',
|
||||
html: `
|
||||
<p>Hi there,</p>
|
||||
<p>Someone entered the wrong password for your Puter account
|
||||
<b>{{username}}</b> many times, so we have paused password sign-in to it for
|
||||
now. Browsers you have signed in with before can still sign in.</p>
|
||||
<p>If this wasn't you, <a href="{{reset_link}}">reset your password</a> and
|
||||
<a href="{{sessions_link}}">check where you are signed in</a>. Resetting your
|
||||
password lifts the pause.</p>
|
||||
<p>Sincerely,</p>
|
||||
<p>Puter</p>
|
||||
`,
|
||||
},
|
||||
new_sign_in_notification: {
|
||||
subject: 'New sign-in to your Puter account',
|
||||
html: `
|
||||
<p>Hi there,</p>
|
||||
<p>Your Puter account <b>{{username}}</b> was just signed in to.</p>
|
||||
<p>When: {{time}}<br />
|
||||
IP address: {{ip}}<br />
|
||||
Device: {{device}}</p>
|
||||
<p>If this was you, there's nothing to do.</p>
|
||||
<p>If this wasn't you, <a href="{{security_link}}">change your password and
|
||||
review where you are signed in</a>. If you can't sign in anymore,
|
||||
<a href="{{reset_link}}">reset your password</a>.</p>
|
||||
<p>Sincerely,</p>
|
||||
<p>Puter</p>
|
||||
`,
|
||||
},
|
||||
credential_exposure_notification: {
|
||||
subject: 'Your Puter password appears in a known data breach',
|
||||
html: `
|
||||
<p>Hi there,</p>
|
||||
<p>The password you use for your Puter account <b>{{username}}</b> appears in
|
||||
publicly known breach data from other websites. This does not mean Puter was
|
||||
breached. It means the same password was exposed somewhere else, so other
|
||||
people may know it.</p>
|
||||
<p>Please <a href="{{security_link}}">change your password</a> to one you
|
||||
don't use anywhere else, and turn on two-factor authentication on the same
|
||||
page.</p>
|
||||
<p>Sincerely,</p>
|
||||
<p>Puter</p>
|
||||
`,
|
||||
},
|
||||
email_verification_code: {
|
||||
subject: '{{code}} is your confirmation code',
|
||||
html: `
|
||||
|
||||
@@ -204,29 +204,6 @@ describe('EventClient', () => {
|
||||
errSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('reverts the listed fields a throwing listener set, and keeps the rest', async () => {
|
||||
const errSpy = vi
|
||||
.spyOn(console, 'error')
|
||||
.mockImplementation(() => {});
|
||||
target.on(key, (_k, data) => {
|
||||
(data as { allow: boolean }).allow = false;
|
||||
});
|
||||
target.on(key, (_k, data) => {
|
||||
const event = data as { gate: boolean; note: string };
|
||||
event.gate = true;
|
||||
event.note = 'seen';
|
||||
throw new Error('boom');
|
||||
});
|
||||
const data = { allow: true, gate: false, note: '' };
|
||||
const meta: { listener_failed?: boolean } = {};
|
||||
await target.emitAndWait(key, data, meta, {
|
||||
revertOnFailure: ['allow', 'gate'],
|
||||
});
|
||||
expect(data).toEqual({ allow: false, gate: false, note: 'seen' });
|
||||
expect(meta.listener_failed).toBe(true);
|
||||
errSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('awaits wildcard listeners too', async () => {
|
||||
let resolved = false;
|
||||
target.on(`${key}.*`, async () => {
|
||||
|
||||
@@ -87,19 +87,14 @@ export class EventClient extends PuterClient {
|
||||
*
|
||||
* Listeners run sequentially in the order they're registered so an earlier
|
||||
* handler's mutation is visible to later ones. A listener that throws is
|
||||
* logged and the chain continues, with `meta.listener_failed` set. With
|
||||
* `revertOnFailure`, those fields of `data` go back to what they held
|
||||
* before a listener that threw, so only completed listeners' outputs
|
||||
* count.
|
||||
* logged and the chain continues, with `meta.listener_failed` set.
|
||||
*/
|
||||
async emitAndWait<T extends keyof EventMap>(
|
||||
key: T,
|
||||
data: EventMap[T],
|
||||
meta: EventMetadata,
|
||||
{ revertOnFailure }: { revertOnFailure?: readonly string[] } = {},
|
||||
) {
|
||||
if (!this.hasListeners(key)) return;
|
||||
const fields = data as Record<string, unknown>;
|
||||
// Spanned because callers block on listeners — this is where time
|
||||
// spent in extension hooks (e.g. `ip.validate`) gets attributed.
|
||||
return withSpan('event.emitAndWait', { 'event.key': key }, async () => {
|
||||
@@ -107,15 +102,9 @@ export class EventClient extends PuterClient {
|
||||
// A wildcard listener observes the event; it does not implement it.
|
||||
const isExact = matchKey === key;
|
||||
for (const listener of this.#listenersFor(matchKey)) {
|
||||
const before = revertOnFailure?.map(
|
||||
(name) => [name, fields[name]] as const,
|
||||
);
|
||||
try {
|
||||
await listener(key, data, meta);
|
||||
} catch (e) {
|
||||
for (const [name, value] of before ?? []) {
|
||||
fields[name] = value;
|
||||
}
|
||||
// The chain continues, but the caller is told: a hook
|
||||
// that could not run is not a hook that approved.
|
||||
if (isExact) meta.listener_failed = true;
|
||||
|
||||
@@ -21,7 +21,6 @@ import type {
|
||||
} from 'express';
|
||||
import { Actor } from '../../core';
|
||||
import type { EmailTemplateName } from '../email/templates';
|
||||
import type { SenderAuthOutcome } from '../../services/email/mailbox';
|
||||
import type { UsageInput } from '../../services/metering/types';
|
||||
import { FSEntry } from '../../stores/fs/FSEntry';
|
||||
|
||||
@@ -82,9 +81,6 @@ export type KvOp = 'set' | 'del' | 'expire';
|
||||
/** A flush empties a whole namespace, so it is never reported per key. */
|
||||
export type KvMutation = KvOp | 'flush';
|
||||
|
||||
/** How an interactive sign-in proved the account. */
|
||||
export type SignInMethod = 'password' | 'otp' | 'recovery_code' | 'oidc';
|
||||
|
||||
export type EventMap = {
|
||||
// ---- Server lifecycle ----
|
||||
serverStart: Record<string, never>;
|
||||
@@ -162,9 +158,9 @@ export type EventMap = {
|
||||
'app.new-icon': { app_uid: string; data_url: string };
|
||||
'app.opened': { app_uid: string; user_id: number; ts: number };
|
||||
/**
|
||||
* A user removed an app: its grants and worker sessions are gone, and its
|
||||
* workers in the account are taken down. With `deleteData`, the user asked
|
||||
* for the app's data in their account to go too. Awaited by the emitter.
|
||||
* A user removed an app: its grants and worker sessions are gone. With
|
||||
* `deleteData`, the user asked for the app's data in their account to go
|
||||
* too. Awaited by the emitter.
|
||||
*/
|
||||
'app.uninstalled': {
|
||||
userId: number;
|
||||
@@ -315,110 +311,6 @@ export type EventMap = {
|
||||
requires_card_verification: boolean;
|
||||
accepted: boolean;
|
||||
};
|
||||
/**
|
||||
* A password sign-in attempt (`POST /login`), via `emitAndWait` before the
|
||||
* account is looked up or the password checked, so a refusal looks the same
|
||||
* for every account. A listener refuses with `allow = false`; one that
|
||||
* throws counts as no opinion, and what it set is undone.
|
||||
*/
|
||||
'puter.signin.attempt': {
|
||||
/** The request; the password is gone from its body. */
|
||||
req: unknown;
|
||||
ip: string | null;
|
||||
user_agent: string | null;
|
||||
tls_fingerprint: string | null;
|
||||
/** Device fingerprint the client sent, shape-checked. */
|
||||
fingerprint: string | null;
|
||||
/** The request body without its credential fields. */
|
||||
data: Record<string, unknown>;
|
||||
allow: boolean;
|
||||
message: string | null;
|
||||
code: string | null;
|
||||
/** Shown to a refused user as the Request Code. */
|
||||
trail_id: string | undefined;
|
||||
};
|
||||
/**
|
||||
* Before the session of an interactive sign-in is minted (not signup), via
|
||||
* `emitAndWait`. A listener refuses with `allow = false`, or asks for phone
|
||||
* or card verification; a listener that throws counts as no opinion, and
|
||||
* what it set is undone.
|
||||
*/
|
||||
'puter.signin.validate': {
|
||||
/** The request; a verified password or code is gone from its body. */
|
||||
req: unknown;
|
||||
/** The account row without credentials or single-use tokens. */
|
||||
user: Record<string, unknown>;
|
||||
method: SignInMethod;
|
||||
ip: string | null;
|
||||
user_agent: string | null;
|
||||
tls_fingerprint: string | null;
|
||||
/** Device fingerprint the client sent, shape-checked. */
|
||||
fingerprint: string | null;
|
||||
/** The request body without its credential fields. */
|
||||
data: Record<string, unknown>;
|
||||
/** Whole days since the account's last activity; null if unknown. */
|
||||
dormant_days: number | null;
|
||||
is_dormant: boolean;
|
||||
known_device: boolean;
|
||||
/** The request carries the signal for a credential known to have leaked. */
|
||||
credential_exposed: boolean;
|
||||
/** Every signal the edge proxy raised on the request. */
|
||||
request_signals: readonly string[];
|
||||
account: {
|
||||
has_verified_phone: boolean;
|
||||
has_verified_card: boolean;
|
||||
otp_enabled: boolean;
|
||||
oidc_linked: boolean;
|
||||
email_confirmed: boolean;
|
||||
is_org_seat: boolean;
|
||||
is_temp: boolean;
|
||||
};
|
||||
allow: boolean;
|
||||
message: string | null;
|
||||
code: string | null;
|
||||
requires_phone_verification: boolean;
|
||||
requires_card_verification: boolean;
|
||||
/** Shown to a refused user as the Request Code. */
|
||||
trail_id: string | undefined;
|
||||
};
|
||||
/** After a sign-in's session exists, signups included. */
|
||||
'puter.signin': {
|
||||
user_id: number;
|
||||
user_uuid: string;
|
||||
username: string;
|
||||
method: SignInMethod | 'signup';
|
||||
ip: string | null;
|
||||
user_agent: string | null;
|
||||
tls_fingerprint: string | null;
|
||||
fingerprint: string | null;
|
||||
dormant_days: number | null;
|
||||
is_dormant: boolean;
|
||||
known_device: boolean;
|
||||
session_uuid: string;
|
||||
};
|
||||
/** A sign-in's password is known to be exposed, once per source. */
|
||||
'puter.signin.credential_exposed': {
|
||||
user_id: number;
|
||||
user_uuid: string;
|
||||
username: string;
|
||||
source: 'edge' | 'pwned_passwords';
|
||||
/** Whether the owner was emailed for this sign-in. */
|
||||
notified: boolean;
|
||||
};
|
||||
/** A request from a resolved actor carried request signals. */
|
||||
'puter.request.flagged': {
|
||||
signals: readonly string[];
|
||||
user_id: number | null;
|
||||
user_uuid: string | null;
|
||||
username: string | null;
|
||||
app_uid: string | null;
|
||||
/** HTTP method. */
|
||||
method: string;
|
||||
/** The matched route pattern, else the path. */
|
||||
route: string;
|
||||
ip: string | null;
|
||||
tls_fingerprint: string | null;
|
||||
};
|
||||
'email.validate': {
|
||||
email: string;
|
||||
allow: boolean;
|
||||
@@ -438,29 +330,6 @@ export type EventMap = {
|
||||
raw: Buffer;
|
||||
handled: boolean;
|
||||
};
|
||||
// Inbound mail about to be filed in a mailbox, emitted via `emitAndWait`.
|
||||
// A listener may refuse it, or record a sender check that is stored with
|
||||
// the message as the `X-Puter-Sender-Auth` header. A failed listener is
|
||||
// no opinion: the message is filed.
|
||||
'email.ingress.validate': {
|
||||
/** The envelope recipient. */
|
||||
to: string;
|
||||
/** The envelope sender (MAIL FROM), when the relay passed one. */
|
||||
envelopeFrom: string | null;
|
||||
/** The header block exactly as received, through its blank line. */
|
||||
head: Buffer;
|
||||
/** Size of the whole message in bytes. */
|
||||
size: number;
|
||||
/**
|
||||
* The whole message as received. Holds it in memory, so call it only
|
||||
* when the head says the body matters.
|
||||
*/
|
||||
readMessage: () => Promise<Buffer>;
|
||||
allow: boolean;
|
||||
/** Why it was refused; the relay hands it back to the sender. */
|
||||
message: string | null;
|
||||
senderAuth: SenderAuthOutcome | null;
|
||||
};
|
||||
'user.save_account': {
|
||||
user_id: number;
|
||||
old_username?: string;
|
||||
@@ -550,19 +419,9 @@ export type EventMap = {
|
||||
user_uid: string;
|
||||
email?: string;
|
||||
setup_intent_id: string;
|
||||
/**
|
||||
* Set when only this card may verify: the listener refuses any other,
|
||||
* and one whose details weren't entered for this confirmation.
|
||||
*/
|
||||
required_fingerprint: string | null;
|
||||
enabled: boolean | null;
|
||||
verified: boolean;
|
||||
reason: string | null;
|
||||
/**
|
||||
* The card's details were entered for this confirmation, not reused
|
||||
* from a saved card.
|
||||
*/
|
||||
card_entered: boolean;
|
||||
fingerprint: string | null;
|
||||
funding: string | null;
|
||||
country: string | null;
|
||||
@@ -1039,8 +898,6 @@ export type EventMap = {
|
||||
};
|
||||
/** `uuid` is the entry the thumbnail belongs to. */
|
||||
'thumbnail.created': { url: string; uuid: string };
|
||||
/** A thumbnail `uuid`'s entry stopped pointing at; `url` is as stored. */
|
||||
'thumbnail.discarded': { url: string; uuid: string };
|
||||
'thumbnail.upload.prepare': {
|
||||
items: { index: number; item_uid: string }[];
|
||||
uploadUrl?: string;
|
||||
|
||||
@@ -1,197 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import http from 'node:http';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { INTERNAL_AUTH_HEADER } from '../../core/http/middleware/internalAuth.js';
|
||||
import type { PuterServer } from '../../server.js';
|
||||
import type { ActiveCall } from '../../services/alarm/AlarmCallService.js';
|
||||
import { allocateEphemeralPort, setupTestServer } from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { formatCalls } from './AlarmController.js';
|
||||
|
||||
const SECRET = 'alarm-calls-test-secret';
|
||||
|
||||
/** Alarms raised in a real server, polled over a real socket. */
|
||||
describe('GET /alarms/calls', () => {
|
||||
let server: PuterServer;
|
||||
let port: number;
|
||||
|
||||
beforeAll(async () => {
|
||||
port = await allocateEphemeralPort();
|
||||
server = await setupTestServer(
|
||||
{
|
||||
port,
|
||||
domain: 'puter.localhost',
|
||||
origin: `http://puter.localhost:${port}`,
|
||||
api_base_url: `http://api.puter.localhost:${port}`,
|
||||
pager: {
|
||||
calls: {
|
||||
secret: SECRET,
|
||||
rules: {
|
||||
'billing:*': 'billing',
|
||||
'ai:outage:*': 'ai',
|
||||
},
|
||||
},
|
||||
},
|
||||
} as unknown as IConfig,
|
||||
{ listen: true },
|
||||
);
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
}, 120_000);
|
||||
|
||||
afterAll(async () => {
|
||||
vi.restoreAllMocks();
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
const get = (path: string, headers: Record<string, string> = {}) =>
|
||||
new Promise<{ status: number; body: string }>((resolve, reject) => {
|
||||
const req = http.request(
|
||||
{
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
path,
|
||||
headers: {
|
||||
host: `api.puter.localhost:${port}`,
|
||||
...headers,
|
||||
},
|
||||
},
|
||||
(res) => {
|
||||
let body = '';
|
||||
res.setEncoding('utf8');
|
||||
res.on('data', (chunk) => (body += chunk));
|
||||
res.on('end', () =>
|
||||
resolve({ status: res.statusCode ?? 0, body }),
|
||||
);
|
||||
},
|
||||
);
|
||||
req.on('error', reject);
|
||||
req.end();
|
||||
});
|
||||
|
||||
const poll = (query = '') =>
|
||||
get(`/alarms/calls${query}`, { [INTERNAL_AUTH_HEADER]: SECRET });
|
||||
|
||||
it('refuses a request without the secret', async () => {
|
||||
expect((await get('/alarms/calls')).status).toBe(403);
|
||||
expect(
|
||||
(
|
||||
await get('/alarms/calls', {
|
||||
[INTERNAL_AUTH_HEADER]: 'wrong',
|
||||
})
|
||||
).status,
|
||||
).toBe(403);
|
||||
});
|
||||
|
||||
it('rejects a group that cannot name one', async () => {
|
||||
expect((await poll('?group=a%20b')).status).toBe(400);
|
||||
expect((await poll('?group=a&group=b')).status).toBe(400);
|
||||
});
|
||||
|
||||
it('answers 200 while nothing is calling', async () => {
|
||||
const res = await poll('?group=billing');
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toBe('no active calls');
|
||||
});
|
||||
|
||||
it('answers 503 for the group an alarm calls, until it expires', async () => {
|
||||
server.clients.alarm.create(
|
||||
'billing:topup_stuck:tx1',
|
||||
'Top-up stuck crediting',
|
||||
{},
|
||||
'info',
|
||||
);
|
||||
// Not a call: no rule names it.
|
||||
server.clients.alarm.create('other:thing', 'unrelated', {}, 'info');
|
||||
|
||||
await vi.waitFor(async () =>
|
||||
expect((await poll('?group=billing')).status).toBe(503),
|
||||
);
|
||||
const res = await poll('?group=billing');
|
||||
expect(res.body).toContain('1 active call');
|
||||
expect(res.body).toContain('[billing] Top-up stuck crediting');
|
||||
|
||||
expect((await poll('?group=ai')).status).toBe(200);
|
||||
const all = await poll();
|
||||
expect(all.status).toBe(503);
|
||||
expect(all.body).not.toContain('unrelated');
|
||||
|
||||
const {
|
||||
calls: [call],
|
||||
} = await server.services.alarmCalls.listActive('billing');
|
||||
expect(call).toMatchObject({
|
||||
group: 'billing',
|
||||
id: 'billing:topup_stuck:tx1',
|
||||
severity: 'info',
|
||||
links: [],
|
||||
});
|
||||
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
try {
|
||||
vi.setSystemTime(Date.now() + 31 * 60 * 1000);
|
||||
expect((await poll('?group=billing')).status).toBe(200);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('formatCalls', () => {
|
||||
const call = (over: Partial<ActiveCall> = {}): ActiveCall => ({
|
||||
group: 'billing',
|
||||
id: 'billing:stuck',
|
||||
shortId: 'amber-delta-fox',
|
||||
message: 'Top-up stuck',
|
||||
severity: 'info',
|
||||
count: 1,
|
||||
links: [],
|
||||
at: 0,
|
||||
...over,
|
||||
});
|
||||
|
||||
it('links each call to its message, or names it by short id', () => {
|
||||
expect(
|
||||
formatCalls([
|
||||
call({ links: ['https://chat/msg/1'], count: 3 }),
|
||||
call({ group: 'ai', message: 'Provider down' }),
|
||||
]),
|
||||
).toBe(
|
||||
'2 active calls\n' +
|
||||
'[billing] Top-up stuck (x3) https://chat/msg/1\n' +
|
||||
'[ai] Provider down amber-delta-fox',
|
||||
);
|
||||
});
|
||||
|
||||
it('marks a listing that stopped early', () => {
|
||||
expect(formatCalls([], false)).toBe('call listing incomplete');
|
||||
expect(formatCalls([call()], false)).toMatch(/^1\+ active call\n/);
|
||||
});
|
||||
|
||||
it('stays short enough for a monitor to keep', () => {
|
||||
const body = formatCalls(
|
||||
Array.from({ length: 20 }, (_, i) =>
|
||||
call({ message: `${'x'.repeat(200)} ${i}` }),
|
||||
),
|
||||
);
|
||||
expect(body.length).toBeLessThanOrEqual(480);
|
||||
expect(body).toMatch(/\n\+\d+ more$/);
|
||||
expect(body.startsWith('20 active calls')).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -1,91 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import { CALL_GROUP_PATTERN } from '../../clients/alarm/severity.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { Controller, Get } from '../../core/http/decorators.js';
|
||||
import type { ActiveCall } from '../../services/alarm/AlarmCallService.js';
|
||||
import { PuterController } from '../types.js';
|
||||
|
||||
// Monitors tend to keep a failing response only while it is short.
|
||||
const MAX_BODY_LENGTH = 480;
|
||||
const MAX_LINE_MESSAGE = 120;
|
||||
|
||||
@Controller('/alarms')
|
||||
export class AlarmController extends PuterController {
|
||||
/**
|
||||
* GET /alarms/calls?group= — for an uptime monitor to turn into a phone
|
||||
* call: 503 while an alarm in `group` (any group when omitted) is calling,
|
||||
* or when the listing couldn't be finished, 200 otherwise. The body is
|
||||
* plain text, one line per call.
|
||||
*/
|
||||
@Get('/calls', {
|
||||
subdomain: 'api',
|
||||
internalAuth: (config) => config.pager?.calls?.secret,
|
||||
// The secret gates the route; this only bounds guessing.
|
||||
rateLimit: {
|
||||
scope: 'alarm-calls',
|
||||
limit: 600,
|
||||
window: 60_000,
|
||||
key: 'ip',
|
||||
},
|
||||
})
|
||||
async listCalls(req: Request, res: Response): Promise<void> {
|
||||
const { calls, complete } = await this.services.alarmCalls.listActive(
|
||||
parseGroup(req.query.group),
|
||||
);
|
||||
res.set('Cache-Control', 'no-store');
|
||||
res.type('text/plain');
|
||||
res.status(calls.length > 0 || !complete ? 503 : 200).send(
|
||||
formatCalls(calls, complete),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function parseGroup(raw: unknown): string | undefined {
|
||||
if (raw === undefined) return undefined;
|
||||
if (typeof raw === 'string' && CALL_GROUP_PATTERN.test(raw)) return raw;
|
||||
throw new HttpError(400, 'Invalid `group`', { legacyCode: 'bad_request' });
|
||||
}
|
||||
|
||||
/** An incomplete listing reads as at least what it found. */
|
||||
export function formatCalls(calls: ActiveCall[], complete = true): string {
|
||||
if (calls.length === 0) {
|
||||
return complete ? 'no active calls' : 'call listing incomplete';
|
||||
}
|
||||
|
||||
const count = `${calls.length}${complete ? '' : '+'}`;
|
||||
let body = `${count} active call${calls.length === 1 ? '' : 's'}`;
|
||||
for (const [i, call] of calls.entries()) {
|
||||
const message =
|
||||
call.message.length > MAX_LINE_MESSAGE
|
||||
? `${call.message.slice(0, MAX_LINE_MESSAGE - 1)}…`
|
||||
: call.message;
|
||||
const repeat = call.count > 1 ? ` (x${call.count})` : '';
|
||||
const where = call.links[0] ?? call.shortId;
|
||||
const line = `\n[${call.group}] ${message}${repeat} ${where}`;
|
||||
const more = `\n+${calls.length - i} more`;
|
||||
if (body.length + line.length + more.length > MAX_BODY_LENGTH) {
|
||||
return body + more;
|
||||
}
|
||||
body += line;
|
||||
}
|
||||
return body;
|
||||
}
|
||||
@@ -114,8 +114,6 @@ type CardConfirmOverride = (data: {
|
||||
enabled: boolean | null;
|
||||
verified: boolean;
|
||||
reason: string | null;
|
||||
fingerprint?: string | null;
|
||||
card_entered?: boolean;
|
||||
}) => void;
|
||||
let cardSetupOverride: CardSetupOverride | null = null;
|
||||
let cardConfirmOverride: CardConfirmOverride | null = null;
|
||||
@@ -208,8 +206,6 @@ interface MockRes {
|
||||
clearCookie(name: string): MockRes;
|
||||
send(text: string): MockRes;
|
||||
end(): MockRes;
|
||||
/** Work deferred past the response never runs here. */
|
||||
once(event: string, listener: () => void): MockRes;
|
||||
}
|
||||
|
||||
const makeRes = (): MockRes => {
|
||||
@@ -248,9 +244,6 @@ const makeRes = (): MockRes => {
|
||||
this.headersSent = true;
|
||||
return this;
|
||||
},
|
||||
once() {
|
||||
return this;
|
||||
},
|
||||
};
|
||||
return res;
|
||||
};
|
||||
@@ -6206,6 +6199,7 @@ describe('AuthController SMS → card fallback', () => {
|
||||
const { user, actor } = await makeUserAndActor({
|
||||
requires_card_verification: 1,
|
||||
requires_phone_verification: 1,
|
||||
phone: '+14155550123',
|
||||
});
|
||||
await openFallback(user.id);
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
@@ -6318,6 +6312,7 @@ describe('AuthController SMS → card fallback', () => {
|
||||
const { user, actor } = await makeUserAndActor({
|
||||
requires_card_verification: 1,
|
||||
requires_phone_verification: 1,
|
||||
phone: '+14155550123',
|
||||
});
|
||||
await openFallback(user.id);
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
@@ -6348,6 +6343,7 @@ describe('AuthController SMS → card fallback', () => {
|
||||
it('clears the phone gate via card even when card was not required', async () => {
|
||||
const { user, actor } = await makeUserAndActor({
|
||||
requires_phone_verification: 1,
|
||||
phone: '+14155550123',
|
||||
});
|
||||
await openFallback(user.id);
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
@@ -6372,349 +6368,6 @@ describe('AuthController SMS → card fallback', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('AuthController phone on file', () => {
|
||||
const ON_FILE = '+14155550123';
|
||||
const stubPrelude = (over: Record<string, unknown> = {}) => ({
|
||||
isConfigured: () => true,
|
||||
isCountrySupported: () => true,
|
||||
defaultCountry: 'US',
|
||||
createVerification: vi.fn(async () => ({ status: 'success' })),
|
||||
checkVerification: vi.fn(async () => ({ status: 'success' })),
|
||||
...over,
|
||||
});
|
||||
const withPrelude = async (
|
||||
prelude: unknown,
|
||||
fn: () => Promise<void>,
|
||||
): Promise<void> => {
|
||||
const ctrl = controller as { clients: { prelude: unknown } };
|
||||
const real = ctrl.clients.prelude;
|
||||
ctrl.clients.prelude = prelude;
|
||||
try {
|
||||
await fn();
|
||||
} finally {
|
||||
ctrl.clients.prelude = real;
|
||||
}
|
||||
};
|
||||
const withFallbackConfig = async (
|
||||
value: unknown,
|
||||
fn: () => Promise<void>,
|
||||
): Promise<void> => {
|
||||
const cfg = (controller as { config: Record<string, unknown> }).config;
|
||||
const prev = cfg.phone_verification_card_fallback;
|
||||
cfg.phone_verification_card_fallback = value;
|
||||
try {
|
||||
await fn();
|
||||
} finally {
|
||||
cfg.phone_verification_card_fallback = prev;
|
||||
}
|
||||
};
|
||||
const onFileAccount = (extra: Record<string, unknown> = {}) =>
|
||||
makeUserAndActor({
|
||||
phone: ON_FILE,
|
||||
requires_phone_verification: 1,
|
||||
...extra,
|
||||
});
|
||||
const confirmCard = async (
|
||||
actor: Actor,
|
||||
fingerprint: string,
|
||||
entered = true,
|
||||
): Promise<MockRes> => {
|
||||
const res = makeRes();
|
||||
await withCardConfirmOverride(
|
||||
(data) => {
|
||||
data.enabled = true;
|
||||
data.verified = true;
|
||||
data.fingerprint = fingerprint;
|
||||
data.card_entered = entered;
|
||||
},
|
||||
() =>
|
||||
controller.handleCardVerificationConfirm(
|
||||
makeReq({ setup_intent_id: 'seti_1' }, { actor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
return res;
|
||||
};
|
||||
|
||||
it('sends the code to the number on file, whatever number the request names', async () => {
|
||||
const { user, actor } = await onFileAccount();
|
||||
const prelude = stubPrelude();
|
||||
await withPrelude(prelude, async () => {
|
||||
await controller.handleSendConfirmPhone(
|
||||
makeReq({ phone: '+14155550999' }, { actor }),
|
||||
makeRes(),
|
||||
);
|
||||
});
|
||||
expect(prelude.createVerification).toHaveBeenCalledWith(
|
||||
ON_FILE,
|
||||
expect.anything(),
|
||||
);
|
||||
const { res: staged } = await server.stores.kv.get({
|
||||
key: `phone-verify-pending:${user.id}`,
|
||||
});
|
||||
expect(staged).toBe(ON_FILE);
|
||||
});
|
||||
|
||||
it('confirms only against the number on file', async () => {
|
||||
const { user, actor } = await onFileAccount();
|
||||
// A number staged before the step was set doesn't count.
|
||||
await server.stores.kv.set({
|
||||
key: `phone-verify-pending:${user.id}`,
|
||||
value: '+14155550999',
|
||||
});
|
||||
const prelude = stubPrelude();
|
||||
await withPrelude(prelude, async () => {
|
||||
const res = makeRes();
|
||||
await controller.handleConfirmPhone(
|
||||
makeReq({ code: '123456' }, { actor }),
|
||||
res,
|
||||
);
|
||||
expect(res.body).toMatchObject({ phone_verified: true });
|
||||
});
|
||||
expect(prelude.checkVerification).toHaveBeenCalledWith(
|
||||
ON_FILE,
|
||||
'123456',
|
||||
);
|
||||
const after = await server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(after).toMatchObject({
|
||||
phone: ON_FILE,
|
||||
requires_phone_verification: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('opens the card fallback for a lost number without sending anything', async () => {
|
||||
const { user, actor } = await onFileAccount({
|
||||
card_fingerprint: 'fp_on_file',
|
||||
});
|
||||
const prelude = stubPrelude();
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
await withPrelude(prelude, async () => {
|
||||
const res = makeRes();
|
||||
await controller.handleSendConfirmPhone(
|
||||
makeReq({ lostNumber: true }, { actor }),
|
||||
res,
|
||||
);
|
||||
expect(res.body).toEqual({ card_fallback_available: true });
|
||||
});
|
||||
});
|
||||
expect(prelude.createVerification).not.toHaveBeenCalled();
|
||||
const { res: flag } = await server.stores.kv.get({
|
||||
key: `card-fallback-open:${user.id}`,
|
||||
});
|
||||
expect(flag).toBe(true);
|
||||
});
|
||||
|
||||
it('points a lost number at support when the account has no card on file', async () => {
|
||||
const { actor } = await onFileAccount();
|
||||
const prelude = stubPrelude();
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
await withPrelude(prelude, async () => {
|
||||
await expect(
|
||||
controller.handleSendConfirmPhone(
|
||||
makeReq({ lostNumber: true }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 409,
|
||||
code: 'phone_on_file_unavailable',
|
||||
});
|
||||
});
|
||||
});
|
||||
expect(prelude.createVerification).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('clears the step only with the card already on file', async () => {
|
||||
const { user, actor } = await onFileAccount({
|
||||
card_fingerprint: 'fp_on_file',
|
||||
});
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
await withPrelude(stubPrelude(), async () => {
|
||||
await controller.handleSendConfirmPhone(
|
||||
makeReq({ lostNumber: true }, { actor }),
|
||||
makeRes(),
|
||||
);
|
||||
});
|
||||
// Another card, and the right card reused without being entered.
|
||||
for (const [fingerprint, entered] of [
|
||||
['fp_other', true],
|
||||
['fp_on_file', false],
|
||||
] as const) {
|
||||
expect(
|
||||
(await confirmCard(actor, fingerprint, entered)).body,
|
||||
).toEqual({
|
||||
card_verified: false,
|
||||
reason: 'card_not_on_file',
|
||||
});
|
||||
}
|
||||
expect(
|
||||
(await server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_phone_verification,
|
||||
).toBe(true);
|
||||
expect((await confirmCard(actor, 'fp_on_file')).body).toMatchObject(
|
||||
{ card_verified: true, phone_verified: true },
|
||||
);
|
||||
});
|
||||
// The number counts as proven for the next hour.
|
||||
expect(
|
||||
(
|
||||
await server.stores.kv.get({
|
||||
key: `phone-on-file-proven:${user.id}`,
|
||||
})
|
||||
).res,
|
||||
).toMatchObject({ phone: ON_FILE });
|
||||
expect(
|
||||
(await server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_phone_verification,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('marks the number proven only when it is the one on file', async () => {
|
||||
const proven = async (id: number) =>
|
||||
(await server.stores.kv.get({ key: `phone-on-file-proven:${id}` }))
|
||||
.res ?? null;
|
||||
|
||||
const onFile = await onFileAccount();
|
||||
await withPrelude(stubPrelude(), async () => {
|
||||
await controller.handleConfirmPhone(
|
||||
makeReq({ code: '123456' }, { actor: onFile.actor }),
|
||||
makeRes(),
|
||||
);
|
||||
});
|
||||
expect(await proven(onFile.user.id)).toMatchObject({ phone: ON_FILE });
|
||||
|
||||
// A number entered for the first time proves nothing about the past.
|
||||
const fresh = await makeUserAndActor({
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
await server.stores.kv.set({
|
||||
key: `phone-verify-pending:${fresh.user.id}`,
|
||||
value: ON_FILE,
|
||||
});
|
||||
await withPrelude(stubPrelude(), async () => {
|
||||
await controller.handleConfirmPhone(
|
||||
makeReq({ code: '123456' }, { actor: fresh.actor }),
|
||||
makeRes(),
|
||||
);
|
||||
});
|
||||
expect(await proven(fresh.user.id)).toBeNull();
|
||||
});
|
||||
|
||||
it('lets no new card through while a number on file waits, whatever the gate order', async () => {
|
||||
// Card-first order reaches the card check without the fallback.
|
||||
const { user, actor } = await onFileAccount({
|
||||
requires_card_verification: 1,
|
||||
});
|
||||
await server.stores.user.updateMetadata(user.id, {
|
||||
emailGateLast: true,
|
||||
});
|
||||
expect((await confirmCard(actor, 'fp_new')).body).toEqual({
|
||||
card_verified: false,
|
||||
reason: 'card_not_on_file',
|
||||
});
|
||||
expect(
|
||||
(await server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_card_verification,
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('offers no card for a number on file from a fallback opened earlier', async () => {
|
||||
const { user, actor } = await onFileAccount();
|
||||
await server.stores.kv.set({
|
||||
key: `card-fallback-open:${user.id}`,
|
||||
value: true,
|
||||
});
|
||||
await withFallbackConfig({ enabled: true }, async () => {
|
||||
await expect(
|
||||
controller.handleCardVerificationSetup(
|
||||
makeReq({}, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 409 });
|
||||
});
|
||||
});
|
||||
|
||||
it('asks a card step on an account with a card for that card', async () => {
|
||||
const { user, actor } = await makeUserAndActor({
|
||||
requires_card_verification: 1,
|
||||
card_fingerprint: 'fp_on_file',
|
||||
});
|
||||
for (const [fingerprint, entered] of [
|
||||
['fp_other', true],
|
||||
['fp_on_file', false],
|
||||
] as const) {
|
||||
expect((await confirmCard(actor, fingerprint, entered)).body).toEqual({
|
||||
card_verified: false,
|
||||
reason: 'card_not_on_file',
|
||||
});
|
||||
}
|
||||
expect((await confirmCard(actor, 'fp_on_file')).body).toMatchObject({
|
||||
card_verified: true,
|
||||
});
|
||||
expect(
|
||||
(await server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_card_verification,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('sends at most two codes a day to a number on file', async () => {
|
||||
const { actor } = await onFileAccount();
|
||||
const prelude = stubPrelude();
|
||||
await withPrelude(prelude, async () => {
|
||||
for (let i = 0; i < 2; i++) {
|
||||
await controller.handleSendConfirmPhone(
|
||||
makeReq({}, { actor }),
|
||||
makeRes(),
|
||||
);
|
||||
}
|
||||
await expect(
|
||||
controller.handleSendConfirmPhone(makeReq({}, { actor }), makeRes()),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 429,
|
||||
code: 'phone_on_file_send_limit',
|
||||
});
|
||||
});
|
||||
expect(prelude.createVerification).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('points a lost number at support when there is no card fallback', async () => {
|
||||
const { actor } = await onFileAccount();
|
||||
const prelude = stubPrelude();
|
||||
await withFallbackConfig({ enabled: false }, async () => {
|
||||
await withPrelude(prelude, async () => {
|
||||
await expect(
|
||||
controller.handleSendConfirmPhone(
|
||||
makeReq({ lostNumber: true }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 409,
|
||||
code: 'phone_on_file_unavailable',
|
||||
});
|
||||
});
|
||||
});
|
||||
expect(prelude.createVerification).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('lets an account with no number on file name one', async () => {
|
||||
const { actor } = await makeUserAndActor({
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
const prelude = stubPrelude();
|
||||
await withPrelude(prelude, async () => {
|
||||
await controller.handleSendConfirmPhone(
|
||||
makeReq({ phone: '+14155550999', lostNumber: true }, { actor }),
|
||||
makeRes(),
|
||||
);
|
||||
});
|
||||
expect(prelude.createVerification).toHaveBeenCalledWith(
|
||||
'+14155550999',
|
||||
expect.anything(),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AuthController.handleConfirmEmail', () => {
|
||||
it('throws 400 when code is missing', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
@@ -6868,20 +6521,13 @@ describe('AuthController.handleConfirmEmail', () => {
|
||||
email_confirmed: false,
|
||||
},
|
||||
} as Actor;
|
||||
// Mailed, so the code is current and the address check is what refuses.
|
||||
await server.services.signup.sendConfirmationEmail(claimant, 'resend', {
|
||||
code: '123456',
|
||||
});
|
||||
|
||||
await expect(
|
||||
controller.handleConfirmEmail(
|
||||
makeReq({ code: '123456' }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'email_already_in_use',
|
||||
});
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
|
||||
const untouched = await server.stores.user.getById(owner.id, {
|
||||
force: true,
|
||||
@@ -7055,17 +6701,6 @@ describe('AuthController password recovery', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('send-pass-recovery-email: 400 for a username that is not a string', async () => {
|
||||
for (const username of [['a', 'b'], { username: 1 }, 7]) {
|
||||
await expect(
|
||||
controller.handleSendPassRecoveryEmail(
|
||||
makeReq({ username }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
}
|
||||
});
|
||||
|
||||
it('send-pass-recovery-email: returns the generic message even for an unknown username (no leak)', async () => {
|
||||
const res = makeRes();
|
||||
await controller.handleSendPassRecoveryEmail(
|
||||
@@ -7753,7 +7388,7 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
});
|
||||
|
||||
it('change-email: 400 on missing/invalid email and on a confirmed-account collision', async () => {
|
||||
const { actor } = await makeUserAndActor({ email_confirmed: 1, requires_email_confirmation: 0 });
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleChangeEmail(makeReq({}, { actor }), makeRes()),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
@@ -7798,7 +7433,7 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
});
|
||||
|
||||
it('change-email: stages the new email + token on success', async () => {
|
||||
const { user, actor } = await makeUserAndActor({ email_confirmed: 1, requires_email_confirmation: 0 });
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const newEmail = `ch_${uniq()}@test.local`;
|
||||
const res = makeRes();
|
||||
await controller.handleChangeEmail(
|
||||
@@ -7859,7 +7494,7 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
});
|
||||
|
||||
it('change_email/confirm: completes the swap when the token matches the staged row', async () => {
|
||||
const { user, actor } = await makeUserAndActor({ email_confirmed: 1, requires_email_confirmation: 0 });
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const newEmail = `chc_${uniq()}@test.local`;
|
||||
await controller.handleChangeEmail(
|
||||
makeReq({ new_email: newEmail }, { actor }),
|
||||
@@ -8909,22 +8544,6 @@ describe('AuthController dev-app permission flows', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('revoke-dev-app: 400 on a parameter of the wrong type', async () => {
|
||||
const { actor } = await makeUserAndActor();
|
||||
for (const body of [
|
||||
{ app_uid: 'app-x', permission: ['fs:read'] },
|
||||
{ app_uid: { uid: 'app-x' }, permission: 'fs:read' },
|
||||
{ app_uid: 'app-x', permission: 'fs:read', meta: 'note' },
|
||||
]) {
|
||||
await expect(
|
||||
controller.handleRevokeDevApp(
|
||||
makeReq(body, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
}
|
||||
});
|
||||
|
||||
it('grant/revoke-dev-app: an unregistered `origin` cannot be redirected onto an app squatting its synthetic uid', async () => {
|
||||
// Same hole the user-app handlers close: `appUidFromOrigin`
|
||||
// synthesises `app-<uuidv5(origin)>` for an origin with no app row,
|
||||
@@ -9999,8 +9618,8 @@ describe('AuthController.handleSignup additional branches', () => {
|
||||
// No phone/card gate outstanding: stuck on the email alone.
|
||||
expect(fields.sole_gate).toBe(true);
|
||||
expect(fields.email_domain).toBe('test.local');
|
||||
expect(raised![3]).toBe('info');
|
||||
expect(raised![4]).toEqual({ dedup: true, channel: 'email' });
|
||||
expect(raised![3]).toBe('warning');
|
||||
expect(raised![4]).toEqual({ dedup: true });
|
||||
} finally {
|
||||
alarm.mockRestore();
|
||||
}
|
||||
@@ -10289,8 +9908,8 @@ describe('AuthController user-protected mutations: additional branches', () => {
|
||||
it('change-email: 400 when an unconfirmed-but-password-holding account already owns the email', async () => {
|
||||
// Other user: password set, email NOT confirmed → still blocks
|
||||
// (existing.password !== null branch).
|
||||
const { user: other } = await makeUserAndActor({ email_confirmed: 1, requires_email_confirmation: 0 });
|
||||
const { actor } = await makeUserAndActor({ email_confirmed: 1, requires_email_confirmation: 0 });
|
||||
const { user: other } = await makeUserAndActor();
|
||||
const { actor } = await makeUserAndActor();
|
||||
await expect(
|
||||
controller.handleChangeEmail(
|
||||
makeReq({ new_email: other.email! }, { actor }),
|
||||
@@ -12124,7 +11743,7 @@ describe('AuthController account mail streams', () => {
|
||||
});
|
||||
|
||||
it('sends both change-email messages on the auth stream', async () => {
|
||||
const { user, actor } = await makeUserAndActor({ email_confirmed: 1, requires_email_confirmation: 0 });
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const newEmail = `cs_${uniq()}@test.local`;
|
||||
const send = spySend();
|
||||
const sendRaw = vi
|
||||
|
||||
@@ -28,7 +28,6 @@ import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { antiCsrf } from '../../core/http/middleware/antiCsrf.js';
|
||||
import { generateCaptcha } from '../../core/http/middleware/captcha.js';
|
||||
import {
|
||||
assertVerifiedAccount,
|
||||
DEFAULT_ADMIN_USERNAMES,
|
||||
hasVerifiedCard,
|
||||
hasVerifiedPhone,
|
||||
@@ -38,10 +37,8 @@ import { isAppActor, isPlainUserActor, makeActor } from '../../core/actor.js';
|
||||
import {
|
||||
checkRateLimit,
|
||||
consumeRouteRateLimit,
|
||||
listConfiguredRateLimitBackends,
|
||||
peekRateLimit,
|
||||
peekRouteRateLimit,
|
||||
reserveRateLimit,
|
||||
} from '../../core/http/middleware/rateLimit.js';
|
||||
import {
|
||||
signStepUpToken,
|
||||
@@ -60,8 +57,6 @@ import { isOwnedEmailConflict } from '../../stores/user/UserStore.js';
|
||||
import type { CardFallbackDeps } from '../../util/cardFallback.js';
|
||||
import {
|
||||
CARD_FALLBACK_OPEN_TTL_SECONDS,
|
||||
PHONE_ON_FILE_SEND_WINDOW_MS,
|
||||
PHONE_ON_FILE_SENDS_PER_DAY,
|
||||
SEND_PHONE_RATE_LIMIT,
|
||||
SEND_PHONE_RATE_WINDOW_MS,
|
||||
cardFallbackAfterAttempts,
|
||||
@@ -71,19 +66,11 @@ import {
|
||||
isCardFallbackEnabled,
|
||||
phoneAttemptsKey,
|
||||
} from '../../util/cardFallback.js';
|
||||
import {
|
||||
hasPhoneFromBeforeDormancy,
|
||||
isCardOnFileStep,
|
||||
isEmailGateLast,
|
||||
isEmailPending,
|
||||
isPhoneOnFileStep,
|
||||
PHONE_ON_FILE_PROVEN_TTL_SECONDS,
|
||||
phoneOnFileProvenKey,
|
||||
} from '../../util/verificationOrder.js';
|
||||
import { isEmailGateLast } from '../../util/verificationOrder.js';
|
||||
import { sessionCookieFlags } from '../../util/cookieFlags.js';
|
||||
import { cleanEmail, isStorableEmail } from '../../util/email.js';
|
||||
import { isGodmodeApp } from '../../util/godmodeApps.js';
|
||||
import { parsePhone, phoneLastDigits } from '../../util/phone.js';
|
||||
import { parsePhone } from '../../util/phone.js';
|
||||
import { deleteIfEqual } from '../../util/redisLock.js';
|
||||
import {
|
||||
bonusCodeInvalidError,
|
||||
@@ -92,11 +79,7 @@ import {
|
||||
normalizeBonusCode,
|
||||
} from '../../util/signupBonus.js';
|
||||
import { isTemporaryPasswordExpired } from '../../util/temporaryPassword.js';
|
||||
import {
|
||||
buildUserDetails,
|
||||
passwordTag,
|
||||
scrubSensitive,
|
||||
} from '../../util/userDetails.js';
|
||||
import { buildUserDetails, scrubSensitive } from '../../util/userDetails.js';
|
||||
import {
|
||||
assertValidUsername,
|
||||
generateUsername,
|
||||
@@ -132,15 +115,7 @@ import type { ReauthReason } from '../../services/auth/AuthService';
|
||||
import { normalizeAbsolutePath } from '../../services/fs/resolveNode.js';
|
||||
import type { FSEntry } from '../../stores/fs/FSEntry.js';
|
||||
import { PuterController } from '../types.js';
|
||||
import type { SignInInput } from '../../services/auth/SignInService.js';
|
||||
import { claimOnce } from '../../util/kvClaim.js';
|
||||
import { knownDeviceStatus } from './knownDevice.js';
|
||||
import {
|
||||
rememberProvenDevice,
|
||||
runAfterResponse,
|
||||
signInOf,
|
||||
startWebSession,
|
||||
} from './webSession.js';
|
||||
import { startWebSession } from './webSession.js';
|
||||
import { completeOidcLoginReturn } from './oidcLoginReturn.js';
|
||||
|
||||
const FINGERPRINT_MAX_LENGTH = 128;
|
||||
@@ -300,109 +275,6 @@ const SECOND_FACTOR_FAILURE_LIMIT = {
|
||||
window: 30 * 60_000,
|
||||
} as const;
|
||||
|
||||
interface FailureLimit {
|
||||
scope: string;
|
||||
limit: number;
|
||||
window: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wrong passwords per account on `/login`, from every network together. While a
|
||||
* window is spent, password sign-in is refused whatever the password, except
|
||||
* from a device known to the account. Each attempt reserves a slot in the
|
||||
* region's limiter, kept only when the password is wrong. Keyed by
|
||||
* {@link passwordBudgetSubject}.
|
||||
*/
|
||||
const PASSWORD_FAILURE_LIMITS: readonly FailureLimit[] = [
|
||||
{ scope: 'login-password-failure', limit: 20, window: 60 * 60_000 },
|
||||
{
|
||||
scope: 'login-password-failure-day',
|
||||
limit: 50,
|
||||
window: 24 * 60 * 60_000,
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
* The same windows in the system KV, which every region reads: charged on a
|
||||
* wrong password, so guesses spread over regions share one budget.
|
||||
*/
|
||||
const SHARED_PASSWORD_FAILURE_LIMITS: readonly FailureLimit[] = [
|
||||
{ scope: 'login-password-failure-global', limit: 20, window: 60 * 60_000 },
|
||||
{
|
||||
scope: 'login-password-failure-global-day',
|
||||
limit: 50,
|
||||
window: 24 * 60 * 60_000,
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
* Whose wrong-password budget an attempt spends: the account under its current
|
||||
* password, so any password replacement starts a fresh budget everywhere.
|
||||
*/
|
||||
const passwordBudgetSubject = (user: UserRow): string =>
|
||||
`${user.uuid}:${passwordTag(user) ?? 'none'}`;
|
||||
|
||||
// Redis reserves atomically; the KV backend lists, then writes.
|
||||
const regionalPasswordBackend = (): 'redis' | undefined =>
|
||||
listConfiguredRateLimitBackends().available.includes('redis')
|
||||
? 'redis'
|
||||
: undefined;
|
||||
|
||||
/**
|
||||
* One lockout alarm and owner email a day per account and password, in every
|
||||
* region.
|
||||
*/
|
||||
const PASSWORD_LOCK_NOTICE_SECONDS = 24 * 60 * 60;
|
||||
const passwordLockNoticeKey = (subject: string): string =>
|
||||
`login-lock-notice:${subject}`;
|
||||
|
||||
/** Wrong email confirmation codes per account. Charged only on a wrong code. */
|
||||
const CONFIRM_CODE_FAILURE_LIMITS: readonly FailureLimit[] = [
|
||||
{ scope: 'confirm-email-failure', limit: 10, window: 60 * 60_000 },
|
||||
{
|
||||
scope: 'confirm-email-failure-day',
|
||||
limit: 20,
|
||||
window: 24 * 60 * 60_000,
|
||||
},
|
||||
];
|
||||
|
||||
/** Whether every window of a failure budget has room left. Fails open. */
|
||||
const failureBudgetOpen = async (
|
||||
limits: readonly FailureLimit[],
|
||||
subject: string,
|
||||
backend?: 'kv' | 'redis',
|
||||
): Promise<boolean> =>
|
||||
(
|
||||
await Promise.all(
|
||||
limits.map(({ scope, limit, window }) =>
|
||||
peekRateLimit(`${scope}:${subject}`, limit, window, backend),
|
||||
),
|
||||
)
|
||||
).every(Boolean);
|
||||
|
||||
const chargeFailure = async (
|
||||
limits: readonly FailureLimit[],
|
||||
subject: string,
|
||||
backend?: 'kv',
|
||||
): Promise<void> => {
|
||||
await Promise.all(
|
||||
limits.map(({ scope, limit, window }) =>
|
||||
checkRateLimit(`${scope}:${subject}`, limit, window, backend),
|
||||
),
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Drop credentials a handler has finished with, parsed and raw, so nothing
|
||||
* after it sees them.
|
||||
*/
|
||||
const dropUsedCredentials = (req: Request, ...fields: string[]): void => {
|
||||
delete req.rawBody;
|
||||
const body: unknown = req.body;
|
||||
if (!body || typeof body !== 'object') return;
|
||||
for (const field of fields) delete (body as Record<string, unknown>)[field];
|
||||
};
|
||||
|
||||
/** Recovery mails per account per hour, across every requester. */
|
||||
const PASS_RECOVERY_EMAIL_TARGET_LIMIT = {
|
||||
scope: 'send-pass-recovery-email-target',
|
||||
@@ -660,20 +532,13 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
if (username && typeof username !== 'string') {
|
||||
throw new HttpError(400, 'username must be a string.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
// From here the password lives in this local only; no listener reads
|
||||
// it off the request.
|
||||
dropUsedCredentials(req, 'password');
|
||||
// Before the lookup, so a refusal reads the same for every account.
|
||||
await this.services.signIn.checkAttempt(req);
|
||||
|
||||
// Look up user
|
||||
let user;
|
||||
if (username) {
|
||||
if (typeof username !== 'string')
|
||||
throw new HttpError(400, 'username must be a string.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
user = await this.stores.user.getByUsername(username);
|
||||
} else {
|
||||
user = await this.stores.user.getByEmail(email);
|
||||
@@ -708,43 +573,16 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
// The budget is reserved alongside the compare, so a burst can't all
|
||||
// pass on its last slot. A 2FA account's checks run at its code.
|
||||
const budget = passwordBudgetSubject(user);
|
||||
if (!user.otp_enabled) {
|
||||
this.services.signIn.preloadAccountFacts(req, user, 'password');
|
||||
}
|
||||
const [compared, reserved] = await Promise.allSettled([
|
||||
bcrypt.compare(password, user.password as string),
|
||||
this.#reservePasswordAttempt(req, user, budget),
|
||||
]);
|
||||
if (compared.status === 'rejected') {
|
||||
if (reserved.status === 'fulfilled') await reserved.value.release();
|
||||
throw compared.reason;
|
||||
}
|
||||
if (reserved.status === 'rejected') throw reserved.reason;
|
||||
const passwordMatch = compared.value;
|
||||
const attempt = reserved.value;
|
||||
if (attempt.locked) {
|
||||
void attempt.release();
|
||||
throw new HttpError(
|
||||
429,
|
||||
'Too many incorrect passwords for this account. Reset your password, or try again later.',
|
||||
{
|
||||
legacyCode: 'too_many_requests',
|
||||
code: 'login_locked',
|
||||
fields: { action: 'reset_password' },
|
||||
},
|
||||
);
|
||||
}
|
||||
// Verify password
|
||||
const passwordMatch = await bcrypt.compare(
|
||||
password,
|
||||
user.password as string,
|
||||
);
|
||||
if (!passwordMatch) {
|
||||
await this.#recordPasswordFailure(req, user, budget);
|
||||
throw new HttpError(401, 'Incorrect password.', {
|
||||
legacyCode: 'password_mismatch',
|
||||
});
|
||||
}
|
||||
// A right password spends nothing.
|
||||
void attempt.release();
|
||||
// An administrator-issued temporary password that was never used dies
|
||||
// rather than becoming a standing credential the team holds.
|
||||
if (isTemporaryPasswordExpired(user)) {
|
||||
@@ -754,13 +592,6 @@ export class AuthController extends PuterController {
|
||||
{ legacyCode: 'temporary_password_expired' },
|
||||
);
|
||||
}
|
||||
// Keeps only what the exposure checks need, not the password.
|
||||
const credentialCheck = this.services.signIn.prepareCredentialCheck(
|
||||
req,
|
||||
user,
|
||||
password,
|
||||
);
|
||||
if (credentialCheck) runAfterResponse(res, credentialCheck);
|
||||
|
||||
const reauthAuthId =
|
||||
this.#verifySuppliedReauthToken(req.body.reauth_token)?.authId ??
|
||||
@@ -791,7 +622,7 @@ export class AuthController extends PuterController {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.#completeLogin(req, res, user, 'password');
|
||||
await this.#completeLogin(req, res, user);
|
||||
}
|
||||
|
||||
// -- Login: OTP verification -------------------------------------
|
||||
@@ -858,17 +689,10 @@ export class AuthController extends PuterController {
|
||||
return;
|
||||
}
|
||||
await this.#consumeOtpLoginToken(token);
|
||||
dropUsedCredentials(req, 'token', 'code');
|
||||
|
||||
await this.#enforceAuthIdMatch(req, user, decoded.auth_id ?? null);
|
||||
|
||||
await this.#completeLogin(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
'otp',
|
||||
decoded.oidcContinuation,
|
||||
);
|
||||
await this.#completeLogin(req, res, user, decoded.oidcContinuation);
|
||||
}
|
||||
|
||||
// -- Login: recovery code ----------------------------------------
|
||||
@@ -951,17 +775,10 @@ export class AuthController extends PuterController {
|
||||
res.json({ proceed: false });
|
||||
return;
|
||||
}
|
||||
dropUsedCredentials(req, 'token', 'code');
|
||||
|
||||
await this.#enforceAuthIdMatch(req, user, decoded.auth_id ?? null);
|
||||
|
||||
await this.#completeLogin(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
'recovery_code',
|
||||
decoded.oidcContinuation,
|
||||
);
|
||||
await this.#completeLogin(req, res, user, decoded.oidcContinuation);
|
||||
}
|
||||
|
||||
// -- Signup ------------------------------------------------------
|
||||
@@ -1056,7 +873,7 @@ export class AuthController extends PuterController {
|
||||
legacyCode: 'account_suspended',
|
||||
});
|
||||
}
|
||||
await this.#completeLogin(req, res, existing, 'signup');
|
||||
await this.#completeLogin(req, res, existing);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1131,7 +948,7 @@ export class AuthController extends PuterController {
|
||||
fingerprint,
|
||||
bonusCode,
|
||||
});
|
||||
await this.#completeLogin(req, res, user, 'signup', undefined, {
|
||||
await this.#completeLogin(req, res, user, undefined, {
|
||||
...(confirmationEmailSent !== undefined
|
||||
? { confirmation_email_sent: confirmationEmailSent }
|
||||
: {}),
|
||||
@@ -1314,23 +1131,6 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!(await failureBudgetOpen(CONFIRM_CODE_FAILURE_LIMITS, user.uuid))
|
||||
) {
|
||||
throw new HttpError(
|
||||
429,
|
||||
'Too many incorrect codes. Try again later.',
|
||||
{ legacyCode: 'too_many_requests' },
|
||||
);
|
||||
}
|
||||
// Checked before the code, so no guess at an expired code counts.
|
||||
if (!(await this.services.signup.isConfirmationCodeCurrent(user.id))) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'This confirmation code has expired. Request a new code.',
|
||||
{ code: 'confirmation_code_expired' },
|
||||
);
|
||||
}
|
||||
// Reject before comparing when no code is stored: `String(null)` would
|
||||
// otherwise equal a submitted `"null"` and confirm the email without
|
||||
// the real code.
|
||||
@@ -1338,7 +1138,6 @@ export class AuthController extends PuterController {
|
||||
!user.email_confirm_code ||
|
||||
String(user.email_confirm_code) !== String(code)
|
||||
) {
|
||||
await chargeFailure(CONFIRM_CODE_FAILURE_LIMITS, user.uuid);
|
||||
res.json({
|
||||
email_confirmed: false,
|
||||
original_client_socket_id,
|
||||
@@ -1453,8 +1252,6 @@ export class AuthController extends PuterController {
|
||||
*/
|
||||
private async recordPhoneAttemptForFallback(user: {
|
||||
id: number;
|
||||
phone?: string | null;
|
||||
card_fingerprint?: string | null;
|
||||
requires_phone_verification?: boolean | number | null;
|
||||
}): Promise<boolean> {
|
||||
const attempts = await this.bumpPhoneAttempts(user.id);
|
||||
@@ -1468,13 +1265,9 @@ export class AuthController extends PuterController {
|
||||
*/
|
||||
private async openCardFallback(user: {
|
||||
id: number;
|
||||
phone?: string | null;
|
||||
card_fingerprint?: string | null;
|
||||
requires_phone_verification?: boolean | number | null;
|
||||
}): Promise<boolean> {
|
||||
if (!user.requires_phone_verification) return false;
|
||||
// A number on file can only be stood in for by the card on file.
|
||||
if (isPhoneOnFileStep(user) && !user.card_fingerprint) return false;
|
||||
const deps = this.cardFallbackDeps();
|
||||
if (!(await isCardFallbackEnabled(this.config, deps))) return false;
|
||||
try {
|
||||
@@ -1556,21 +1349,6 @@ export class AuthController extends PuterController {
|
||||
'Card verification must be completed first.',
|
||||
{ legacyCode: 'conflict' },
|
||||
);
|
||||
// An account proving the number it already has gets the code there and
|
||||
// can't name another. Losing that number opens the card fallback.
|
||||
const onFile = isPhoneOnFileStep(user);
|
||||
if (onFile && req.body?.lostNumber === true) {
|
||||
if (await this.openCardFallback(user)) {
|
||||
res.json({ card_fallback_available: true });
|
||||
return;
|
||||
}
|
||||
throw new HttpError(
|
||||
409,
|
||||
'Email [email protected] to verify your account another way.',
|
||||
{ legacyCode: 'conflict', code: 'phone_on_file_unavailable' },
|
||||
);
|
||||
}
|
||||
|
||||
if (!this.clients.prelude?.isConfigured())
|
||||
throw await this.smsSendError(
|
||||
503,
|
||||
@@ -1583,7 +1361,7 @@ export class AuthController extends PuterController {
|
||||
// Parse to E.164 (Prelude's required form + the stored form) and the
|
||||
// country, so we can apply the per-country cost cap.
|
||||
const parsed = parsePhone(
|
||||
onFile ? user.phone : req.body?.phone,
|
||||
req.body?.phone,
|
||||
this.clients.prelude.defaultCountry,
|
||||
);
|
||||
if (!parsed)
|
||||
@@ -1619,28 +1397,6 @@ export class AuthController extends PuterController {
|
||||
{ legacyCode: 'phone_country_not_supported' as never },
|
||||
);
|
||||
|
||||
// Codes to a number on file are sent without the owner typing it, so
|
||||
// the account gets few of them.
|
||||
if (
|
||||
onFile &&
|
||||
!(await checkRateLimit(
|
||||
`phone-on-file-send:${user.id}`,
|
||||
PHONE_ON_FILE_SENDS_PER_DAY,
|
||||
PHONE_ON_FILE_SEND_WINDOW_MS,
|
||||
))
|
||||
) {
|
||||
const fallback = await this.openCardFallback(user);
|
||||
throw new HttpError(
|
||||
429,
|
||||
'Too many codes were sent to this number today. Try again tomorrow.',
|
||||
{
|
||||
legacyCode: 'too_many_requests',
|
||||
code: 'phone_on_file_send_limit',
|
||||
fields: fallback ? { card_fallback_available: true } : {},
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// Counted before the abuse / Prelude checks so a blocked attempt still
|
||||
// counts toward the fallback threshold.
|
||||
const fallbackAvailable =
|
||||
@@ -1817,9 +1573,6 @@ export class AuthController extends PuterController {
|
||||
res.json({
|
||||
...fallbackFields,
|
||||
...(deliveryChannel ? { channel: deliveryChannel } : {}),
|
||||
// Where the code went, for a client that didn't know the account
|
||||
// had a number to prove.
|
||||
...(onFile ? { phone_on_file: phoneLastDigits(parsed.e164) } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1859,20 +1612,17 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
// The number being verified is the one staged at send time (KV
|
||||
// pending), not the user row — we don't persist an unverified number.
|
||||
// An account proving the number on file proves only that one.
|
||||
// Fall back to a row value for accounts that already have one on file
|
||||
// (legacy / a number persisted by a prior verified flow).
|
||||
const pendingPhoneKey = `phone-verify-pending:${user.id}`;
|
||||
let pendingPhone: string | null = isPhoneOnFileStep(user)
|
||||
? (user.phone as string)
|
||||
: null;
|
||||
if (!pendingPhone) {
|
||||
try {
|
||||
const { res: staged } = await this.stores.kv.get({
|
||||
key: pendingPhoneKey,
|
||||
});
|
||||
if (typeof staged === 'string' && staged) pendingPhone = staged;
|
||||
} catch (e) {
|
||||
console.warn('[confirm-phone] pending read failed:', e);
|
||||
}
|
||||
let pendingPhone: string | null = null;
|
||||
try {
|
||||
const { res: staged } = await this.stores.kv.get({
|
||||
key: pendingPhoneKey,
|
||||
});
|
||||
if (typeof staged === 'string' && staged) pendingPhone = staged;
|
||||
} catch (e) {
|
||||
console.warn('[confirm-phone] pending read failed:', e);
|
||||
}
|
||||
if (!pendingPhone) pendingPhone = user.phone ?? null;
|
||||
if (!pendingPhone)
|
||||
@@ -1905,18 +1655,10 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
|
||||
// Verified — persist the number now (and only now) and clear the gate.
|
||||
const provedOnFile = isPhoneOnFileStep(user);
|
||||
if (provedOnFile) await this.#beforeProofOnFile(req);
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_phone_verification: 0,
|
||||
phone: pendingPhone,
|
||||
});
|
||||
if (provedOnFile) {
|
||||
await this.#afterProofOnFile(req, user, {
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
await this.#markPhoneOnFileProven(req, user);
|
||||
}
|
||||
await this.sendDeferredConfirmation(user.id);
|
||||
|
||||
// Run the verified-event listeners synchronously (emitAndWait, not
|
||||
@@ -2136,28 +1878,13 @@ export class AuthController extends PuterController {
|
||||
{ legacyCode: 'conflict' },
|
||||
);
|
||||
|
||||
// While a number on file waits to be proven, the only card that
|
||||
// verifies is the one already on file, entered again. A card step on
|
||||
// an account that has a card asks for that card too.
|
||||
const onFileStep = isPhoneOnFileStep(user);
|
||||
if (onFileStep && !user.card_fingerprint) {
|
||||
res.json({ card_verified: false, reason: 'card_not_on_file' });
|
||||
return;
|
||||
}
|
||||
const requiredFingerprint =
|
||||
onFileStep || isCardOnFileStep(user)
|
||||
? (user.card_fingerprint as string)
|
||||
: null;
|
||||
|
||||
const confirmEvent = {
|
||||
user_id: user.id,
|
||||
user_uid: user.uuid,
|
||||
email: user.email!,
|
||||
setup_intent_id,
|
||||
required_fingerprint: requiredFingerprint,
|
||||
enabled: null as boolean | null,
|
||||
verified: false,
|
||||
card_entered: false,
|
||||
reason: null as string | null,
|
||||
fingerprint: null as string | null,
|
||||
funding: null as string | null,
|
||||
@@ -2199,15 +1926,6 @@ export class AuthController extends PuterController {
|
||||
res.json({ card_verified: false, reason: confirmEvent.reason });
|
||||
return;
|
||||
}
|
||||
// Checked here too, so the rule doesn't rest on the listener alone.
|
||||
if (
|
||||
requiredFingerprint !== null &&
|
||||
(confirmEvent.fingerprint !== requiredFingerprint ||
|
||||
confirmEvent.card_entered !== true)
|
||||
) {
|
||||
res.json({ card_verified: false, reason: 'card_not_on_file' });
|
||||
return;
|
||||
}
|
||||
|
||||
// A fallback card clears the phone gate too — the point of the
|
||||
// fallback. `fallbackEligible &&` makes the invariant local instead
|
||||
@@ -2215,21 +1933,10 @@ export class AuthController extends PuterController {
|
||||
// ever clear a phone gate.
|
||||
const clearedPhoneGate =
|
||||
fallbackEligible && Boolean(user.requires_phone_verification);
|
||||
if (requiredFingerprint !== null) await this.#beforeProofOnFile(req);
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_card_verification: 0,
|
||||
...(clearedPhoneGate ? { requires_phone_verification: 0 } : {}),
|
||||
});
|
||||
// Only the gates this request cleared go back on if it fails.
|
||||
if (requiredFingerprint !== null)
|
||||
await this.#afterProofOnFile(req, user, {
|
||||
...(user.requires_card_verification
|
||||
? { requires_card_verification: 1 }
|
||||
: {}),
|
||||
...(clearedPhoneGate ? { requires_phone_verification: 1 } : {}),
|
||||
});
|
||||
if (clearedPhoneGate && onFileStep)
|
||||
await this.#markPhoneOnFileProven(req, user);
|
||||
await this.sendDeferredConfirmation(user.id);
|
||||
|
||||
// Awaited like `user.phone-verified`, so whatever a listener grants on
|
||||
@@ -2310,10 +2017,6 @@ export class AuthController extends PuterController {
|
||||
|
||||
let user;
|
||||
if (username) {
|
||||
if (typeof username !== 'string')
|
||||
throw new HttpError(400, 'username must be a string.', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
user = await this.stores.user.getByUsername(username);
|
||||
} else {
|
||||
if (!isStorableEmail(email))
|
||||
@@ -2583,7 +2286,6 @@ export class AuthController extends PuterController {
|
||||
await this.services.auth.revokeInteractiveSessionsForUserId(
|
||||
user.id as number,
|
||||
);
|
||||
await this.#sendPasswordChangeNotification(user);
|
||||
|
||||
res.send('Password successfully updated.');
|
||||
}
|
||||
@@ -2639,7 +2341,20 @@ export class AuthController extends PuterController {
|
||||
// rows); only the session that changed the password survives.
|
||||
await this.services.auth.revokeAllSessions(req.actor!);
|
||||
|
||||
await this.#sendPasswordChangeNotification(user);
|
||||
if (this.clients.email && user.email) {
|
||||
try {
|
||||
await this.clients.email.send(
|
||||
user.email,
|
||||
'password_change_notification',
|
||||
{
|
||||
username: user.username,
|
||||
},
|
||||
{ stream: 'auth' },
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[change-password] notification send failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
res.send('Password successfully updated.');
|
||||
}
|
||||
@@ -2777,9 +2492,6 @@ export class AuthController extends PuterController {
|
||||
|
||||
@Post('/user-protected/change-email', {
|
||||
userProtected: true,
|
||||
// A pending email step is checked in the handler; every other gate
|
||||
// still applies there.
|
||||
allowUnconfirmed: true,
|
||||
rateLimit: {
|
||||
scope: 'change-email-start',
|
||||
limit: 10,
|
||||
@@ -2807,7 +2519,6 @@ export class AuthController extends PuterController {
|
||||
throw new HttpError(404, 'User not found', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
await this.#assertEmailChangeAllowed(req, user);
|
||||
this.#requireAccountTwoFactor(user, req.body?.code);
|
||||
|
||||
const { new_email } = req.body ?? {};
|
||||
@@ -2890,100 +2601,6 @@ export class AuthController extends PuterController {
|
||||
res.json({});
|
||||
}
|
||||
|
||||
/**
|
||||
* A dormant account whose address was never confirmed can replace it while
|
||||
* the code is pending, so a dead or mistyped one isn't a dead end. The
|
||||
* password alone doesn't prove the account is the caller's: it also takes
|
||||
* the number the account had before, proven again within the hour (by its
|
||||
* code, or by the card on file). Without that the code is still required.
|
||||
*/
|
||||
async #assertEmailChangeAllowed(
|
||||
req: Request,
|
||||
user: UserRow,
|
||||
): Promise<void> {
|
||||
if (!isEmailPending(user)) {
|
||||
assertVerifiedAccount(user);
|
||||
return;
|
||||
}
|
||||
// A second factor can't be enabled before the address is confirmed,
|
||||
// and a provider-only account would lose its sign-in when the change
|
||||
// unlinks its providers.
|
||||
if (user.password === null || !hasPhoneFromBeforeDormancy(user))
|
||||
assertVerifiedAccount(user);
|
||||
assertVerifiedAccount({ ...user, requires_email_confirmation: false });
|
||||
|
||||
let proven = false;
|
||||
try {
|
||||
const { res } = await this.stores.kv.get({
|
||||
key: phoneOnFileProvenKey(user.id),
|
||||
});
|
||||
// The proof counts for the sign-in that gave it, not the account.
|
||||
const proof = res as { phone?: unknown; signIn?: unknown } | null;
|
||||
const signIn = signInOf(req);
|
||||
proven =
|
||||
Boolean(user.phone) &&
|
||||
Boolean(signIn) &&
|
||||
proof?.phone === user.phone &&
|
||||
proof?.signIn === signIn;
|
||||
} catch (e) {
|
||||
console.warn('[change-email] phone proof unreadable:', e);
|
||||
}
|
||||
if (proven) return;
|
||||
// Ask for the number again, where a code can be sent at all.
|
||||
if (!this.clients.prelude?.isConfigured()) assertVerifiedAccount(user);
|
||||
await this.stores.user.update(user.id, {
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
throw new HttpError(
|
||||
403,
|
||||
'Please verify your phone number to continue',
|
||||
{ legacyCode: 'phone_verification_required' },
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The phone or card on file was just proven. The gate is on the account, so
|
||||
* clearing it lets every session through: the account's other sign-ins are
|
||||
* signed out first, while the gate still holds them.
|
||||
*/
|
||||
async #beforeProofOnFile(req: Request): Promise<void> {
|
||||
await this.services.auth.revokeAllSessions(req.actor!);
|
||||
}
|
||||
|
||||
/**
|
||||
* The gate is clear. Sign-ins made since the first pass are signed out too,
|
||||
* and this one's device is remembered. If they can't be signed out, `gates`
|
||||
* goes back on and the request fails.
|
||||
*/
|
||||
async #afterProofOnFile(
|
||||
req: Request,
|
||||
user: UserRow,
|
||||
gates: Record<string, number>,
|
||||
): Promise<void> {
|
||||
try {
|
||||
await this.services.auth.revokeAllSessions(req.actor!);
|
||||
} catch (e) {
|
||||
await this.stores.user.update(user.id, gates);
|
||||
throw e;
|
||||
}
|
||||
await rememberProvenDevice(req, user.uuid, { kv: this.stores.kv });
|
||||
}
|
||||
|
||||
/** Best effort: a failed write only means the number is asked for again. */
|
||||
async #markPhoneOnFileProven(req: Request, user: UserRow): Promise<void> {
|
||||
try {
|
||||
await this.stores.kv.set({
|
||||
key: phoneOnFileProvenKey(user.id),
|
||||
value: { phone: user.phone, signIn: signInOf(req) },
|
||||
expireAt:
|
||||
Math.floor(Date.now() / 1000) +
|
||||
PHONE_ON_FILE_PROVEN_TTL_SECONDS,
|
||||
});
|
||||
} catch (e) {
|
||||
console.warn('[verification] phone proof not recorded:', e);
|
||||
}
|
||||
}
|
||||
|
||||
@Get('/change_email/confirm', {
|
||||
subdomain: ['api', ''],
|
||||
rateLimit: {
|
||||
@@ -3086,22 +2703,6 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
|
||||
await this.stores.oidc.unlinkAllByUserId(user.id);
|
||||
// Replacing an address that was never confirmed confirms the account,
|
||||
// once however many times the link is followed at once.
|
||||
if (
|
||||
!user.email_confirmed &&
|
||||
(await claimOnce(
|
||||
this.stores.kv,
|
||||
`change-email-confirmed:${decoded.token}`,
|
||||
60 * 60,
|
||||
))
|
||||
) {
|
||||
await this.services.signup.announceEmailConfirmed({
|
||||
...user,
|
||||
email: newEmail,
|
||||
email_confirmed: true,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
this.clients.event?.emit(
|
||||
@@ -4158,12 +3759,6 @@ export class AuthController extends PuterController {
|
||||
async handleRevokeDevApp(req: Request, res: Response): Promise<void> {
|
||||
let { app_uid } = req.body ?? {};
|
||||
const { origin, permission, meta } = req.body ?? {};
|
||||
this.#validateAppPermissionParams({
|
||||
app_uid,
|
||||
origin,
|
||||
permission,
|
||||
meta,
|
||||
});
|
||||
if (origin && !app_uid) {
|
||||
// Registered apps only — see handleGrantDevApp.
|
||||
app_uid = await this.#registeredAppUidFromOrigin(origin);
|
||||
@@ -5181,134 +4776,6 @@ export class AuthController extends PuterController {
|
||||
await checkRateLimit(`${scope}:${userUid}`, limit, window);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reserve this attempt in the account's wrong-password budget. Locked when
|
||||
* a regional window refuses the reservation or a shared window is spent,
|
||||
* unless the device signed in to the account before (whose wrong guesses
|
||||
* still count). The device read also serves the sign-in that may follow.
|
||||
* `release` gives the reservation back; it never rejects.
|
||||
*/
|
||||
async #reservePasswordAttempt(
|
||||
req: Request,
|
||||
user: UserRow,
|
||||
budget: string,
|
||||
): Promise<{ locked: boolean; release: () => Promise<void> }> {
|
||||
const regional = regionalPasswordBackend();
|
||||
const [reservations, sharedOpen, device] = await Promise.all([
|
||||
Promise.all(
|
||||
PASSWORD_FAILURE_LIMITS.map(({ scope, limit, window }) =>
|
||||
reserveRateLimit(
|
||||
`${scope}:${budget}`,
|
||||
limit,
|
||||
window,
|
||||
regional,
|
||||
),
|
||||
),
|
||||
),
|
||||
failureBudgetOpen(SHARED_PASSWORD_FAILURE_LIMITS, budget, 'kv'),
|
||||
knownDeviceStatus(req, user.uuid, {
|
||||
token: this.services.token,
|
||||
kv: this.stores.kv,
|
||||
}),
|
||||
]);
|
||||
const spent = !sharedOpen || reservations.some((r) => !r.ok);
|
||||
return {
|
||||
// An account with no email can't reset its way out, so a lock
|
||||
// would let anyone keep it out of new browsers.
|
||||
locked: spent && !device.known && Boolean(user.email),
|
||||
release: async () => {
|
||||
await Promise.all(reservations.map((r) => r.release()));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** The reservation charged this region; the shared windows are charged here. */
|
||||
async #recordPasswordFailure(
|
||||
req: Request,
|
||||
user: UserRow,
|
||||
budget: string,
|
||||
): Promise<void> {
|
||||
await chargeFailure(SHARED_PASSWORD_FAILURE_LIMITS, budget, 'kv');
|
||||
const open = await Promise.all([
|
||||
failureBudgetOpen(
|
||||
PASSWORD_FAILURE_LIMITS,
|
||||
budget,
|
||||
regionalPasswordBackend(),
|
||||
),
|
||||
failureBudgetOpen(SHARED_PASSWORD_FAILURE_LIMITS, budget, 'kv'),
|
||||
]);
|
||||
if (open.every(Boolean)) return;
|
||||
await this.#notifyPasswordLock(req, user, budget);
|
||||
}
|
||||
|
||||
/**
|
||||
* Alarm and email the owner that password sign-in is locked, once a day per
|
||||
* account and password. Best effort: a failed notice never changes the
|
||||
* answer.
|
||||
*/
|
||||
async #notifyPasswordLock(
|
||||
req: Request,
|
||||
user: UserRow,
|
||||
budget: string,
|
||||
): Promise<void> {
|
||||
const claimed = await claimOnce(
|
||||
this.stores.kv,
|
||||
passwordLockNoticeKey(budget),
|
||||
PASSWORD_LOCK_NOTICE_SECONDS,
|
||||
);
|
||||
if (!claimed) return;
|
||||
try {
|
||||
this.clients.alarm?.create(
|
||||
`auth:login-locked:${user.uuid}`,
|
||||
user.email
|
||||
? 'Password sign-in locked after repeated wrong passwords'
|
||||
: 'Repeated wrong passwords on an account with no email (not locked)',
|
||||
{
|
||||
username: user.username,
|
||||
user_uuid: user.uuid,
|
||||
ip: req.ip || req.socket?.remoteAddress || null,
|
||||
},
|
||||
'warning',
|
||||
{ dedup: true },
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[login] lockout alarm failed:', e);
|
||||
}
|
||||
if (!this.clients.email || !user.email) return;
|
||||
const origin = this.config.origin ?? '';
|
||||
try {
|
||||
await this.clients.email.send(
|
||||
user.email,
|
||||
'login_locked_notification',
|
||||
{
|
||||
username: user.username,
|
||||
reset_link: `${origin}/action/password-recovery`,
|
||||
sessions_link: `${origin}/dashboard#security`,
|
||||
},
|
||||
{ stream: 'auth' },
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[login] lockout notice send failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
async #sendPasswordChangeNotification(user: {
|
||||
email?: string | null;
|
||||
username: string;
|
||||
}): Promise<void> {
|
||||
if (!this.clients.email || !user.email) return;
|
||||
try {
|
||||
await this.clients.email.send(
|
||||
user.email,
|
||||
'password_change_notification',
|
||||
{ username: user.username },
|
||||
{ stream: 'auth' },
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[password] change notification send failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* An `otp-login` JWT completes one sign-in; a second redemption is refused.
|
||||
* Fails closed: while the store can't say whether the token was used, the
|
||||
@@ -5394,38 +4861,21 @@ export class AuthController extends PuterController {
|
||||
async #completeLogin(
|
||||
req: Request,
|
||||
res: Response,
|
||||
resolvedUser: UserRow,
|
||||
method: SignInInput['method'],
|
||||
user: UserRow,
|
||||
oidcContinuation?: unknown,
|
||||
extra: { confirmation_email_sent?: boolean } = {},
|
||||
): Promise<void> {
|
||||
const redirectUrl =
|
||||
oidcContinuation === undefined
|
||||
? undefined
|
||||
: completeOidcLoginReturn(res, resolvedUser, oidcContinuation, {
|
||||
: completeOidcLoginReturn(res, user, oidcContinuation, {
|
||||
config: this.config,
|
||||
oidc: this.services.oidc,
|
||||
});
|
||||
// The row it hands back carries the gates the sign-in just set, so
|
||||
// the client opens the right verification flow straight away.
|
||||
const {
|
||||
guiToken: gui_token,
|
||||
user,
|
||||
orgSeat,
|
||||
} = await startWebSession(
|
||||
req,
|
||||
res,
|
||||
resolvedUser,
|
||||
{
|
||||
config: this.config,
|
||||
auth: this.services.auth,
|
||||
signIn: this.services.signIn,
|
||||
token: this.services.token,
|
||||
kv: this.stores.kv,
|
||||
user: this.stores.user,
|
||||
},
|
||||
method,
|
||||
);
|
||||
const gui_token = await startWebSession(req, res, user, {
|
||||
config: this.config,
|
||||
auth: this.services.auth,
|
||||
});
|
||||
|
||||
// The whoami payload, so a no-reload login can store it as
|
||||
// window.user verbatim and skip a round-trip on first paint.
|
||||
@@ -5437,7 +4887,7 @@ export class AuthController extends PuterController {
|
||||
stores: this.stores,
|
||||
services: this.services,
|
||||
},
|
||||
{ isUser: true, orgSeat },
|
||||
{ isUser: true },
|
||||
);
|
||||
scrubSensitive(details);
|
||||
|
||||
|
||||
@@ -1,262 +0,0 @@
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
createTestUser,
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
|
||||
describe('changing an email that is still waiting for its code', () => {
|
||||
let env: PuterTestEnv;
|
||||
const password = 'pending-email-password';
|
||||
const links = new Map<string, string>();
|
||||
const PHONE = '+14155550123';
|
||||
// What a dormant sign-in records about the number the account had.
|
||||
const BEFORE = { phoneAtDormantSignIn: PHONE };
|
||||
let smsConfigured: ReturnType<typeof vi.spyOn>;
|
||||
const checkVerification = vi.fn(async () => ({ status: 'success' }));
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
vi.spyOn(env.server.clients.email, 'send').mockImplementation(
|
||||
async (to, template, values) => {
|
||||
if (template === 'email_verification_link')
|
||||
links.set(to, String(values?.link));
|
||||
return null;
|
||||
},
|
||||
);
|
||||
const prelude = env.server.clients.prelude;
|
||||
smsConfigured = vi.spyOn(prelude, 'isConfigured').mockReturnValue(true);
|
||||
vi.spyOn(prelude, 'checkVerification').mockImplementation(
|
||||
checkVerification as never,
|
||||
);
|
||||
}, 120_000);
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
const account = async ({
|
||||
metadata,
|
||||
...patch
|
||||
}: Record<string, unknown> & { metadata?: Record<string, unknown> }) => {
|
||||
const username = `pendmail_${uuidv4().slice(0, 8)}`;
|
||||
const credentials = await createTestUser(env.server, {
|
||||
username,
|
||||
password,
|
||||
});
|
||||
const user = (await env.server.stores.user.getByUsername(username))!;
|
||||
await env.server.stores.user.update(user.id, {
|
||||
email: `${username}@example.com`,
|
||||
email_confirmed: false,
|
||||
requires_email_confirmation: true,
|
||||
...patch,
|
||||
});
|
||||
if (metadata)
|
||||
await env.server.stores.user.updateMetadata(user.id, metadata);
|
||||
return { ...credentials, id: user.id };
|
||||
};
|
||||
|
||||
const change = (token: string, newEmail: string) =>
|
||||
fetch(new URL('/user-protected/change-email', env.origin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
cookie: `puter_auth_token=${token}`,
|
||||
},
|
||||
body: JSON.stringify({ password, new_email: newEmail }),
|
||||
});
|
||||
|
||||
const staged = async (id: number) =>
|
||||
(await env.server.stores.user.getById(id, { force: true }))!
|
||||
.unconfirmed_change_email;
|
||||
|
||||
it.each([
|
||||
['nothing verified', {}],
|
||||
['a number with no record from before', { phone: PHONE }],
|
||||
[
|
||||
'a number other than the one it had before',
|
||||
{ phone: '+14155550199', metadata: BEFORE },
|
||||
],
|
||||
['only a card', { card_fingerprint: 'fp_1' }],
|
||||
])('keeps asking for the code from an account with %s', async (_, patch) => {
|
||||
const user = await account(patch);
|
||||
const response = await change(user.token, `${uuidv4()}@example.com`);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.json()).toMatchObject({
|
||||
code: 'email_confirmation_required',
|
||||
});
|
||||
expect(await staged(user.id)).toBeFalsy();
|
||||
// No phone step was started for it either.
|
||||
expect(
|
||||
(await env.server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_phone_verification,
|
||||
).toBeFalsy();
|
||||
});
|
||||
|
||||
it('keeps asking a provider-only account for the code', async () => {
|
||||
const user = await account({
|
||||
phone: PHONE,
|
||||
metadata: BEFORE,
|
||||
password: null,
|
||||
});
|
||||
const proof = env.server.services.token.sign(
|
||||
'oidc-state',
|
||||
{
|
||||
user_uuid: (await env.server.stores.user.getById(user.id))!
|
||||
.uuid,
|
||||
purpose: 'revalidate',
|
||||
},
|
||||
{ expiresIn: '5m' },
|
||||
);
|
||||
const response = await fetch(
|
||||
new URL('/user-protected/change-email', env.origin),
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
cookie: `puter_auth_token=${user.token}; puter_revalidation=${proof}`,
|
||||
},
|
||||
body: JSON.stringify({ new_email: `${uuidv4()}@example.com` }),
|
||||
},
|
||||
);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.json()).toMatchObject({
|
||||
code: 'email_confirmation_required',
|
||||
});
|
||||
expect(await staged(user.id)).toBeFalsy();
|
||||
});
|
||||
|
||||
it('asks for the number it had before, then lets the proven account replace the address', async () => {
|
||||
const user = await account({ phone: PHONE, metadata: BEFORE });
|
||||
const newEmail = `${uuidv4()}@example.com`;
|
||||
const confirmed: unknown[] = [];
|
||||
env.server.clients.event.on('user.email-confirmed', (_key, data) => {
|
||||
confirmed.push(data);
|
||||
});
|
||||
|
||||
// Having the number isn't enough: it has to be proven now.
|
||||
const asked = await change(user.token, newEmail);
|
||||
expect(asked.status).toBe(403);
|
||||
expect(await asked.json()).toMatchObject({
|
||||
code: 'phone_verification_required',
|
||||
});
|
||||
expect(await staged(user.id)).toBeFalsy();
|
||||
expect(
|
||||
(await env.server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_phone_verification,
|
||||
).toBe(true);
|
||||
|
||||
// The code sent to that number clears the step.
|
||||
const proof = await fetch(new URL('/confirm-phone', env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${user.token}`,
|
||||
},
|
||||
body: JSON.stringify({ code: '123456' }),
|
||||
});
|
||||
expect(await proof.json()).toMatchObject({ phone_verified: true });
|
||||
expect(checkVerification).toHaveBeenLastCalledWith(PHONE, '123456');
|
||||
|
||||
expect((await change(user.token, newEmail)).status).toBe(200);
|
||||
expect(
|
||||
(await fetch(links.get(newEmail)!, { redirect: 'manual' })).status,
|
||||
).toBe(200);
|
||||
const after = await env.server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
});
|
||||
expect(after).toMatchObject({
|
||||
email: newEmail,
|
||||
email_confirmed: true,
|
||||
requires_email_confirmation: false,
|
||||
});
|
||||
// The first confirmed address, so its listeners hear about it.
|
||||
await vi.waitFor(() =>
|
||||
expect(confirmed).toContainEqual({
|
||||
user_id: user.id,
|
||||
user_uid: after!.uuid,
|
||||
email: newEmail,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not count a proof another sign-in gave', async () => {
|
||||
const user = await account({
|
||||
phone: PHONE,
|
||||
metadata: BEFORE,
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
const proof = await fetch(new URL('/confirm-phone', env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${user.token}`,
|
||||
},
|
||||
body: JSON.stringify({ code: '123456' }),
|
||||
});
|
||||
expect(await proof.json()).toMatchObject({ phone_verified: true });
|
||||
|
||||
// Signed in afterwards with the password alone.
|
||||
const row = (await env.server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
}))!;
|
||||
const later = await env.server.services.auth.createSessionToken(
|
||||
row,
|
||||
{},
|
||||
);
|
||||
const refused = await change(later.token, `${uuidv4()}@example.com`);
|
||||
expect(refused.status).toBe(403);
|
||||
expect(await refused.json()).toMatchObject({
|
||||
code: 'phone_verification_required',
|
||||
});
|
||||
expect(await staged(user.id)).toBeFalsy();
|
||||
});
|
||||
|
||||
it('keeps asking for the code where no phone code can be sent', async () => {
|
||||
const user = await account({ phone: PHONE, metadata: BEFORE });
|
||||
smsConfigured.mockReturnValue(false);
|
||||
try {
|
||||
const response = await change(
|
||||
user.token,
|
||||
`${uuidv4()}@example.com`,
|
||||
);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.json()).toMatchObject({
|
||||
code: 'email_confirmation_required',
|
||||
});
|
||||
} finally {
|
||||
smsConfigured.mockReturnValue(true);
|
||||
}
|
||||
expect(
|
||||
(await env.server.stores.user.getById(user.id, { force: true }))!
|
||||
.requires_phone_verification,
|
||||
).toBeFalsy();
|
||||
});
|
||||
|
||||
it('still sends a pending phone step first', async () => {
|
||||
const user = await account({
|
||||
phone: PHONE,
|
||||
metadata: BEFORE,
|
||||
requires_phone_verification: true,
|
||||
});
|
||||
const response = await change(user.token, `${uuidv4()}@example.com`);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.json()).toMatchObject({
|
||||
code: 'phone_verification_required',
|
||||
});
|
||||
expect(await staged(user.id)).toBeFalsy();
|
||||
});
|
||||
|
||||
it('keeps refusing a confirmed account with a pending phone step', async () => {
|
||||
const user = await account({
|
||||
email_confirmed: true,
|
||||
requires_email_confirmation: false,
|
||||
requires_phone_verification: true,
|
||||
});
|
||||
const response = await change(user.token, `${uuidv4()}@example.com`);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.json()).toMatchObject({
|
||||
code: 'phone_verification_required',
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,199 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import {
|
||||
createTestUser,
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
|
||||
const MINUTE = 60_000;
|
||||
|
||||
describe('email confirmation codes', () => {
|
||||
let env: PuterTestEnv;
|
||||
const codes = new Map<string, string>();
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
}, 120_000);
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
beforeEach(() => {
|
||||
vi.spyOn(env.server.clients.email, 'send').mockImplementation(
|
||||
async (to, template, values) => {
|
||||
if (template === 'email_verification_code') {
|
||||
codes.set(to, String(values?.code));
|
||||
}
|
||||
return null;
|
||||
},
|
||||
);
|
||||
vi.useFakeTimers({ toFake: ['Date'], now: Date.now() });
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const later = (ms: number) => vi.setSystemTime(Date.now() + ms);
|
||||
|
||||
const account = async () => {
|
||||
const username = `confirm_${uuidv4().slice(0, 8)}`;
|
||||
const email = `${username}@example.com`;
|
||||
const { token } = await createTestUser(env.server, {
|
||||
username,
|
||||
password: 'confirm-code-password',
|
||||
});
|
||||
const user = (await env.server.stores.user.getByUsername(username))!;
|
||||
await env.server.stores.user.update(user.id, {
|
||||
email,
|
||||
email_confirmed: false,
|
||||
requires_email_confirmation: true,
|
||||
});
|
||||
const call = (path: string, body: Record<string, unknown> = {}) =>
|
||||
fetch(new URL(path, env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${token}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
const send = async () => {
|
||||
expect((await call('/send-confirm-email')).status).toBe(200);
|
||||
return codes.get(email)!;
|
||||
};
|
||||
const confirm = async (code: string) => {
|
||||
const response = await call('/confirm-email', { code });
|
||||
return {
|
||||
status: response.status,
|
||||
body: (await response.json()) as Record<string, unknown>,
|
||||
};
|
||||
};
|
||||
const miss = async (times: number, code: string) => {
|
||||
for (let i = 0; i < times; i++) {
|
||||
const wrong = code === '000000' ? '111111' : '000000';
|
||||
expect(await confirm(wrong)).toEqual({
|
||||
status: 200,
|
||||
body: { email_confirmed: false },
|
||||
});
|
||||
}
|
||||
};
|
||||
const confirmed = async () =>
|
||||
Boolean(
|
||||
(await env.server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
}))!.email_confirmed,
|
||||
);
|
||||
return { send, confirm, miss, confirmed };
|
||||
};
|
||||
|
||||
it('honors a code for an hour from its send', async () => {
|
||||
const { send, confirm, confirmed } = await account();
|
||||
const code = await send();
|
||||
later(59 * MINUTE);
|
||||
expect((await confirm(code)).body.email_confirmed).toBe(true);
|
||||
expect(await confirmed()).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses an expired code with an error the resend path recovers from', async () => {
|
||||
const { send, confirm, confirmed } = await account();
|
||||
const code = await send();
|
||||
later(61 * MINUTE);
|
||||
expect(await confirm(code)).toMatchObject({
|
||||
status: 400,
|
||||
body: { code: 'confirmation_code_expired' },
|
||||
});
|
||||
expect(await confirmed()).toBe(false);
|
||||
|
||||
const fresh = await send();
|
||||
expect((await confirm(fresh)).body.email_confirmed).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses a code that was never mailed', async () => {
|
||||
const { confirm } = await account();
|
||||
expect((await confirm('123456')).body).toMatchObject({
|
||||
code: 'confirmation_code_expired',
|
||||
});
|
||||
});
|
||||
|
||||
it('confirms after nine wrong codes', async () => {
|
||||
const { send, confirm, miss } = await account();
|
||||
const code = await send();
|
||||
await miss(9, code);
|
||||
expect((await confirm(code)).body.email_confirmed).toBe(true);
|
||||
});
|
||||
|
||||
it('after ten wrong codes refuses even the right one until the hour passes', async () => {
|
||||
const { send, confirm, miss, confirmed } = await account();
|
||||
const code = await send();
|
||||
await miss(10, code);
|
||||
// Past the route's own ten-minute window, so only the code budget answers.
|
||||
later(11 * MINUTE);
|
||||
expect((await confirm(code)).status).toBe(429);
|
||||
expect(await confirmed()).toBe(false);
|
||||
|
||||
later(50 * MINUTE);
|
||||
const fresh = await send();
|
||||
expect((await confirm(fresh)).body.email_confirmed).toBe(true);
|
||||
});
|
||||
|
||||
it('after twenty wrong codes in a day refuses until the day passes', async () => {
|
||||
const { send, confirm, miss } = await account();
|
||||
await miss(10, await send());
|
||||
later(61 * MINUTE);
|
||||
await miss(10, await send());
|
||||
later(61 * MINUTE);
|
||||
expect((await confirm(await send())).status).toBe(429);
|
||||
|
||||
later(24 * 60 * MINUTE);
|
||||
expect((await confirm(await send())).body.email_confirmed).toBe(true);
|
||||
});
|
||||
|
||||
it('honors the code when the marker cannot be read', async () => {
|
||||
const { send, confirm } = await account();
|
||||
const code = await send();
|
||||
const get = env.server.stores.kv.get.bind(env.server.stores.kv);
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(env.server.stores.kv, 'get').mockImplementation(
|
||||
async (args, opts) => {
|
||||
if (
|
||||
typeof args.key === 'string' &&
|
||||
args.key.startsWith('email-confirm-code-sent:')
|
||||
) {
|
||||
throw new Error('kv down');
|
||||
}
|
||||
return get(args, opts);
|
||||
},
|
||||
);
|
||||
later(2 * 60 * MINUTE);
|
||||
expect((await confirm(code)).body.email_confirmed).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -1,401 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import { signStepUpToken } from '../../core/http/middleware/stepUpSession.js';
|
||||
import { configContainer } from '../../exports.js';
|
||||
import {
|
||||
createTestUser,
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import {
|
||||
isKnownDevice,
|
||||
KNOWN_DEVICE_COOKIE_NAME,
|
||||
knownDeviceKey,
|
||||
knownDevicesMarkerKey,
|
||||
knownDeviceStatus,
|
||||
readDeviceCookie,
|
||||
rememberDevice,
|
||||
} from './knownDevice.js';
|
||||
|
||||
/** `name=value` of the device cookie a response sets, if it sets one. */
|
||||
const deviceCookieOf = (response: globalThis.Response): string | undefined =>
|
||||
response.headers
|
||||
.getSetCookie()
|
||||
.join(', ')
|
||||
.match(
|
||||
new RegExp(`(?:^|,\\s*)(${KNOWN_DEVICE_COOKIE_NAME}=[^;]+)`),
|
||||
)?.[1];
|
||||
|
||||
describe('known devices', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
}, 120_000);
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const deps = () => ({
|
||||
config: configContainer as IConfig,
|
||||
token: env.server.services.token,
|
||||
kv: env.server.stores.kv,
|
||||
});
|
||||
|
||||
/** A request carrying `cookies`, and a response that keeps what it sets. */
|
||||
const exchange = (cookies: Record<string, string> = {}) => {
|
||||
const set: Record<string, { value: string; opts: unknown }> = {};
|
||||
const req = { cookies, headers: {} } as unknown as Request;
|
||||
const res = {
|
||||
cookie: (name: string, value: string, opts: unknown) => {
|
||||
set[name] = { value, opts };
|
||||
},
|
||||
} as unknown as Response;
|
||||
return { req, res, set };
|
||||
};
|
||||
|
||||
const deviceIdOf = (cookie: string) =>
|
||||
readDeviceCookie(
|
||||
exchange({ [KNOWN_DEVICE_COOKIE_NAME]: cookie }).req,
|
||||
env.server.services.token,
|
||||
)!.deviceId;
|
||||
|
||||
/** The account's record of the device behind `cookie`, if any. */
|
||||
const recorded = async (userUuid: string, cookie: string) =>
|
||||
(
|
||||
await env.server.stores.kv.get({
|
||||
key: knownDeviceKey(userUuid, deviceIdOf(cookie)),
|
||||
})
|
||||
).res as { seenAt: number } | null;
|
||||
|
||||
const signedInDevice = async (userUuid: string) => {
|
||||
const first = exchange();
|
||||
await rememberDevice(first.req, first.res, userUuid, deps());
|
||||
return first.set[KNOWN_DEVICE_COOKIE_NAME]!.value;
|
||||
};
|
||||
|
||||
const isKnown = (userUuid: string, cookie: string) =>
|
||||
isKnownDevice(
|
||||
exchange({ [KNOWN_DEVICE_COOKIE_NAME]: cookie }).req,
|
||||
userUuid,
|
||||
deps(),
|
||||
);
|
||||
|
||||
it('issues an HttpOnly, SameSite=Lax, long-lived cookie and records the device', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const { req, res, set } = exchange();
|
||||
expect(await rememberDevice(req, res, userUuid, deps())).toBe(false);
|
||||
|
||||
const cookie = set[KNOWN_DEVICE_COOKIE_NAME]!;
|
||||
expect(cookie.opts).toMatchObject({
|
||||
httpOnly: true,
|
||||
sameSite: 'lax',
|
||||
maxAge: 2 * 365 * 24 * 60 * 60 * 1000,
|
||||
});
|
||||
expect(await recorded(userUuid, cookie.value)).toEqual({
|
||||
seenAt: expect.any(Number),
|
||||
});
|
||||
expect(
|
||||
(
|
||||
await env.server.stores.kv.get({
|
||||
key: knownDevicesMarkerKey(userUuid),
|
||||
})
|
||||
).res,
|
||||
).toBeTruthy();
|
||||
});
|
||||
|
||||
it('a recorded device costs one read and no write, and keeps its cookie', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const value = await signedInDevice(userUuid);
|
||||
const get = vi.spyOn(env.server.stores.kv, 'get');
|
||||
const writes = [
|
||||
vi.spyOn(env.server.stores.kv, 'set'),
|
||||
vi.spyOn(env.server.stores.kv, 'batchPut'),
|
||||
vi.spyOn(env.server.stores.kv, 'updateReservedItemIf'),
|
||||
];
|
||||
|
||||
const again = exchange({ [KNOWN_DEVICE_COOKIE_NAME]: value });
|
||||
expect(await isKnownDevice(again.req, userUuid, deps())).toBe(true);
|
||||
expect(
|
||||
await rememberDevice(again.req, again.res, userUuid, deps()),
|
||||
).toBe(true);
|
||||
expect(get).toHaveBeenCalledTimes(1);
|
||||
for (const write of writes) expect(write).not.toHaveBeenCalled();
|
||||
expect(again.set).toEqual({});
|
||||
});
|
||||
|
||||
it('renews the cookie and the record once they are a month old', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const value = await signedInDevice(userUuid);
|
||||
const before = await recorded(userUuid, value);
|
||||
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.setSystemTime(Date.now() + 31 * 24 * 60 * 60 * 1000);
|
||||
const later = exchange({ [KNOWN_DEVICE_COOKIE_NAME]: value });
|
||||
expect(
|
||||
await rememberDevice(later.req, later.res, userUuid, deps()),
|
||||
).toBe(true);
|
||||
expect(later.set[KNOWN_DEVICE_COOKIE_NAME]).toBeDefined();
|
||||
const after = await recorded(userUuid, value);
|
||||
expect(after!.seenAt).toBeGreaterThan(before!.seenAt);
|
||||
});
|
||||
|
||||
it('keeps every device of concurrent sign-ins to one account', async () => {
|
||||
const userUuid = uuidv4();
|
||||
await signedInDevice(userUuid);
|
||||
const a = exchange();
|
||||
const b = exchange();
|
||||
// Both read the account's devices before either records its own.
|
||||
await Promise.all([
|
||||
knownDeviceStatus(a.req, userUuid, deps()),
|
||||
knownDeviceStatus(b.req, userUuid, deps()),
|
||||
]);
|
||||
await Promise.all([
|
||||
rememberDevice(a.req, a.res, userUuid, deps()),
|
||||
rememberDevice(b.req, b.res, userUuid, deps()),
|
||||
]);
|
||||
for (const { set } of [a, b]) {
|
||||
expect(
|
||||
await isKnown(userUuid, set[KNOWN_DEVICE_COOKIE_NAME]!.value),
|
||||
).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps any number of devices, each for a year after it last renewed', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const first = await signedInDevice(userUuid);
|
||||
for (let i = 0; i < 24; i++) await signedInDevice(userUuid);
|
||||
expect(await isKnown(userUuid, first)).toBe(true);
|
||||
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.setSystemTime(Date.now() + 366 * 24 * 60 * 60 * 1000);
|
||||
expect(await isKnown(userUuid, first)).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps the account marker as long as its newest device', async () => {
|
||||
const userUuid = uuidv4();
|
||||
await signedInDevice(userUuid);
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
// The marker is too fresh to renew on its own by then.
|
||||
vi.setSystemTime(Date.now() + 20 * 24 * 60 * 60 * 1000);
|
||||
await signedInDevice(userUuid);
|
||||
|
||||
vi.setSystemTime(Date.now() + 350 * 24 * 60 * 60 * 1000);
|
||||
const newBrowser = exchange();
|
||||
expect(
|
||||
await knownDeviceStatus(newBrowser.req, userUuid, deps()),
|
||||
).toEqual({ known: false, anyRecorded: true });
|
||||
});
|
||||
|
||||
it('never shortens the account marker when an older sign-in records last', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const dynamo = env.server.clients.dynamo;
|
||||
let holding = true;
|
||||
let releaseOlder!: () => void;
|
||||
const older = new Promise<void>((resolve) => {
|
||||
releaseOlder = resolve;
|
||||
});
|
||||
const held = vi.fn();
|
||||
for (const method of ['put', 'batchPut', 'update'] as const) {
|
||||
const write = dynamo[method].bind(dynamo) as (
|
||||
...args: unknown[]
|
||||
) => Promise<unknown>;
|
||||
vi.spyOn(dynamo, method).mockImplementation((async (
|
||||
...args: unknown[]
|
||||
) => {
|
||||
if (holding) {
|
||||
held();
|
||||
await older;
|
||||
}
|
||||
return write(...args);
|
||||
}) as never);
|
||||
}
|
||||
|
||||
const start = Date.now();
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.setSystemTime(start);
|
||||
const a = exchange();
|
||||
const first = rememberDevice(a.req, a.res, userUuid, deps());
|
||||
await vi.waitFor(() => expect(held).toHaveBeenCalled());
|
||||
|
||||
holding = false;
|
||||
vi.setSystemTime(start + 60_000);
|
||||
const b = exchange();
|
||||
await rememberDevice(b.req, b.res, userUuid, deps());
|
||||
releaseOlder();
|
||||
await first;
|
||||
|
||||
// Past the older sign-in's year, inside the newer one's.
|
||||
vi.setSystemTime(start + 365 * 24 * 60 * 60 * 1000 + 30_000);
|
||||
expect(
|
||||
await knownDeviceStatus(exchange().req, userUuid, deps()),
|
||||
).toEqual({ known: false, anyRecorded: true });
|
||||
});
|
||||
|
||||
it('keeps devices per account', async () => {
|
||||
const value = await signedInDevice(uuidv4());
|
||||
const other = exchange({ [KNOWN_DEVICE_COOKIE_NAME]: value });
|
||||
expect(await isKnownDevice(other.req, uuidv4(), deps())).toBe(false);
|
||||
});
|
||||
|
||||
it('reads only cookies it signed for this purpose', async () => {
|
||||
const token = env.server.services.token;
|
||||
const value = await signedInDevice(uuidv4());
|
||||
const forged = `${value.slice(0, -4)}AAAA`;
|
||||
const stepUp = signStepUpToken(token, { uuid: uuidv4() });
|
||||
const session = env.users.user.token;
|
||||
const unsigned = token.sign('known-device', {
|
||||
purpose: 'known-device',
|
||||
device_id: 'not-a-uuid',
|
||||
});
|
||||
for (const cookie of [forged, stepUp, session, unsigned]) {
|
||||
expect(
|
||||
readDeviceCookie(
|
||||
exchange({ [KNOWN_DEVICE_COOKIE_NAME]: cookie }).req,
|
||||
token,
|
||||
),
|
||||
).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
it('answers unknown when the record cannot be read', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const value = await signedInDevice(userUuid);
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(env.server.stores.kv, 'get').mockRejectedValue(
|
||||
new Error('kv down'),
|
||||
);
|
||||
const { req } = exchange({ [KNOWN_DEVICE_COOKIE_NAME]: value });
|
||||
expect(await isKnownDevice(req, userUuid, deps())).toBe(false);
|
||||
});
|
||||
|
||||
it('reports whether the device and any device are recorded', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const status = (cookies: Record<string, string> = {}) =>
|
||||
knownDeviceStatus(exchange(cookies).req, userUuid, deps());
|
||||
expect(await status()).toEqual({ known: false, anyRecorded: false });
|
||||
|
||||
const value = await signedInDevice(userUuid);
|
||||
expect(await status()).toEqual({ known: false, anyRecorded: true });
|
||||
expect(await status({ [KNOWN_DEVICE_COOKIE_NAME]: value })).toEqual({
|
||||
known: true,
|
||||
anyRecorded: true,
|
||||
});
|
||||
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(env.server.stores.kv, 'get').mockRejectedValue(
|
||||
new Error('kv down'),
|
||||
);
|
||||
expect(await status({ [KNOWN_DEVICE_COOKIE_NAME]: value })).toEqual({
|
||||
known: false,
|
||||
anyRecorded: false,
|
||||
});
|
||||
});
|
||||
|
||||
describe('signed-in page loads', () => {
|
||||
const load = (cookie: string, dest = 'document') =>
|
||||
fetch(new URL('/', env.origin), {
|
||||
headers: { cookie, 'sec-fetch-dest': dest },
|
||||
});
|
||||
|
||||
const account = async () => {
|
||||
const username = `device_${uuidv4().slice(0, 8)}`;
|
||||
const { token } = await createTestUser(env.server, {
|
||||
username,
|
||||
password: 'device-page-password',
|
||||
});
|
||||
const user =
|
||||
(await env.server.stores.user.getByUsername(username))!;
|
||||
return { session: `puter_auth_token=${token}`, uuid: user.uuid };
|
||||
};
|
||||
|
||||
it('record a browser that signed in before devices were recorded', async () => {
|
||||
const { session, uuid } = await account();
|
||||
const first = await load(session);
|
||||
expect(first.status).toBe(200);
|
||||
const device = deviceCookieOf(first);
|
||||
expect(device).toBeDefined();
|
||||
const cookieValue = device!.slice(
|
||||
`${KNOWN_DEVICE_COOKIE_NAME}=`.length,
|
||||
);
|
||||
await vi.waitFor(async () =>
|
||||
expect(await recorded(uuid, cookieValue)).toBeTruthy(),
|
||||
);
|
||||
|
||||
const read = vi.spyOn(env.server.stores.kv, 'get');
|
||||
const deviceWrite = vi.spyOn(env.server.stores.kv, 'set');
|
||||
const markerWrite = vi.spyOn(
|
||||
env.server.stores.kv,
|
||||
'updateReservedItemIf',
|
||||
);
|
||||
const again = await load(`${session}; ${device}`);
|
||||
expect(again.status).toBe(200);
|
||||
expect(deviceCookieOf(again)).toBeUndefined();
|
||||
const deviceKey = knownDeviceKey(uuid, deviceIdOf(cookieValue));
|
||||
const ofDevice = ([arg]: [{ key: string | string[] }]) =>
|
||||
[arg.key].flat().includes(deviceKey);
|
||||
await vi.waitFor(() =>
|
||||
expect(read.mock.calls.filter(ofDevice)).toHaveLength(1),
|
||||
);
|
||||
// Lets a write that would follow the read land before checking.
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
expect(read.mock.calls.filter(ofDevice)).toHaveLength(1);
|
||||
expect(
|
||||
deviceWrite.mock.calls.filter(([arg]) => arg.key === deviceKey),
|
||||
).toHaveLength(0);
|
||||
expect(
|
||||
markerWrite.mock.calls.filter(
|
||||
([key]) => key === knownDevicesMarkerKey(uuid),
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('leave framed and anonymous loads alone', async () => {
|
||||
const { session, uuid } = await account();
|
||||
const framed = await load(session, 'iframe');
|
||||
expect(deviceCookieOf(framed)).toBeUndefined();
|
||||
const anonymous = await load('', 'document');
|
||||
expect(deviceCookieOf(anonymous)).toBeUndefined();
|
||||
expect(
|
||||
(
|
||||
await env.server.stores.kv.get({
|
||||
key: knownDevicesMarkerKey(uuid),
|
||||
})
|
||||
).res,
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,301 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import { v4 as uuidv4, validate as validateUuid } from 'uuid';
|
||||
import type { SignInDevice } from '../../services/auth/SignInService.js';
|
||||
import type { TokenService } from '../../services/auth/TokenService.js';
|
||||
import type { SystemKVStore } from '../../stores/systemKv/SystemKVStore.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { sessionCookieFlags } from '../../util/cookieFlags.js';
|
||||
|
||||
/**
|
||||
* Known devices: a signed HttpOnly cookie holds a random device id, and the
|
||||
* system KV keeps one record per (account, device) that completed a sign-in,
|
||||
* plus a marker that the account has any. Each lives a year past its last
|
||||
* renewal. A missing or unreadable record only makes a device read as new.
|
||||
*/
|
||||
|
||||
export const KNOWN_DEVICE_COOKIE_NAME = 'puter_device';
|
||||
|
||||
// Its own scope and `purpose`, and no auth `type` claim, so the auth probe
|
||||
// never accepts the cookie as a session token.
|
||||
const DEVICE_TOKEN_SCOPE = 'known-device';
|
||||
const DEVICE_TOKEN_PURPOSE = 'known-device';
|
||||
const COOKIE_TTL_SECONDS = 2 * 365 * 24 * 60 * 60;
|
||||
const RECORD_TTL_SECONDS = 365 * 24 * 60 * 60;
|
||||
/** A cookie or record this old is renewed on its next use, so both slide. */
|
||||
const RENEW_AFTER_SECONDS = 30 * 24 * 60 * 60;
|
||||
|
||||
export const knownDeviceKey = (userUuid: string, deviceId: string): string =>
|
||||
`known-device:${userUuid}:${deviceId}`;
|
||||
|
||||
/** Present while the account has at least one recorded device. */
|
||||
export const knownDevicesMarkerKey = (userUuid: string): string =>
|
||||
`known-devices-any:${userUuid}`;
|
||||
|
||||
export interface KnownDeviceDeps {
|
||||
token: TokenService;
|
||||
kv: SystemKVStore;
|
||||
}
|
||||
|
||||
interface DeviceRecord {
|
||||
/** Unix seconds the record was last renewed. */
|
||||
seenAt: number;
|
||||
}
|
||||
|
||||
/** The device id the request's cookie carries, or null when absent or not ours. */
|
||||
export function readDeviceCookie(
|
||||
req: Request,
|
||||
token: TokenService,
|
||||
): { deviceId: string; issuedAt: number } | null {
|
||||
const raw = req.cookies?.[KNOWN_DEVICE_COOKIE_NAME];
|
||||
if (typeof raw !== 'string' || !raw) return null;
|
||||
try {
|
||||
const claims = token.verify<{
|
||||
purpose?: unknown;
|
||||
device_id?: unknown;
|
||||
iat?: unknown;
|
||||
}>(DEVICE_TOKEN_SCOPE, raw);
|
||||
if (
|
||||
claims.purpose !== DEVICE_TOKEN_PURPOSE ||
|
||||
typeof claims.device_id !== 'string' ||
|
||||
!validateUuid(claims.device_id)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
deviceId: claims.device_id,
|
||||
issuedAt: typeof claims.iat === 'number' ? claims.iat : 0,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const parseRecord = (value: unknown): DeviceRecord | null =>
|
||||
typeof (value as DeviceRecord | null)?.seenAt === 'number'
|
||||
? (value as DeviceRecord)
|
||||
: null;
|
||||
|
||||
interface AccountDevices {
|
||||
/** The record of the request's cookie device; null without one. */
|
||||
device: DeviceRecord | null;
|
||||
marker: DeviceRecord | null;
|
||||
}
|
||||
|
||||
// One read per request and account (the device record and the marker in one
|
||||
// batch), shared by the lockout check, the sign-in, and whatever else asks
|
||||
// during that request. Null when it failed.
|
||||
const accountReads = new WeakMap<
|
||||
Request,
|
||||
Map<string, Promise<AccountDevices | null>>
|
||||
>();
|
||||
|
||||
function readAccountDevices(
|
||||
req: Request,
|
||||
userUuid: string,
|
||||
deps: KnownDeviceDeps,
|
||||
): Promise<AccountDevices | null> {
|
||||
let reads = accountReads.get(req);
|
||||
if (!reads) {
|
||||
reads = new Map();
|
||||
accountReads.set(req, reads);
|
||||
}
|
||||
let read = reads.get(userUuid);
|
||||
if (!read) {
|
||||
const deviceId = readDeviceCookie(req, deps.token)?.deviceId;
|
||||
read = (async () => {
|
||||
try {
|
||||
if (!deviceId) {
|
||||
const { res } = await deps.kv.get({
|
||||
key: knownDevicesMarkerKey(userUuid),
|
||||
});
|
||||
return { device: null, marker: parseRecord(res) };
|
||||
}
|
||||
const { res } = await deps.kv.get({
|
||||
key: [
|
||||
knownDeviceKey(userUuid, deviceId),
|
||||
knownDevicesMarkerKey(userUuid),
|
||||
],
|
||||
});
|
||||
const [device, marker] = (res as unknown[]).map(parseRecord);
|
||||
return { device: device ?? null, marker: marker ?? null };
|
||||
} catch (e) {
|
||||
console.warn('[known-device] record read failed:', e);
|
||||
return null;
|
||||
}
|
||||
})();
|
||||
reads.set(userUuid, read);
|
||||
}
|
||||
return read;
|
||||
}
|
||||
|
||||
/**
|
||||
* What the account's records say about the request's device. An unreadable
|
||||
* record reads as no devices at all.
|
||||
*/
|
||||
export async function knownDeviceStatus(
|
||||
req: Request,
|
||||
userUuid: string,
|
||||
deps: KnownDeviceDeps,
|
||||
): Promise<SignInDevice> {
|
||||
const read = await readAccountDevices(req, userUuid, deps);
|
||||
return {
|
||||
known: Boolean(read?.device),
|
||||
anyRecorded: Boolean(read?.device || read?.marker),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the request comes from a device that completed a sign-in on this
|
||||
* account. False when there is no valid cookie or the record can't be read.
|
||||
*/
|
||||
export async function isKnownDevice(
|
||||
req: Request,
|
||||
userUuid: string,
|
||||
deps: KnownDeviceDeps,
|
||||
): Promise<boolean> {
|
||||
if (!readDeviceCookie(req, deps.token)) return false;
|
||||
return (await knownDeviceStatus(req, userUuid, deps)).known;
|
||||
}
|
||||
|
||||
/**
|
||||
* The request's device id, issuing or renewing its cookie. Records nothing: the
|
||||
* id only counts once {@link rememberDevice} or {@link recordKnownDevice} puts it
|
||||
* on an account.
|
||||
*/
|
||||
export function issueDeviceCookie(
|
||||
req: Request,
|
||||
res: Response,
|
||||
deps: Pick<KnownDeviceDeps, 'token'> & { config: IConfig },
|
||||
): string {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const cookie = readDeviceCookie(req, deps.token);
|
||||
const deviceId = cookie?.deviceId ?? uuidv4();
|
||||
if (!cookie || now - cookie.issuedAt >= RENEW_AFTER_SECONDS) {
|
||||
res.cookie(
|
||||
KNOWN_DEVICE_COOKIE_NAME,
|
||||
deps.token.sign(
|
||||
DEVICE_TOKEN_SCOPE,
|
||||
{ purpose: DEVICE_TOKEN_PURPOSE, device_id: deviceId },
|
||||
{ expiresIn: COOKIE_TTL_SECONDS },
|
||||
),
|
||||
{
|
||||
...sessionCookieFlags(deps.config, { crossSite: false }),
|
||||
httpOnly: true,
|
||||
maxAge: COOKIE_TTL_SECONDS * 1000,
|
||||
},
|
||||
);
|
||||
}
|
||||
return deviceId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a device on the account by id, for a request that doesn't carry its
|
||||
* cookie. Same records as {@link rememberDevice}.
|
||||
*/
|
||||
export async function recordKnownDevice(
|
||||
userUuid: string,
|
||||
deviceId: string,
|
||||
deps: Pick<KnownDeviceDeps, 'kv'>,
|
||||
): Promise<void> {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const fresh: DeviceRecord = { seenAt: now };
|
||||
const expireAt = now + RECORD_TTL_SECONDS;
|
||||
try {
|
||||
await Promise.all([
|
||||
deps.kv.set({
|
||||
key: knownDeviceKey(userUuid, deviceId),
|
||||
value: fresh,
|
||||
expireAt,
|
||||
}),
|
||||
deps.kv.updateReservedItemIf(
|
||||
knownDevicesMarkerKey(userUuid),
|
||||
{ set: { value: fresh, ttl: expireAt } },
|
||||
'attribute_not_exists(#ttl) OR #ttl < :ttl',
|
||||
{ ':ttl': expireAt },
|
||||
{ '#ttl': 'ttl' },
|
||||
),
|
||||
]);
|
||||
} catch (e) {
|
||||
console.warn('[known-device] record update failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record the request's device on the account, issuing or renewing its cookie.
|
||||
* Call only once the request has proved the account (a completed sign-in, or a
|
||||
* session it already holds). The cookie is set before the first await, so a
|
||||
* caller may leave the record update running. Resolves to whether the device
|
||||
* was already recorded; a device recorded and renewed recently costs one read
|
||||
* and no write.
|
||||
*/
|
||||
export async function rememberDevice(
|
||||
req: Request,
|
||||
res: Response,
|
||||
userUuid: string,
|
||||
deps: KnownDeviceDeps & { config: IConfig },
|
||||
): Promise<boolean> {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const cookie = readDeviceCookie(req, deps.token);
|
||||
const deviceId = issueDeviceCookie(req, res, deps);
|
||||
// Without a cookie the device is new, and only the marker was read.
|
||||
const read = await readAccountDevices(req, userUuid, deps);
|
||||
const device = cookie ? (read?.device ?? null) : null;
|
||||
const stale = (record: DeviceRecord | null | undefined) =>
|
||||
!record || now - record.seenAt >= RENEW_AFTER_SECONDS;
|
||||
// Each write touches only this device's record and the marker, so
|
||||
// concurrent sign-ins never drop each other's devices. The marker is
|
||||
// rewritten with every device record, so it never expires first.
|
||||
const fresh: DeviceRecord = { seenAt: now };
|
||||
const expireAt = now + RECORD_TTL_SECONDS;
|
||||
const writeDevice = stale(device);
|
||||
const writeMarker = writeDevice || stale(read?.marker);
|
||||
if (!writeDevice && !writeMarker) return true;
|
||||
try {
|
||||
await Promise.all([
|
||||
writeDevice &&
|
||||
deps.kv.set({
|
||||
key: knownDeviceKey(userUuid, deviceId),
|
||||
value: fresh,
|
||||
expireAt,
|
||||
}),
|
||||
// Only ever extends it: an older sign-in landing last is refused.
|
||||
writeMarker &&
|
||||
deps.kv.updateReservedItemIf(
|
||||
knownDevicesMarkerKey(userUuid),
|
||||
{ set: { value: fresh, ttl: expireAt } },
|
||||
'attribute_not_exists(#ttl) OR #ttl < :ttl',
|
||||
{ ':ttl': expireAt },
|
||||
{ '#ttl': 'ttl' },
|
||||
),
|
||||
]);
|
||||
accountReads.get(req)?.set(
|
||||
userUuid,
|
||||
Promise.resolve({
|
||||
device: writeDevice ? fresh : device,
|
||||
marker: writeMarker ? fresh : (read?.marker ?? null),
|
||||
}),
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[known-device] record update failed:', e);
|
||||
}
|
||||
return Boolean(device);
|
||||
}
|
||||
@@ -1,436 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import bcrypt from 'bcrypt';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import {
|
||||
checkRateLimit,
|
||||
configureRateLimit,
|
||||
peekRateLimit,
|
||||
} from '../../core/http/middleware/rateLimit.js';
|
||||
import {
|
||||
createTestUser,
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { passwordTag } from '../../util/userDetails.js';
|
||||
import { KNOWN_DEVICE_COOKIE_NAME } from './knownDevice.js';
|
||||
|
||||
/** `name=value` of the device cookie a response sets, if it sets one. */
|
||||
const deviceCookieOf = (response: Response): string | undefined => {
|
||||
const match = response.headers
|
||||
.getSetCookie()
|
||||
.join(', ')
|
||||
.match(new RegExp(`(?:^|,\\s*)(${KNOWN_DEVICE_COOKIE_NAME}=[^;]+)`));
|
||||
return match?.[1];
|
||||
};
|
||||
|
||||
// The account budget in this region (Redis) and as every region sees it
|
||||
// (system KV).
|
||||
const HOUR = 'login-password-failure';
|
||||
const DAY = 'login-password-failure-day';
|
||||
const GLOBAL_HOUR = 'login-password-failure-global';
|
||||
const GLOBAL_DAY = 'login-password-failure-global-day';
|
||||
const HOUR_MS = 60 * 60_000;
|
||||
|
||||
/** A budget's key under the account's current password. */
|
||||
const budgetKey = (
|
||||
scope: string,
|
||||
user: { uuid: string; password?: unknown },
|
||||
) => `${scope}:${user.uuid}:${passwordTag(user)}`;
|
||||
|
||||
describe('per-account wrong-password budget on /login', () => {
|
||||
let env: PuterTestEnv;
|
||||
const password = 'lockout-correct-password';
|
||||
const sent: Array<{ to: string; template: string; values: unknown }> = [];
|
||||
let nextAddress = 0;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Trusting the proxy header lets each request come from its own
|
||||
// address, the way a caller rotating proxies would.
|
||||
env = await setupPuterTestEnv({ trust_proxy: true } as IConfig);
|
||||
}, 120_000);
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
sent.length = 0;
|
||||
});
|
||||
|
||||
const captureEmail = () =>
|
||||
vi
|
||||
.spyOn(env.server.clients.email, 'send')
|
||||
.mockImplementation(async (to, template, values) => {
|
||||
sent.push({ to, template, values });
|
||||
return null;
|
||||
});
|
||||
|
||||
const freshAddress = () => {
|
||||
nextAddress++;
|
||||
return `10.${(nextAddress >> 16) & 255}.${(nextAddress >> 8) & 255}.${nextAddress & 255}`;
|
||||
};
|
||||
|
||||
const login = async (
|
||||
username: string,
|
||||
attempt: string,
|
||||
cookie?: string,
|
||||
) => {
|
||||
const response = await fetch(new URL('/login', env.origin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'x-forwarded-for': freshAddress(),
|
||||
...(cookie ? { cookie } : {}),
|
||||
},
|
||||
body: JSON.stringify({ username, password: attempt }),
|
||||
});
|
||||
const deviceCookie = deviceCookieOf(response);
|
||||
return {
|
||||
status: response.status,
|
||||
body: (await response.json()) as Record<string, unknown>,
|
||||
deviceCookie,
|
||||
// The session cookie is set first.
|
||||
sessionCookie: response.headers.getSetCookie()[0]?.split(';')[0],
|
||||
};
|
||||
};
|
||||
|
||||
const account = async () => {
|
||||
const username = `lockout_${uuidv4().slice(0, 8)}`;
|
||||
await createTestUser(env.server, { username, password });
|
||||
const user = (await env.server.stores.user.getByUsername(username))!;
|
||||
await env.server.stores.user.update(user.id, {
|
||||
email: `${username}@example.com`,
|
||||
email_confirmed: true,
|
||||
});
|
||||
return { username, user };
|
||||
};
|
||||
|
||||
const missPassword = async (username: string, times: number) => {
|
||||
for (let i = 0; i < times; i++) {
|
||||
const { status, body } = await login(username, `wrong-${i}-guess`);
|
||||
expect(status).toBe(401);
|
||||
expect(body.code).toBe('password_mismatch');
|
||||
}
|
||||
};
|
||||
|
||||
it('locks on the attempt after the twentieth wrong password, not before', async () => {
|
||||
const { username } = await account();
|
||||
await missPassword(username, 19);
|
||||
expect((await login(username, password)).status).toBe(200);
|
||||
|
||||
// The correct password spent nothing, so twenty more wrong ones fit.
|
||||
await missPassword(username, 1);
|
||||
const locked = await login(username, 'wrong-again');
|
||||
expect(locked.status).toBe(429);
|
||||
expect(locked.body).toMatchObject({
|
||||
code: 'too_many_requests',
|
||||
errorCode: 'login_locked',
|
||||
action: 'reset_password',
|
||||
});
|
||||
});
|
||||
|
||||
it('never locks an account with no email, which has no reset path', async () => {
|
||||
const username = `lockout_${uuidv4().slice(0, 8)}`;
|
||||
await createTestUser(env.server, { username, password });
|
||||
const user = (await env.server.stores.user.getByUsername(username))!;
|
||||
await env.server.stores.user.update(user.id, { email: null });
|
||||
const alarm = vi.spyOn(env.server.clients.alarm, 'create');
|
||||
|
||||
await missPassword(username, 21);
|
||||
const signedIn = await login(username, password);
|
||||
expect(signedIn.status).toBe(200);
|
||||
// Still counted, and still raised.
|
||||
expect(alarm).toHaveBeenCalledWith(
|
||||
`auth:login-locked:${user.uuid}`,
|
||||
expect.stringContaining('not locked'),
|
||||
expect.anything(),
|
||||
'warning',
|
||||
expect.anything(),
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses the correct password while locked, from a device the account has not seen', async () => {
|
||||
const { username } = await account();
|
||||
await missPassword(username, 20);
|
||||
const refused = await login(username, password);
|
||||
expect(refused.status).toBe(429);
|
||||
expect(refused.body.errorCode).toBe('login_locked');
|
||||
expect(refused.body.token).toBeUndefined();
|
||||
expect(refused.deviceCookie).toBeUndefined();
|
||||
});
|
||||
|
||||
it('a spent day window locks the account too', async () => {
|
||||
const { username, user } = await account();
|
||||
for (let i = 0; i < 50; i++) {
|
||||
await checkRateLimit(
|
||||
budgetKey(DAY, user),
|
||||
50,
|
||||
24 * 60 * 60_000,
|
||||
);
|
||||
}
|
||||
expect((await login(username, password)).status).toBe(429);
|
||||
});
|
||||
|
||||
it('admits only the guesses that fit, however many arrive at once', async () => {
|
||||
const { username } = await account();
|
||||
await missPassword(username, 18);
|
||||
const burst = await Promise.all(
|
||||
Array.from({ length: 6 }, (_, i) =>
|
||||
login(username, `burst-${i}-guess`),
|
||||
),
|
||||
);
|
||||
const statuses = burst.map((r) => r.status);
|
||||
expect(statuses.filter((s) => s === 401)).toHaveLength(2);
|
||||
expect(statuses.filter((s) => s === 429)).toHaveLength(4);
|
||||
});
|
||||
|
||||
it('locks on wrong passwords counted in another region', async () => {
|
||||
const { username, user } = await account();
|
||||
for (let i = 0; i < 20; i++) {
|
||||
await checkRateLimit(
|
||||
budgetKey(GLOBAL_HOUR, user),
|
||||
20,
|
||||
60 * 60_000,
|
||||
'kv',
|
||||
);
|
||||
}
|
||||
const locked = await login(username, password);
|
||||
expect(locked.status).toBe(429);
|
||||
expect(locked.body.errorCode).toBe('login_locked');
|
||||
});
|
||||
|
||||
it('counts each wrong password where every region reads it, and a right one nowhere', async () => {
|
||||
const { username, user } = await account();
|
||||
await missPassword(username, 2);
|
||||
expect((await login(username, password)).status).toBe(200);
|
||||
for (const scope of [GLOBAL_HOUR, GLOBAL_DAY]) {
|
||||
const key = budgetKey(scope, user);
|
||||
expect(await peekRateLimit(key, 2, 60 * 60_000, 'kv')).toBe(false);
|
||||
expect(await peekRateLimit(key, 3, 60 * 60_000, 'kv')).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('lets a device that signed in before through the lockout', async () => {
|
||||
const { username } = await account();
|
||||
const first = await login(username, password);
|
||||
expect(first.status).toBe(200);
|
||||
expect(first.deviceCookie).toBeDefined();
|
||||
const device = first.deviceCookie!;
|
||||
|
||||
await missPassword(username, 20);
|
||||
expect((await login(username, password)).status).toBe(429);
|
||||
|
||||
// Its wrong guesses still answer as wrong, and its right one signs in.
|
||||
const miss = await login(username, 'wrong-from-device', device);
|
||||
expect(miss.status).toBe(401);
|
||||
const signedIn = await login(username, password, device);
|
||||
expect(signedIn.status).toBe(200);
|
||||
expect(signedIn.body.next_step).toBe('complete');
|
||||
});
|
||||
|
||||
it("doesn't take another account's device as known", async () => {
|
||||
const owner = await account();
|
||||
const stranger = await account();
|
||||
const device = (await login(stranger.username, password)).deviceCookie!;
|
||||
await missPassword(owner.username, 20);
|
||||
expect((await login(owner.username, password, device)).status).toBe(
|
||||
429,
|
||||
);
|
||||
});
|
||||
|
||||
it('alarms and emails the owner once when the lock trips', async () => {
|
||||
captureEmail();
|
||||
const alarm = vi.spyOn(env.server.clients.alarm, 'create');
|
||||
const { username, user } = await account();
|
||||
const device = (await login(username, password)).deviceCookie!;
|
||||
|
||||
await missPassword(username, 19);
|
||||
expect(
|
||||
alarm.mock.calls.filter(([id]) => id.includes(user.uuid)),
|
||||
).toHaveLength(0);
|
||||
await missPassword(username, 1);
|
||||
// A known device keeps failing past the lock; nothing new goes out.
|
||||
expect(
|
||||
(await login(username, 'wrong-from-device', device)).status,
|
||||
).toBe(401);
|
||||
|
||||
const raised = alarm.mock.calls.filter(
|
||||
([id]) => id === `auth:login-locked:${user.uuid}`,
|
||||
);
|
||||
expect(raised).toHaveLength(1);
|
||||
expect(raised[0]![2]).toEqual({
|
||||
username,
|
||||
user_uuid: user.uuid,
|
||||
ip: expect.any(String),
|
||||
});
|
||||
expect(raised[0]![3]).toBe('warning');
|
||||
const notices = sent.filter(
|
||||
(m) => m.template === 'login_locked_notification',
|
||||
);
|
||||
expect(notices).toHaveLength(1);
|
||||
expect(notices[0]).toMatchObject({
|
||||
to: `${username}@example.com`,
|
||||
values: {
|
||||
username,
|
||||
reset_link: `${env.origin}/action/password-recovery`,
|
||||
sessions_link: `${env.origin}/dashboard#security`,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps the suspended-account answer while locked', async () => {
|
||||
const { username, user } = await account();
|
||||
await missPassword(username, 20);
|
||||
await env.server.stores.user.update(user.id, { suspended: true });
|
||||
const response = await login(username, password);
|
||||
expect(response.status).toBe(401);
|
||||
expect(response.body.code).toBe('account_suspended');
|
||||
});
|
||||
|
||||
it('a completed password reset lifts the lock in every region and tells the owner', async () => {
|
||||
captureEmail();
|
||||
const { username, user } = await account();
|
||||
await missPassword(username, 20);
|
||||
expect((await login(username, password)).status).toBe(429);
|
||||
for (let i = 0; i < 30; i++) {
|
||||
await checkRateLimit(
|
||||
budgetKey(GLOBAL_DAY, user),
|
||||
50,
|
||||
24 * 60 * 60_000,
|
||||
'kv',
|
||||
);
|
||||
}
|
||||
|
||||
const recoveryToken = uuidv4();
|
||||
await env.server.stores.user.update(user.id, {
|
||||
pass_recovery_token: recoveryToken,
|
||||
});
|
||||
const token = env.server.services.token.sign(
|
||||
'otp',
|
||||
{
|
||||
token: recoveryToken,
|
||||
user_uid: user.uuid,
|
||||
email: `${username}@example.com`,
|
||||
purpose: 'pass-recovery',
|
||||
},
|
||||
{ expiresIn: '1h' },
|
||||
);
|
||||
const reset = await fetch(
|
||||
new URL('/set-pass-using-token', env.origin),
|
||||
{
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({ token, password: 'a-new-password' }),
|
||||
},
|
||||
);
|
||||
expect(reset.status).toBe(200);
|
||||
|
||||
expect((await login(username, 'a-new-password')).status).toBe(200);
|
||||
expect(
|
||||
sent.filter(
|
||||
(m) =>
|
||||
m.template === 'password_change_notification' &&
|
||||
m.to === `${username}@example.com`,
|
||||
),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('a password change from a signed-in session lifts the lock', async () => {
|
||||
const { username } = await account();
|
||||
const { sessionCookie } = await login(username, password);
|
||||
await missPassword(username, 20);
|
||||
expect((await login(username, password)).status).toBe(429);
|
||||
|
||||
const change = await fetch(
|
||||
new URL('/user-protected/change-password', env.origin),
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
cookie: sessionCookie!,
|
||||
},
|
||||
body: JSON.stringify({ password, new_pass: 'a-changed-password' }),
|
||||
},
|
||||
);
|
||||
expect(change.status).toBe(200);
|
||||
expect((await login(username, 'a-changed-password')).status).toBe(200);
|
||||
});
|
||||
|
||||
it('a password set outside the sign-in routes (a team reset) lifts the lock, and a new lock notifies again', async () => {
|
||||
captureEmail();
|
||||
const { username, user } = await account();
|
||||
await missPassword(username, 20);
|
||||
expect((await login(username, password)).status).toBe(429);
|
||||
|
||||
await env.server.stores.user.update(user.id, {
|
||||
password: await bcrypt.hash('a-team-reset-password', 8),
|
||||
});
|
||||
expect((await login(username, 'a-team-reset-password')).status).toBe(
|
||||
200,
|
||||
);
|
||||
|
||||
await missPassword(username, 20);
|
||||
expect(
|
||||
sent.filter((m) => m.template === 'login_locked_notification'),
|
||||
).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('gives the reservation back when the password check fails', async () => {
|
||||
const { username, user } = await account();
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(bcrypt, 'compare').mockRejectedValueOnce(
|
||||
new Error('bcrypt failed') as never,
|
||||
);
|
||||
expect((await login(username, password)).status).toBe(500);
|
||||
expect(
|
||||
await peekRateLimit(budgetKey(HOUR, user), 1, HOUR_MS, 'redis'),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('reserves on Redis even where the limiter defaults to the KV store', async () => {
|
||||
const { redis } = env.server.clients;
|
||||
const { kv } = env.server.stores;
|
||||
configureRateLimit({ default: 'kv', redis, kv });
|
||||
try {
|
||||
const { username, user } = await account();
|
||||
await missPassword(username, 1);
|
||||
const key = budgetKey(HOUR, user);
|
||||
expect(await peekRateLimit(key, 1, HOUR_MS, 'redis')).toBe(false);
|
||||
expect(await peekRateLimit(key, 1, HOUR_MS, 'kv')).toBe(true);
|
||||
} finally {
|
||||
configureRateLimit({
|
||||
default: 'redis',
|
||||
redis,
|
||||
kv,
|
||||
metering: env.server.services.metering,
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large.
Load diff
@@ -24,10 +24,6 @@ import { configContainer } from '../../exports.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import {
|
||||
KNOWN_DEVICE_COOKIE_NAME,
|
||||
knownDevicesMarkerKey,
|
||||
} from './knownDevice.js';
|
||||
import { startWebSession } from './webSession.js';
|
||||
|
||||
let server: PuterServer;
|
||||
@@ -55,7 +51,6 @@ describe('startWebSession', () => {
|
||||
cookie: (name: string, value: string, opts: unknown) => {
|
||||
cookies.push({ name, value, opts });
|
||||
},
|
||||
once: () => res,
|
||||
} as unknown as Response;
|
||||
const req = {
|
||||
ip: '192.0.2.10',
|
||||
@@ -63,32 +58,16 @@ describe('startWebSession', () => {
|
||||
} as unknown as Request;
|
||||
const config = configContainer as IConfig;
|
||||
|
||||
const { guiToken } = await startWebSession(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
{
|
||||
config,
|
||||
auth: server.services.auth,
|
||||
signIn: server.services.signIn,
|
||||
token: server.services.token,
|
||||
kv: server.stores.kv,
|
||||
},
|
||||
'password',
|
||||
);
|
||||
const guiToken = await startWebSession(req, res, user, {
|
||||
config,
|
||||
auth: server.services.auth,
|
||||
});
|
||||
|
||||
expect(typeof guiToken).toBe('string');
|
||||
expect(cookies.map((c) => c.name).sort()).toEqual(
|
||||
[
|
||||
config.cookie_name ?? 'puter_token',
|
||||
KNOWN_DEVICE_COOKIE_NAME,
|
||||
].sort(),
|
||||
);
|
||||
const session = cookies.find(
|
||||
(c) => c.name === (config.cookie_name ?? 'puter_token'),
|
||||
)!;
|
||||
expect(session.value).not.toBe(guiToken);
|
||||
expect(session.opts).toMatchObject({ httpOnly: true });
|
||||
expect(cookies).toHaveLength(1);
|
||||
expect(cookies[0]!.name).toBe(config.cookie_name ?? 'puter_token');
|
||||
expect(cookies[0]!.value).not.toBe(guiToken);
|
||||
expect(cookies[0]!.opts).toMatchObject({ httpOnly: true });
|
||||
|
||||
const sessions = (await server.stores.session.getByUserId(
|
||||
user.id,
|
||||
@@ -100,85 +79,4 @@ describe('startWebSession', () => {
|
||||
last_user_agent: 'ws-agent',
|
||||
});
|
||||
});
|
||||
|
||||
it('records no device for a temporary account', async () => {
|
||||
const username = `ws_${Math.random().toString(36).slice(2, 10)}`;
|
||||
const user = await server.stores.user.create({
|
||||
username,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email: null,
|
||||
});
|
||||
const names: string[] = [];
|
||||
const res = {
|
||||
cookie: (name: string) => {
|
||||
names.push(name);
|
||||
},
|
||||
once: () => res,
|
||||
} as unknown as Response;
|
||||
const req = { ip: '192.0.2.11', headers: {} } as unknown as Request;
|
||||
const config = configContainer as IConfig;
|
||||
|
||||
await startWebSession(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
{
|
||||
config,
|
||||
auth: server.services.auth,
|
||||
signIn: server.services.signIn,
|
||||
token: server.services.token,
|
||||
kv: server.stores.kv,
|
||||
},
|
||||
'signup',
|
||||
);
|
||||
|
||||
expect(names).toEqual([config.cookie_name ?? 'puter_token']);
|
||||
});
|
||||
|
||||
it("leaves the device alone on a cross-site POST, which can't carry its cookie", async () => {
|
||||
const username = `ws_${Math.random().toString(36).slice(2, 10)}`;
|
||||
const user = await server.stores.user.create({
|
||||
username,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email: `${username}@test.local`,
|
||||
});
|
||||
const names: string[] = [];
|
||||
const res = {
|
||||
cookie: (name: string) => {
|
||||
names.push(name);
|
||||
},
|
||||
once: () => res,
|
||||
} as unknown as Response;
|
||||
const req = {
|
||||
ip: '192.0.2.12',
|
||||
method: 'POST',
|
||||
headers: { 'sec-fetch-site': 'cross-site' },
|
||||
} as unknown as Request;
|
||||
const config = configContainer as IConfig;
|
||||
|
||||
await startWebSession(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
{
|
||||
config,
|
||||
auth: server.services.auth,
|
||||
signIn: server.services.signIn,
|
||||
token: server.services.token,
|
||||
kv: server.stores.kv,
|
||||
},
|
||||
'oidc',
|
||||
);
|
||||
|
||||
expect(names).toEqual([config.cookie_name ?? 'puter_token']);
|
||||
expect(
|
||||
(
|
||||
await server.stores.kv.get({
|
||||
key: knownDevicesMarkerKey(user.uuid),
|
||||
})
|
||||
).res,
|
||||
).toBeFalsy();
|
||||
});
|
||||
});
|
||||
@@ -18,167 +18,31 @@
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import { onResponseClose } from '../../core/http/onResponseClose.js';
|
||||
import type { AuthService } from '../../services/auth/AuthService.js';
|
||||
import type {
|
||||
SignInDevice,
|
||||
SignInInput,
|
||||
SignInResult,
|
||||
SignInService,
|
||||
} from '../../services/auth/SignInService.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import type { UserRow, UserStore } from '../../stores/user/UserStore.js';
|
||||
import type { UserRow } from '../../stores/user/UserStore.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { sessionCookieFlags } from '../../util/cookieFlags.js';
|
||||
import {
|
||||
isCardOnFileStep,
|
||||
isPhoneOnFileStep,
|
||||
} from '../../util/verificationOrder.js';
|
||||
import {
|
||||
issueDeviceCookie,
|
||||
type KnownDeviceDeps,
|
||||
knownDeviceStatus,
|
||||
recordKnownDevice,
|
||||
rememberDevice,
|
||||
} from './knownDevice.js';
|
||||
|
||||
/** Run `task` once the response has gone out; a failure is only logged. */
|
||||
export function runAfterResponse(
|
||||
res: Response,
|
||||
task: () => Promise<void>,
|
||||
): void {
|
||||
onResponseClose(res, () => {
|
||||
void (async () => {
|
||||
try {
|
||||
await task();
|
||||
} catch (e) {
|
||||
console.warn('[auth] after-response task failed:', e);
|
||||
}
|
||||
})();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete an interactive sign-in this request proved: the sign-in pipeline
|
||||
* runs (and may refuse it), then a web session opens with its HTTP-only cookie.
|
||||
* Returns the GUI token, the only credential the client sees, and the row with
|
||||
* any verification gate the sign-in set.
|
||||
* Open a web session for a sign-in this request completed, recording where it
|
||||
* came from, and set its HTTP-only cookie. Returns the GUI token, the only
|
||||
* credential the client sees.
|
||||
*/
|
||||
export async function startWebSession(
|
||||
req: Request,
|
||||
res: Response,
|
||||
user: UserRow,
|
||||
deps: {
|
||||
config: IConfig;
|
||||
auth: AuthService;
|
||||
signIn: SignInService;
|
||||
user: Pick<UserStore, 'getById'>;
|
||||
} & KnownDeviceDeps,
|
||||
method: SignInInput['method'],
|
||||
): Promise<SignInResult> {
|
||||
// A temporary account has no credential to guard or address to notify.
|
||||
// A cross-site POST (a provider's form_post callback) can't carry the
|
||||
// Lax device cookie, so the device is unknowable there.
|
||||
const recordDevice =
|
||||
(user.password !== null || user.email !== null) &&
|
||||
!(
|
||||
req.method === 'POST' &&
|
||||
req.headers?.['sec-fetch-site'] === 'cross-site'
|
||||
);
|
||||
const device: Promise<SignInDevice | null> = recordDevice
|
||||
? knownDeviceStatus(req, user.uuid, deps)
|
||||
: Promise.resolve(null);
|
||||
return deps.signIn.signIn({
|
||||
req,
|
||||
user,
|
||||
method,
|
||||
device,
|
||||
openSession: async (sessionUser) => {
|
||||
const { session, token, gui_token } =
|
||||
await deps.auth.createSessionToken(sessionUser, {
|
||||
ip: req.ip || req.socket?.remoteAddress,
|
||||
user_agent: req.headers?.['user-agent'],
|
||||
origin: req.headers?.origin,
|
||||
host: req.headers?.host,
|
||||
});
|
||||
const owesProof = owesProofOnFile(sessionUser);
|
||||
// A sign-in that still owes proof of the phone or card on file
|
||||
// hasn't shown it is the owner's, so its device isn't remembered
|
||||
// yet; the id is kept for the request that proves it.
|
||||
if (owesProof && recordDevice) {
|
||||
try {
|
||||
await deps.kv.set({
|
||||
key: unprovenDeviceKey(String(session.uuid)),
|
||||
value: issueDeviceCookie(req, res, deps),
|
||||
expireAt:
|
||||
Math.floor(Date.now() / 1000) +
|
||||
UNPROVEN_DEVICE_TTL_SECONDS,
|
||||
});
|
||||
} catch (e) {
|
||||
console.warn('[known-device] pending id not kept:', e);
|
||||
}
|
||||
}
|
||||
// The proof signs every other session out. One that lands while
|
||||
// this sign-in runs would miss this session, so a sign-in that
|
||||
// finds the gate gone isn't kept, and gets no session cookie.
|
||||
if (owesProof) {
|
||||
const fresh = await deps.user.getById(user.id, { force: true });
|
||||
if (!fresh || !owesProofOnFile(fresh)) {
|
||||
await deps.auth.revokeSession(String(session.uuid));
|
||||
throw new HttpError(409, 'Please sign in again.', {
|
||||
legacyCode: 'conflict',
|
||||
code: 'signin_retry',
|
||||
});
|
||||
}
|
||||
}
|
||||
res.cookie(deps.config.cookie_name ?? 'puter_token', token, {
|
||||
...sessionCookieFlags(deps.config),
|
||||
httpOnly: true,
|
||||
});
|
||||
if (!owesProof && recordDevice)
|
||||
await rememberDevice(req, res, user.uuid, deps);
|
||||
return { sessionUuid: String(session.uuid), guiToken: gui_token };
|
||||
},
|
||||
afterResponse: (task) => runAfterResponse(res, task),
|
||||
deps: { config: IConfig; auth: AuthService },
|
||||
): Promise<string> {
|
||||
const { token, gui_token } = await deps.auth.createSessionToken(user, {
|
||||
ip: req.ip || req.socket?.remoteAddress,
|
||||
user_agent: req.headers?.['user-agent'],
|
||||
origin: req.headers?.origin,
|
||||
host: req.headers?.host,
|
||||
});
|
||||
}
|
||||
|
||||
const owesProofOnFile = (user: UserRow): boolean =>
|
||||
isPhoneOnFileStep(user) || isCardOnFileStep(user);
|
||||
|
||||
/**
|
||||
* The sign-in a request belongs to: its web session, or the one its token was
|
||||
* handed out under.
|
||||
*/
|
||||
export const signInOf = (req: Request): string | null =>
|
||||
req.actor?.session?.parentUid ?? req.actor?.session?.uid ?? null;
|
||||
|
||||
/** The device id of a session opened while proof on file was owed. */
|
||||
const unprovenDeviceKey = (sessionUuid: string): string =>
|
||||
`signin-unproven-device:${sessionUuid}`;
|
||||
// As long as an idle web session lasts.
|
||||
const UNPROVEN_DEVICE_TTL_SECONDS = 180 * 24 * 60 * 60;
|
||||
|
||||
/**
|
||||
* Remember the device of the session that just proved the phone or card on
|
||||
* file. Best effort: a device that isn't recorded is only asked again.
|
||||
*/
|
||||
export async function rememberProvenDevice(
|
||||
req: Request,
|
||||
userUuid: string,
|
||||
deps: Pick<KnownDeviceDeps, 'kv'>,
|
||||
): Promise<void> {
|
||||
const session = req.actor?.session;
|
||||
for (const sessionUuid of [session?.uid, session?.parentUid]) {
|
||||
if (!sessionUuid) continue;
|
||||
try {
|
||||
const key = unprovenDeviceKey(sessionUuid);
|
||||
const { res } = await deps.kv.get({ key });
|
||||
if (typeof res !== 'string' || !res) continue;
|
||||
await recordKnownDevice(userUuid, res, deps);
|
||||
await deps.kv.del({ key });
|
||||
} catch (e) {
|
||||
console.warn('[known-device] proven device not recorded:', e);
|
||||
}
|
||||
}
|
||||
res.cookie(deps.config.cookie_name ?? 'puter_token', token, {
|
||||
...sessionCookieFlags(deps.config),
|
||||
httpOnly: true,
|
||||
});
|
||||
return gui_token;
|
||||
}
|
||||
@@ -5,17 +5,12 @@ import { useBenchEnv, type BenchEnv } from '../../benchUtil.js';
|
||||
describe('legacy fs', () => {
|
||||
const home = () => `/${env.users.user.username}`;
|
||||
|
||||
let openedUid = '';
|
||||
// A 1x1 PNG, which is what the thumbnails extension will accept.
|
||||
const thumbnail =
|
||||
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==';
|
||||
|
||||
const seed = async (env: BenchEnv) => {
|
||||
const owner = await env.server.stores.user.getByUsername(
|
||||
env.users.user.username,
|
||||
);
|
||||
const content = Buffer.from('opened by a benchmark');
|
||||
const { fsEntry } = await env.server.services.fs.write(owner!.id, {
|
||||
await env.server.services.fs.write(owner!.id, {
|
||||
fileMetadata: {
|
||||
path: `${home()}/Documents/opened.txt`,
|
||||
size: content.byteLength,
|
||||
@@ -23,7 +18,6 @@ describe('legacy fs', () => {
|
||||
},
|
||||
fileContent: content,
|
||||
});
|
||||
openedUid = fsEntry.uuid;
|
||||
};
|
||||
|
||||
const env = useBenchEnv({ seed });
|
||||
@@ -44,12 +38,6 @@ describe('legacy fs', () => {
|
||||
});
|
||||
});
|
||||
|
||||
env.scenario("set a file's thumbnail from the dashboard", async () => {
|
||||
await env.request('/update-fsentry-thumbnail', {
|
||||
json: { uid: openedUid, thumbnail },
|
||||
});
|
||||
});
|
||||
|
||||
env.scenario('read my storage usage', async () => {
|
||||
await env.request('/df', { json: {} });
|
||||
});
|
||||
|
||||
@@ -619,279 +619,6 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
|
||||
expect(seen).toEqual([expect.objectContaining({ uuid })]);
|
||||
});
|
||||
|
||||
const captureEvents = async (
|
||||
name: 'thumbnail.discarded' | 'outer.gui.item.updated',
|
||||
run: () => Promise<unknown>,
|
||||
): Promise<unknown[]> => {
|
||||
const seen: unknown[] = [];
|
||||
const listener = (_key: string, data: unknown) => {
|
||||
seen.push(data);
|
||||
};
|
||||
server.clients.event.on(name, listener as never);
|
||||
try {
|
||||
await run();
|
||||
} finally {
|
||||
server.clients.event.off(name, listener as never);
|
||||
}
|
||||
return seen;
|
||||
};
|
||||
|
||||
const setThumbnail = (
|
||||
actor: Actor,
|
||||
body: Record<string, unknown>,
|
||||
): Promise<unknown> =>
|
||||
withActor(actor, () =>
|
||||
controller.updateFsentryThumbnail(
|
||||
makeReq({ body, actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
|
||||
it('stores the thumbnail while the file has the expected modified time', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/expected-modified.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
const entry = await server.stores.fsEntry.getEntryByUuid(uuid);
|
||||
|
||||
const thumbnail = 'data:image/png;base64,aGVsbG8=';
|
||||
await setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail,
|
||||
expectedModified: entry!.modified,
|
||||
});
|
||||
|
||||
expect(
|
||||
(await server.stores.fsEntry.getEntryByUuidFromPrimary(uuid))
|
||||
?.thumbnail,
|
||||
).toBe(thumbnail);
|
||||
});
|
||||
|
||||
it('refuses a thumbnail made from contents the file no longer has', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/changed-since.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
const entry = await server.stores.fsEntry.getEntryByUuid(uuid);
|
||||
|
||||
const thumbnail = 'data:image/png;base64,aGVsbG8=';
|
||||
let rejection: unknown;
|
||||
const discarded = await captureEvents('thumbnail.discarded', () =>
|
||||
setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail,
|
||||
expectedModified: entry!.modified - 1,
|
||||
}).catch((error: unknown) => {
|
||||
rejection = error;
|
||||
}),
|
||||
);
|
||||
|
||||
expect(rejection).toMatchObject({
|
||||
statusCode: 409,
|
||||
legacyCode: 'conflict',
|
||||
});
|
||||
// The upload the extension made for it is dropped with it.
|
||||
expect(discarded).toEqual([{ uuid, url: thumbnail }]);
|
||||
expect(
|
||||
(await server.stores.fsEntry.getEntryByUuidFromPrimary(uuid))
|
||||
?.thumbnail ?? null,
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects an expectedModified that is not a number', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/bad-modified.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
|
||||
await expect(
|
||||
setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail: 'data:image/png;base64,aGVsbG8=',
|
||||
expectedModified: '123',
|
||||
}),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
message: '`expectedModified` must be a whole number of seconds',
|
||||
});
|
||||
});
|
||||
|
||||
it('discards the thumbnail it replaces, and only that one', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/replaced-thumb.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
const first = 'data:image/png;base64,Rmlyc3Q=';
|
||||
const second = 'data:image/png;base64,U2Vjb25k';
|
||||
|
||||
const discardedFirst = await captureEvents('thumbnail.discarded', () =>
|
||||
setThumbnail(actor, { uid: uuid, thumbnail: first }),
|
||||
);
|
||||
const discardedSecond = await captureEvents(
|
||||
'thumbnail.discarded',
|
||||
() => setThumbnail(actor, { uid: uuid, thumbnail: second }),
|
||||
);
|
||||
|
||||
expect(discardedFirst).toEqual([]);
|
||||
expect(discardedSecond).toEqual([{ uuid, url: first }]);
|
||||
});
|
||||
|
||||
it("tells the owner's open clients about the new thumbnail", async () => {
|
||||
const owner = await makeUser();
|
||||
const writer = await makeUser();
|
||||
const path = `/${owner.username}/Documents/announced-thumb.txt`;
|
||||
const { uuid } = await writeFileEntry(owner.actor, path, 'body');
|
||||
await server.services.permission.grantUserUserPermission(
|
||||
owner.actor,
|
||||
writer.username,
|
||||
`fs:${uuid}:write`,
|
||||
{},
|
||||
);
|
||||
|
||||
const thumbnail = 'data:image/png;base64,aGVsbG8=';
|
||||
const updates = await captureEvents('outer.gui.item.updated', () =>
|
||||
setThumbnail(writer.actor, { uid: uuid, thumbnail }),
|
||||
);
|
||||
|
||||
expect(updates).toEqual([
|
||||
expect.objectContaining({
|
||||
user_id_list: [owner.userId],
|
||||
response: expect.objectContaining({ uid: uuid, thumbnail }),
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([1795043242.5, 1e30, -1, 2 ** 31])(
|
||||
'rejects an expectedModified of %s, which no file has',
|
||||
async (expectedModified) => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/odd-modified.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
|
||||
await expect(
|
||||
setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail: 'data:image/png;base64,aGVsbG8=',
|
||||
expectedModified,
|
||||
}),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
},
|
||||
);
|
||||
|
||||
const duringThumbnailCreated = async (
|
||||
listener: (data: { url: string | null; uuid: string }) => unknown,
|
||||
run: () => Promise<unknown>,
|
||||
): Promise<void> => {
|
||||
const wrapped = (_key: string, data: unknown) =>
|
||||
listener(data as { url: string | null; uuid: string });
|
||||
server.clients.event.on('thumbnail.created', wrapped as never);
|
||||
try {
|
||||
await run();
|
||||
} finally {
|
||||
server.clients.event.off('thumbnail.created', wrapped as never);
|
||||
}
|
||||
};
|
||||
|
||||
it('announces the entry as it is after the thumbnail is stored', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/renamed-meanwhile.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
|
||||
let updates: unknown[] = [];
|
||||
// A rename that lands while the thumbnail is being uploaded.
|
||||
await duringThumbnailCreated(
|
||||
() =>
|
||||
withActor(actor, () =>
|
||||
controller.rename(
|
||||
makeReq({
|
||||
body: { uid: uuid, new_name: 'new-name.txt' },
|
||||
actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
),
|
||||
async () => {
|
||||
updates = await captureEvents('outer.gui.item.updated', () =>
|
||||
setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail: 'data:image/png;base64,aGVsbG8=',
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
expect(updates.at(-1)).toMatchObject({
|
||||
response: {
|
||||
uid: uuid,
|
||||
name: 'new-name.txt',
|
||||
path: `/${username}/Documents/new-name.txt`,
|
||||
thumbnail: 'data:image/png;base64,aGVsbG8=',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps the current thumbnail when the new one could not be saved', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/unsaved-thumb.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
const current = 'data:image/png;base64,Rmlyc3Q=';
|
||||
await setThumbnail(actor, { uid: uuid, thumbnail: current });
|
||||
|
||||
let rejection: unknown;
|
||||
let updates: unknown[] = [];
|
||||
await duringThumbnailCreated(
|
||||
(data) => {
|
||||
data.url = 's3://bucket/thumbnails/never-uploaded';
|
||||
throw new Error('storage is down');
|
||||
},
|
||||
async () => {
|
||||
updates = await captureEvents('outer.gui.item.updated', () =>
|
||||
setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail: 'data:image/png;base64,U2Vjb25k',
|
||||
}).catch((error: unknown) => {
|
||||
rejection = error;
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
expect(rejection).toMatchObject({ statusCode: 500 });
|
||||
expect(updates).toEqual([]);
|
||||
expect(
|
||||
(await server.stores.fsEntry.getEntryByUuidFromPrimary(uuid))
|
||||
?.thumbnail,
|
||||
).toBe(current);
|
||||
});
|
||||
|
||||
it('keeps the current thumbnail when the new one is not a usable image', async () => {
|
||||
const { actor, username } = await makeUser();
|
||||
const path = `/${username}/Documents/unusable-thumb.txt`;
|
||||
const { uuid } = await writeFileEntry(actor, path, 'body');
|
||||
const current = 'data:image/png;base64,Rmlyc3Q=';
|
||||
await setThumbnail(actor, { uid: uuid, thumbnail: current });
|
||||
|
||||
let rejection: unknown;
|
||||
// What the thumbnails extension does with an image it can't decode.
|
||||
await duringThumbnailCreated(
|
||||
(data) => {
|
||||
data.url = null;
|
||||
},
|
||||
() =>
|
||||
setThumbnail(actor, {
|
||||
uid: uuid,
|
||||
thumbnail: 'data:image/png;base64,U2Vjb25k',
|
||||
}).catch((error: unknown) => {
|
||||
rejection = error;
|
||||
}),
|
||||
);
|
||||
|
||||
expect(rejection).toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
expect(
|
||||
(await server.stores.fsEntry.getEntryByUuidFromPrimary(uuid))
|
||||
?.thumbnail,
|
||||
).toBe(current);
|
||||
});
|
||||
|
||||
it("refuses to retag another user's entry", async () => {
|
||||
const owner = await makeUser();
|
||||
const stranger = await makeUser();
|
||||
|
||||
@@ -90,7 +90,6 @@ import {
|
||||
FS_SIGNED_READ_LIMIT,
|
||||
FS_SIGNED_WRITE_LIMIT,
|
||||
FS_STAT_LIMIT,
|
||||
FS_THUMBNAIL_LIMIT,
|
||||
} from './limits.js';
|
||||
import {
|
||||
asRecord,
|
||||
@@ -140,9 +139,6 @@ const BATCH_MUTATING_OPS = new Set(['mkdir', 'shortcut', 'move', 'delete']);
|
||||
/** Rows per `readdir-subdomains` read; the response pages until done. */
|
||||
const SUBDOMAIN_PAGE_SIZE = 500;
|
||||
|
||||
/** The largest `fsentries.modified` every database engine's column holds. */
|
||||
const MAX_MODIFIED_SECONDS = 2 ** 31 - 1;
|
||||
|
||||
// An op's `as` names the path it produced; later ops in the same batch refer
|
||||
// to it as `$name` or `$name/rest`, since a deduped mkdir can land elsewhere.
|
||||
function resolveBatchRef(raw: string, refs: Map<string, string>): string {
|
||||
@@ -479,7 +475,7 @@ export class LegacyFSController extends PuterController {
|
||||
);
|
||||
router.post(
|
||||
'/update-fsentry-thumbnail',
|
||||
{ ...apiOptions, rateLimit: FS_THUMBNAIL_LIMIT },
|
||||
{ ...apiOptions, rateLimit: FS_HELPER_LIMIT },
|
||||
this.updateFsentryThumbnail,
|
||||
);
|
||||
|
||||
@@ -1199,10 +1195,9 @@ export class LegacyFSController extends PuterController {
|
||||
res: Response,
|
||||
): Promise<void> => {
|
||||
const actor = this.#requireActor(req);
|
||||
const { uid, thumbnail, expectedModified } = asRecord(req.body) as {
|
||||
const { uid, thumbnail } = asRecord(req.body) as {
|
||||
uid?: string;
|
||||
thumbnail?: string;
|
||||
expectedModified?: unknown;
|
||||
};
|
||||
if (!uid)
|
||||
throw new HttpError(400, 'Missing `uid`', {
|
||||
@@ -1220,21 +1215,6 @@ export class LegacyFSController extends PuterController {
|
||||
throw new HttpError(400, '`thumbnail` must be a data: URL', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
// The `modified` time of the contents the thumbnail was made from, so
|
||||
// a thumbnail of an older version isn't stored on a newer one. Only a
|
||||
// value the column can hold reaches the query.
|
||||
if (
|
||||
expectedModified !== undefined &&
|
||||
(typeof expectedModified !== 'number' ||
|
||||
!Number.isInteger(expectedModified) ||
|
||||
expectedModified < 0 ||
|
||||
expectedModified > MAX_MODIFIED_SECONDS)
|
||||
)
|
||||
throw new HttpError(
|
||||
400,
|
||||
'`expectedModified` must be a whole number of seconds',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
|
||||
const entry = await this.stores.fsEntry.getEntryByUuid(uid);
|
||||
if (!entry || !entry.path)
|
||||
@@ -1253,63 +1233,13 @@ export class LegacyFSController extends PuterController {
|
||||
// `event.url` from a data URL to an `s3://` pointer, and the DB
|
||||
// write below needs to see that rewrite.
|
||||
const event = { url: thumbnail, uuid: entry.uuid };
|
||||
const meta: import('../../clients/event/types.js').EventMetadata = {};
|
||||
await this.clients.event.emitAndWait('thumbnail.created', event, meta);
|
||||
const discard = (url: string | null) => {
|
||||
if (url) {
|
||||
this.clients.event.emit(
|
||||
'thumbnail.discarded',
|
||||
{ uuid: entry.uuid, url },
|
||||
{},
|
||||
);
|
||||
}
|
||||
};
|
||||
// Storing a pointer whose upload failed, or the null left for an image
|
||||
// that can't be decoded, would replace a thumbnail that works.
|
||||
if (meta.listener_failed) {
|
||||
if (event.url !== thumbnail) discard(event.url);
|
||||
throw new HttpError(500, 'The thumbnail could not be saved', {
|
||||
legacyCode: 'internal_error',
|
||||
});
|
||||
}
|
||||
// The listener's rewrite isn't bound by the event's declared type.
|
||||
if (typeof event.url !== 'string' || !event.url)
|
||||
throw new HttpError(400, '`thumbnail` is not a usable image', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
await this.clients.event.emitAndWait('thumbnail.created', event, {});
|
||||
|
||||
// Not the owner-scoped update: a shared writer passed the check above.
|
||||
// The entry is read again alongside it: a rename that landed during
|
||||
// the upload would otherwise be announced below with the name and
|
||||
// path it replaced.
|
||||
const [stored, current] = await Promise.all([
|
||||
this.stores.fsEntry.updateEntryThumbnailByUuid(
|
||||
entry.uuid,
|
||||
event.url,
|
||||
typeof expectedModified === 'number'
|
||||
? { modified: expectedModified }
|
||||
: {},
|
||||
),
|
||||
this.stores.fsEntry.getEntryByUuid(entry.uuid),
|
||||
]);
|
||||
if (!stored) {
|
||||
discard(event.url);
|
||||
if (typeof expectedModified !== 'number')
|
||||
throw new HttpError(404, `Entry not found: uid=${uid}`, {
|
||||
legacyCode: 'subject_does_not_exist',
|
||||
});
|
||||
throw new HttpError(
|
||||
409,
|
||||
'The file changed after this thumbnail was made',
|
||||
{ legacyCode: 'conflict' },
|
||||
);
|
||||
}
|
||||
if (entry.thumbnail !== event.url) discard(entry.thumbnail);
|
||||
// Other open tabs show the new thumbnail instead of each making one.
|
||||
await this.#emitGuiEvent('outer.gui.item.updated', {
|
||||
...(current ?? entry),
|
||||
thumbnail: event.url,
|
||||
});
|
||||
await this.stores.fsEntry.updateEntryThumbnailByUuid(
|
||||
entry.uuid,
|
||||
event.url,
|
||||
);
|
||||
res.json({ thumbnail: event.url });
|
||||
};
|
||||
|
||||
|
||||
@@ -179,13 +179,6 @@ export const FS_SIGN_LIMIT = userWindow('fs:sign', 300, 150, 60);
|
||||
/** Low-frequency GUI helpers. */
|
||||
export const FS_HELPER_LIMIT = userWindow('fs:helper', 120, 60, 30);
|
||||
|
||||
/**
|
||||
* The GUI fills in missing thumbnails as folders are viewed, so this gets its
|
||||
* own counter: a folder of photos must not use up the helper budget that
|
||||
* opening a file needs.
|
||||
*/
|
||||
export const FS_THUMBNAIL_LIMIT = userWindow('fs:thumbnail', 120, 60, 30);
|
||||
|
||||
/**
|
||||
* Puter.js polls this on a timer to decide whether to purge its FS cache. Same
|
||||
* ceiling for free and paid — it is a single cache read.
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
import express from 'express';
|
||||
import path from 'node:path';
|
||||
import { PuterController } from '../types.js';
|
||||
import { rememberDevice } from '../auth/knownDevice.js';
|
||||
import { isPlainUserActor } from '../../core/actor.js';
|
||||
import { toAppShellView } from '../../util/appShellView.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter';
|
||||
@@ -183,7 +182,6 @@ export class HomepageController extends PuterController {
|
||||
actor?: Parameters<typeof homepage.send>[0]['actor'];
|
||||
}
|
||||
).actor ?? null;
|
||||
this.#seedKnownDevice(req, res, actor);
|
||||
await homepage.send(
|
||||
{
|
||||
req,
|
||||
@@ -231,15 +229,7 @@ export class HomepageController extends PuterController {
|
||||
const guestApps = await this.services.recommendedApps
|
||||
.getRecommendedApps()
|
||||
.catch((): Array<Record<string, unknown>> => []);
|
||||
res.send(
|
||||
homepage.renderAppBrowserEmbed({
|
||||
guestApps,
|
||||
hostOrigin: embedHostOrigin(
|
||||
this.config,
|
||||
req.get('referer'),
|
||||
),
|
||||
}),
|
||||
);
|
||||
res.send(homepage.renderAppBrowserEmbed({ guestApps }));
|
||||
});
|
||||
|
||||
router.get('/embed/feedback', {}, (req, res) => {
|
||||
@@ -411,36 +401,6 @@ export class HomepageController extends PuterController {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Records the device behind a signed-in GUI page load, so browsers signed
|
||||
* in before devices were recorded count as known. Top-level loads only: a
|
||||
* framed page can't keep the SameSite=Lax cookie. The cookie is set before
|
||||
* the page is sent; the record update runs on without holding the page.
|
||||
*/
|
||||
#seedKnownDevice(
|
||||
req: express.Request,
|
||||
res: express.Response,
|
||||
actor: ShellContext['actor'],
|
||||
): void {
|
||||
if (req.tokenSource !== 'cookie') return;
|
||||
if (req.headers['sec-fetch-dest'] !== 'document') return;
|
||||
if (!actor || !isPlainUserActor(actor)) return;
|
||||
if (actor.session?.kind !== 'web' || actor.session.parentUid) return;
|
||||
const { uuid, hasPassword, email } = actor.user;
|
||||
if (!uuid || (!hasPassword && !email)) return;
|
||||
void (async () => {
|
||||
try {
|
||||
await rememberDevice(req, res, uuid, {
|
||||
config: this.config,
|
||||
token: this.services.token,
|
||||
kv: this.stores.kv,
|
||||
});
|
||||
} catch (e) {
|
||||
console.warn('[known-device] page-load record failed:', e);
|
||||
}
|
||||
})();
|
||||
}
|
||||
|
||||
/**
|
||||
* Lets the GUI skip `/get-gui-token`, which reads the session cookie: the
|
||||
* digest of the token it would hand over, null when there is no session
|
||||
|
||||
@@ -17,7 +17,6 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { AlarmController } from './alarm/AlarmController.js';
|
||||
import { AppController } from './apps/AppController.js';
|
||||
import { AppFeedbackController } from './feedback/AppFeedbackController.js';
|
||||
import { AuthController } from './auth/AuthController.js';
|
||||
@@ -64,7 +63,6 @@ export const puterControllers = {
|
||||
oidc: OIDCController,
|
||||
wisp: WispController,
|
||||
peer: PeerController,
|
||||
alarm: AlarmController,
|
||||
// Last so its catch-all static fallback doesn't shadow earlier routes.
|
||||
homepage: HomepageController,
|
||||
} satisfies IPuterControllerRegistry;
|
||||
@@ -35,7 +35,6 @@ import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import type { OIDCService } from '../../services/auth/OIDCService.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { KNOWN_DEVICE_COOKIE_NAME } from '../auth/knownDevice.js';
|
||||
|
||||
const TEST_ORIGIN = 'http://test.local';
|
||||
|
||||
@@ -134,8 +133,6 @@ interface CapturedResponse {
|
||||
redirectUrl?: string;
|
||||
headers: Record<string, string>;
|
||||
cookies: Array<{ name: string; value: string; opts?: unknown }>;
|
||||
/** The known-device cookie a sign-in sets, kept apart from `cookies`. */
|
||||
deviceCookie?: string;
|
||||
clearedCookies: Array<{ name: string; opts?: unknown }>;
|
||||
contentType?: string;
|
||||
}
|
||||
@@ -200,11 +197,7 @@ const makeRes = () => {
|
||||
return res;
|
||||
}),
|
||||
cookie: vi.fn((name: string, value: string, opts?: unknown) => {
|
||||
if (name === KNOWN_DEVICE_COOKIE_NAME) {
|
||||
captured.deviceCookie = value;
|
||||
} else {
|
||||
captured.cookies.push({ name, value, opts });
|
||||
}
|
||||
captured.cookies.push({ name, value, opts });
|
||||
return res;
|
||||
}),
|
||||
clearCookie: vi.fn((name: string, opts?: unknown) => {
|
||||
@@ -212,7 +205,6 @@ const makeRes = () => {
|
||||
return res;
|
||||
}),
|
||||
type: vi.fn(() => res),
|
||||
once: vi.fn(() => res),
|
||||
};
|
||||
return { res: res as unknown as Response, captured };
|
||||
};
|
||||
@@ -607,7 +599,6 @@ describe('OIDCController login callback', () => {
|
||||
// Session cookie was set with the configured cookie name.
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(captured.cookies[0]?.value).toBeTruthy();
|
||||
expect(captured.deviceCookie).toBeTruthy();
|
||||
// Same-origin redirect target is preserved.
|
||||
expect(captured.redirectUrl).toBe('http://test.local/');
|
||||
|
||||
@@ -1426,193 +1417,6 @@ describe('OIDCController signup veto (abuse harness)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('OIDCController sign-in validate', () => {
|
||||
const stubIdp = (sub: string, email: string) => {
|
||||
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
|
||||
access_token: 'access',
|
||||
id_token: 'id',
|
||||
} as never);
|
||||
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
|
||||
sub,
|
||||
email,
|
||||
email_verified: true,
|
||||
} as never);
|
||||
};
|
||||
|
||||
const onValidate = (listener: (data: Record<string, unknown>) => void) => {
|
||||
const handler = async (_key: string, data: unknown) =>
|
||||
listener(data as Record<string, unknown>);
|
||||
server.clients.event.on('puter.signin.validate', handler);
|
||||
return () => server.clients.event.off('puter.signin.validate', handler);
|
||||
};
|
||||
|
||||
it('a refused sign-in redirects with signin_blocked and the request code, minting nothing', async () => {
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const email = `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
const created = await runWithContext({ req: makeReq({}) }, () =>
|
||||
oidc().createUserFromOIDC('custom', {
|
||||
sub,
|
||||
email,
|
||||
email_verified: true,
|
||||
}),
|
||||
);
|
||||
const user = created.user!;
|
||||
stubIdp(sub, email);
|
||||
const seen: Array<Record<string, unknown>> = [];
|
||||
const stop = onValidate((data) => {
|
||||
if ((data.user as { uuid?: string }).uuid !== user.uuid) return;
|
||||
seen.push({ ...data });
|
||||
data.allow = false;
|
||||
data.trail_id = 'trail-oidc-1';
|
||||
});
|
||||
try {
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/login',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
expect(seen).toHaveLength(1);
|
||||
expect(seen[0]).toMatchObject({
|
||||
method: 'oidc',
|
||||
data: {},
|
||||
account: { oidc_linked: true },
|
||||
});
|
||||
expect(captured.redirectStatus).toBe(302);
|
||||
expect(captured.redirectUrl).toContain('action=login');
|
||||
expect(captured.redirectUrl).toContain('message=signin_blocked');
|
||||
expect(captured.redirectUrl).toContain('request_code=trail-oidc-1');
|
||||
expect(captured.cookies).toHaveLength(0);
|
||||
expect(captured.deviceCookie).toBeUndefined();
|
||||
} finally {
|
||||
stop();
|
||||
}
|
||||
});
|
||||
|
||||
it('a sign-in that has to be made again redirects back to its page with no session', async () => {
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const email = `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
const created = await runWithContext({ req: makeReq({}) }, () =>
|
||||
oidc().createUserFromOIDC('custom', {
|
||||
sub,
|
||||
email,
|
||||
email_verified: true,
|
||||
}),
|
||||
);
|
||||
const user = created.user!;
|
||||
await server.stores.user.update(user.id, {
|
||||
phone: '+14155550123',
|
||||
requires_phone_verification: 1,
|
||||
});
|
||||
stubIdp(sub, email);
|
||||
// The proof on file is given while this sign-in opens its session.
|
||||
const create = server.services.auth.createSessionToken.bind(
|
||||
server.services.auth,
|
||||
);
|
||||
vi.spyOn(
|
||||
server.services.auth,
|
||||
'createSessionToken',
|
||||
).mockImplementationOnce(async (...args) => {
|
||||
await server.stores.user.update(user.id, {
|
||||
requires_phone_verification: 0,
|
||||
});
|
||||
return create(...(args as Parameters<typeof create>));
|
||||
});
|
||||
const sessions = async () =>
|
||||
((await server.stores.session.getByUserId(user.id)) as unknown[])
|
||||
.length;
|
||||
const before = await sessions();
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/login',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.redirectStatus).toBe(302);
|
||||
expect(
|
||||
new URL(String(captured.redirectUrl), TEST_ORIGIN).pathname,
|
||||
).toBe('/');
|
||||
expect(captured.redirectUrl).toContain('action=login');
|
||||
expect(captured.redirectUrl).toContain('message=unauthorized');
|
||||
expect(captured.cookies).toHaveLength(0);
|
||||
expect(await sessions()).toBe(before);
|
||||
});
|
||||
|
||||
it('never asks puter.signin.attempt listeners', async () => {
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const email = `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
await runWithContext({ req: makeReq({}) }, () =>
|
||||
oidc().createUserFromOIDC('custom', {
|
||||
sub,
|
||||
email,
|
||||
email_verified: true,
|
||||
}),
|
||||
);
|
||||
stubIdp(sub, email);
|
||||
let attempts = 0;
|
||||
const refuse = async (_key: string, data: unknown) => {
|
||||
attempts++;
|
||||
(data as { allow: boolean }).allow = false;
|
||||
};
|
||||
server.clients.event.on('puter.signin.attempt', refuse);
|
||||
try {
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/login',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
expect(attempts).toBe(0);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
} finally {
|
||||
server.clients.event.off('puter.signin.attempt', refuse);
|
||||
}
|
||||
});
|
||||
|
||||
it('is not asked about the sign-in that creates the account', async () => {
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const email = `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
stubIdp(sub, email);
|
||||
const seen: unknown[] = [];
|
||||
const stop = onValidate((data) => {
|
||||
if ((data.user as { email?: string }).email === email)
|
||||
seen.push(data);
|
||||
});
|
||||
try {
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/signup',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(seen).toHaveLength(0);
|
||||
} finally {
|
||||
stop();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('OIDCController browser binding', () => {
|
||||
const NONCE_COOKIE = 'puter_oidc_nonce';
|
||||
|
||||
|
||||
@@ -23,7 +23,6 @@ import type { Request, Response } from 'express';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { PuterController } from '../types.js';
|
||||
import { SignInRefusedError } from '../../services/auth/SignInService.js';
|
||||
import { startWebSession } from '../auth/webSession.js';
|
||||
import {
|
||||
completeOidcLoginReturn,
|
||||
@@ -65,7 +64,6 @@ const ALLOWED_ERRORS = [
|
||||
'account_suspended',
|
||||
'unauthorized',
|
||||
'signup_blocked',
|
||||
'signin_blocked',
|
||||
'bonus_code_invalid',
|
||||
] as const;
|
||||
|
||||
@@ -623,14 +621,7 @@ export class OIDCController extends PuterController {
|
||||
);
|
||||
}
|
||||
|
||||
await this.#finishLogin(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
stateDecoded,
|
||||
'login',
|
||||
resolved.created ? 'signup' : 'oidc',
|
||||
);
|
||||
await this.#finishLogin(req, res, user, stateDecoded);
|
||||
};
|
||||
router.get('/auth/oidc/callback/login', cbOpts, loginCb);
|
||||
router.post('/auth/oidc/callback/login', cbOpts, loginCb);
|
||||
@@ -694,14 +685,7 @@ export class OIDCController extends PuterController {
|
||||
);
|
||||
}
|
||||
|
||||
await this.#finishLogin(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
stateDecoded,
|
||||
'signup',
|
||||
resolved.created ? 'signup' : 'oidc',
|
||||
);
|
||||
await this.#finishLogin(req, res, user, stateDecoded);
|
||||
};
|
||||
router.get('/auth/oidc/callback/signup', cbOpts, signupCb);
|
||||
router.post('/auth/oidc/callback/signup', cbOpts, signupCb);
|
||||
@@ -840,11 +824,7 @@ if (window.opener) {
|
||||
attempt = 0,
|
||||
): Promise<
|
||||
| { error: string; code?: string; requestCode?: string }
|
||||
| {
|
||||
user: import('../../stores/user/UserStore.js').UserRow;
|
||||
/** Created by this callback, so its sign-in is the signup's. */
|
||||
created?: boolean;
|
||||
}
|
||||
| { user: import('../../stores/user/UserStore.js').UserRow }
|
||||
> {
|
||||
// 1. Existing provider/sub link.
|
||||
const linked = await this.services.oidc.findUserByProviderSub(
|
||||
@@ -918,7 +898,7 @@ if (window.opener) {
|
||||
requestCode: outcome.requestCode,
|
||||
};
|
||||
}
|
||||
return { user: outcome.user, created: true };
|
||||
return { user: outcome.user };
|
||||
}
|
||||
|
||||
async #processCallback(
|
||||
@@ -1007,8 +987,6 @@ if (window.opener) {
|
||||
res: Response,
|
||||
user: UserRow,
|
||||
stateDecoded: Record<string, unknown>,
|
||||
sourceFlow: 'login' | 'signup',
|
||||
method: 'oidc' | 'signup',
|
||||
): Promise<void> {
|
||||
const origin = (this.config.origin ?? '').replace(/\/$/, '');
|
||||
const continuation: OidcLoginContinuation = {
|
||||
@@ -1064,38 +1042,10 @@ if (window.opener) {
|
||||
res.redirect(302, `${origin}/action/oidc-2fa#${fragment}`);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await startWebSession(
|
||||
req,
|
||||
res,
|
||||
user,
|
||||
{
|
||||
config: this.config,
|
||||
auth: this.services.auth,
|
||||
signIn: this.services.signIn,
|
||||
token: this.services.token,
|
||||
kv: this.stores.kv,
|
||||
user: this.stores.user,
|
||||
},
|
||||
method,
|
||||
);
|
||||
} catch (e) {
|
||||
// A sign-in that has to be made again lands back on its page.
|
||||
const retry = e instanceof HttpError && e.code === 'signin_retry';
|
||||
if (!(e instanceof SignInRefusedError) && !retry) throw e;
|
||||
return res.redirect(
|
||||
302,
|
||||
buildErrorRedirectUrl(
|
||||
origin,
|
||||
sourceFlow,
|
||||
'other',
|
||||
retry ? 'unauthorized' : 'signin_blocked',
|
||||
stateDecoded,
|
||||
retry ? undefined : (e as SignInRefusedError).requestCode,
|
||||
(p) => this.#mintPopupReturn(res, p),
|
||||
),
|
||||
);
|
||||
}
|
||||
await startWebSession(req, res, user, {
|
||||
config: this.config,
|
||||
auth: this.services.auth,
|
||||
});
|
||||
res.redirect(
|
||||
302,
|
||||
completeOidcLoginReturn(res, user, continuation, {
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
import { describe } from 'vitest';
|
||||
import { useBenchEnv, type BenchEnv } from '../../benchUtil.js';
|
||||
|
||||
describe('profile', () => {
|
||||
let etag = '';
|
||||
|
||||
const seed = async (env: BenchEnv) => {
|
||||
const user = await env.server.stores.user.getByUsername(
|
||||
env.users.user.username,
|
||||
);
|
||||
await env.server.services.profile.updateProfile(user!, {
|
||||
displayName: 'Bench',
|
||||
bio: 'Read by a benchmark',
|
||||
});
|
||||
const response = await env.request('/profile');
|
||||
etag = response.headers.get('etag') ?? '';
|
||||
};
|
||||
|
||||
const env = useBenchEnv({ seed });
|
||||
|
||||
env.scenario('read my own profile', async () => {
|
||||
await env.request('/profile');
|
||||
});
|
||||
|
||||
// A browser returning to the dashboard revalidates the copy it holds.
|
||||
env.scenario('revalidate my own profile', async () => {
|
||||
const response = await fetch(new URL('/profile', env.apiOrigin), {
|
||||
headers: {
|
||||
authorization: `Bearer ${env.users.user.token}`,
|
||||
'if-none-match': etag,
|
||||
// Without it fetch sends `no-cache`, which never gets a 304.
|
||||
'cache-control': 'max-age=0',
|
||||
},
|
||||
});
|
||||
await response.text();
|
||||
if (response.status !== 304) {
|
||||
throw new Error(`GET /profile answered ${response.status}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -17,8 +17,7 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import http from 'node:http';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
|
||||
const PICTURE =
|
||||
@@ -150,162 +149,4 @@ describe('profile routes over HTTP', () => {
|
||||
});
|
||||
expect(anonymous.status).toBe(401);
|
||||
});
|
||||
|
||||
describe('revalidation', () => {
|
||||
// `fetch` adds `cache-control: no-cache` beside If-None-Match, which
|
||||
// rules out a 304; this sends what a browser revalidating its copy does.
|
||||
const revalidate = (query: string, etag: string, token?: string) =>
|
||||
new Promise<{ status: number; etag?: string; body: string }>(
|
||||
(resolve, reject) => {
|
||||
const url = new URL(`/profile${query}`, env.apiOrigin);
|
||||
const req = http.request(
|
||||
{
|
||||
host: url.hostname,
|
||||
port: url.port,
|
||||
path: `${url.pathname}${url.search}`,
|
||||
headers: {
|
||||
host: url.host,
|
||||
'if-none-match': etag,
|
||||
...(token
|
||||
? { authorization: `Bearer ${token}` }
|
||||
: {}),
|
||||
},
|
||||
},
|
||||
(res) => {
|
||||
let body = '';
|
||||
res.setEncoding('utf8');
|
||||
res.on('data', (chunk) => (body += chunk));
|
||||
res.on('end', () =>
|
||||
resolve({
|
||||
status: res.statusCode!,
|
||||
etag: res.headers.etag,
|
||||
body,
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
req.on('error', reject);
|
||||
req.end();
|
||||
},
|
||||
);
|
||||
|
||||
it('answers 304 to the current tag without reading the stored profile', async () => {
|
||||
await post({ bio: 'tagged' }, env.users.user.token);
|
||||
const first = await get('', env.users.user.token);
|
||||
expect(first.headers.get('cache-control')).toBe(
|
||||
'private, no-cache',
|
||||
);
|
||||
const etag = first.headers.get('etag')!;
|
||||
expect(etag).toMatch(/^W\/"[^"]+"$/);
|
||||
|
||||
const read = vi.spyOn(env.server.services.fs, 'readContent');
|
||||
try {
|
||||
const again = await revalidate('', etag, env.users.user.token);
|
||||
expect(again.status).toBe(304);
|
||||
expect(again.body).toBe('');
|
||||
expect(read).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
read.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('sends the new profile once it changes', async () => {
|
||||
await post({ bio: 'before' }, env.users.user.token);
|
||||
const etag = (await get('', env.users.user.token)).headers.get(
|
||||
'etag',
|
||||
)!;
|
||||
await post({ bio: 'after' }, env.users.user.token);
|
||||
|
||||
const again = await revalidate('', etag, env.users.user.token);
|
||||
expect(again.status).toBe(200);
|
||||
expect(again.etag).not.toBe(etag);
|
||||
expect(JSON.parse(again.body)).toMatchObject({ bio: 'after' });
|
||||
});
|
||||
|
||||
// The file is written before its entry, and an entry can be read
|
||||
// from a cache or replica that has not caught up.
|
||||
describe('while the file entry is behind the file', () => {
|
||||
const profileEntry = async () =>
|
||||
(await env.server.stores.fsEntry.getEntryByPath(
|
||||
`/system/profiles/${userUuid}.profile`,
|
||||
))!;
|
||||
const serveEntry = (entry: unknown) => {
|
||||
const getEntryByPath =
|
||||
env.server.stores.fsEntry.getEntryByPath.bind(
|
||||
env.server.stores.fsEntry,
|
||||
);
|
||||
return vi
|
||||
.spyOn(env.server.stores.fsEntry, 'getEntryByPath')
|
||||
.mockImplementation(async (path, options) =>
|
||||
path.startsWith('/system/profiles/')
|
||||
? (entry as never)
|
||||
: getEntryByPath(path, options),
|
||||
);
|
||||
};
|
||||
|
||||
it('tags a response by the profile it carries', async () => {
|
||||
await post({ bio: 'first' }, env.users.user.token);
|
||||
const firstTag = (
|
||||
await get('', env.users.user.token)
|
||||
).headers.get('etag')!;
|
||||
const behind = await profileEntry();
|
||||
await post({ bio: 'second' }, env.users.user.token);
|
||||
|
||||
const spy = serveEntry(behind);
|
||||
try {
|
||||
const response = await get('', env.users.user.token);
|
||||
expect(await response.json()).toMatchObject({
|
||||
bio: 'second',
|
||||
});
|
||||
expect(response.headers.get('etag')).not.toBe(firstTag);
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('still answers 304 for a profile stored before tags were kept, after reading it', async () => {
|
||||
await post({ bio: 'kept' }, env.users.user.token);
|
||||
const untagged = { ...(await profileEntry()), metadata: null };
|
||||
const spy = serveEntry(untagged);
|
||||
const read = vi.spyOn(env.server.services.fs, 'readContent');
|
||||
try {
|
||||
const etag = (
|
||||
await get('', env.users.user.token)
|
||||
).headers.get('etag')!;
|
||||
const again = await revalidate(
|
||||
'',
|
||||
etag,
|
||||
env.users.user.token,
|
||||
);
|
||||
expect(again.status).toBe(304);
|
||||
expect(read).toHaveBeenCalledTimes(2);
|
||||
} finally {
|
||||
read.mockRestore();
|
||||
spy.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
it('tags an account that has stored no profile', async () => {
|
||||
const first = await get('', env.users.other.token);
|
||||
expect(await first.json()).toEqual({
|
||||
picture: null,
|
||||
displayName: null,
|
||||
bio: null,
|
||||
});
|
||||
const etag = first.headers.get('etag')!;
|
||||
const again = await revalidate('', etag, env.users.other.token);
|
||||
expect(again.status).toBe(304);
|
||||
});
|
||||
|
||||
it('still hides a profile whose owner left their plan, whatever tag the reader holds', async () => {
|
||||
const query = `?username=${env.users.user.username}`;
|
||||
const etag = await withSubscription('business', async () => {
|
||||
const shown = await get(query);
|
||||
expect(shown.status).toBe(200);
|
||||
return shown.headers.get('etag')!;
|
||||
});
|
||||
expect((await revalidate(query, etag)).status).toBe(404);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -66,18 +66,12 @@ describe('ProfileController own profile', () => {
|
||||
const getOwn = async (uuid: string) => {
|
||||
let body: unknown;
|
||||
const res = {
|
||||
set: () => res,
|
||||
vary: () => res,
|
||||
json: (value: unknown) => {
|
||||
body = value;
|
||||
},
|
||||
} as unknown as Response;
|
||||
await controller.get(
|
||||
{
|
||||
query: {},
|
||||
fresh: false,
|
||||
actor: { user: { uuid } },
|
||||
} as unknown as Request,
|
||||
{ query: {}, actor: { user: { uuid } } } as unknown as Request,
|
||||
res,
|
||||
);
|
||||
return body;
|
||||
|
||||
@@ -21,8 +21,8 @@ import type { Request, Response } from 'express';
|
||||
import { Controller, Get, Post } from '../../core/http/decorators.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import type {
|
||||
OpenedProfile,
|
||||
ProfileService,
|
||||
UserProfile,
|
||||
} from '../../services/profile/ProfileService.js';
|
||||
import { PuterController } from '../types.js';
|
||||
|
||||
@@ -42,8 +42,7 @@ const notFound = () =>
|
||||
export class ProfileController extends PuterController {
|
||||
/**
|
||||
* GET /profile?username=<name> — a user's profile. Without `username`, the
|
||||
* signed-in user's own. Answers 304 to a request holding the current tag
|
||||
* without reading the stored profile.
|
||||
* signed-in user's own.
|
||||
*/
|
||||
@Get('/', {
|
||||
subdomain: 'api',
|
||||
@@ -60,17 +59,17 @@ export class ProfileController extends PuterController {
|
||||
const username = req.query.username;
|
||||
|
||||
let target;
|
||||
let ownProfile: PromiseSettledResult<OpenedProfile> | undefined;
|
||||
let ownProfile: PromiseSettledResult<UserProfile> | undefined;
|
||||
if (username === undefined) {
|
||||
if (!caller?.uuid) {
|
||||
throw new HttpError(401, 'Sign in, or pass a username', {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
}
|
||||
// Opened alongside the account; discarded if there's no account.
|
||||
// Read alongside the account; discarded if there's no account.
|
||||
const [found, profile] = await Promise.allSettled([
|
||||
this.stores.user.getByUuid(caller.uuid),
|
||||
service.openProfile({ uuid: caller.uuid }),
|
||||
service.getProfile({ uuid: caller.uuid }),
|
||||
]);
|
||||
if (found.status === 'rejected') throw found.reason;
|
||||
target = found.value;
|
||||
@@ -92,27 +91,12 @@ export class ProfileController extends PuterController {
|
||||
if (!isOwner && !(await service.isPubliclyVisible(target))) {
|
||||
throw notFound();
|
||||
}
|
||||
let profile: OpenedProfile;
|
||||
if (ownProfile && isOwner) {
|
||||
if (ownProfile.status === 'rejected') throw ownProfile.reason;
|
||||
profile = ownProfile.value;
|
||||
} else {
|
||||
profile = await service.openProfile(target);
|
||||
res.json(ownProfile.value);
|
||||
return;
|
||||
}
|
||||
// Stored and revalidated on every read, by the caller's browser only.
|
||||
res.set('Cache-Control', 'private, no-cache');
|
||||
res.vary('Accept-Encoding');
|
||||
if (profile.etag) {
|
||||
res.set('ETag', profile.etag);
|
||||
if (req.fresh) {
|
||||
res.status(304).end();
|
||||
return;
|
||||
}
|
||||
}
|
||||
// Tagged by what was read; `res.json` answers 304 when that matches.
|
||||
const { profile: body, etag } = await profile.read();
|
||||
res.set('ETag', etag);
|
||||
res.json(body);
|
||||
res.json(await service.getProfile(target));
|
||||
}
|
||||
|
||||
/** POST /profile — update the signed-in user's own profile. */
|
||||
|
||||
@@ -7,11 +7,6 @@ describe('sharing', () => {
|
||||
// The share scenario takes a fresh recipient each run, so no run reads
|
||||
// what the run before it wrote.
|
||||
const recipients: string[] = [];
|
||||
// One standing share per run of the withdraw scenario, from an owner of
|
||||
// its own so the other scenarios' listings stay as they were.
|
||||
let revocable = '';
|
||||
let revoker = '';
|
||||
const holders: string[] = [];
|
||||
|
||||
const dir = () => `/${env.users.user.username}/Documents/shared`;
|
||||
|
||||
@@ -52,41 +47,6 @@ describe('sharing', () => {
|
||||
mode: 'read',
|
||||
},
|
||||
});
|
||||
const revokerUser = await createTestUser(env.server, {
|
||||
username: 'sharebenchrevoker',
|
||||
password: 'share-bench-password',
|
||||
});
|
||||
revoker = revokerUser.token;
|
||||
const content = Buffer.from('share bench');
|
||||
const revokerRow = await env.server.stores.user.getByUsername(
|
||||
revokerUser.username,
|
||||
);
|
||||
({
|
||||
fsEntry: { uuid: revocable },
|
||||
} = await fs.write(revokerRow!.id, {
|
||||
fileMetadata: {
|
||||
path: `/${revokerUser.username}/Documents/revocable.txt`,
|
||||
size: content.byteLength,
|
||||
contentType: 'text/plain',
|
||||
},
|
||||
fileContent: content,
|
||||
}));
|
||||
// Two warm-up runs and seven recorded ones.
|
||||
for (let i = 0; i < 9; i++) {
|
||||
const { username } = await createTestUser(env.server, {
|
||||
username: `sharebenchholder${i}`,
|
||||
password: 'share-bench-password',
|
||||
});
|
||||
holders.push(username);
|
||||
}
|
||||
await env.request('/share', {
|
||||
json: {
|
||||
recipients: holders,
|
||||
items: [{ uid: revocable }],
|
||||
mode: 'read',
|
||||
},
|
||||
token: revoker,
|
||||
});
|
||||
};
|
||||
|
||||
const env = useBenchEnv({ seed });
|
||||
@@ -129,15 +89,4 @@ describe('sharing', () => {
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
let withdrawals = 0;
|
||||
env.scenario('withdraw a file share from a user', async () => {
|
||||
await env.request('/share/revoke', {
|
||||
json: {
|
||||
recipients: [holders[withdrawals++]],
|
||||
items: [{ uid: revocable }],
|
||||
},
|
||||
token: revoker,
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -213,54 +213,6 @@ describe('share endpoints over HTTP', () => {
|
||||
expect(after.items.find((i) => i.uid_entry === file.uid)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('withdraws every mode a recipient holds with one delete and one cache bump', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
const file = await makeFile(owner);
|
||||
for (const mode of ['read', 'write']) {
|
||||
const shared = await post('/share', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
mode,
|
||||
});
|
||||
expect(shared.status).toBe(200);
|
||||
}
|
||||
|
||||
const writes = vi.spyOn(env.server.clients.db, 'write');
|
||||
const emits = vi.spyOn(env.server.clients.event, 'emit');
|
||||
try {
|
||||
const res = await post('/share/revoke', owner.token, {
|
||||
recipients: [recipient.username],
|
||||
items: [{ uid: file.uid }],
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
expect(await res.json()).toMatchObject({ revoked: 1 });
|
||||
|
||||
const grantDeletes = writes.mock.calls.filter(([sql]) =>
|
||||
/^DELETE FROM `user_to_user_permissions`/.test(String(sql)),
|
||||
);
|
||||
expect(grantDeletes).toHaveLength(1);
|
||||
const bumps = emits.mock.calls.filter(
|
||||
([key]) => key === 'outer.permission.generationBumped',
|
||||
);
|
||||
expect(bumps).toHaveLength(1);
|
||||
} finally {
|
||||
writes.mockRestore();
|
||||
emits.mockRestore();
|
||||
}
|
||||
|
||||
const read = await fetch(
|
||||
`${env.apiOrigin}/read?${new URLSearchParams({ file: file.path })}`,
|
||||
{
|
||||
headers: {
|
||||
authorization: `Bearer ${recipient.token}`,
|
||||
origin: env.apiOrigin,
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(read.status).not.toBe(200);
|
||||
});
|
||||
|
||||
describe('GET /share/shared-by-me', () => {
|
||||
it('lists what the caller shared out, across unrelated items', async () => {
|
||||
const owner = await makeUser();
|
||||
|
||||
@@ -55,11 +55,7 @@ export interface ActorAccessToken {
|
||||
}
|
||||
|
||||
export interface Actor {
|
||||
/**
|
||||
* The account. Token auth builds it from the user's row without the
|
||||
* password hash, and sets `hasPassword` in its place.
|
||||
*/
|
||||
user: Partial<UserRow> & { hasPassword?: boolean };
|
||||
user: Partial<UserRow>;
|
||||
/**
|
||||
* The app this actor carries _directly_, empty on an access token an app
|
||||
* issued. Answers "is this an app acting as itself" — `isAppActor` is
|
||||
|
||||
@@ -118,8 +118,6 @@ export interface HttpErrorOptions {
|
||||
* it. `noAlarm` still wins. Never serialized to the client.
|
||||
*/
|
||||
alarmSeverity?: PagerSeverity;
|
||||
/** Chat channel key for the alarm, as `AlarmOptions.channel`. */
|
||||
alarmChannel?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -150,7 +148,6 @@ export class HttpError extends Error {
|
||||
readonly fields?: Record<string, unknown>;
|
||||
readonly noAlarm?: boolean;
|
||||
readonly alarmSeverity?: PagerSeverity;
|
||||
readonly alarmChannel?: string;
|
||||
|
||||
constructor(
|
||||
statusCode: number,
|
||||
@@ -168,7 +165,6 @@ export class HttpError extends Error {
|
||||
this.fields = options.fields;
|
||||
this.noAlarm = options.noAlarm;
|
||||
this.alarmSeverity = options.alarmSeverity;
|
||||
this.alarmChannel = options.alarmChannel;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -111,13 +111,6 @@ declare global {
|
||||
*/
|
||||
tlsFingerprint?: string;
|
||||
|
||||
/**
|
||||
* Signals the proxy in front of the server raised on this request,
|
||||
* read from the headers named by `requestSignalHeaders`. Empty when
|
||||
* there are none.
|
||||
*/
|
||||
requestSignals?: readonly string[];
|
||||
|
||||
/**
|
||||
* Parsed cookies, populated by the global `cookie-parser`
|
||||
* middleware.
|
||||
|
||||
@@ -53,32 +53,4 @@ describe('auth', () => {
|
||||
throw new Error(`POST /login answered ${response.status}: ${body}`);
|
||||
}
|
||||
});
|
||||
|
||||
// A deployment whose extensions validate sign-ins, so the payload's reads
|
||||
// (known device, OIDC link, seat) run before the session is minted.
|
||||
env.scenario('log in with a sign-in validate listener', async () => {
|
||||
const { username, password } = env.users.user;
|
||||
const listener = async () => {};
|
||||
env.server.clients.event.on('puter.signin.validate', listener);
|
||||
try {
|
||||
const response = await fetch(new URL('/login', env.origin), {
|
||||
method: 'POST',
|
||||
// Its own client, so the scenario above leaves its login
|
||||
// limit untouched.
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'user-agent': 'auth-bench-validate',
|
||||
},
|
||||
body: JSON.stringify({ username, password }),
|
||||
});
|
||||
const body = await response.text();
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`POST /login answered ${response.status}: ${body}`,
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
env.server.clients.event.off('puter.signin.validate', listener);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -79,12 +79,10 @@ describe('createFingerprintMiddleware', () => {
|
||||
});
|
||||
|
||||
it('drops a malformed fingerprint to undefined', () => {
|
||||
expect(
|
||||
run({ body: { fingerprint: 'bad fp!' } }).req.deviceFingerprint,
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
run({ body: { fingerprint: 'short' } }).req.deviceFingerprint,
|
||||
).toBeUndefined();
|
||||
expect(run({ body: { fingerprint: 'bad fp!' } }).req.deviceFingerprint)
|
||||
.toBeUndefined();
|
||||
expect(run({ body: { fingerprint: 'short' } }).req.deviceFingerprint)
|
||||
.toBeUndefined();
|
||||
expect(
|
||||
run({ body: { fingerprint: 123 as unknown as string } }).req
|
||||
.deviceFingerprint,
|
||||
@@ -112,160 +110,4 @@ describe('createFingerprintMiddleware', () => {
|
||||
).req.tlsFingerprint,
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
describe('request signals', () => {
|
||||
const signalHeaders = {
|
||||
'X-Edge-Bot': 'bot',
|
||||
'x-edge-leaked': 'credential_exposed',
|
||||
'x-edge-bot-alt': 'bot',
|
||||
};
|
||||
|
||||
it('is empty without configured headers', () => {
|
||||
const { req } = run({ headers: { 'x-edge-bot': '1' } });
|
||||
expect(req.requestSignals).toEqual([]);
|
||||
});
|
||||
|
||||
it('are off, with one warning, when no value is configured', () => {
|
||||
const warn = vi
|
||||
.spyOn(console, 'warn')
|
||||
.mockImplementation(() => {});
|
||||
try {
|
||||
const middleware = createFingerprintMiddleware({
|
||||
requestSignalHeaders: signalHeaders,
|
||||
});
|
||||
expect(warn).toHaveBeenCalledTimes(1);
|
||||
for (const value of ['1', 'yes']) {
|
||||
const req = {
|
||||
headers: { 'x-edge-bot': value, 'x-edge-leaked': value },
|
||||
} as unknown as Request;
|
||||
middleware(req, {} as Response, vi.fn());
|
||||
expect(req.requestSignals).toEqual([]);
|
||||
}
|
||||
expect(warn).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('counts each signal once, and never a header holding an empty value', () => {
|
||||
const { req } = run(
|
||||
{
|
||||
headers: {
|
||||
'x-edge-bot': 'proxy-secret',
|
||||
'x-edge-bot-alt': 'proxy-secret',
|
||||
'x-edge-leaked': '',
|
||||
},
|
||||
},
|
||||
{
|
||||
requestSignalHeaders: signalHeaders,
|
||||
requestSignalValue: 'proxy-secret',
|
||||
},
|
||||
);
|
||||
expect(req.requestSignals).toEqual(['bot']);
|
||||
});
|
||||
|
||||
it('counts only the configured value when one is set', () => {
|
||||
const opts = {
|
||||
requestSignalHeaders: signalHeaders,
|
||||
requestSignalValue: 'proxy-secret',
|
||||
};
|
||||
expect(
|
||||
run(
|
||||
{
|
||||
headers: {
|
||||
'x-edge-bot': 'proxy-secret',
|
||||
'x-edge-leaked': 'proxy-secret',
|
||||
},
|
||||
},
|
||||
opts,
|
||||
).req.requestSignals,
|
||||
).toEqual(['bot', 'credential_exposed']);
|
||||
expect(
|
||||
run(
|
||||
{
|
||||
headers: {
|
||||
'x-edge-bot': 'forged',
|
||||
'x-edge-leaked': 'proxy-secret-',
|
||||
},
|
||||
},
|
||||
opts,
|
||||
).req.requestSignals,
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
const flaggedRun = (
|
||||
headers: Record<string, string>,
|
||||
actor: unknown,
|
||||
) => {
|
||||
const emit = vi.fn();
|
||||
const closers: Array<() => void> = [];
|
||||
const res = {
|
||||
once: vi.fn((_event: string, listener: () => void) => {
|
||||
closers.push(listener);
|
||||
}),
|
||||
};
|
||||
const middleware = createFingerprintMiddleware({
|
||||
requestSignalHeaders: signalHeaders,
|
||||
requestSignalValue: 'proxy-secret',
|
||||
events: { emit } as never,
|
||||
});
|
||||
const req = {
|
||||
headers,
|
||||
method: 'POST',
|
||||
path: '/drivers/call',
|
||||
ip: '198.51.100.4',
|
||||
actor,
|
||||
} as unknown as Request;
|
||||
middleware(req, res as unknown as Response, vi.fn());
|
||||
return { req, res, emit, close: () => closers.forEach((c) => c()) };
|
||||
};
|
||||
|
||||
it('announces a flagged request with an actor once it closes', () => {
|
||||
const { req, emit, close } = flaggedRun(
|
||||
{ 'x-edge-bot': 'proxy-secret' },
|
||||
{
|
||||
user: { id: 7, uuid: 'user-uuid', username: 'alice' },
|
||||
effectiveApp: { uid: 'app-uid' },
|
||||
},
|
||||
);
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
(req as unknown as { route: { path: string } }).route = {
|
||||
path: '/drivers/call',
|
||||
};
|
||||
close();
|
||||
expect(emit).toHaveBeenCalledWith(
|
||||
'puter.request.flagged',
|
||||
{
|
||||
signals: ['bot'],
|
||||
user_id: 7,
|
||||
user_uuid: 'user-uuid',
|
||||
username: 'alice',
|
||||
app_uid: 'app-uid',
|
||||
method: 'POST',
|
||||
route: '/drivers/call',
|
||||
ip: '198.51.100.4',
|
||||
tls_fingerprint: null,
|
||||
},
|
||||
{},
|
||||
);
|
||||
});
|
||||
|
||||
it('stays quiet for a flagged request without an actor', () => {
|
||||
const { emit, close } = flaggedRun(
|
||||
{ 'x-edge-bot': 'proxy-secret' },
|
||||
undefined,
|
||||
);
|
||||
close();
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('adds no work to a request without signals', () => {
|
||||
const { res, emit } = flaggedRun(
|
||||
{ 'user-agent': 'UA/1.0' },
|
||||
{ user: { id: 7, uuid: 'user-uuid', username: 'alice' } },
|
||||
);
|
||||
expect(res.once).not.toHaveBeenCalled();
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -17,10 +17,7 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, RequestHandler } from 'express';
|
||||
import type { EventClient } from '../../../clients/event/EventClient.js';
|
||||
import { secretsEqual } from '../../../util/secureCompare.js';
|
||||
import { onResponseClose } from '../onResponseClose.js';
|
||||
import type { RequestHandler } from 'express';
|
||||
import { computeNetworkFingerprint } from './rateLimit.js';
|
||||
import '../expressAugmentation';
|
||||
|
||||
@@ -68,66 +65,6 @@ function readDeviceFingerprint(req: {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const NO_SIGNALS: readonly string[] = Object.freeze([]);
|
||||
|
||||
/** `[header, signal]` pairs from `requestSignalHeaders`, header lowercased. */
|
||||
const signalHeaderPairs = (
|
||||
headers: Record<string, string> | undefined,
|
||||
): Array<[string, string]> =>
|
||||
Object.entries(headers ?? {}).flatMap(([header, signal]) =>
|
||||
typeof header === 'string' &&
|
||||
header &&
|
||||
typeof signal === 'string' &&
|
||||
signal
|
||||
? [[header.toLowerCase(), signal] as [string, string]]
|
||||
: [],
|
||||
);
|
||||
|
||||
function readRequestSignals(
|
||||
req: { headers?: Record<string, unknown> },
|
||||
pairs: Array<[string, string]>,
|
||||
expectedValue: string,
|
||||
): readonly string[] {
|
||||
let signals: string[] | null = null;
|
||||
for (const [header, signal] of pairs) {
|
||||
const value = req.headers?.[header];
|
||||
if (typeof value !== 'string' || !value) continue;
|
||||
if (!secretsEqual(value, expectedValue)) continue;
|
||||
signals ??= [];
|
||||
if (!signals.includes(signal)) signals.push(signal);
|
||||
}
|
||||
return signals ?? NO_SIGNALS;
|
||||
}
|
||||
|
||||
/** Announce a flagged request once it is done, when it had an actor. */
|
||||
function emitFlaggedOnClose(
|
||||
req: Request,
|
||||
res: Parameters<typeof onResponseClose>[0],
|
||||
events: EventClient,
|
||||
): void {
|
||||
const path = req.path;
|
||||
onResponseClose(res, () => {
|
||||
const actor = req.actor;
|
||||
if (!actor?.user) return;
|
||||
const route = req.route?.path;
|
||||
events.emit(
|
||||
'puter.request.flagged',
|
||||
{
|
||||
signals: req.requestSignals ?? NO_SIGNALS,
|
||||
user_id: actor.user.id ?? null,
|
||||
user_uuid: actor.user.uuid ?? null,
|
||||
username: actor.user.username ?? null,
|
||||
app_uid: actor.effectiveApp?.uid ?? null,
|
||||
method: req.method,
|
||||
route: typeof route === 'string' ? route : path,
|
||||
ip: req.ip || req.socket?.remoteAddress || null,
|
||||
tls_fingerprint: req.tlsFingerprint ?? null,
|
||||
},
|
||||
{},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Stamp request-scoped fingerprints onto `req` so any downstream gate, handler,
|
||||
* or service (via the ALS `Context.get('req')`) can read them without
|
||||
@@ -145,45 +82,19 @@ function emitFlaggedOnClose(
|
||||
* - `req.tlsFingerprint` — set only when `tlsFingerprintHeader` is configured and
|
||||
* the request carries a well-shaped value in that header. It names the
|
||||
* client's TLS stack, so every client on the same build shares it.
|
||||
* - `req.requestSignals` — the signals of the `requestSignalHeaders` the request
|
||||
* carries holding `requestSignalValue`; always empty without that value. A
|
||||
* flagged request with an actor emits `puter.request.flagged` once it closes;
|
||||
* an unflagged one costs nothing more.
|
||||
*
|
||||
* Install AFTER the body parsers (so the body fingerprint is readable) and
|
||||
* before `requestContext` (so the snapshot into ALS already carries them).
|
||||
* Never rejects — a missing/invalid device fingerprint is simply absent.
|
||||
*/
|
||||
export const createFingerprintMiddleware = (
|
||||
opts: {
|
||||
tlsFingerprintHeader?: string;
|
||||
requestSignalHeaders?: Record<string, string>;
|
||||
requestSignalValue?: string;
|
||||
events?: EventClient;
|
||||
} = {},
|
||||
opts: { tlsFingerprintHeader?: string } = {},
|
||||
): RequestHandler => {
|
||||
const tlsHeader = opts.tlsFingerprintHeader?.toLowerCase();
|
||||
const signalValue = opts.requestSignalValue || undefined;
|
||||
let signalPairs = signalHeaderPairs(opts.requestSignalHeaders);
|
||||
// Without the shared value any client could raise a signal itself.
|
||||
if (signalPairs.length > 0 && !signalValue) {
|
||||
console.warn(
|
||||
'[fingerprint] requestSignalHeaders is set without requestSignalValue; request signals are off',
|
||||
);
|
||||
signalPairs = [];
|
||||
}
|
||||
const { events } = opts;
|
||||
return (req, res, next) => {
|
||||
return (req, _res, next) => {
|
||||
req.networkFingerprint = computeNetworkFingerprint(req);
|
||||
req.deviceFingerprint = readDeviceFingerprint(req);
|
||||
req.tlsFingerprint = readTlsFingerprint(req, tlsHeader);
|
||||
req.requestSignals =
|
||||
signalValue && signalPairs.length > 0
|
||||
? readRequestSignals(req, signalPairs, signalValue)
|
||||
: NO_SIGNALS;
|
||||
if (req.requestSignals.length > 0 && events) {
|
||||
emitFlaggedOnClose(req, res, events);
|
||||
}
|
||||
next();
|
||||
};
|
||||
};
|
||||
@@ -147,7 +147,6 @@ export const createLargeJsonBodyParser = ({
|
||||
code: 'large_body_capacity',
|
||||
// Load shedding, not a fault: record it, don't page.
|
||||
alarmSeverity: 'info',
|
||||
alarmChannel: 'platform',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
|
||||
@@ -25,7 +25,6 @@ import {
|
||||
HOSTED_ASSET_COOKIES,
|
||||
type AuthService,
|
||||
} from '../../../services/auth/AuthService';
|
||||
import { FULL_API_ACCESS } from '../../../services/permission/consts';
|
||||
import { PuterServer } from '../../../server';
|
||||
import { setupTestServer } from '../../../testUtil';
|
||||
import type { IConfig } from '../../../types';
|
||||
@@ -834,15 +833,10 @@ describe('resolvePrivateIdentity', () => {
|
||||
|
||||
it('returns the sticky private-cookie identity when the token matches the expected app/subdomain/host', async () => {
|
||||
const user = await makeUser();
|
||||
const { session } = await authService.createSessionToken(
|
||||
user as never,
|
||||
{},
|
||||
);
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
const token = await authService.createHostedAssetToken('private', {
|
||||
appUid,
|
||||
userUid: user.uuid,
|
||||
sessionUuid: (session as { uuid: string }).uuid,
|
||||
subdomain: 'beans',
|
||||
host: 'beans.app.puter.localhost',
|
||||
});
|
||||
@@ -865,14 +859,9 @@ describe('resolvePrivateIdentity', () => {
|
||||
|
||||
it('falls through to req.actor when the private cookie is for a different app', async () => {
|
||||
const user = await makeUser();
|
||||
const { session } = await authService.createSessionToken(
|
||||
user as never,
|
||||
{},
|
||||
);
|
||||
const wrongToken = await authService.createHostedAssetToken('private', {
|
||||
appUid: `app-${uuidv4()}`,
|
||||
userUid: user.uuid,
|
||||
sessionUuid: (session as { uuid: string }).uuid,
|
||||
subdomain: 'beans',
|
||||
host: 'beans.app.puter.localhost',
|
||||
});
|
||||
@@ -1022,7 +1011,7 @@ describe('resolvePrivateIdentity', () => {
|
||||
expect(out.source).toBe('none');
|
||||
});
|
||||
|
||||
it('accepts a full-access token, which holds the account, bound to its own row', async () => {
|
||||
it('accepts a full-access token, which holds the account', async () => {
|
||||
const user = await makeUser();
|
||||
const issuer = makeActor({ user: { uuid: user.uuid } });
|
||||
const out = await resolvePrivateIdentity({
|
||||
@@ -1034,7 +1023,6 @@ describe('resolvePrivateIdentity', () => {
|
||||
uid: `tok-${uuidv4()}`,
|
||||
issuer,
|
||||
fullAccess: true,
|
||||
sessionUid: 'token-row-1',
|
||||
},
|
||||
}),
|
||||
}),
|
||||
@@ -1044,83 +1032,6 @@ describe('resolvePrivateIdentity', () => {
|
||||
});
|
||||
expect(out.source).toBe('session-cookie');
|
||||
expect(out.userUid).toBe(user.uuid);
|
||||
expect(out.sessionUuid).toBe('token-row-1');
|
||||
});
|
||||
|
||||
it('ignores a full-access token that names no row of its own to bind a cookie to', async () => {
|
||||
const user = await makeUser();
|
||||
const issuer = makeActor({ user: { uuid: user.uuid } });
|
||||
const out = await resolvePrivateIdentity({
|
||||
req: reqOf({
|
||||
cookies: {},
|
||||
actor: makeActor({
|
||||
user: { uuid: user.uuid },
|
||||
accessToken: {
|
||||
uid: `tok-${uuidv4()}`,
|
||||
issuer,
|
||||
fullAccess: true,
|
||||
},
|
||||
}),
|
||||
}),
|
||||
authService,
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
expectedAppUid: `app-target-${uuidv4()}`,
|
||||
});
|
||||
expect(out.source).toBe('none');
|
||||
});
|
||||
|
||||
describe('a cookie minted from a full-access token in the query', () => {
|
||||
const mint = async () => {
|
||||
const user = await makeUser();
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
const expected = {
|
||||
expectedAppUid: appUid,
|
||||
expectedSubdomain: 'beans',
|
||||
expectedPrivateHost: 'beans.app.puter.localhost',
|
||||
};
|
||||
const token = await authService.createAccessToken(
|
||||
makeActor({ user: { id: user.id, uuid: user.uuid } }),
|
||||
[[FULL_API_ACCESS]],
|
||||
);
|
||||
const viaQuery = await resolvePrivateIdentity({
|
||||
req: reqOf({ query: { auth_token: token } }),
|
||||
authService,
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
...expected,
|
||||
});
|
||||
expect(viaQuery.source).toBe('query');
|
||||
const cookie = await authService.createHostedAssetToken('private', {
|
||||
appUid,
|
||||
userUid: viaQuery.userUid!,
|
||||
sessionUuid: viaQuery.sessionUuid,
|
||||
subdomain: 'beans',
|
||||
host: 'beans.app.puter.localhost',
|
||||
});
|
||||
const withCookie = () =>
|
||||
resolvePrivateIdentity({
|
||||
req: reqOf({
|
||||
cookies: {
|
||||
[HOSTED_ASSET_COOKIES.private.name]: cookie,
|
||||
},
|
||||
}),
|
||||
authService,
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
...expected,
|
||||
});
|
||||
return { user, token, viaQuery, withCookie };
|
||||
};
|
||||
|
||||
it('is refused once the token is revoked', async () => {
|
||||
const { user, token, viaQuery, withCookie } = await mint();
|
||||
expect(viaQuery.sessionUuid).toBeTruthy();
|
||||
expect((await withCookie()).source).toBe('private-cookie');
|
||||
|
||||
await authService.revokeAccessToken(
|
||||
makeActor({ user: { id: user.id, uuid: user.uuid } }),
|
||||
token,
|
||||
);
|
||||
expect((await withCookie()).source).toBe('none');
|
||||
});
|
||||
});
|
||||
|
||||
it("ignores a bootstrap query-token whose actor.app.uid doesn't match the target app", async () => {
|
||||
|
||||
@@ -400,7 +400,7 @@ export async function resolvePrivateIdentity(opts: {
|
||||
return {
|
||||
source: 'session-cookie',
|
||||
userUid: existingActor.user.uuid,
|
||||
sessionUuid: boundSessionUid(existingActor),
|
||||
sessionUuid: existingActor.session?.uid,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -419,7 +419,7 @@ export async function resolvePrivateIdentity(opts: {
|
||||
return {
|
||||
source: 'session-cookie',
|
||||
userUid: actor.user.uuid,
|
||||
sessionUuid: boundSessionUid(actor),
|
||||
sessionUuid: actor.session?.uid,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
@@ -441,7 +441,7 @@ export async function resolvePrivateIdentity(opts: {
|
||||
return {
|
||||
source: bootstrap.source,
|
||||
userUid: actor.user.uuid,
|
||||
sessionUuid: boundSessionUid(actor),
|
||||
sessionUuid: actor.session?.uid,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
@@ -466,30 +466,19 @@ export async function resolvePrivateIdentity(opts: {
|
||||
* as a plain session on any private host.
|
||||
*
|
||||
* A scoped access token (a read URL, whoever issued it) is never the account
|
||||
* here. A full-access one is, when it names its own session row: the hosted
|
||||
* cookie minted from it is bound to that row, so it ends with the token.
|
||||
* here: the hosted cookie minted from it would outlive the token's revocation.
|
||||
*/
|
||||
function actorMatchesExpectedApp(
|
||||
actor: Actor,
|
||||
expectedAppUid: string | undefined,
|
||||
): boolean {
|
||||
if (
|
||||
isAccessTokenActor(actor) &&
|
||||
(!isAccountContext(actor) || !actor.accessToken?.sessionUid)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (isAccessTokenActor(actor) && !isAccountContext(actor)) return false;
|
||||
if (!expectedAppUid) return true;
|
||||
const app = actor.effectiveApp;
|
||||
if (!app) return true;
|
||||
return app.uid === expectedAppUid;
|
||||
}
|
||||
|
||||
/** The session row a hosted cookie minted for `actor` is bound to. */
|
||||
function boundSessionUid(actor: Actor): string | undefined {
|
||||
return actor.session?.uid ?? actor.accessToken?.sessionUid;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fresh authenticated identities replace the sticky public billing cookie when
|
||||
* the visitor switches accounts.
|
||||
@@ -522,7 +511,7 @@ export async function resolvePublicHostedIdentity(opts: {
|
||||
return {
|
||||
source: 'session-cookie',
|
||||
userUid: existingActor.user.uuid,
|
||||
sessionUuid: boundSessionUid(existingActor),
|
||||
sessionUuid: existingActor.session?.uid,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -540,7 +529,7 @@ export async function resolvePublicHostedIdentity(opts: {
|
||||
return {
|
||||
source: 'session-cookie',
|
||||
userUid: actor.user.uuid,
|
||||
sessionUuid: boundSessionUid(actor),
|
||||
sessionUuid: actor.session?.uid,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
@@ -616,7 +605,7 @@ export async function resolvePublicHostedIdentity(opts: {
|
||||
return {
|
||||
source: bootstrap.source,
|
||||
userUid: actor.user.uuid,
|
||||
sessionUuid: boundSessionUid(actor),
|
||||
sessionUuid: actor.session?.uid,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
|
||||
@@ -2010,7 +2010,6 @@ describe('createPuterSiteMiddleware — private app sign-in token', () => {
|
||||
let host: string;
|
||||
let appUid: string;
|
||||
let token: string;
|
||||
let sessionUuid: string;
|
||||
const allow = (
|
||||
_key: string,
|
||||
data: { appUid: string; result: { allowed: boolean } },
|
||||
@@ -2041,11 +2040,9 @@ describe('createPuterSiteMiddleware — private app sign-in token', () => {
|
||||
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`, `is_private`) VALUES (?, ?, ?, ?, ?, ?)',
|
||||
[appUid, sub, sub, `http://${host}/`, owner.id, 1],
|
||||
);
|
||||
const signedIn = await server.services.auth.createSessionToken(
|
||||
({ token } = await server.services.auth.createSessionToken(
|
||||
visitor as never,
|
||||
);
|
||||
token = signedIn.token;
|
||||
sessionUuid = (signedIn.session as { uuid: string }).uuid;
|
||||
));
|
||||
server.clients.event.on('app.privateAccess.check', allow as never);
|
||||
});
|
||||
|
||||
@@ -2109,7 +2106,6 @@ describe('createPuterSiteMiddleware — private app sign-in token', () => {
|
||||
{
|
||||
appUid,
|
||||
userUid: visitor.uuid,
|
||||
sessionUuid,
|
||||
subdomain: sub,
|
||||
host,
|
||||
},
|
||||
@@ -2119,34 +2115,6 @@ describe('createPuterSiteMiddleware — private app sign-in token', () => {
|
||||
cookies: { [cookieName()]: cookie },
|
||||
});
|
||||
expect(out.redirected).toEqual({ status: 302, url: '/?a=1&b=x%20y' });
|
||||
expect(out.cookies[cookieName()]).toBeUndefined();
|
||||
});
|
||||
|
||||
it('replaces a cookie that names no session with one bound to the sign-in', async () => {
|
||||
const unbound = server.services.token.sign('hosted-asset', {
|
||||
kind: 'private',
|
||||
version: '2',
|
||||
user_uid: visitor.uuid,
|
||||
app_uid: appUid,
|
||||
subdomain: sub,
|
||||
host,
|
||||
});
|
||||
const out = await visit({
|
||||
dest: 'iframe',
|
||||
cookies: { [cookieName()]: unbound },
|
||||
});
|
||||
expect(out.statusCode).toBe(200);
|
||||
const minted = out.cookies[cookieName()];
|
||||
expect(minted).toBeTruthy();
|
||||
const claims = await server.services.auth.verifyHostedAssetToken(
|
||||
'private',
|
||||
String(minted),
|
||||
{ appUid },
|
||||
);
|
||||
expect(claims).toMatchObject({
|
||||
userUid: visitor.uuid,
|
||||
sessionUuid: expect.any(String),
|
||||
});
|
||||
});
|
||||
|
||||
it('serves without redirecting when the cookie could not be minted', async () => {
|
||||
|
||||
@@ -101,8 +101,7 @@ export function sweepMemoryWindows() {
|
||||
sweep.unref?.();
|
||||
}
|
||||
|
||||
/** Records a hit and returns its timestamp, or null when the bucket is full. */
|
||||
function admitMemory(key, limit, windowMs) {
|
||||
async function checkMemory(key, limit, windowMs) {
|
||||
const now = Date.now();
|
||||
const cutoff = now - windowMs;
|
||||
let entry = memoryWindows.get(key);
|
||||
@@ -121,24 +120,9 @@ function admitMemory(key, limit, windowMs) {
|
||||
}
|
||||
const timestamps = entry.ts;
|
||||
while (timestamps.length > 0 && timestamps[0] < cutoff) timestamps.shift();
|
||||
if (timestamps.length >= limit) return null;
|
||||
if (timestamps.length >= limit) return false;
|
||||
timestamps.push(now);
|
||||
return now;
|
||||
}
|
||||
|
||||
async function checkMemory(key, limit, windowMs) {
|
||||
return admitMemory(key, limit, windowMs) !== null;
|
||||
}
|
||||
|
||||
/** `checkMemory` that returns a function giving the hit back, or null. */
|
||||
async function reserveMemory(key, limit, windowMs) {
|
||||
const at = admitMemory(key, limit, windowMs);
|
||||
if (at === null) return null;
|
||||
return async () => {
|
||||
const timestamps = memoryWindows.get(key)?.ts;
|
||||
const i = timestamps ? timestamps.lastIndexOf(at) : -1;
|
||||
if (i >= 0) timestamps.splice(i, 1);
|
||||
};
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -290,12 +274,7 @@ function multiResult(results, i) {
|
||||
return entry;
|
||||
}
|
||||
|
||||
async function checkRedis(redis, key, limit, windowMs) {
|
||||
return (await reserveRedis(redis, key, limit, windowMs)) !== null;
|
||||
}
|
||||
|
||||
/** `checkRedis` that returns a function giving the hit back, or null. */
|
||||
async function reserveRedis(
|
||||
async function checkRedis(
|
||||
/** @type {import('ioredis').Cluster} */
|
||||
redis,
|
||||
/** @type {string} */
|
||||
@@ -326,11 +305,9 @@ async function reserveRedis(
|
||||
const count = Number(multiResult(results, 2));
|
||||
if (count > limit) {
|
||||
sendAsUndo(() => redis.zrem(redisKey, member)).catch(noop);
|
||||
return null;
|
||||
return false;
|
||||
}
|
||||
return async () => {
|
||||
await redis.zrem(redisKey, member);
|
||||
};
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -376,11 +353,6 @@ async function peekRedis(redis, key, limit, windowMs) {
|
||||
// -- KV backend ------------------------------------------------------
|
||||
|
||||
async function checkKv(kv, key, limit, windowMs) {
|
||||
return (await reserveKv(kv, key, limit, windowMs)) !== null;
|
||||
}
|
||||
|
||||
/** `checkKv` that returns a function giving the hit back, or null. */
|
||||
async function reserveKv(kv, key, limit, windowMs) {
|
||||
const prefix = `rate:${key}:`;
|
||||
// `list` filters by TTL already, so a non-expired row ⇒ in-window.
|
||||
// Cap the fetch at `limit + 1` — once we know it's over, the exact
|
||||
@@ -393,18 +365,15 @@ async function reserveKv(kv, key, limit, windowMs) {
|
||||
reverse: true,
|
||||
});
|
||||
const keys = Array.isArray(res) ? res : (res?.items ?? []);
|
||||
if (keys.length >= limit) return null;
|
||||
if (keys.length >= limit) return false;
|
||||
|
||||
const now = Date.now();
|
||||
const row = `${prefix}${now}:${crypto.randomUUID()}`;
|
||||
await kv.set({
|
||||
key: row,
|
||||
key: `${prefix}${now}:${crypto.randomUUID()}`,
|
||||
value: 1,
|
||||
expireAt: Math.ceil((now + windowMs) / 1000),
|
||||
});
|
||||
return async () => {
|
||||
await kv.del({ key: row });
|
||||
};
|
||||
return true;
|
||||
}
|
||||
|
||||
// `checkKv` for many units: one read, then a row per admitted unit. Newest
|
||||
@@ -614,10 +583,6 @@ function instrumentBackendPair(name, pair) {
|
||||
withSpan('rate_limit.peek', attrs, () =>
|
||||
pair.peek(key, limit, windowMs),
|
||||
),
|
||||
reserve: (key, limit, windowMs) =>
|
||||
withSpan('rate_limit.reserve', attrs, () =>
|
||||
pair.reserve(key, limit, windowMs),
|
||||
),
|
||||
claim: (key, limit, windowMs, units) =>
|
||||
withSpan('rate_limit.claim', attrs, () =>
|
||||
pair.claim(key, limit, windowMs, units),
|
||||
@@ -636,7 +601,6 @@ function instrumentBackendPair(name, pair) {
|
||||
const memoryBackendPair = instrumentBackendPair('memory', {
|
||||
rate: checkMemory,
|
||||
peek: peekMemory,
|
||||
reserve: reserveMemory,
|
||||
claim: claimMemory,
|
||||
count: claimCountMemory,
|
||||
acquire: acquireMemoryConcurrent,
|
||||
@@ -687,8 +651,6 @@ export function configureRateLimit({
|
||||
checkRedis(redis, key, limit, windowMs),
|
||||
peek: (key, limit, windowMs) =>
|
||||
peekRedis(redis, key, limit, windowMs),
|
||||
reserve: (key, limit, windowMs) =>
|
||||
reserveRedis(redis, key, limit, windowMs),
|
||||
claim: (key, limit, windowMs, units) =>
|
||||
claimRedis(redis, key, limit, windowMs, units),
|
||||
count: (key, limit, windowMs, units) =>
|
||||
@@ -700,8 +662,6 @@ export function configureRateLimit({
|
||||
backends.kv = instrumentBackendPair('kv', {
|
||||
rate: (key, limit, windowMs) => checkKv(kv, key, limit, windowMs),
|
||||
peek: (key, limit) => peekKv(kv, key, limit),
|
||||
reserve: (key, limit, windowMs) =>
|
||||
reserveKv(kv, key, limit, windowMs),
|
||||
claim: (key, limit, windowMs, units) =>
|
||||
claimKv(kv, key, limit, windowMs, units),
|
||||
count: (key, limit, windowMs, units) =>
|
||||
@@ -1017,40 +977,6 @@ export async function checkRateLimit(
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* `checkRateLimit` whose hit can be given back: `{ ok, release }`, where
|
||||
* `release()` returns an admitted hit to the bucket and never rejects. For a
|
||||
* budget charged by an outcome known only after the work, such as a failed
|
||||
* credential check: counting the hit first means concurrent callers can't all
|
||||
* pass on the last slot. Fails open, admitting with a no-op `release`.
|
||||
*/
|
||||
export async function reserveRateLimit(key, limit, windowMs, backend) {
|
||||
let giveBack;
|
||||
try {
|
||||
giveBack = await resolveBackend(backend).reserve(key, limit, windowMs);
|
||||
} catch (err) {
|
||||
console.error(
|
||||
'[rate-limit] imperative reserve failed, failing open:',
|
||||
err,
|
||||
);
|
||||
return { ok: true, release: noop };
|
||||
}
|
||||
if (!giveBack) return { ok: false, release: noop };
|
||||
let released = false;
|
||||
return {
|
||||
ok: true,
|
||||
release: async () => {
|
||||
if (released) return;
|
||||
released = true;
|
||||
try {
|
||||
await giveBack();
|
||||
} catch (err) {
|
||||
console.error('[rate-limit] imperative release failed:', err);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* `checkRateLimit` for `units` hits at once, in one backend round trip.
|
||||
* Resolves to how many were admitted (0..units) and spent; the rest were not
|
||||
|
||||
@@ -47,7 +47,6 @@ import {
|
||||
listConfiguredRateLimitBackends,
|
||||
peekRateLimit,
|
||||
rateLimitGate,
|
||||
reserveRateLimit,
|
||||
sweepMemoryWindows,
|
||||
} from './rateLimit.js';
|
||||
|
||||
@@ -1781,91 +1780,6 @@ describe('peekRateLimit', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('reserveRateLimit', () => {
|
||||
beforeEach(() => configureRateLimit());
|
||||
afterEach(() => configureRateLimit());
|
||||
|
||||
it.each([
|
||||
['memory', () => ({})],
|
||||
['redis', () => ({ redis: new RedisMock() })],
|
||||
])(
|
||||
'admits only what fits however many reserve at once, and takes a release back (%s)',
|
||||
async (name, wiring) => {
|
||||
const cfg = wiring();
|
||||
configureRateLimit(cfg);
|
||||
const key = `reserve-${name}-${Math.random()}`;
|
||||
expect(await checkRateLimit(key, 3, 60_000, name)).toBe(true);
|
||||
const held = await Promise.all(
|
||||
Array.from({ length: 5 }, () =>
|
||||
reserveRateLimit(key, 3, 60_000, name),
|
||||
),
|
||||
);
|
||||
expect(held.filter((r) => r.ok)).toHaveLength(2);
|
||||
expect(await peekRateLimit(key, 3, 60_000, name)).toBe(false);
|
||||
|
||||
const admitted = held.filter((r) => r.ok);
|
||||
await admitted[0].release();
|
||||
await admitted[0].release();
|
||||
expect(await reserveRateLimit(key, 3, 60_000, name)).toMatchObject({
|
||||
ok: true,
|
||||
});
|
||||
expect(await peekRateLimit(key, 3, 60_000, name)).toBe(false);
|
||||
await cfg.redis?.quit?.();
|
||||
},
|
||||
);
|
||||
|
||||
it('releases each overlapping memory reservation from its own slot', async () => {
|
||||
const key = `reserve-memory-overlap-${Math.random()}`;
|
||||
const clock = vi.spyOn(Date, 'now').mockReturnValue(1_000_000);
|
||||
try {
|
||||
const first = reserveRateLimit(key, 2, 60_000, 'memory');
|
||||
clock.mockReturnValue(1_000_001);
|
||||
const second = reserveRateLimit(key, 2, 60_000, 'memory');
|
||||
const held = await Promise.all([first, second]);
|
||||
expect(held.map((r) => r.ok)).toEqual([true, true]);
|
||||
|
||||
await Promise.all(held.map((r) => r.release()));
|
||||
expect(await peekRateLimit(key, 1, 60_000, 'memory')).toBe(true);
|
||||
} finally {
|
||||
clock.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('takes a kv hit back', async () => {
|
||||
const server = await setupTestServer();
|
||||
try {
|
||||
configureRateLimit({ kv: server.stores.kv });
|
||||
const key = `reserve-kv-${Math.random().toString(36).slice(2, 10)}`;
|
||||
const first = await reserveRateLimit(key, 1, 60_000, 'kv');
|
||||
expect(first.ok).toBe(true);
|
||||
expect((await reserveRateLimit(key, 1, 60_000, 'kv')).ok).toBe(
|
||||
false,
|
||||
);
|
||||
await first.release();
|
||||
expect(await peekRateLimit(key, 1, 60_000, 'kv')).toBe(true);
|
||||
} finally {
|
||||
configureRateLimit();
|
||||
await server.shutdown();
|
||||
}
|
||||
});
|
||||
|
||||
it('fails open when the backend throws', async () => {
|
||||
configureRateLimit({
|
||||
default: 'redis',
|
||||
redis: {
|
||||
multi: () => {
|
||||
throw new Error('redis down');
|
||||
},
|
||||
},
|
||||
});
|
||||
const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
const reserved = await reserveRateLimit('boom', 1, 60_000);
|
||||
expect(reserved.ok).toBe(true);
|
||||
await expect(reserved.release()).resolves.toBeUndefined();
|
||||
spy.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
// ── Failure paths on the driver helpers ─────────────────────────────
|
||||
|
||||
describe('driver helpers — backend failures', () => {
|
||||
|
||||
@@ -557,7 +557,6 @@ describe('ChatCompletionDriver exhausted-chain classification', () => {
|
||||
});
|
||||
expect(err.noAlarm).toBeFalsy();
|
||||
expect(err.alarmSeverity).toBe('info');
|
||||
expect(err.alarmChannel).toBe('ai_providers');
|
||||
expect(err.message).toBe('Model not available: fake');
|
||||
expect(JSON.stringify(attemptsFrom(err))).not.toMatch(
|
||||
/http|request id/i,
|
||||
|
||||
@@ -938,85 +938,6 @@ describe('ChatCompletionDriver.complete credit gate and max_tokens cap', () => {
|
||||
}
|
||||
});
|
||||
|
||||
describe('when the balance would cut the reply short', () => {
|
||||
// 8192 output tokens at 2000 µ¢ each: ~16.4M µ¢ for the full reply.
|
||||
const capme = async () => {
|
||||
vi.spyOn(
|
||||
FakeChatProvider.prototype,
|
||||
'models',
|
||||
).mockResolvedValueOnce([
|
||||
{
|
||||
id: 'capme',
|
||||
aliases: [],
|
||||
costs_currency: 'usd-cents',
|
||||
costs: { input_tokens: 1000, output_tokens: 2000 },
|
||||
max_tokens: 8192,
|
||||
},
|
||||
]);
|
||||
return makeDriver();
|
||||
};
|
||||
const completeCapme = async (d: ChatCompletionDriver) => {
|
||||
const completeSpy = vi
|
||||
.spyOn(FakeChatProvider.prototype, 'complete')
|
||||
.mockResolvedValueOnce({
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
},
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
finish_reason: 'stop',
|
||||
} as never);
|
||||
await withTestActor(() =>
|
||||
d.complete({
|
||||
model: 'capme',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
return (completeSpy.mock.calls[0]![0] as ICompleteArguments)
|
||||
.max_tokens;
|
||||
};
|
||||
|
||||
it('asks for a top-up and gives the reply its full output limit', async () => {
|
||||
const d = await capme();
|
||||
const headroom = vi
|
||||
.spyOn(server.services.metering, 'getUsageHeadroom')
|
||||
.mockResolvedValueOnce({ balance: 100_000, held: 0 })
|
||||
.mockResolvedValue({ balance: 100_000_000, held: 0 });
|
||||
const refill = vi.fn(
|
||||
(_key: string, refusal: { refilled: boolean }) => {
|
||||
refusal.refilled = true;
|
||||
},
|
||||
);
|
||||
server.clients.event.on('metering.out-of-credits', refill as never);
|
||||
try {
|
||||
expect(await completeCapme(d)).toBe(8192);
|
||||
expect(refill).toHaveBeenCalledTimes(1);
|
||||
expect(headroom).toHaveBeenCalledTimes(2);
|
||||
} finally {
|
||||
server.clients.event.off(
|
||||
'metering.out-of-credits',
|
||||
refill as never,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('asks no listener when the balance covers the full reply', async () => {
|
||||
const d = await capme();
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'getUsageHeadroom',
|
||||
).mockResolvedValue({ balance: 100_000_000, held: 0 });
|
||||
const refill = vi.fn();
|
||||
server.clients.event.on('metering.out-of-credits', refill);
|
||||
try {
|
||||
expect(await completeCapme(d)).toBe(8192);
|
||||
expect(refill).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
server.clients.event.off('metering.out-of-credits', refill);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
it('caps `max_tokens` so output cannot exceed remaining credits', async () => {
|
||||
// Inject a model with canonical `output_tokens` cost so the
|
||||
// `outputTokenCost > 0` branch triggers (fake-chat:costly uses
|
||||
|
||||
@@ -514,12 +514,7 @@ const classifyAttempts = (
|
||||
fields: { ...fields, upstreamStatus: 404 },
|
||||
// Our provider account lost access to the model; low alarm so
|
||||
// someone notices.
|
||||
...(accessDenied
|
||||
? {
|
||||
alarmSeverity: 'info' as const,
|
||||
alarmChannel: 'ai_providers',
|
||||
}
|
||||
: {}),
|
||||
...(accessDenied ? { alarmSeverity: 'info' as const } : {}),
|
||||
});
|
||||
}
|
||||
if (
|
||||
@@ -1788,8 +1783,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
*
|
||||
* Rejects when the account can't afford the approximate input cost, keeps
|
||||
* subscriber-only models gated, and tightens `args.max_tokens` so the
|
||||
* output this attempt can produce is bounded by the remaining balance. A
|
||||
* balance short of either is first offered to top-up listeners.
|
||||
* output this attempt can produce is bounded by the remaining balance.
|
||||
*
|
||||
* Returns a hold on what the attempt can cost at worst, so requests this
|
||||
* account is running in parallel see the spend before it is recorded. The
|
||||
@@ -1865,21 +1859,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
const extras = ((pricing.extraCost ?? 0) + requestFee) * costFactor;
|
||||
const minimumCredits = Number(model.minimumCredits || 1);
|
||||
const needed = Math.max(approximateInputCost + extras, minimumCredits);
|
||||
// What the request would get with money no object.
|
||||
const limit = Math.min(
|
||||
requestedMaxTokens ?? Number.POSITIVE_INFINITY,
|
||||
outputCeiling(model, promptTokenEstimate),
|
||||
positiveOrInfinity(pricing.maxTokens),
|
||||
);
|
||||
// The prompt plus output run to `limit`. A balance short of it would
|
||||
// cut the reply off, which the caller is told the same way as a refusal.
|
||||
const fullCost =
|
||||
outputTokenCost > 0 && Number.isFinite(limit)
|
||||
? Math.max(
|
||||
needed,
|
||||
approximateInputCost + extras + limit * outputTokenCost,
|
||||
)
|
||||
: needed;
|
||||
|
||||
// One read of the balance and its holds serves the whole gate: the
|
||||
// affordability check here and the output cap below, net of holds,
|
||||
@@ -1889,7 +1868,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
// Short of funds rather than of what running requests hold: a top-up
|
||||
// made at this moment is worth reading the balance once more.
|
||||
if (
|
||||
balance < fullCost &&
|
||||
balance < needed &&
|
||||
(await creditsRefilled(this.clients.event, actor))
|
||||
) {
|
||||
({ balance, held } = await metering.getUsageHeadroom(actor));
|
||||
@@ -1915,6 +1894,12 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
if (outputTokenCost > 0) {
|
||||
const affordableOutputTokens = (credits: number): number =>
|
||||
(credits - approximateInputCost - extras) / outputTokenCost;
|
||||
// What the request would get with money no object.
|
||||
const limit = Math.min(
|
||||
requestedMaxTokens ?? Number.POSITIVE_INFINITY,
|
||||
outputCeiling(model, promptTokenEstimate),
|
||||
positiveOrInfinity(pricing.maxTokens),
|
||||
);
|
||||
const cap = Math.floor(
|
||||
Math.min(limit, affordableOutputTokens(remainingCredits)),
|
||||
);
|
||||
|
||||
@@ -60,7 +60,7 @@ import { setupTestServer } from '../../../../testUtil.js';
|
||||
import { generateDefaultFsentries } from '../../../../util/userProvisioning.js';
|
||||
import type { OCRDriver } from './OCRDriver.js';
|
||||
import { OCR_COSTS } from './costs.js';
|
||||
import textractUsWest2 from '../../providers/aws/ocr/fixtures/textract-us-west-2.json';
|
||||
import textractUsWest2 from '../../providers/awsTextract/ocr/fixtures/textract-us-west-2.json';
|
||||
|
||||
// ── SDK mocks ───────────────────────────────────────────────────────
|
||||
//
|
||||
@@ -1153,7 +1153,7 @@ describe('OCRDriver.recognize (mistral)', () => {
|
||||
|
||||
await withActor(actor, () =>
|
||||
driver.recognize({
|
||||
source: dataUrl(buildPdf(1), 'application/pdf'),
|
||||
source: dataUrl(Buffer.from('x'), 'application/pdf'),
|
||||
provider: 'mistral',
|
||||
bboxAnnotationFormat: {
|
||||
type: 'json_schema',
|
||||
@@ -1361,78 +1361,26 @@ describe('OCRDriver credit pre-flight', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
describe('a document whose pages it cannot count', () => {
|
||||
// A PDF whose page tree is hidden in an unreadable stream.
|
||||
const unreadablePdf = Buffer.alloc(256 * 1024, 0xff);
|
||||
it('falls back to 20 pages per MB for documents it cannot count', async () => {
|
||||
const { actor } = await makeUser();
|
||||
mistralOcrProcessMock.mockResolvedValue({ pages: [] });
|
||||
const unreadablePdf = Buffer.alloc(1024 * 1024, 0xff);
|
||||
unreadablePdf.write('%PDF-1.7\n', 'latin1');
|
||||
const docx = dataUrl(
|
||||
Buffer.alloc(64 * 1024),
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
);
|
||||
const recognize = (actor: Actor, source: unknown, pages?: number[]) =>
|
||||
withActor(actor, () =>
|
||||
driver.recognize({
|
||||
source,
|
||||
provider: 'mistral',
|
||||
...(pages ? { pages } : {}),
|
||||
}),
|
||||
const docx =
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document';
|
||||
|
||||
for (const source of [
|
||||
pdfSource(unreadablePdf),
|
||||
dataUrl(Buffer.alloc(512 * 1024), docx),
|
||||
// Small unreadable input stays at one page.
|
||||
pdfSource(Buffer.from('x')),
|
||||
]) {
|
||||
await withActor(actor, () =>
|
||||
driver.recognize({ source, provider: 'mistral' }),
|
||||
);
|
||||
const sentPages = () =>
|
||||
mistralOcrProcessMock.mock.calls.map(([payload]) => payload.pages);
|
||||
const remainingPages = (pages: number) =>
|
||||
vi
|
||||
.spyOn(server.services.metering, 'getRemainingUsage')
|
||||
.mockResolvedValue(perPage * pages + perPage - 1);
|
||||
}
|
||||
|
||||
it('holds the page limit when the balance covers it', async () => {
|
||||
const { actor } = await makeUser();
|
||||
mistralOcrProcessMock.mockResolvedValue({ pages: [] });
|
||||
remainingPages(5000);
|
||||
admitSpy.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
|
||||
await recognize(actor, pdfSource(unreadablePdf));
|
||||
await recognize(actor, docx);
|
||||
|
||||
expect(checkedCosts()).toEqual([perPage * 1000, perPage * 1000]);
|
||||
expect(sentPages()).toEqual([undefined, undefined]);
|
||||
});
|
||||
|
||||
it('reads only as many leading pages as the balance covers', async () => {
|
||||
const { actor } = await makeUser();
|
||||
mistralOcrProcessMock.mockResolvedValue({ pages: [] });
|
||||
remainingPages(7);
|
||||
|
||||
await recognize(actor, pdfSource(unreadablePdf));
|
||||
await recognize(actor, docx);
|
||||
|
||||
expect(checkedCosts()).toEqual([perPage * 7, perPage * 7]);
|
||||
expect(sentPages()).toEqual([
|
||||
[0, 1, 2, 3, 4, 5, 6],
|
||||
[0, 1, 2, 3, 4, 5, 6],
|
||||
]);
|
||||
});
|
||||
|
||||
it('refuses when the balance does not cover one page', async () => {
|
||||
const { actor } = await makeUser();
|
||||
remainingPages(0);
|
||||
|
||||
await expect(
|
||||
recognize(actor, pdfSource(unreadablePdf)),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
expect(mistralOcrProcessMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('holds only the selected pages', async () => {
|
||||
const { actor } = await makeUser();
|
||||
mistralOcrProcessMock.mockResolvedValue({ pages: [] });
|
||||
|
||||
await recognize(actor, pdfSource(unreadablePdf), [0, 1, 2]);
|
||||
|
||||
expect(checkedCosts()).toEqual([perPage * 3]);
|
||||
expect(sentPages()).toEqual([[0, 1, 2]]);
|
||||
});
|
||||
expect(checkedCosts()).toEqual([perPage * 20, perPage * 10, perPage]);
|
||||
});
|
||||
|
||||
it('counts an image, and any Textract input, as one page', async () => {
|
||||
|
||||
@@ -37,7 +37,7 @@ import { readProviderKey } from '../../../../util/aiProviderKey.js';
|
||||
import {
|
||||
TEXTRACT_PAGE_USAGE_TYPE,
|
||||
TextractOcrProvider,
|
||||
} from '../../providers/aws/ocr/TextractOcrProvider.js';
|
||||
} from '../../providers/awsTextract/ocr/TextractOcrProvider.js';
|
||||
import { MistralOcrProvider } from '../../providers/mistral/ocr/MistralOcrProvider.js';
|
||||
import { badRequest } from './common.js';
|
||||
import { OCR_COSTS } from './costs.js';
|
||||
@@ -135,9 +135,11 @@ export class OCRDriver extends PuterDriver {
|
||||
const metering = this.#aiMetering;
|
||||
|
||||
const textract = providers['aws-textract'] as
|
||||
Record<string, unknown> | undefined;
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
const textractAws = (textract?.aws ?? textract) as
|
||||
Record<string, unknown> | undefined;
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
const textractCredentials = awsCredentialsFrom(textractAws);
|
||||
if (textractCredentials) {
|
||||
this.#textract = new TextractOcrProvider(metering, {
|
||||
|
||||
@@ -24,7 +24,11 @@ import type { CreditHold } from '../../../../services/metering/types.js';
|
||||
import { mimeFromName } from '../../../../util/fileSigning.js';
|
||||
import type { LoadedFile, ResolvedFileInput } from '../../../util/fileInput.js';
|
||||
import type { AiMeteringService } from '../../../../util/aiCostFactor.js';
|
||||
import { OCR_MAX_PAGES, type OcrProviderId } from './models.js';
|
||||
import {
|
||||
OCR_MAX_INPUT_BYTES,
|
||||
OCR_MAX_PAGES,
|
||||
type OcrProviderId,
|
||||
} from './models.js';
|
||||
import { countPdfPages } from './pdfPages.js';
|
||||
|
||||
export const badRequest = (message: string) =>
|
||||
@@ -50,20 +54,27 @@ export const isDocumentInput = (loaded: LoadedFile): boolean => {
|
||||
|
||||
/**
|
||||
* The most pages a call can bill, known before the provider runs: a PDF's page
|
||||
* count, capped by a `pages` selection and by what the provider reads in one
|
||||
* call. Null for a document whose pages can't be counted, such as a PDF with an
|
||||
* encrypted page tree, when no pages are selected. Reads the input only for a
|
||||
* provider that takes more than one page.
|
||||
* count, or a size-based guess for other documents, capped by a `pages`
|
||||
* selection and by what the provider reads in one call. Reads the input only
|
||||
* for a provider that takes more than one page.
|
||||
*/
|
||||
export const estimateOcrPages = async (
|
||||
input: ResolvedFileInput,
|
||||
provider: OcrProviderId,
|
||||
selection?: unknown,
|
||||
): Promise<number | null> => {
|
||||
): Promise<number> => {
|
||||
const maxPages = OCR_MAX_PAGES[provider];
|
||||
if (maxPages <= 1) return 1;
|
||||
const loaded = await input.load();
|
||||
if (!isDocumentInput(loaded)) return 1;
|
||||
// A document we can't count is billed in proportion to its size, reaching
|
||||
// the provider's page limit at its size limit (20 pages per MB for
|
||||
// Mistral). Small files stay cheap; a large one can't pass as one page.
|
||||
const documentPages =
|
||||
(await countPdfPages(loaded.buffer, maxPages)) ??
|
||||
Math.ceil(
|
||||
(loaded.buffer.length * maxPages) / OCR_MAX_INPUT_BYTES[provider],
|
||||
);
|
||||
const selected = Array.isArray(selection)
|
||||
? new Set(
|
||||
selection.filter(
|
||||
@@ -71,12 +82,7 @@ export const estimateOcrPages = async (
|
||||
),
|
||||
).size
|
||||
: 0;
|
||||
const counted = await countPdfPages(loaded.buffer, maxPages);
|
||||
if (counted === null && !selected) return null;
|
||||
return Math.max(
|
||||
1,
|
||||
Math.min(counted ?? maxPages, maxPages, selected || Infinity),
|
||||
);
|
||||
return Math.max(1, Math.min(documentPages, maxPages, selected || Infinity));
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -46,7 +46,7 @@ import { PuterServer } from '../../../../server.js';
|
||||
import { setupTestServer, withAiCostFactorHook } from '../../../../testUtil.js';
|
||||
import { readStreamResult } from '../../shared/integrationTestUtil.js';
|
||||
import type { TTSDriver } from './TTSDriver.js';
|
||||
import pollyPriceList from '../../providers/aws/textToSpeech/fixtures/us-west-2.json';
|
||||
import pollyPriceList from '../../providers/awsPolly/textToSpeech/fixtures/us-west-2.json';
|
||||
|
||||
// ── SDK mocks ──────────────────────────────────────────────────────
|
||||
//
|
||||
@@ -486,10 +486,6 @@ describe('TTSDriver Gemini registration with a custom apiBaseUrl', () => {
|
||||
) => {
|
||||
const booted = await setupTestServer({ providers } as never);
|
||||
try {
|
||||
await booted.services.metering.updateAddonCredit(
|
||||
SYSTEM_ACTOR.user!.uuid!,
|
||||
100_000_000,
|
||||
);
|
||||
const passthrough = fetchSpy.getMockImplementation()!;
|
||||
fetchSpy.mockImplementation((input, init) =>
|
||||
String(input).startsWith(GOOGLE)
|
||||
@@ -726,7 +722,6 @@ describe('TTSDriver cost factor', () => {
|
||||
driver.synthesize({
|
||||
text: 'hello there',
|
||||
provider: 'openai',
|
||||
model: 'tts-1',
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
@@ -32,7 +32,7 @@ import { usesGoogleGeminiEndpoint } from '../../providers/gemini/geminiRest.js';
|
||||
import { ProviderRegistry } from '../../shared/providerRegistry.js';
|
||||
import { readProviderKey } from '../../../../util/aiProviderKey.js';
|
||||
import { TTS_CATALOG, TTS_DRIVER_ALIASES } from './providerAliases.js';
|
||||
import { AWSPollyTTSProvider } from '../../providers/aws/textToSpeech/AWSPollyTTSProvider.js';
|
||||
import { AWSPollyTTSProvider } from '../../providers/awsPolly/textToSpeech/AWSPollyTTSProvider.js';
|
||||
import { ElevenLabsTTSProvider } from '../../providers/elevenlabs/textToSpeech/ElevenLabsTTSProvider.js';
|
||||
import { GeminiTTSProvider } from '../../providers/gemini/textToSpeech/GeminiTTSProvider.js';
|
||||
import { KlingTTSProvider } from '../../providers/kling/textToSpeech/KlingTTSProvider.js';
|
||||
@@ -258,7 +258,8 @@ export class TTSDriver extends PuterDriver {
|
||||
|
||||
const polly = providers['aws-polly'];
|
||||
const pollyAws = (polly?.aws ?? polly) as
|
||||
Record<string, unknown> | undefined;
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
const pollyCredentials = awsCredentialsFrom(pollyAws);
|
||||
if (pollyCredentials) {
|
||||
register(
|
||||
@@ -267,7 +268,8 @@ export class TTSDriver extends PuterDriver {
|
||||
new AWSPollyTTSProvider(m, {
|
||||
credentials: pollyCredentials,
|
||||
region: (pollyAws?.region ?? polly?.region) as
|
||||
string | undefined,
|
||||
| string
|
||||
| undefined,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -303,7 +305,8 @@ export class TTSDriver extends PuterDriver {
|
||||
new KlingTTSProvider(m, {
|
||||
apiKey,
|
||||
apiBaseUrl: providers['kling']?.apiBaseUrl as
|
||||
string | undefined,
|
||||
| string
|
||||
| undefined,
|
||||
}),
|
||||
providers['kling'],
|
||||
),
|
||||
|
||||
@@ -294,10 +294,6 @@ describe('clampReasoningEffort', () => {
|
||||
['gpt-5.5', 'max', 'xhigh'],
|
||||
['gpt-5', 'high', 'high'],
|
||||
['gpt-6-astra-1', 'none', 'low'],
|
||||
...(['gpt-6.2-astra', 'gpt-6.2-sol'] as const).flatMap((id) => [
|
||||
[id, 'none', 'low'] as const,
|
||||
[id, 'max', 'max'] as const,
|
||||
]),
|
||||
...(['none', 'low', 'medium', 'high', 'xhigh', 'max'] as const).map(
|
||||
(effort) => ['gpt-6.1-terra', effort, effort] as const,
|
||||
),
|
||||
|
||||
@@ -227,7 +227,7 @@ const REASONING_EFFORT_TABLE: {
|
||||
allowed: readonly ReasoningEffort[];
|
||||
}[] = [
|
||||
{
|
||||
pattern: /^gpt-6-astra|^gpt-6\.1-sol|^gpt-6\.2-(astra|sol)/,
|
||||
pattern: /^gpt-6-astra|^gpt-6\.1-sol/,
|
||||
allowed: ['low', 'medium', 'high', 'xhigh', 'max'],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -195,7 +195,6 @@ describe('AlibabaProvider model catalog', () => {
|
||||
['qwen3.8-flash', 15, 47, 1.6],
|
||||
['qwen3.8-omni-flash', 15, 47, 1.6],
|
||||
['qwen3.8-max-0902', 200, 600, 25],
|
||||
['qwen4-max', 150, 450, 7.5],
|
||||
// No cache discount: cached reads bill at the input rate.
|
||||
['qwen3.7-max-2026-05-17', 250, 750, undefined],
|
||||
['deepseek-v4.1-flash', 30, 120, undefined],
|
||||
@@ -576,57 +575,44 @@ describe('AlibabaProvider.complete non-stream output', () => {
|
||||
expect(overrides).toMatchObject({ cached_tokens: 0 });
|
||||
});
|
||||
|
||||
it.each(['qwen3.8-max', 'qwen4-max'])(
|
||||
'accounts for cached tokens in %s completions',
|
||||
async (modelId) => {
|
||||
axiosGetMock.mockResolvedValueOnce({
|
||||
data: {
|
||||
success: true,
|
||||
output: {
|
||||
total: 1,
|
||||
page_no: 1,
|
||||
models: [{ model: modelId, capabilities: ['TG'] }],
|
||||
},
|
||||
it('accounts for cached_tokens when prompt_tokens_details is present', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
});
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 50,
|
||||
completion_tokens: 20,
|
||||
prompt_tokens_details: { cached_tokens: 15 },
|
||||
},
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: modelId,
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const model = ALIBABA_MODELS.find((m) => m.id === modelId)!;
|
||||
const [usage, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
// Cached reads come out of `prompt_tokens`; they're priced once.
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 35,
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 50,
|
||||
completion_tokens: 20,
|
||||
cached_tokens: 15,
|
||||
});
|
||||
expect(prefix).toBe(`alibaba:${modelId}`);
|
||||
expect(overrides).toEqual({
|
||||
prompt_tokens: 35 * Number(model.costs.prompt_tokens),
|
||||
completion_tokens: 20 * Number(model.costs.completion_tokens),
|
||||
cached_tokens: 15 * Number(model.costs.cached_tokens ?? 0),
|
||||
});
|
||||
},
|
||||
);
|
||||
prompt_tokens_details: { cached_tokens: 15 },
|
||||
},
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'qwen3.8-max',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
const model = ALIBABA_MODELS.find((m) => m.id === 'qwen3.8-max')!;
|
||||
const [usage, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
// Cached reads come out of `prompt_tokens`; they're priced once.
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 35,
|
||||
completion_tokens: 20,
|
||||
cached_tokens: 15,
|
||||
});
|
||||
expect(prefix).toBe('alibaba:qwen3.8-max');
|
||||
expect(overrides).toEqual({
|
||||
prompt_tokens: 35 * Number(model.costs.prompt_tokens),
|
||||
completion_tokens: 20 * Number(model.costs.completion_tokens),
|
||||
cached_tokens: 15 * Number(model.costs.cached_tokens ?? 0),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ── Streaming deltas ────────────────────────────────────────────────
|
||||
@@ -960,50 +946,6 @@ describe('Alibaba official catalog', () => {
|
||||
expect(model).toBeNull();
|
||||
});
|
||||
|
||||
it.each([
|
||||
['qwen3.6-plus-2026-04-02', 'qwen3.6-plus'],
|
||||
['qwen-plus-latest', 'qwen-plus-2025-12-01'],
|
||||
])(
|
||||
'lists %s once, as the priced %s, and keeps its alibaba: name routable',
|
||||
async (alias, curatedId) => {
|
||||
axiosGetMock.mockReset();
|
||||
axiosGetMock.mockResolvedValueOnce({
|
||||
data: {
|
||||
success: true,
|
||||
output: {
|
||||
page_no: 1,
|
||||
total: 1,
|
||||
models: [
|
||||
{
|
||||
model: alias,
|
||||
capabilities: ['TG', 'Reasoning'],
|
||||
inference_metadata: {
|
||||
request_modality: ['Text'],
|
||||
response_modality: ['Text'],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
const { provider } = makeProvider();
|
||||
const models = await provider.models();
|
||||
expect(models.some((model) => model.id === alias)).toBe(false);
|
||||
const curated = ALIBABA_MODELS.find(
|
||||
(model) => model.id === curatedId,
|
||||
)!;
|
||||
const listed = models.filter(
|
||||
(model) => model.puterId === curated.puterId,
|
||||
);
|
||||
expect(listed).toHaveLength(1);
|
||||
expect(listed[0].costs).toBe(curated.costs);
|
||||
// The chat driver routes by aliases, so the alibaba: spelling must
|
||||
// stay one.
|
||||
expect(listed[0].aliases).toContain(`alibaba:qwen/${alias}`);
|
||||
expect(curated.aliases).not.toContain(`alibaba:qwen/${alias}`);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
'ASR',
|
||||
|
||||
@@ -85,11 +85,6 @@ const positiveLimit = (value: unknown): number | undefined =>
|
||||
const chatCapabilities = ['TG', 'Reasoning', 'VU'];
|
||||
|
||||
const curatedPuterIds = new Set(ALIBABA_MODELS.map((model) => model.puterId));
|
||||
const curatedAliasOwners = new Map(
|
||||
ALIBABA_MODELS.flatMap((model) =>
|
||||
(model.aliases ?? []).map((alias) => [alias, model.id] as const),
|
||||
),
|
||||
);
|
||||
|
||||
const inferPuterId = (id: string): string => {
|
||||
if (id.includes('/')) return `alibaba:${id}`;
|
||||
@@ -272,21 +267,11 @@ export const fetchAlibabaCatalog = async (
|
||||
for (const raw of entries) {
|
||||
if (typeof raw.model !== 'string' || !raw.model.trim())
|
||||
throw new Error('Invalid Alibaba model id');
|
||||
const id = raw.model.trim();
|
||||
seen.add(id);
|
||||
// A snapshot or -latest name that a curated model carries as an alias
|
||||
// names that model, so its alibaba: spelling becomes one more alias.
|
||||
const owner = models.get(curatedAliasOwners.get(id) ?? '');
|
||||
if (owner) {
|
||||
const name = inferPuterId(id);
|
||||
if (name !== owner.puterId && !owner.aliases?.includes(name))
|
||||
models.set(owner.id, {
|
||||
...owner,
|
||||
aliases: [...(owner.aliases ?? []), name],
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const model = mapAlibabaCatalogModel(raw, models.get(id));
|
||||
seen.add(raw.model.trim());
|
||||
const model = mapAlibabaCatalogModel(
|
||||
raw,
|
||||
models.get(raw.model.trim()),
|
||||
);
|
||||
if (model) models.set(model.id, model);
|
||||
}
|
||||
if (models.size > 20_000)
|
||||
|
||||
@@ -45,30 +45,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
},
|
||||
},
|
||||
|
||||
// International USD rates: https://www.alibabacloud.com/en/press-room/alibaba-unveils-qwen4-max
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen4-max',
|
||||
id: 'qwen4-max',
|
||||
name: 'Qwen4-Max',
|
||||
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-11-16',
|
||||
aliases: ['qwen/qwen4-max'],
|
||||
context: 1_000_000,
|
||||
max_tokens: 131_072,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen4-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 150,
|
||||
completion_tokens: 450,
|
||||
cached_tokens: 7.5,
|
||||
},
|
||||
},
|
||||
|
||||
// -- Plus tier --------------------------------------------------
|
||||
{
|
||||
puterId: 'alibaba:qwen/qwen-plus',
|
||||
|
||||
@@ -1,175 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { cachedRemoteCatalog } from '../../../shared/cachedRemoteCatalog.js';
|
||||
import {
|
||||
fetchAwsPriceList,
|
||||
isAwsPriceRegion,
|
||||
onDemandRates,
|
||||
type AwsPriceDimension,
|
||||
} from '../priceList.js';
|
||||
|
||||
/** A region's Textract prices are tens of KB; anything far past that is not one. */
|
||||
const TEXTRACT_PRICE_LIST_MAX_BYTES = 2 * 1024 * 1024;
|
||||
|
||||
/**
|
||||
* Most regions priced from the feed at once. Calls run in the configured region
|
||||
* and the regions of the deployment's buckets, so a handful is plenty; any
|
||||
* region past the cap is billed at the curated rate.
|
||||
*/
|
||||
export const TEXTRACT_PRICE_MAX_REGIONS = 8;
|
||||
|
||||
// DetectDocumentText only: `<prefix>-SyncTextPagesProcessed`. Async and
|
||||
// AnalyzeDocument variants, and prefixed promotional usage, don't match.
|
||||
const USAGE_TYPE_PATTERN = /^[A-Z0-9]+-SyncTextPagesProcessed$/;
|
||||
|
||||
const RANGE_PATTERN = /^\d+$/;
|
||||
|
||||
/** The one group this file prices: regular synchronous DetectDocumentText. */
|
||||
const SYNC_TEXT = 'sync-text';
|
||||
|
||||
const syncTextGroup = (
|
||||
product: Record<string, unknown>,
|
||||
attrs: Record<string, unknown>,
|
||||
): string | undefined =>
|
||||
product.productFamily === 'API Request' &&
|
||||
attrs.apiType === 'Sync' &&
|
||||
attrs.entityType === 'Text' &&
|
||||
typeof attrs.usagetype === 'string' &&
|
||||
USAGE_TYPE_PATTERN.test(attrs.usagetype) &&
|
||||
(attrs.platopagedatatype === undefined ||
|
||||
attrs.platopagedatatype === 'SyncTextPagesProcessed')
|
||||
? SYNC_TEXT
|
||||
: undefined;
|
||||
|
||||
/**
|
||||
* A term's first volume tier. Later tiers depend on the account's monthly
|
||||
* volume, which a single call can't know, so they're never priced from.
|
||||
*/
|
||||
const firstTier = (
|
||||
dimensions: AwsPriceDimension[],
|
||||
): AwsPriceDimension | undefined => {
|
||||
const firstTiers = dimensions.filter((d) => d.beginRange === '0');
|
||||
if (firstTiers.length !== 1) return undefined;
|
||||
const tier = firstTiers[0]!;
|
||||
if (
|
||||
tier.endRange !== 'Inf' &&
|
||||
!(
|
||||
typeof tier.endRange === 'string' &&
|
||||
RANGE_PATTERN.test(tier.endRange) &&
|
||||
Number(tier.endRange) > 0
|
||||
)
|
||||
)
|
||||
return undefined;
|
||||
return tier;
|
||||
};
|
||||
|
||||
/**
|
||||
* Microcents per page for DetectDocumentText in one region's AWS price list:
|
||||
* the first volume tier of the single in-effect on-demand USD term on the
|
||||
* single regular synchronous text SKU. Undefined when that rate is missing,
|
||||
* malformed, or ambiguous. Throws when the document isn't a Textract price
|
||||
* list.
|
||||
*/
|
||||
export const parseTextractPriceList = (
|
||||
doc: unknown,
|
||||
region: string,
|
||||
now: number = Date.now(),
|
||||
): number | undefined =>
|
||||
onDemandRates(doc, {
|
||||
service: 'Textract',
|
||||
region,
|
||||
unit: 'Pages',
|
||||
now,
|
||||
group: syncTextGroup,
|
||||
dimension: firstTier,
|
||||
}).get(SYNC_TEXT);
|
||||
|
||||
/**
|
||||
* Fetches one region's public Textract price list and reads its rate. Null when
|
||||
* AWS publishes no list for the region, as for every region without Textract.
|
||||
*/
|
||||
export const fetchTextractPriceList = async (
|
||||
region: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<number | null> => {
|
||||
const doc = await fetchAwsPriceList(
|
||||
'Textract',
|
||||
region,
|
||||
TEXTRACT_PRICE_LIST_MAX_BYTES,
|
||||
signal,
|
||||
);
|
||||
if (doc === null) return null;
|
||||
const rate = parseTextractPriceList(doc, region);
|
||||
if (rate === undefined) {
|
||||
throw new Error(
|
||||
`AWS Textract price list has no usable DetectDocumentText rate for ${region}`,
|
||||
);
|
||||
}
|
||||
return rate;
|
||||
};
|
||||
|
||||
export interface TextractRates {
|
||||
/**
|
||||
* Microcents per page in a region: its last good feed rate, else the
|
||||
* curated one. Null when AWS publishes no price list for the region, which
|
||||
* means it doesn't run Textract there.
|
||||
*/
|
||||
rate(region: string): Promise<number | null>;
|
||||
/** The rate {@link rate} last settled on for a region, without fetching. */
|
||||
peek(region: string): number;
|
||||
}
|
||||
|
||||
/**
|
||||
* DetectDocumentText rates per region, each region's price list cached and
|
||||
* refreshed on its own. A region whose feed fails or omits the rate keeps its
|
||||
* last good value, or `fallback` until it has one.
|
||||
*/
|
||||
export const textractRates = (
|
||||
fallback: number,
|
||||
maxRegions: number = TEXTRACT_PRICE_MAX_REGIONS,
|
||||
): TextractRates => {
|
||||
const catalogs = new Map<string, () => Promise<number | null>>();
|
||||
const latest = new Map<string, number | null>();
|
||||
|
||||
const catalogFor = (region: string) => {
|
||||
const existing = catalogs.get(region);
|
||||
if (existing) return existing;
|
||||
if (!isAwsPriceRegion(region) || catalogs.size >= maxRegions)
|
||||
return undefined;
|
||||
const catalog = cachedRemoteCatalog<number | null>({
|
||||
name: `AWS Textract ${region} price list`,
|
||||
fallback,
|
||||
fetch: (signal) => fetchTextractPriceList(region, signal),
|
||||
});
|
||||
catalogs.set(region, catalog);
|
||||
return catalog;
|
||||
};
|
||||
|
||||
return {
|
||||
async rate(region) {
|
||||
const catalog = catalogFor(region);
|
||||
if (!catalog) return fallback;
|
||||
const rate = await catalog();
|
||||
latest.set(region, rate);
|
||||
return rate;
|
||||
},
|
||||
peek: (region) => latest.get(region) ?? fallback,
|
||||
};
|
||||
};
|
||||
@@ -1,157 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import pollyUsWest2 from './textToSpeech/fixtures/us-west-2.json';
|
||||
import textractUsWest2 from './ocr/fixtures/textract-us-west-2.json';
|
||||
import {
|
||||
fetchAwsPriceList,
|
||||
isAwsPriceRegion,
|
||||
onDemandRates,
|
||||
type OnDemandRateQuery,
|
||||
} from './priceList.js';
|
||||
|
||||
// The fixtures' terms take effect on 2026-05-01.
|
||||
const NOW = Date.parse('2026-10-13T00:00:00Z');
|
||||
|
||||
// Groups every product by its usage type and bills from a single flat tier.
|
||||
const byUsageType: OnDemandRateQuery = {
|
||||
service: 'Polly',
|
||||
region: 'us-west-2',
|
||||
unit: 'Characters',
|
||||
now: NOW,
|
||||
group: (_product, attrs) =>
|
||||
typeof attrs.usagetype === 'string' ? attrs.usagetype : undefined,
|
||||
dimension: (dimensions) =>
|
||||
dimensions.length === 1 ? dimensions[0] : undefined,
|
||||
};
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
describe('isAwsPriceRegion', () => {
|
||||
it('accepts plain AWS region codes only', () => {
|
||||
for (const region of ['us-west-2', 'ap-southeast-5', 'us-gov-west-1']) {
|
||||
expect(isAwsPriceRegion(region)).toBe(true);
|
||||
}
|
||||
for (const region of ['', 'US-WEST-2', '../us-west-2', 'us-west-2/x']) {
|
||||
expect(isAwsPriceRegion(region)).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('onDemandRates', () => {
|
||||
it('prices each group from its one in-effect term', () => {
|
||||
const rates = onDemandRates(pollyUsWest2, byUsageType);
|
||||
expect(rates.get('USW2-SynthesizeSpeech-Characters')).toBe(400);
|
||||
expect(rates.get('USW2-SynthesizeSpeechNeural-Characters')).toBe(1600);
|
||||
});
|
||||
|
||||
it('leaves a group unpriced when two SKUs price it', () => {
|
||||
const rates = onDemandRates(pollyUsWest2, {
|
||||
...byUsageType,
|
||||
group: () => 'all',
|
||||
});
|
||||
expect(rates.has('all')).toBe(true);
|
||||
expect(rates.get('all')).toBe(undefined);
|
||||
});
|
||||
|
||||
it('only reads products in the service and region asked for', () => {
|
||||
expect(
|
||||
onDemandRates(pollyUsWest2, { ...byUsageType, region: 'us-east-1' })
|
||||
.size,
|
||||
).toBe(0);
|
||||
});
|
||||
|
||||
it('only reads dimensions priced in the unit asked for', () => {
|
||||
const rates = onDemandRates(pollyUsWest2, {
|
||||
...byUsageType,
|
||||
unit: 'Pages',
|
||||
});
|
||||
expect([...rates.values()].every((rate) => rate === undefined)).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("throws on another service's price list", () => {
|
||||
expect(() => onDemandRates(textractUsWest2, byUsageType)).toThrow(
|
||||
'Not an AWS Polly price list',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('fetchAwsPriceList', () => {
|
||||
const fetchList = (region = 'us-west-2', maxBytes = 1024 * 1024) =>
|
||||
fetchAwsPriceList(
|
||||
'Textract',
|
||||
region,
|
||||
maxBytes,
|
||||
new AbortController().signal,
|
||||
);
|
||||
|
||||
it("reads the service's region price list without following redirects", async () => {
|
||||
const fetchMock = vi.fn(async () => Response.json({ ok: true }));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
await expect(fetchList()).resolves.toEqual({ ok: true });
|
||||
expect(fetchMock).toHaveBeenCalledWith(
|
||||
'https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonTextract/current/us-west-2/index.json',
|
||||
expect.objectContaining({ redirect: 'error' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('never requests an invalid region', async () => {
|
||||
const fetchMock = vi.fn();
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
await expect(fetchList('../../evil')).rejects.toThrow('Invalid');
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('reads a missing price list as null', async () => {
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
vi.fn(async () => new Response('NoSuchKey', { status: 404 })),
|
||||
);
|
||||
await expect(fetchList()).resolves.toBe(null);
|
||||
});
|
||||
|
||||
it('rejects an error status', async () => {
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
vi.fn(async () => new Response('denied', { status: 403 })),
|
||||
);
|
||||
await expect(fetchList()).rejects.toThrow('403');
|
||||
});
|
||||
|
||||
it('rejects a body over the size cap', async () => {
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
vi.fn(async () => new Response(' '.repeat(65))),
|
||||
);
|
||||
await expect(fetchList('us-west-2', 64)).rejects.toThrow('size cap');
|
||||
});
|
||||
|
||||
it('rejects a body that is not JSON', async () => {
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
vi.fn(async () => new Response('<html>')),
|
||||
);
|
||||
await expect(fetchList()).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
@@ -1,230 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { readBoundedBody } from '../../../util/fileInput.js';
|
||||
import { usdToUcents } from '../../shared/catalogPrice.js';
|
||||
|
||||
/**
|
||||
* An AWS service whose public price list we bill from; its offer code is
|
||||
* `Amazon<service>`.
|
||||
*/
|
||||
export type AwsPriceListService = 'Polly' | 'Textract';
|
||||
|
||||
const PRICE_LIST_BASE_URL =
|
||||
'https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws';
|
||||
|
||||
// The region lands in the price list URL, so only plain AWS region codes pass.
|
||||
const REGION_PATTERN = /^[a-z]{2}(-gov)?-[a-z]+-\d{1,2}$/;
|
||||
|
||||
export const isAwsPriceRegion = (region: string): boolean =>
|
||||
REGION_PATTERN.test(region);
|
||||
|
||||
const isRecord = (value: unknown): value is Record<string, unknown> =>
|
||||
typeof value === 'object' && value !== null && !Array.isArray(value);
|
||||
|
||||
/** One price dimension of an on-demand term, checked down to its USD price. */
|
||||
export interface AwsPriceDimension {
|
||||
beginRange: unknown;
|
||||
endRange: unknown;
|
||||
usd: unknown;
|
||||
}
|
||||
|
||||
export interface OnDemandRateQuery {
|
||||
service: AwsPriceListService;
|
||||
region: string;
|
||||
/** The unit every dimension must be priced in, e.g. `Characters`. */
|
||||
unit: string;
|
||||
now: number;
|
||||
/** The group a product's rate belongs to, or undefined to skip the product. */
|
||||
group(
|
||||
product: Record<string, unknown>,
|
||||
attributes: Record<string, unknown>,
|
||||
): string | undefined;
|
||||
/**
|
||||
* The dimension a term bills from, or undefined when its tiers can't be
|
||||
* billed from.
|
||||
*/
|
||||
dimension(dimensions: AwsPriceDimension[]): AwsPriceDimension | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* The dimensions of one on-demand term, `null` when the term isn't in effect
|
||||
* yet, or undefined when its shape isn't one we can bill from.
|
||||
*/
|
||||
const termDimensions = (
|
||||
sku: string,
|
||||
termKey: string,
|
||||
term: unknown,
|
||||
unit: string,
|
||||
now: number,
|
||||
): AwsPriceDimension[] | null | undefined => {
|
||||
if (!isRecord(term) || term.sku !== sku) return undefined;
|
||||
if (
|
||||
typeof term.offerTermCode !== 'string' ||
|
||||
termKey !== `${sku}.${term.offerTermCode}`
|
||||
)
|
||||
return undefined;
|
||||
if (
|
||||
term.termAttributes !== undefined &&
|
||||
(!isRecord(term.termAttributes) ||
|
||||
Object.keys(term.termAttributes).length > 0)
|
||||
)
|
||||
return undefined;
|
||||
const effective =
|
||||
typeof term.effectiveDate === 'string'
|
||||
? Date.parse(term.effectiveDate)
|
||||
: NaN;
|
||||
if (!Number.isFinite(effective)) return undefined;
|
||||
if (effective > now) return null;
|
||||
|
||||
if (!isRecord(term.priceDimensions)) return undefined;
|
||||
const dimensions: AwsPriceDimension[] = [];
|
||||
for (const [dimensionKey, dimension] of Object.entries(
|
||||
term.priceDimensions,
|
||||
)) {
|
||||
if (!isRecord(dimension) || dimension.rateCode !== dimensionKey)
|
||||
return undefined;
|
||||
if (!dimensionKey.startsWith(`${termKey}.`)) return undefined;
|
||||
if (dimension.unit !== unit) return undefined;
|
||||
if (
|
||||
dimension.appliesTo !== undefined &&
|
||||
!(
|
||||
Array.isArray(dimension.appliesTo) &&
|
||||
dimension.appliesTo.length === 0
|
||||
)
|
||||
)
|
||||
return undefined;
|
||||
const price = dimension.pricePerUnit;
|
||||
if (!isRecord(price) || Object.keys(price).join() !== 'USD')
|
||||
return undefined;
|
||||
dimensions.push({
|
||||
beginRange: dimension.beginRange,
|
||||
endRange: dimension.endRange,
|
||||
usd: price.USD,
|
||||
});
|
||||
}
|
||||
return dimensions;
|
||||
};
|
||||
|
||||
/** Microcents for a positive USD price, undefined for anything else. */
|
||||
const positiveUcents = (usd: unknown): number | undefined => {
|
||||
if (typeof usd !== 'string') return undefined;
|
||||
const ucents = usdToUcents(usd);
|
||||
return ucents !== undefined && ucents > 0 ? ucents : undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
* Microcents per unit for each group of one region's on-demand products. A
|
||||
* group's rate is undefined unless exactly one well-formed, in-effect term
|
||||
* prices it. Throws when the document isn't the service's price list.
|
||||
*/
|
||||
export const onDemandRates = (
|
||||
doc: unknown,
|
||||
query: OnDemandRateQuery,
|
||||
): Map<string, number | undefined> => {
|
||||
const { service, region, unit, now } = query;
|
||||
const offerCode = `Amazon${service}`;
|
||||
if (
|
||||
!isRecord(doc) ||
|
||||
doc.offerCode !== offerCode ||
|
||||
doc.formatVersion !== 'v1.0' ||
|
||||
!isRecord(doc.products) ||
|
||||
!isRecord(doc.terms) ||
|
||||
!isRecord(doc.terms.OnDemand)
|
||||
) {
|
||||
throw new Error(`Not an AWS ${service} price list`);
|
||||
}
|
||||
const onDemand = doc.terms.OnDemand;
|
||||
|
||||
const candidates = new Map<string, Array<number | undefined>>();
|
||||
for (const [sku, product] of Object.entries(doc.products)) {
|
||||
if (!isRecord(product) || product.sku !== sku) continue;
|
||||
const attrs = product.attributes;
|
||||
if (!isRecord(attrs)) continue;
|
||||
if (attrs.servicecode !== offerCode || attrs.regionCode !== region)
|
||||
continue;
|
||||
if ('electronStatus' in attrs) continue;
|
||||
const group = query.group(product, attrs);
|
||||
if (group === undefined) continue;
|
||||
|
||||
const terms = onDemand[sku];
|
||||
const rates: Array<number | undefined> = [];
|
||||
if (isRecord(terms)) {
|
||||
for (const [termKey, term] of Object.entries(terms)) {
|
||||
const dimensions = termDimensions(
|
||||
sku,
|
||||
termKey,
|
||||
term,
|
||||
unit,
|
||||
now,
|
||||
);
|
||||
if (dimensions === null) continue;
|
||||
const dimension = dimensions && query.dimension(dimensions);
|
||||
rates.push(dimension && positiveUcents(dimension.usd));
|
||||
}
|
||||
}
|
||||
// A matching SKU with no usable term poisons its group rather than
|
||||
// letting a sibling SKU's rate stand alone.
|
||||
let found = candidates.get(group);
|
||||
if (!found) candidates.set(group, (found = []));
|
||||
found.push(...(rates.length ? rates : [undefined]));
|
||||
}
|
||||
|
||||
const rates = new Map<string, number | undefined>();
|
||||
for (const [group, found] of candidates) {
|
||||
rates.set(group, found.length === 1 ? found[0] : undefined);
|
||||
}
|
||||
return rates;
|
||||
};
|
||||
|
||||
/**
|
||||
* Fetches one region's public price list for a service as parsed JSON. Null
|
||||
* when AWS publishes no list for the region.
|
||||
*/
|
||||
export const fetchAwsPriceList = async (
|
||||
service: AwsPriceListService,
|
||||
region: string,
|
||||
maxBytes: number,
|
||||
signal: AbortSignal,
|
||||
): Promise<unknown> => {
|
||||
if (!isAwsPriceRegion(region)) {
|
||||
throw new Error(`Invalid AWS region for the ${service} price list`);
|
||||
}
|
||||
const response = await fetch(
|
||||
`${PRICE_LIST_BASE_URL}/Amazon${service}/current/${region}/index.json`,
|
||||
{
|
||||
signal,
|
||||
redirect: 'error',
|
||||
headers: { accept: 'application/json' },
|
||||
},
|
||||
);
|
||||
if (!response.ok) {
|
||||
await response.body?.cancel();
|
||||
if (response.status === 404) return null;
|
||||
throw new Error(
|
||||
`AWS ${service} price list returned ${response.status}`,
|
||||
);
|
||||
}
|
||||
const body = await readBoundedBody(
|
||||
response,
|
||||
maxBytes,
|
||||
() => new Error(`AWS ${service} price list exceeds the size cap`),
|
||||
);
|
||||
return JSON.parse(body.toString('utf8'));
|
||||
};
|
||||
@@ -1,119 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { fetchAwsPriceList, onDemandRates } from '../priceList.js';
|
||||
|
||||
/** A region's Polly prices are a few KB; anything far past that is not one. */
|
||||
const POLLY_PRICE_LIST_MAX_BYTES = 1024 * 1024;
|
||||
|
||||
// Regular synthesis only: `<prefix>-SynthesizeSpeech[Engine]-Characters`.
|
||||
// Promotional variants (`USW2-ai-SynthesizeSpeech…`) don't match.
|
||||
const USAGE_TYPE_PATTERN =
|
||||
/^[A-Z0-9]+-SynthesizeSpeech(Neural|LongForm|Generative)?-Characters$/;
|
||||
|
||||
const ENGINE_BY_USAGE_SUFFIX: Record<string, string> = {
|
||||
'': 'standard',
|
||||
Neural: 'neural',
|
||||
LongForm: 'long-form',
|
||||
Generative: 'generative',
|
||||
};
|
||||
|
||||
const ENGINE_BY_ATTRIBUTE: Record<string, string> = {
|
||||
Standard: 'standard',
|
||||
Neural: 'neural',
|
||||
'Long-Form': 'long-form',
|
||||
Generative: 'generative',
|
||||
};
|
||||
|
||||
const PRODUCT_FAMILIES = new Set(['AmazonPolly', 'Synthesize Speech']);
|
||||
|
||||
/** Microcents per character, keyed by Polly engine id. */
|
||||
export type PollyRates = Record<string, number>;
|
||||
|
||||
/** The engine a product prices, or undefined when it isn't regular synthesis. */
|
||||
const productEngine = (
|
||||
product: Record<string, unknown>,
|
||||
attrs: Record<string, unknown>,
|
||||
): string | undefined => {
|
||||
if (
|
||||
typeof product.productFamily !== 'string' ||
|
||||
!PRODUCT_FAMILIES.has(product.productFamily)
|
||||
)
|
||||
return undefined;
|
||||
if (typeof attrs.usagetype !== 'string') return undefined;
|
||||
const usage = USAGE_TYPE_PATTERN.exec(attrs.usagetype);
|
||||
if (!usage) return undefined;
|
||||
const engine = ENGINE_BY_USAGE_SUFFIX[usage[1] ?? ''];
|
||||
if (
|
||||
typeof attrs.engine !== 'string' ||
|
||||
ENGINE_BY_ATTRIBUTE[attrs.engine] !== engine
|
||||
)
|
||||
return undefined;
|
||||
return engine;
|
||||
};
|
||||
|
||||
/**
|
||||
* Per-engine rates from one region's AWS price list. An engine appears only
|
||||
* when exactly one well-formed, in-effect, flat USD-per-character rate prices
|
||||
* it; anything missing, malformed, or ambiguous is left out for the caller to
|
||||
* fill from its curated rates. Throws when the document isn't a Polly price
|
||||
* list.
|
||||
*/
|
||||
export const parsePollyPriceList = (
|
||||
doc: unknown,
|
||||
region: string,
|
||||
now: number = Date.now(),
|
||||
): PollyRates => {
|
||||
const found = onDemandRates(doc, {
|
||||
service: 'Polly',
|
||||
region,
|
||||
unit: 'Characters',
|
||||
now,
|
||||
group: productEngine,
|
||||
// A flat rate: one tier covering every character.
|
||||
dimension: (dimensions) =>
|
||||
dimensions.length === 1 &&
|
||||
dimensions[0]!.beginRange === '0' &&
|
||||
dimensions[0]!.endRange === 'Inf'
|
||||
? dimensions[0]
|
||||
: undefined,
|
||||
});
|
||||
const rates: PollyRates = {};
|
||||
for (const [engine, rate] of found) {
|
||||
if (rate !== undefined) rates[engine] = rate;
|
||||
}
|
||||
return rates;
|
||||
};
|
||||
|
||||
/** Fetches and parses one region's public Polly price list. */
|
||||
export const fetchPollyPriceList = async (
|
||||
region: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<PollyRates> => {
|
||||
const doc = await fetchAwsPriceList(
|
||||
'Polly',
|
||||
region,
|
||||
POLLY_PRICE_LIST_MAX_BYTES,
|
||||
signal,
|
||||
);
|
||||
if (doc === null) {
|
||||
throw new Error(`AWS publishes no Polly price list for ${region}`);
|
||||
}
|
||||
return parsePollyPriceList(doc, region);
|
||||
};
|
||||
Loaded 100 of 445 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user