mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Nariman Jelveh
|
a290dd446f |
No files matched your search
+2
-2
@@ -2,9 +2,9 @@
|
||||
# docker-compose.full.yml up -d`. None of the defaults below are safe for
|
||||
# anything beyond a local laptop test.
|
||||
|
||||
# ── Public-facing ports (Caddy) ---------------------------------------
|
||||
# ── Public-facing ports (nginx) ---------------------------------------
|
||||
HTTP_PORT=80
|
||||
# HTTPS_PORT=443 # uncomment after you enable TLS in caddy/Caddyfile
|
||||
# HTTPS_PORT=443 # uncomment after you enable TLS in nginx/nginx.conf
|
||||
|
||||
# ── MariaDB ------------------------------------------------------------
|
||||
MARIADB_ROOT_PASSWORD=replace-with-strong-password
|
||||
|
||||
@@ -87,8 +87,6 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
context: .
|
||||
push: true
|
||||
provenance: mode=max
|
||||
sbom: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
|
||||
@@ -10,7 +10,6 @@ Use these as the source of truth before exploring further:
|
||||
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), and extensions.
|
||||
- [doc/contributing-apis.md](doc/contributing-apis.md) — adding and maintaining public APIs end to end (backend surface → puter.js → types → docs → tests). Follow it for any API work.
|
||||
- [doc/pagination.md](doc/pagination.md) — the one pagination convention for list APIs (limit/cursor/offset/includeTotal, envelope shape, cursor semantics).
|
||||
- [doc/alarms.md](doc/alarms.md) — raising alarms and picking a severity (what pages, what only gets recorded), plus the config that routes them.
|
||||
- [doc/self-hosting.md](doc/self-hosting.md) — running Puter outside hosted infra.
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) — testing, security, AI-assisted code, PR conventions, Boy Scout Rule.
|
||||
- [SECURITY.md](SECURITY.md) — how to report vulnerabilities (do not file them publicly).
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
# Reverse proxy in front of Puter — mirrors what the prod ALB does:
|
||||
# accepts every Host header, forwards to the Puter container, and lets
|
||||
# the Puter app handle subdomain-based routing internally (api.*,
|
||||
# site.*, app.*, dev.*, plus the per-user subdomains under those).
|
||||
#
|
||||
# To enable TLS:
|
||||
# 1. Drop a wildcard fullchain.pem + privkey.pem into ./puter/tls/
|
||||
# (see "Step 3 — TLS" in doc/self-hosting.md).
|
||||
# 2. Uncomment the `:443` block at the bottom of this file, and swap the
|
||||
# `:80` block for the redirect shown alongside it.
|
||||
# 3. Uncomment the `443:443` port mapping under the `caddy` service in
|
||||
# docker-compose.yml, and HTTPS_PORT in .env.
|
||||
# 4. Set `"protocol": "https"` and `"pub_port": 443` in config.json.
|
||||
{
|
||||
# Certs are supplied by the operator, not issued by Caddy. Puter serves
|
||||
# per-user sites and apps on dynamic subdomains (<name>.site.<domain>,
|
||||
# <name>.app.<domain>) — only a DNS-01 wildcard cert covers those, and
|
||||
# DNS-01 needs a provider plugin that isn't in the stock caddy image.
|
||||
# `off` stops Caddy attempting ACME on boot (it would fail and leave the
|
||||
# site unreachable) and stops it inventing its own http→https redirects.
|
||||
auto_https off
|
||||
}
|
||||
|
||||
# Shared handling, imported by the HTTP and HTTPS site blocks below so the
|
||||
# two can't drift apart.
|
||||
(puter_routes) {
|
||||
# Rough size cap that mirrors prod ALB defaults; tune for your uploads.
|
||||
# Puter chunks large uploads, so 1 GiB per request is plenty.
|
||||
request_body {
|
||||
max_size 1024MiB
|
||||
}
|
||||
|
||||
# RustFS — see the `s3` service in docker-compose.yml. Browsers PUT/GET
|
||||
# here for presigned-URL uploads / downloads. Matched on the `s3.`
|
||||
# subdomain of whatever the install domain is, so signature verification
|
||||
# works (Caddy preserves the original Host end-to-end) and HTTPS stays
|
||||
# clean — no mixed content from a port-9000 host publish.
|
||||
@s3 header_regexp Host ^s3\.
|
||||
handle @s3 {
|
||||
reverse_proxy s3:9000
|
||||
}
|
||||
|
||||
# Everything else, on every Host, goes to Puter — which routes on that
|
||||
# Host internally. Caddy forwards it unchanged and adds X-Forwarded-For
|
||||
# / -Proto / -Host, which is what `trust_proxy` in config.json counts.
|
||||
handle {
|
||||
reverse_proxy puter:4100 {
|
||||
# Stream responses through unbuffered — Puter uses SSE and
|
||||
# socket.io. WebSocket upgrades need no config of their own;
|
||||
# Caddy proxies them by default.
|
||||
flush_interval -1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ── HTTP (port 80) ─────────────────────────────────────────────────────
|
||||
# A site address with no hostname is the catch-all: it answers for every
|
||||
# Host, which is what Puter's subdomain routing requires.
|
||||
:80 {
|
||||
import puter_routes
|
||||
}
|
||||
|
||||
# ── HTTPS (port 443) — uncomment after dropping certs in ./puter/tls/ ──
|
||||
# Replace the `:80` block above with a redirect to force HTTPS everywhere:
|
||||
#
|
||||
# :80 {
|
||||
# redir https://{host}{uri} permanent
|
||||
# }
|
||||
#
|
||||
# :443 {
|
||||
# tls /etc/caddy/tls/fullchain.pem /etc/caddy/tls/privkey.pem
|
||||
# import puter_routes
|
||||
# }
|
||||
@@ -5,7 +5,9 @@
|
||||
"protocol": "http",
|
||||
"domain": "puter.localhost",
|
||||
"cookie_name": "puter_auth_token",
|
||||
"jwt_secret": "dev-jwt-secret-change-me",
|
||||
"jwt_secret_v2": "dev-jwt-secret-v2-change-me",
|
||||
"allow_v1_tokens": true,
|
||||
"url_signature_secret": "dev-url-signature-secret-change-me",
|
||||
"allow_all_host_values": true,
|
||||
"allow_no_host_header": true,
|
||||
|
||||
+4
-59
@@ -73,7 +73,11 @@
|
||||
|
||||
// ── Auth / session ──────────────────────────────────────────────────
|
||||
// ALWAYS replace these for any public install — `openssl rand -hex 64`.
|
||||
// `jwt_secret` is legacy verify-only; new tokens sign with `jwt_secret_v2`.
|
||||
"jwt_secret": "change-me",
|
||||
"jwt_secret_v2": "change-me",
|
||||
// Set false to retire v1 tokens entirely (ROLLOUT-1).
|
||||
"allow_v1_tokens": true,
|
||||
"url_signature_secret": "change-me",
|
||||
"cookie_name": "puter_auth_token",
|
||||
"min_pass_length": 6,
|
||||
@@ -251,60 +255,6 @@
|
||||
}
|
||||
},
|
||||
|
||||
// ── Alarms / alerting ───────────────────────────────────────────────
|
||||
// Where system alarms go. Severity is the routing decision — each
|
||||
// transport takes everything at or above its own `minSeverity`:
|
||||
//
|
||||
// critical — an unhandled server error; pages on-call.
|
||||
// error — pages as well; prefer critical or warning.
|
||||
// warning — look at it today; no page.
|
||||
// info — a record in the chat channel only.
|
||||
//
|
||||
// Both transports are off unless enabled, so a self-hosted node just
|
||||
// logs its alarms to the console.
|
||||
"pager": {
|
||||
// Severity for call sites that don't pick one. Default "critical".
|
||||
"defaultSeverity": "critical",
|
||||
|
||||
// Retier or silence an alarm without a deploy. Keys are alarm ids,
|
||||
// or a prefix ending in `*`; the exact id wins over a pattern, and
|
||||
// the longest matching pattern wins among patterns. Values are a
|
||||
// severity or "mute". This is applied last, so it overrides both the
|
||||
// call site and any known-error rule.
|
||||
"severityOverrides": {
|
||||
// "cronMonitor:*": "info",
|
||||
// "http_500:GET:/some/flapping/route:*": "mute"
|
||||
},
|
||||
|
||||
"pagerduty": {
|
||||
"enabled": false,
|
||||
"routingKey": "",
|
||||
// Lowest severity that reaches PagerDuty. Default "warning",
|
||||
// which keeps `info` out of the paging system entirely.
|
||||
"minSeverity": "warning"
|
||||
},
|
||||
|
||||
// Slack incoming webhook — the low-noise destination for everything
|
||||
// that shouldn't page.
|
||||
"slack": {
|
||||
"enabled": false,
|
||||
"webhookUrl": "",
|
||||
// Optional; defaults to the channel the webhook was created for.
|
||||
"channel": "#alerts",
|
||||
"username": "puter-alarms",
|
||||
// Severity window posted to Slack. The ceiling defaults to
|
||||
// "info" when PagerDuty is configured — what pages belongs in
|
||||
// the pager, not in chat — and to "critical" when Slack is the
|
||||
// only transport.
|
||||
"minSeverity": "info",
|
||||
"maxSeverity": "info",
|
||||
// Don't repost the same alarm id within this window. The first
|
||||
// occurrence always posts; the next post that gets through
|
||||
// reports how many occurrences piled up. 0 disables throttling.
|
||||
"repeatThrottleMs": 900000
|
||||
}
|
||||
},
|
||||
|
||||
// ── Rate limiting ───────────────────────────────────────────────────
|
||||
// `memory` for single-node, `redis` for multi-node (default), `kv` for
|
||||
// dynamo-backed counters.
|
||||
@@ -337,10 +287,6 @@
|
||||
"apiKey": "",
|
||||
"apiBaseUrl": "https://openrouter.ai/api/v1"
|
||||
},
|
||||
"infron": {
|
||||
"apiKey": "",
|
||||
"apiBaseUrl": "https://llm.onerouter.pro/v1"
|
||||
},
|
||||
"together-ai": { "apiKey": "" },
|
||||
// Local Ollama. `enabled: false` skips the auto-probe at startup
|
||||
// (otherwise Puter logs ECONNREFUSED on every boot when no Ollama
|
||||
@@ -379,7 +325,6 @@
|
||||
"secret_key": "",
|
||||
"region": "us-west-2"
|
||||
},
|
||||
"speechify": { "apiKey": "" },
|
||||
"aws-textract": {
|
||||
"access_key": "",
|
||||
"secret_key": "",
|
||||
|
||||
-113
@@ -1,113 +0,0 @@
|
||||
# Alarms
|
||||
|
||||
`clients.alarm` is the one way code reports that something is wrong. It
|
||||
de-dupes by alarm id, counts occurrences, and routes each alarm to alert
|
||||
transports by **severity**.
|
||||
|
||||
```ts
|
||||
this.clients.alarm.create(
|
||||
`driver_rate_limit_hit:${iface}:${method}`, // de-dupe key
|
||||
`Driver rate limit hit on ${iface}:${method}`, // what a human reads
|
||||
{ iface, method, userUuid }, // context fields
|
||||
'info', // severity
|
||||
);
|
||||
```
|
||||
|
||||
## Severity is the routing decision
|
||||
|
||||
| Severity | Meaning | Goes to |
|
||||
| ---------- | ------------------------------------------------------ | ----------- |
|
||||
| `critical` | An unhandled server error. Someone gets woken up. | Pager |
|
||||
| `error` | Same urgency as critical; prefer one of the other two. | Pager |
|
||||
| `warning` | Worth a look today. Nobody is paged. | Pager (low) |
|
||||
| `info` | A record of something expected-but-notable. | Chat |
|
||||
|
||||
Each transport declares the severity window it accepts, so the value a call
|
||||
site passes is what decides where the alarm lands. The two windows don't
|
||||
overlap by default: anything that pages lives in the paging system, and chat
|
||||
is the record of what didn't. The bar for `critical` is
|
||||
deliberately high: an unhandled 5xx out of the HTTP error handler is the main
|
||||
thing that still pages. Anything a human can look at tomorrow is `warning`,
|
||||
and anything that's just worth recording is `info`.
|
||||
|
||||
Omitting the severity takes `pager.defaultSeverity` (itself `critical`), so
|
||||
pass one explicitly unless you really mean "page someone".
|
||||
|
||||
### Choosing one
|
||||
|
||||
- Did the server fail to do its job in a way nobody expected? → `critical`
|
||||
- Is a background job, rate, or dependency degraded? → `warning`
|
||||
- Is this a user doing something notable (hitting a limit, tripping an abuse
|
||||
heuristic, overspending)? → `info`
|
||||
|
||||
An extension whose signals are all one tier can default its own local
|
||||
`raiseAlarm` helper to that tier instead of repeating it at every call site —
|
||||
see [extensions/cronMonitor](../../../extensions/cronMonitor/index.js).
|
||||
|
||||
## One incident per occurrence, unless you say otherwise
|
||||
|
||||
Alarms always de-dupe *in process* — repeats of an id bump its occurrence
|
||||
count rather than creating a second alarm. What that means for the pager is a
|
||||
separate decision, and by default every occurrence opens its own PagerDuty
|
||||
incident: two failed scans an hour apart are two things that happened, and
|
||||
closing one shouldn't hide the other.
|
||||
|
||||
Pass `{ dedup: true }` as a fifth argument when repeats of the id really are
|
||||
one recurring fault, and they collapse onto a single incident carrying the
|
||||
occurrence count:
|
||||
|
||||
```ts
|
||||
this.clients.alarm.create(alarmId, message, fields, 'critical', {
|
||||
dedup: true,
|
||||
});
|
||||
```
|
||||
|
||||
The HTTP error handler uses it: its id is route + error signature, so a hot
|
||||
loop of the same crash is one incident with N occurrences instead of N pages.
|
||||
Reach for it anywhere else only when the id is that specific — otherwise a
|
||||
per-request alarm can flood the pager.
|
||||
|
||||
## Configuration
|
||||
|
||||
Everything lives under `pager` in config (see
|
||||
[config.template.jsonc](../config.template.jsonc) for the annotated version).
|
||||
Both transports are off unless enabled, so a self-hosted node just logs
|
||||
alarms to the console.
|
||||
|
||||
```jsonc
|
||||
"pager": {
|
||||
"defaultSeverity": "critical",
|
||||
"severityOverrides": { "cronMonitor:*": "info" },
|
||||
"pagerduty": { "enabled": true, "routingKey": "…", "minSeverity": "warning" },
|
||||
"slack": {
|
||||
"enabled": true,
|
||||
"webhookUrl": "…",
|
||||
"channel": "#alerts",
|
||||
"minSeverity": "info",
|
||||
"maxSeverity": "info",
|
||||
"repeatThrottleMs": 900000,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Slack's `maxSeverity` defaults to `info` whenever PagerDuty is configured, and
|
||||
to `critical` when it isn't — a node with only a webhook still sees
|
||||
everything. Raise it to have chat mirror the paging tiers as well.
|
||||
|
||||
### Retiering without a deploy
|
||||
|
||||
`severityOverrides` is the escape hatch for an alarm that turns out to be
|
||||
noisier or more serious than its call site assumed. Keys are alarm ids or a
|
||||
prefix ending in `*`; the exact id beats a pattern, and the longest matching
|
||||
prefix wins among patterns. Values are a severity, or `mute` to drop the
|
||||
alarm before any transport sees it.
|
||||
|
||||
It is applied *after* the call site's severity and any known-error rule, so
|
||||
config always has the last word.
|
||||
|
||||
### Repeat throttling
|
||||
|
||||
The chat transport won't repost the same alarm id within
|
||||
`repeatThrottleMs` (default 15 minutes). The first occurrence always posts,
|
||||
and the next one that gets through reports how many piled up in between —
|
||||
so a hot loop reads as one message with a count, not a wall of them.
|
||||
+20
-21
@@ -1,6 +1,6 @@
|
||||
# Self-Hosting Puter
|
||||
|
||||
`docker-compose.yml` brings up Puter **plus every external service it needs** — MariaDB, Valkey, DynamoDB-local, RustFS S3, Caddy — wired together. Closest thing to a production deployment you can self-manage on a single host.
|
||||
`docker-compose.yml` brings up Puter **plus every external service it needs** — MariaDB, Valkey, DynamoDB-local, RustFS S3, nginx — wired together. Closest thing to a production deployment you can self-manage on a single host.
|
||||
|
||||
## One-line installer (recommended)
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
curl -fsSL https://raw.githubusercontent.com/HeyPuter/puter/main/install.sh | sh
|
||||
```
|
||||
|
||||
Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker-compose.yml` + `caddy/Caddyfile` from the OSS repo, and runs `docker compose up -d`. Re-running is safe — it won't overwrite existing config (set `PUTER_FORCE=1` to rotate). Use the manual steps below if you want to inspect or tweak each step yourself.
|
||||
Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker-compose.yml` from the OSS repo, and runs `docker compose up -d`. Re-running is safe — it won't overwrite existing config (set `PUTER_FORCE=1` to rotate). Use the manual steps below if you want to inspect or tweak each step yourself.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -20,7 +20,7 @@ Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker
|
||||
|
||||
| Container | Image | Role |
|
||||
| --------------- | ------------------------ | ---------------------------------------------------------- |
|
||||
| `puter-caddy` | `caddy:2.11-alpine` | Reverse proxy on 80 (and 443 if TLS); forwards to Puter |
|
||||
| `puter-nginx` | `nginx:1.27-alpine` | Reverse proxy on 80 (and 443 if TLS); forwards to Puter |
|
||||
| `puter` | `ghcr.io/heyputer/puter` | The app |
|
||||
| `puter-mariadb` | `mariadb:11` | SQL database — schema applied automatically on first boot |
|
||||
| `puter-valkey` | `valkey/valkey:8-alpine` | Redis-compatible cache + rate-limiter |
|
||||
@@ -47,6 +47,7 @@ State lives under `./puter/data/<service>/`.
|
||||
MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
|
||||
MARIADB_PASSWORD=$(openssl rand -hex 32)
|
||||
S3_SECRET_KEY=$(openssl rand -hex 32)
|
||||
JWT_SECRET=$(openssl rand -hex 64)
|
||||
JWT_SECRET_V2=$(openssl rand -hex 64)
|
||||
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
|
||||
|
||||
@@ -64,7 +65,7 @@ S3_SECRET_KEY=$S3_SECRET_KEY
|
||||
S3_BUCKET=puter-local
|
||||
EOF
|
||||
|
||||
mkdir -p puter/config puter/data puter/data/caddy puter/tls
|
||||
mkdir -p puter/config puter/data puter/tls
|
||||
cat > puter/config/config.json <<EOF
|
||||
{
|
||||
"domain": "puter.localhost",
|
||||
@@ -77,7 +78,9 @@ cat > puter/config/config.json <<EOF
|
||||
"private_app_hosting_domain": "app.puter.localhost",
|
||||
"private_app_hosting_domain_alt": "dev.puter.localhost",
|
||||
|
||||
"jwt_secret": "$JWT_SECRET",
|
||||
"jwt_secret_v2": "$JWT_SECRET_V2",
|
||||
"allow_v1_tokens": true,
|
||||
"url_signature_secret": "$URL_SIGNATURE_SECRET",
|
||||
|
||||
"database": {
|
||||
@@ -131,21 +134,21 @@ Replace `puter.localhost`, `site.puter.localhost`, `host.puter.localhost`, `dev.
|
||||
|
||||
Why these knobs:
|
||||
|
||||
- `jwt_secret_v2` — the HMAC secret Puter signs and verifies auth tokens with (`kid: 'v2'` JWT header). The pre-v2 token format is retired: a token signed with the old `jwt_secret` no longer verifies, and holders are asked to sign in again. If you are upgrading from a release that had `jwt_secret`, drop it from your config — it is ignored.
|
||||
- `jwt_secret` + `jwt_secret_v2` — Puter signs new auth tokens with `jwt_secret_v2` (v2 token format, `kid: 'v2'` JWT header). `jwt_secret` is verify-only and lets tokens minted before this version (cookies still in users' browsers, stored API tokens) keep working. `allow_v1_tokens: true` keeps that fallback enabled. Fresh installs can leave it as-is; future versions will flip the flag to `false` and retire `jwt_secret` entirely once legacy tokens have drained.
|
||||
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start:gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
|
||||
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
|
||||
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
|
||||
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
|
||||
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
|
||||
- `s3.s3Config.forcePathStyle: true` — RustFS / MinIO / fauxqs need path-style URLs (`<endpoint>/<bucket>`). Real AWS S3 wants virtual-hosted (`<bucket>.<endpoint>`) — drop this flag (or set `false`) when you swap to real S3.
|
||||
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. Caddy routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
|
||||
- `trust_proxy: 1` — Caddy terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the Caddy container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (Caddy). Bump to `2` if you put Cloudflare in front of Caddy; never set `true` (it trusts every hop and makes XFF forgeable).
|
||||
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. nginx routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
|
||||
- `trust_proxy: 1` — nginx terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the nginx container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (nginx). Bump to `2` if you put Cloudflare in front of nginx; never set `true` (it trusts every hop and makes XFF forgeable).
|
||||
|
||||
> If you ever change `MARIADB_PASSWORD` after first boot, `.env` alone won't update MariaDB — its credentials are baked into `./puter/data/mariadb/` on first init. Either rotate the password inside MariaDB by hand or `docker compose down && rm -rf ./puter/data/mariadb` to start fresh.
|
||||
|
||||
## Step 2 — Point DNS at the server \[Optional\]
|
||||
|
||||
In your DNS provider, add records for the main domain plus the subdomains Puter and Caddy route on (`api.*`, `site.*`, `app.*`, `s3.*`):
|
||||
In your DNS provider, add records for the main domain plus the subdomains Puter and nginx route on (`api.*`, `site.*`, `app.*`, `s3.*`):
|
||||
|
||||
```
|
||||
A puter.localhost → <your server's public IP>
|
||||
@@ -160,7 +163,7 @@ A dev.puter.localhost → <your server's public IP>
|
||||
A *.dev.puter.localhost → <your server's public IP>
|
||||
```
|
||||
|
||||
The wildcards are required — Puter routes via subdomains (`api.*`, `app.*`, etc.) and Caddy routes browser S3 traffic via `s3.*` to RustFS.
|
||||
The wildcards are required — Puter routes via subdomains (`api.*`, `app.*`, etc.) and nginx routes browser S3 traffic via `s3.*` to RustFS.
|
||||
|
||||
## Step 3 — TLS (recommended for public installs) \[Optional\]
|
||||
|
||||
@@ -179,15 +182,13 @@ sudo certbot certonly --manual --preferred-challenges dns \
|
||||
|
||||
The cert needs to cover `*.puter.localhost` so that `s3.puter.localhost` (browser S3 endpoint), plus Puter's own `api.*` / `app.*` subdomains, all validate.
|
||||
|
||||
> **Why not Caddy's automatic HTTPS?** Caddy issues certs over ACME by itself, but only for hostnames it knows up front, and only over HTTP-01 with the stock image. Puter serves every user's site and app on a subdomain it invents at runtime (`<name>.site.<domain>`, `<name>.app.<domain>`), which needs a **wildcard** cert — and wildcards require DNS-01, which requires a DNS-provider plugin that isn't in `caddy:2.11-alpine`. So `caddy/Caddyfile` sets `auto_https off` and reads the cert you supply below. If you'd rather have Caddy manage certs, build an image with the plugin for your DNS provider (`xcaddy build --with github.com/caddy-dns/<provider>`) and swap the `tls` line for a `tls { dns <provider> … }` block.
|
||||
|
||||
Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
|
||||
|
||||
**Wire Caddy to use them:**
|
||||
**Wire nginx to use them:**
|
||||
|
||||
1. Open [caddy/Caddyfile](../caddy/Caddyfile) and uncomment the `# :443 { … }` block at the bottom.
|
||||
2. (Optional but recommended) Replace the plain `:80 { import puter_routes }` block with the `redir` version shown alongside it, to force HTTPS everywhere.
|
||||
3. In [docker-compose.yml](../docker-compose.yml), uncomment the `443:443` port mapping under the `caddy` service.
|
||||
1. Open [nginx/nginx.conf](../nginx/nginx.conf), uncomment **both** `# server { listen 443 ssl … }` blocks (one for `s3.*`, one for the catch-all).
|
||||
2. (Optional) Replace the body of the port-80 blocks with `return 301 https://$host$request_uri;` to force HTTPS everywhere.
|
||||
3. In [docker-compose.yml](../docker-compose.yml), uncomment the `443:443` port mapping under the `nginx` service.
|
||||
4. In `.env`, uncomment `HTTPS_PORT=443`.
|
||||
5. In `config.json`, switch:
|
||||
```json
|
||||
@@ -200,9 +201,9 @@ Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
|
||||
|
||||
## Running behind your own reverse proxy
|
||||
|
||||
The bundled `puter-caddy` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Traefik, nginx, HAProxy, a cloud load balancer, another Caddy). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
|
||||
The bundled `puter-nginx` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Caddy, Traefik, HAProxy, a cloud load balancer, another nginx). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
|
||||
|
||||
You can keep the bundled Caddy as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
|
||||
You can keep the bundled nginx as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
|
||||
|
||||
**The rules that actually matter** (getting any of these wrong is what causes the redirect loops and "Invalid Host header" failures people hit):
|
||||
|
||||
@@ -218,7 +219,7 @@ You can keep the bundled Caddy as the single hop your proxy talks to, or bypass
|
||||
- `X-Forwarded-For` — the real client IP (rate limiting + audit logs).
|
||||
- `Upgrade` / `Connection` — passed through for WebSocket / socket.io upgrades, or the realtime connection silently fails.
|
||||
|
||||
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled Caddy if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
|
||||
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled nginx if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
|
||||
|
||||
6. **Route the wildcard to Puter.** Your proxy must forward `*.<domain>` (covering `api.*`, `app.*`, and `s3.<domain>`) and `*.site.<domain>` (and any other hosting domains) to Puter — same wildcard DNS as Step 2. Puter does the per-subdomain routing internally; the proxy just needs to hand it the traffic with the Host intact.
|
||||
|
||||
@@ -226,8 +227,6 @@ With those in place the rest of the stack is unchanged — `docker compose up -d
|
||||
|
||||
## Step 4 — Bring it up
|
||||
|
||||
Both [docker-compose.yml](../docker-compose.yml) and [caddy/Caddyfile](../caddy/Caddyfile) need to sit next to your `.env` — clone the repo, or copy those two paths out of it. The Caddyfile is bind-mounted read-only; if it's missing, Docker creates a *directory* at that path and `puter-caddy` dies with "not a directory".
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
@@ -470,7 +469,7 @@ Migrations re-apply idempotently across pulls. Volumes are preserved.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Site loads but I get a 502 / "Bad Gateway" from Caddy.**
|
||||
**Site loads but I get "Bad Gateway" / nginx errors.**
|
||||
The puter container failed to come up. `docker compose logs puter` will tell you which dependency rejected it (most often DB password mismatch between `.env` and `config.json`).
|
||||
|
||||
**Login screen says "admin password not set".**
|
||||
|
||||
+12
-22
@@ -2,7 +2,7 @@
|
||||
# Self-hosted Puter — full stack.
|
||||
#
|
||||
# Brings up Puter + every external service it needs:
|
||||
# - caddy : reverse proxy (mirrors prod ALB; handles TLS + Host fan-out)
|
||||
# - nginx : reverse proxy (mirrors prod ALB; handles TLS + Host fan-out)
|
||||
# - valkey : redis-compatible cache / rate-limiter backend
|
||||
# - mariadb : SQL database (Puter applies its schema on first boot)
|
||||
# - dynamo : DynamoDB-local (KV store; Puter creates the table itself)
|
||||
@@ -23,8 +23,7 @@
|
||||
#
|
||||
# Production:
|
||||
# - Always replace the default passwords / S3 keys / Puter secrets.
|
||||
# - Enable TLS on the bundled Caddy (see caddy/Caddyfile) or front the
|
||||
# stack with your own TLS-terminating proxy.
|
||||
# - Front Puter with TLS-terminating reverse proxy (Caddy / nginx).
|
||||
# - Move state-bearing volumes to a backed-up location.
|
||||
|
||||
services:
|
||||
@@ -119,7 +118,7 @@ services:
|
||||
RUSTFS_SECRET_KEY: ${S3_SECRET_KEY:-puter-secret-change-me}
|
||||
volumes:
|
||||
- ./puter/data/s3:/data:z
|
||||
# Internal-only — browsers reach RustFS via Caddy (`s3.<domain>`),
|
||||
# Internal-only — browsers reach RustFS via nginx (`s3.<domain>`),
|
||||
# which preserves the Host header for S3 signature validation and
|
||||
# rides the same TLS termination as Puter. Uncomment to also expose
|
||||
# 9000 directly on the host for `aws-cli` / debugging.
|
||||
@@ -250,7 +249,7 @@ services:
|
||||
condition: service_started
|
||||
s3-init:
|
||||
condition: service_completed_successfully
|
||||
# Internal-only: Caddy reaches it on the compose network. Uncomment
|
||||
# Internal-only: nginx reaches it on the compose network. Uncomment
|
||||
# to also expose port 4100 directly on the host (useful for debugging).
|
||||
# ports:
|
||||
# - "4100:4100"
|
||||
@@ -271,32 +270,23 @@ services:
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
|
||||
caddy:
|
||||
image: caddy:2.11-alpine
|
||||
container_name: puter-caddy
|
||||
nginx:
|
||||
image: nginx:1.27-alpine
|
||||
container_name: puter-nginx
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
puter:
|
||||
condition: service_started
|
||||
ports:
|
||||
- "${HTTP_PORT:-80}:80"
|
||||
# Uncomment when you enable TLS in caddy/Caddyfile:
|
||||
# Uncomment when you enable TLS in nginx/nginx.conf:
|
||||
# - "${HTTPS_PORT:-443}:443"
|
||||
volumes:
|
||||
- ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro,z
|
||||
# TLS certs (wildcard fullchain.pem + privkey.pem). Read-only inside.
|
||||
- ./puter/tls:/etc/caddy/tls:ro,z
|
||||
# Caddy's own state — the cert store it would use if you swap this
|
||||
# config over to ACME. Kept under ./puter/data with everything else
|
||||
# so one directory is the whole backup.
|
||||
- ./puter/data/caddy:/data:z
|
||||
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro,z
|
||||
# TLS certs (fullchain.pem + privkey.pem). Read-only inside.
|
||||
- ./puter/tls:/etc/nginx/tls:ro,z
|
||||
healthcheck:
|
||||
# Hits Caddy's local-only admin API rather than proxying through to
|
||||
# Puter: this reports whether the proxy itself is up and configured,
|
||||
# and keeps working once the `:80` block becomes an HTTPS redirect.
|
||||
# 127.0.0.1, not localhost — the admin endpoint is IPv4-only and
|
||||
# busybox wget tries ::1 first.
|
||||
test: ["CMD-SHELL", "wget -qO- --tries=1 --timeout=2 http://127.0.0.1:2019/config/ >/dev/null || exit 1"]
|
||||
test: ["CMD-SHELL", "wget -qO- --tries=1 --timeout=2 http://localhost/ || exit 1"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
|
||||
@@ -28,7 +28,7 @@ beforeAll(async () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
const seedOwnedApp = async (prefix: string) => {
|
||||
|
||||
@@ -1,555 +0,0 @@
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import { extensionStore } from '../src/backend/extensions.ts';
|
||||
import type { IConfig } from '../src/backend/types';
|
||||
import './devWatcher.ts';
|
||||
|
||||
type Lifecycle = {
|
||||
onServerStart: () => Promise<void>;
|
||||
onServerShutdown: () => Promise<void>;
|
||||
};
|
||||
|
||||
// The extension registers itself on import; that registry entry is the
|
||||
// only handle on the service class.
|
||||
const DevWatcherService = extensionStore.services.devWatcher as unknown as new (
|
||||
config: IConfig,
|
||||
clients: unknown,
|
||||
stores: unknown,
|
||||
services: unknown,
|
||||
) => Lifecycle;
|
||||
|
||||
const makeService = (config: Record<string, unknown>): Lifecycle =>
|
||||
new DevWatcherService(config as unknown as IConfig, {}, {}, {});
|
||||
|
||||
let workdir: string;
|
||||
|
||||
const write = (relative: string, contents: string): string => {
|
||||
const abs = path.join(workdir, relative);
|
||||
writeFileSync(abs, contents);
|
||||
return abs;
|
||||
};
|
||||
|
||||
/** Poll until `predicate` holds or the budget runs out. */
|
||||
const waitFor = async (
|
||||
predicate: () => boolean,
|
||||
label: string,
|
||||
timeoutMs = 15_000,
|
||||
): Promise<void> => {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (predicate()) return;
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
}
|
||||
throw new Error(`timed out waiting for ${label}`);
|
||||
};
|
||||
|
||||
const logLines = (spy: { mock: { calls: unknown[][] } }): string[] =>
|
||||
spy.mock.calls.map((call) => call.map(String).join(' '));
|
||||
|
||||
describe('devWatcher extension', () => {
|
||||
let log: ReturnType<typeof vi.spyOn>;
|
||||
let warn: ReturnType<typeof vi.spyOn>;
|
||||
let error: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
beforeAll(() => {
|
||||
workdir = mkdtempSync(path.join(tmpdir(), 'devwatch-test-'));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
rmSync(workdir, { recursive: true, force: true });
|
||||
delete (extensionStore.services as Record<string, unknown>).devWatcher;
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
log = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
error = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
log.mockRestore();
|
||||
warn.mockRestore();
|
||||
error.mockRestore();
|
||||
});
|
||||
|
||||
const started = () =>
|
||||
logLines(log).some((line) =>
|
||||
line.startsWith('[devwatch] starting watchers from'),
|
||||
);
|
||||
|
||||
// -- start gating -------------------------------------------------
|
||||
|
||||
it('does nothing when the server is not running in dev', async () => {
|
||||
await makeService({ env: 'production' }).onServerStart();
|
||||
expect(started()).toBe(false);
|
||||
});
|
||||
|
||||
it('does nothing when devwatch is explicitly disabled', async () => {
|
||||
await makeService({
|
||||
env: 'dev',
|
||||
devwatch: { enabled: false },
|
||||
}).onServerStart();
|
||||
expect(started()).toBe(false);
|
||||
});
|
||||
|
||||
it('does nothing when the server opts out with no_devwatch', async () => {
|
||||
await makeService({
|
||||
env: 'dev',
|
||||
no_devwatch: true,
|
||||
devwatch: { enabled: true },
|
||||
}).onServerStart();
|
||||
expect(started()).toBe(false);
|
||||
});
|
||||
|
||||
it('starts in dev even with no explicit devwatch block', async () => {
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
await service.onServerStart();
|
||||
expect(started()).toBe(true);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('starts outside dev when devwatch is explicitly enabled', async () => {
|
||||
const service = makeService({
|
||||
env: 'production',
|
||||
devwatch: {
|
||||
enabled: true,
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
await service.onServerStart();
|
||||
expect(started()).toBe(true);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('ignores a devwatch config that is not an object', async () => {
|
||||
// Falls back to `{}`, so the dev-env rule decides — and the
|
||||
// default webpack entries would be used, so pin it off instead.
|
||||
await makeService({
|
||||
env: 'production',
|
||||
devwatch: 'yes',
|
||||
}).onServerStart();
|
||||
expect(started()).toBe(false);
|
||||
});
|
||||
|
||||
it('only starts once even if the lifecycle hook fires again', async () => {
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
await service.onServerStart();
|
||||
await service.onServerStart();
|
||||
expect(
|
||||
logLines(log).filter((line) =>
|
||||
line.startsWith('[devwatch] starting watchers from'),
|
||||
),
|
||||
).toHaveLength(1);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('waits out the configured ready delay before resolving', async () => {
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [],
|
||||
ready_delay_ms: 60,
|
||||
},
|
||||
});
|
||||
const start = Date.now();
|
||||
await service.onServerStart();
|
||||
expect(Date.now() - start).toBeGreaterThanOrEqual(50);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
// -- child commands -----------------------------------------------
|
||||
|
||||
it('spawns a command, line-buffers its output and flushes the tail', async () => {
|
||||
write(
|
||||
'chatty.js',
|
||||
[
|
||||
"process.stdout.write('one\\ntwo\\n');",
|
||||
"process.stderr.write('bad thing\\n');",
|
||||
// No trailing newline: only the stream `end` flushes it.
|
||||
"process.stdout.write('tail-no-newline');",
|
||||
].join('\n'),
|
||||
);
|
||||
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [
|
||||
{
|
||||
name: 'chatty',
|
||||
directory: '.',
|
||||
command: 'node',
|
||||
args: ['chatty.js'],
|
||||
},
|
||||
],
|
||||
webpack: [],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
await service.onServerStart();
|
||||
|
||||
await waitFor(
|
||||
() =>
|
||||
logLines(log).some((l) =>
|
||||
l.includes('[devwatch:chatty:1] tail-no-newline'),
|
||||
),
|
||||
'child stdout',
|
||||
);
|
||||
|
||||
const lines = logLines(log);
|
||||
expect(lines).toContain('[devwatch:chatty:1] one');
|
||||
expect(lines).toContain('[devwatch:chatty:1] two');
|
||||
expect(logLines(warn)).toContain('[devwatch:chatty:2] bad thing');
|
||||
|
||||
await waitFor(
|
||||
() =>
|
||||
logLines(log).some((l) =>
|
||||
l.includes('[devwatch:chatty:exit] process exited'),
|
||||
),
|
||||
'child exit',
|
||||
);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('passes literal and computed env values to the child', async () => {
|
||||
write(
|
||||
'env.js',
|
||||
'process.stdout.write(`STATIC=${process.env.STATIC_VALUE} ORIGIN=${process.env.FROM_CONFIG} MISSING=${process.env.BLOWS_UP}\\n`);',
|
||||
);
|
||||
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
origin: 'http://puter.localhost:4100',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [
|
||||
{
|
||||
name: 'envtest',
|
||||
directory: '.',
|
||||
command: 'node',
|
||||
args: ['env.js'],
|
||||
env: {
|
||||
STATIC_VALUE: 'literal',
|
||||
FROM_CONFIG: ({
|
||||
global_config,
|
||||
}: {
|
||||
global_config: Record<string, unknown> | null;
|
||||
}) => String(global_config?.origin ?? ''),
|
||||
// Reading through a null is the "config not
|
||||
// loaded yet" shape the extension deliberately
|
||||
// stays quiet about.
|
||||
BLOWS_UP: () => {
|
||||
const nothing = null as unknown as {
|
||||
x: string;
|
||||
};
|
||||
return nothing.x;
|
||||
},
|
||||
NOISY: () => {
|
||||
throw new Error('unexpected failure');
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
webpack: [],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
await service.onServerStart();
|
||||
|
||||
await waitFor(
|
||||
() => logLines(log).some((l) => l.includes('STATIC=literal')),
|
||||
'env output',
|
||||
);
|
||||
const line = logLines(log).find((l) => l.includes('STATIC=literal'))!;
|
||||
expect(line).toContain('ORIGIN=http://puter.localhost:4100');
|
||||
expect(line).toContain('MISSING=undefined');
|
||||
|
||||
// A null-property read is expected noise and stays silent; any
|
||||
// other failure is reported.
|
||||
const warnings = logLines(warn);
|
||||
expect(
|
||||
warnings.some((w) => w.includes('could not evaluate env function')),
|
||||
).toBe(true);
|
||||
expect(warnings.some((w) => w.includes('for BLOWS_UP'))).toBe(false);
|
||||
expect(warnings.some((w) => w.includes('for NOISY'))).toBe(true);
|
||||
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('kills a still-running child on shutdown', async () => {
|
||||
write('forever.js', 'setInterval(() => {}, 1000);');
|
||||
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [
|
||||
{
|
||||
name: 'forever',
|
||||
directory: '.',
|
||||
command: 'node',
|
||||
args: ['forever.js'],
|
||||
},
|
||||
],
|
||||
webpack: [],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
await service.onServerStart();
|
||||
await service.onServerShutdown();
|
||||
|
||||
expect(logLines(log)).toContain('[devwatch:forever] stopping process');
|
||||
await waitFor(
|
||||
() =>
|
||||
logLines(log).some((l) =>
|
||||
l.includes('[devwatch:forever:exit] process exited'),
|
||||
),
|
||||
'child killed',
|
||||
);
|
||||
});
|
||||
|
||||
// -- webpack watchers ---------------------------------------------
|
||||
|
||||
const makeWebpackProject = (
|
||||
name: string,
|
||||
configFile: string,
|
||||
contents: string,
|
||||
packageJson?: string,
|
||||
): string => {
|
||||
const dir = path.join(workdir, name);
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
writeFileSync(
|
||||
path.join(
|
||||
(() => {
|
||||
const { mkdirSync } =
|
||||
require('node:fs') as typeof import('node:fs');
|
||||
mkdirSync(path.join(dir, 'src'), { recursive: true });
|
||||
return dir;
|
||||
})(),
|
||||
'src/entry.js',
|
||||
),
|
||||
"console.log('hello');\n",
|
||||
);
|
||||
writeFileSync(path.join(dir, configFile), contents);
|
||||
if (packageJson) {
|
||||
writeFileSync(path.join(dir, 'package.json'), packageJson);
|
||||
}
|
||||
return name;
|
||||
};
|
||||
|
||||
it('compiles a CommonJS webpack config and reports later rebuilds', async () => {
|
||||
const dir = makeWebpackProject(
|
||||
'cjs-project',
|
||||
'webpack.config.cjs',
|
||||
`module.exports = {
|
||||
mode: 'development',
|
||||
entry: './src/entry.js',
|
||||
output: { path: __dirname + '/out', filename: 'bundle.js' },
|
||||
};`,
|
||||
);
|
||||
|
||||
let onConfigSaw: Record<string, unknown> | undefined;
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [
|
||||
{
|
||||
name: 'cjs',
|
||||
directory: dir,
|
||||
onConfig: (cfg: Record<string, unknown>) => {
|
||||
onConfigSaw = cfg;
|
||||
},
|
||||
},
|
||||
],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
|
||||
await service.onServerStart();
|
||||
expect(onConfigSaw).toBeDefined();
|
||||
// The watcher context is anchored at <root>/<directory> so relative
|
||||
// entries resolve regardless of the server's cwd.
|
||||
expect(onConfigSaw!.context).toBe(path.join(workdir, dir));
|
||||
|
||||
// First build is silent by design; force a rebuild and assert the
|
||||
// update line shows up.
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
writeFileSync(
|
||||
path.join(workdir, dir, 'src/entry.js'),
|
||||
"console.log('hello again');\n",
|
||||
);
|
||||
await waitFor(
|
||||
() =>
|
||||
logLines(log).some((l) =>
|
||||
l.includes('[devwatch] updated cjs using Webpack'),
|
||||
),
|
||||
'webpack rebuild',
|
||||
);
|
||||
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('resolves an ESM webpack config declared through package.json type', async () => {
|
||||
const dir = makeWebpackProject(
|
||||
'esm-project',
|
||||
'webpack.config.js',
|
||||
`export default {
|
||||
mode: 'development',
|
||||
entry: './src/entry.js',
|
||||
output: { filename: 'bundle.js' },
|
||||
};`,
|
||||
JSON.stringify({ name: 'esm-project', type: 'module' }),
|
||||
);
|
||||
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [{ directory: dir }],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
|
||||
await service.onServerStart();
|
||||
await service.onServerShutdown();
|
||||
// Falling back to the directory as the display name is the
|
||||
// documented behaviour when `name` is omitted.
|
||||
expect(error).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('calls a config exported as a function and honours an explicit context', async () => {
|
||||
const dir = makeWebpackProject(
|
||||
'fn-project',
|
||||
'webpack.config.cjs',
|
||||
`module.exports = () => ({
|
||||
mode: 'development',
|
||||
context: 'src',
|
||||
entry: './entry.js',
|
||||
name: process.env.WEBPACK_MARKER,
|
||||
output: { path: __dirname + '/out', filename: 'bundle.js' },
|
||||
});`,
|
||||
);
|
||||
|
||||
let seen: Record<string, unknown> | undefined;
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [
|
||||
{
|
||||
name: 'fn',
|
||||
directory: dir,
|
||||
env: { WEBPACK_MARKER: 'set-during-load' },
|
||||
onConfig: (cfg: Record<string, unknown>) => {
|
||||
seen = cfg;
|
||||
},
|
||||
},
|
||||
],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
|
||||
await service.onServerStart();
|
||||
// Relative `context` resolves against <root>/<directory>.
|
||||
expect(seen!.context).toBe(path.join(workdir, dir, 'src'));
|
||||
// The entry's env map is applied while the config module runs, and
|
||||
// restored afterwards.
|
||||
expect(seen!.name).toBe('set-during-load');
|
||||
expect(process.env.WEBPACK_MARKER).toBeUndefined();
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('reports a failing compilation instead of crashing the server', async () => {
|
||||
const dir = makeWebpackProject(
|
||||
'broken-project',
|
||||
'webpack.config.cjs',
|
||||
`module.exports = {
|
||||
mode: 'development',
|
||||
entry: './src/does-not-exist.js',
|
||||
output: { path: __dirname + '/out', filename: 'bundle.js' },
|
||||
};`,
|
||||
);
|
||||
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [{ name: 'broken', directory: dir }],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
|
||||
await service.onServerStart();
|
||||
await waitFor(
|
||||
() =>
|
||||
logLines(error).some((l) =>
|
||||
l.includes('[devwatch] failed to update broken'),
|
||||
),
|
||||
'webpack failure',
|
||||
);
|
||||
expect(
|
||||
logLines(error).some((l) =>
|
||||
l.includes('[devwatch] error information: broken'),
|
||||
),
|
||||
).toBe(true);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
|
||||
it('fails loudly when a directory has no webpack config at all', async () => {
|
||||
const dir = path.join('no-config');
|
||||
const { mkdirSync } = await import('node:fs');
|
||||
mkdirSync(path.join(workdir, dir), { recursive: true });
|
||||
|
||||
const service = makeService({
|
||||
env: 'dev',
|
||||
devwatch: {
|
||||
root: workdir,
|
||||
commands: [],
|
||||
webpack: [{ name: 'missing', directory: dir }],
|
||||
ready_delay_ms: 0,
|
||||
},
|
||||
});
|
||||
|
||||
await expect(service.onServerStart()).rejects.toThrow(
|
||||
'could not find webpack config for: no-config',
|
||||
);
|
||||
await service.onServerShutdown();
|
||||
});
|
||||
});
|
||||
@@ -38,7 +38,7 @@ beforeAll(async () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
const seedUser = async () => {
|
||||
|
||||
@@ -48,7 +48,7 @@ beforeAll(async () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
const seedUser = async () => {
|
||||
|
||||
@@ -71,7 +71,9 @@ describe('serverInfo extension — handleServerInfo', () => {
|
||||
const statfsSpy = vi
|
||||
.spyOn(fs.default, 'statfs')
|
||||
.mockRejectedValue(new Error('statfs unsupported'));
|
||||
const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
const errSpy = vi
|
||||
.spyOn(console, 'error')
|
||||
.mockImplementation(() => {});
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await handleServerInfo({} as Request, res);
|
||||
@@ -86,18 +88,4 @@ describe('serverInfo extension — handleServerInfo', () => {
|
||||
statfsSpy.mockRestore();
|
||||
errSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('reports an unknown CPU model when os.cpus() comes back empty', async () => {
|
||||
const os = await import('node:os');
|
||||
const cpusSpy = vi.spyOn(os.default, 'cpus').mockReturnValue([]);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await handleServerInfo({} as Request, res);
|
||||
|
||||
const cpu = captured.body!.cpu as Record<string, unknown>;
|
||||
expect(cpu.model).toBe('Unknown');
|
||||
expect(cpu.cores).toBe(0);
|
||||
|
||||
cpusSpy.mockRestore();
|
||||
});
|
||||
});
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
PutObjectCommand,
|
||||
type S3Client,
|
||||
} from '@aws-sdk/client-s3';
|
||||
import crypto from 'node:crypto';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
@@ -15,7 +14,6 @@ import {
|
||||
import { PuterServer } from '../src/backend/server.ts';
|
||||
import { setupTestServer } from '../src/backend/testUtil.ts';
|
||||
import {
|
||||
handleFsCopyNodeThumbnail,
|
||||
handleFsRemoveNodeThumbnail,
|
||||
handleThumbnailCreated,
|
||||
handleThumbnailRead,
|
||||
@@ -28,9 +26,6 @@ const TINY_PNG_BASE64 =
|
||||
|
||||
const BUCKET = 'puter-local';
|
||||
|
||||
// Keys the extension will accept back: its own `thumbnails/<uuid>` namespace.
|
||||
const mintedKey = () => `thumbnails/${crypto.randomUUID()}`;
|
||||
|
||||
const streamToBuffer = async (
|
||||
body: { transformToByteArray: () => Promise<Uint8Array> } | undefined,
|
||||
): Promise<Buffer> => {
|
||||
@@ -46,7 +41,7 @@ describe('thumbnails extension — handleThumbnailCreated', () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
it('uploads a valid data: URL thumbnail to S3 and rewrites event.url to an s3:// pointer', async () => {
|
||||
@@ -113,7 +108,7 @@ describe('thumbnails extension — handleThumbnailUploadPrepare', () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
it('returns early when event has no items array', async () => {
|
||||
@@ -193,14 +188,14 @@ describe('thumbnails extension — handleThumbnailRead', () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
it('rewrites an s3:// thumbnail into a presigned https URL', async () => {
|
||||
// Seed an object so the presigned URL points at something real
|
||||
// (the signer itself doesn't validate existence, but this keeps
|
||||
// the test honest).
|
||||
const key = mintedKey();
|
||||
const key = 'thumb-read-test';
|
||||
await s3.send(
|
||||
new PutObjectCommand({
|
||||
Bucket: BUCKET,
|
||||
@@ -225,57 +220,6 @@ describe('thumbnails extension — handleThumbnailRead', () => {
|
||||
expect((entry.thumbnail as string).startsWith('http')).toBe(true);
|
||||
});
|
||||
|
||||
// `fsentries.thumbnail` is writable through the FS API, so a stored
|
||||
// pointer is attacker input. Signing one the extension didn't mint would
|
||||
// hand out a presigned read of an arbitrary object — including another
|
||||
// user's file, whose key is its fsentry uuid in this same bucket.
|
||||
it.each([
|
||||
// Shaped exactly like an fs object key (and like a legacy thumbnail
|
||||
// row) — indistinguishable from a planted pointer, so it fails closed.
|
||||
[
|
||||
"another user's file object",
|
||||
`s3://${BUCKET}/${crypto.randomUUID()}`,
|
||||
],
|
||||
[
|
||||
'a key outside the thumbnails namespace',
|
||||
`s3://${BUCKET}/secrets/dump`,
|
||||
],
|
||||
[
|
||||
'a namespace-lookalike key',
|
||||
`s3://${BUCKET}/thumbnails/../${crypto.randomUUID()}`,
|
||||
],
|
||||
['a non-uuid inside the namespace', `s3://${BUCKET}/thumbnails/etc`],
|
||||
])('refuses to presign a pointer naming %s', async (_label, thumbnail) => {
|
||||
const entry: Record<string, unknown> = { thumbnail };
|
||||
await handleThumbnailRead(entry, {
|
||||
s3,
|
||||
s3Presign,
|
||||
bucketName: BUCKET,
|
||||
bucketEndpoint: 'http://127.0.0.1:4566/puter-local/',
|
||||
db: stubDb,
|
||||
});
|
||||
expect(entry.thumbnail).toBeNull();
|
||||
});
|
||||
|
||||
it('signs against its own bucket, ignoring the one in the pointer', async () => {
|
||||
const key = mintedKey();
|
||||
const entry: Record<string, unknown> = {
|
||||
thumbnail: `s3://attacker-named-bucket/${key}`,
|
||||
};
|
||||
await handleThumbnailRead(entry, {
|
||||
s3,
|
||||
s3Presign,
|
||||
bucketName: BUCKET,
|
||||
bucketEndpoint: 'http://127.0.0.1:4566/puter-local/',
|
||||
db: stubDb,
|
||||
});
|
||||
// Signed for OUR bucket; `attacker-named-bucket` never reached S3.
|
||||
const signed = entry.thumbnail as string;
|
||||
expect(signed.startsWith('http')).toBe(true);
|
||||
expect(signed).not.toContain('attacker-named-bucket');
|
||||
expect(signed).toContain(BUCKET);
|
||||
});
|
||||
|
||||
it('leaves the thumbnail untouched when not s3/https/data', async () => {
|
||||
const entry: Record<string, unknown> = { thumbnail: 'about:blank' };
|
||||
await handleThumbnailRead(entry, {
|
||||
@@ -337,11 +281,11 @@ describe('thumbnails extension — handleFsRemoveNodeThumbnail', () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
it('deletes the S3 object referenced by an s3:// thumbnail URL', async () => {
|
||||
const key = mintedKey();
|
||||
const key = 'thumb-remove-test';
|
||||
await s3.send(
|
||||
new PutObjectCommand({
|
||||
Bucket: BUCKET,
|
||||
@@ -353,7 +297,7 @@ describe('thumbnails extension — handleFsRemoveNodeThumbnail', () => {
|
||||
|
||||
await handleFsRemoveNodeThumbnail(
|
||||
{ target: { thumbnail: `s3://${BUCKET}/${key}` } },
|
||||
{ s3, bucketName: BUCKET },
|
||||
{ s3 },
|
||||
);
|
||||
|
||||
// GetObject should now error because the key was deleted.
|
||||
@@ -362,147 +306,15 @@ describe('thumbnails extension — handleFsRemoveNodeThumbnail', () => {
|
||||
).rejects.toThrow();
|
||||
});
|
||||
|
||||
// The destructive half of the same confused deputy: the stored pointer
|
||||
// decides which object is deleted, so a key the extension didn't mint
|
||||
// must never reach DeleteObject.
|
||||
it('does not delete an object the pointer names but we did not mint', async () => {
|
||||
const victimKey = crypto.randomUUID(); // shaped like an fs object key
|
||||
await s3.send(
|
||||
new PutObjectCommand({
|
||||
Bucket: BUCKET,
|
||||
Key: victimKey,
|
||||
Body: Buffer.from(TINY_PNG_BASE64, 'base64'),
|
||||
ContentType: 'image/png',
|
||||
}),
|
||||
);
|
||||
|
||||
await handleFsRemoveNodeThumbnail(
|
||||
{ target: { thumbnail: `s3://${BUCKET}/${victimKey}` } },
|
||||
{ s3, bucketName: BUCKET },
|
||||
);
|
||||
|
||||
const survivor = await s3.send(
|
||||
new GetObjectCommand({ Bucket: BUCKET, Key: victimKey }),
|
||||
);
|
||||
expect(survivor.ContentType).toBe('image/png');
|
||||
});
|
||||
|
||||
it('is a no-op when the target has no thumbnail', async () => {
|
||||
// Should not throw or attempt a delete.
|
||||
await handleFsRemoveNodeThumbnail(
|
||||
{ target: {} },
|
||||
{ s3, bucketName: BUCKET },
|
||||
);
|
||||
await handleFsRemoveNodeThumbnail({ target: {} }, { s3 });
|
||||
});
|
||||
|
||||
it('is a no-op when the thumbnail URL is not an s3:// pointer', async () => {
|
||||
await handleFsRemoveNodeThumbnail(
|
||||
{ target: { thumbnail: 'https://cdn.example.com/x.png' } },
|
||||
{ s3, bucketName: BUCKET },
|
||||
{ s3 },
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('thumbnails extension — handleFsCopyNodeThumbnail', () => {
|
||||
let server: PuterServer;
|
||||
let s3: S3Client;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
s3 = server.clients.s3.get();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
it('duplicates the shared thumbnail object and repoints the copied row', async () => {
|
||||
const sourceKey = mintedKey();
|
||||
const body = Buffer.from(TINY_PNG_BASE64, 'base64');
|
||||
await s3.send(
|
||||
new PutObjectCommand({
|
||||
Bucket: BUCKET,
|
||||
Key: sourceKey,
|
||||
Body: body,
|
||||
ContentType: 'image/png',
|
||||
}),
|
||||
);
|
||||
|
||||
const copyUuid = crypto.randomUUID();
|
||||
const db = { write: vi.fn().mockResolvedValue(undefined) };
|
||||
await handleFsCopyNodeThumbnail(
|
||||
{
|
||||
copy: {
|
||||
thumbnail: `s3://${BUCKET}/${sourceKey}`,
|
||||
uuid: copyUuid,
|
||||
},
|
||||
},
|
||||
{ s3, bucketName: BUCKET, db },
|
||||
);
|
||||
|
||||
// The copied row was repointed at a fresh object...
|
||||
expect(db.write).toHaveBeenCalledTimes(1);
|
||||
const [, params] = db.write.mock.calls[0] as [string, [string, string]];
|
||||
const [newPointer, updatedUuid] = params;
|
||||
expect(updatedUuid).toBe(copyUuid);
|
||||
expect(newPointer.startsWith(`s3://${BUCKET}/thumbnails/`)).toBe(true);
|
||||
expect(newPointer).not.toBe(`s3://${BUCKET}/${sourceKey}`);
|
||||
|
||||
// ...whose content matches, while the source object survives — so
|
||||
// deleting either entry can no longer break the other's thumbnail.
|
||||
const newKey = newPointer.slice(`s3://${BUCKET}/`.length);
|
||||
const duplicated = await s3.send(
|
||||
new GetObjectCommand({ Bucket: BUCKET, Key: newKey }),
|
||||
);
|
||||
expect(
|
||||
(await streamToBuffer(duplicated.Body as never)).equals(body),
|
||||
).toBe(true);
|
||||
const original = await s3.send(
|
||||
new GetObjectCommand({ Bucket: BUCKET, Key: sourceKey }),
|
||||
);
|
||||
expect(original.ContentType).toBe('image/png');
|
||||
});
|
||||
|
||||
it('drops the pointer when the shared object is already gone', async () => {
|
||||
const copyUuid = crypto.randomUUID();
|
||||
const db = { write: vi.fn().mockResolvedValue(undefined) };
|
||||
await handleFsCopyNodeThumbnail(
|
||||
{
|
||||
copy: {
|
||||
thumbnail: `s3://${BUCKET}/${mintedKey()}`, // never uploaded
|
||||
uuid: copyUuid,
|
||||
},
|
||||
},
|
||||
{ s3, bucketName: BUCKET, db },
|
||||
);
|
||||
|
||||
expect(db.write).toHaveBeenCalledTimes(1);
|
||||
const [sql, params] = db.write.mock.calls[0] as [string, [string]];
|
||||
expect(sql).toContain('NULL');
|
||||
expect(params).toEqual([copyUuid]);
|
||||
});
|
||||
|
||||
it('does not duplicate an object the pointer names but we did not mint', async () => {
|
||||
const foreignKey = crypto.randomUUID(); // shaped like an fs object key
|
||||
const db = { write: vi.fn().mockResolvedValue(undefined) };
|
||||
await handleFsCopyNodeThumbnail(
|
||||
{
|
||||
copy: {
|
||||
thumbnail: `s3://${BUCKET}/${foreignKey}`,
|
||||
uuid: crypto.randomUUID(),
|
||||
},
|
||||
},
|
||||
{ s3, bucketName: BUCKET, db },
|
||||
);
|
||||
expect(db.write).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('is a no-op when the copy has no thumbnail', async () => {
|
||||
const db = { write: vi.fn().mockResolvedValue(undefined) };
|
||||
await handleFsCopyNodeThumbnail(
|
||||
{ copy: { thumbnail: null, uuid: crypto.randomUUID() } },
|
||||
{ s3, bucketName: BUCKET, db },
|
||||
);
|
||||
expect(db.write).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+29
-152
@@ -1,5 +1,4 @@
|
||||
import {
|
||||
CopyObjectCommand,
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
PutObjectCommand,
|
||||
@@ -14,54 +13,6 @@ const clients = extension.import('client');
|
||||
const MAX_THUMBNAIL_BYTES = 2 * 1024 * 1024;
|
||||
const MAX_THUMBNAIL_PIXELS = 64e6;
|
||||
|
||||
// Namespace every object this extension writes. An fs object's key is its
|
||||
// bare fsentry uuid and the default config points `thumbnailStore.name` at
|
||||
// the same bucket as `s3_bucket`, so without a prefix of our own there is no
|
||||
// way to tell a thumbnail we minted from any other object in the deployment.
|
||||
const THUMBNAIL_KEY_PREFIX = 'thumbnails/';
|
||||
const UUID_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
||||
|
||||
const mintThumbnailKey = (): string =>
|
||||
`${THUMBNAIL_KEY_PREFIX}${crypto.randomUUID()}`;
|
||||
|
||||
/**
|
||||
* Extract the object key from a stored thumbnail pointer, or null when the
|
||||
* pointer isn't one this extension minted.
|
||||
*
|
||||
* `fsentries.thumbnail` is writable through the FS API, so neither half of the
|
||||
* stored string is trusted: the bucket is discarded (callers always pass their
|
||||
* own) and the key must sit under {@link THUMBNAIL_KEY_PREFIX} with a random
|
||||
* uuid. Honouring an arbitrary key would lend this extension's storage
|
||||
* credentials to whatever object the caller named — in the shared-bucket layout
|
||||
* that is every user's file, since an fs object's key is its fsentry uuid.
|
||||
* Legacy bare-uuid thumbnails fail the check and are treated as absent; they
|
||||
* are indistinguishable from a planted pointer, so there is nothing safer to do
|
||||
* with them than stop signing them.
|
||||
*/
|
||||
const resolveThumbnailKey = (pointer: string): string | null => {
|
||||
let key: string;
|
||||
if (pointer.startsWith('s3://')) {
|
||||
const rest = pointer.slice('s3://'.length);
|
||||
const slash = rest.indexOf('/');
|
||||
if (slash === -1) return null;
|
||||
key = rest.slice(slash + 1);
|
||||
} else {
|
||||
let pathname: string;
|
||||
try {
|
||||
pathname = new URL(pointer).pathname;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const segments = pathname.replace(/^\/+/, '').split('/');
|
||||
segments.shift(); // bucket
|
||||
key = segments.join('/');
|
||||
}
|
||||
if (!key.startsWith(THUMBNAIL_KEY_PREFIX)) return null;
|
||||
const id = key.slice(THUMBNAIL_KEY_PREFIX.length);
|
||||
return UUID_PATTERN.test(id) ? key : null;
|
||||
};
|
||||
|
||||
// S3 client + bucket config — lazily resolved after boot from config.
|
||||
let s3Client: S3Client | null = null;
|
||||
let s3PresignClient: S3Client | null = null;
|
||||
@@ -161,7 +112,7 @@ export async function handleThumbnailCreated(
|
||||
return;
|
||||
}
|
||||
|
||||
const key = mintThumbnailKey();
|
||||
const key = crypto.randomUUID();
|
||||
event.url = `s3://${deps.bucketName}/${key}`;
|
||||
|
||||
await deps.s3.send(
|
||||
@@ -200,7 +151,7 @@ export const handleThumbnailUploadPrepare = async (
|
||||
continue;
|
||||
}
|
||||
|
||||
const key = mintThumbnailKey();
|
||||
const key = crypto.randomUUID();
|
||||
const command = new PutObjectCommand({
|
||||
Bucket: deps.bucketName,
|
||||
Key: key,
|
||||
@@ -227,27 +178,27 @@ export const handleThumbnailRead = async (
|
||||
if (typeof thumb !== 'string' || !thumb) return;
|
||||
const presignClient = deps.s3Presign;
|
||||
|
||||
if (
|
||||
thumb.startsWith('s3://') ||
|
||||
// Legacy format — remove after full migration
|
||||
(thumb.startsWith('https') &&
|
||||
thumb.includes(new URL(deps.bucketEndpoint).hostname))
|
||||
) {
|
||||
const key = resolveThumbnailKey(thumb);
|
||||
if (!key) {
|
||||
// Not a pointer we minted — refuse to sign it rather than hand
|
||||
// out a presigned read of whatever object it names.
|
||||
entry.thumbnail = null;
|
||||
return;
|
||||
}
|
||||
if (thumb.startsWith('s3://')) {
|
||||
const [bucket, key] = thumb.slice(5).split('/');
|
||||
entry.thumbnail = await getSignedUrl(
|
||||
presignClient,
|
||||
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
|
||||
new GetObjectCommand({ Bucket: bucket, Key: key }),
|
||||
{ expiresIn: 604800 },
|
||||
);
|
||||
} else if (
|
||||
thumb.startsWith('https') &&
|
||||
thumb.includes(new URL(deps.bucketEndpoint).hostname)
|
||||
) {
|
||||
// Legacy format — remove after full migration
|
||||
const [bucket, key] = new URL(thumb).pathname.slice(1).split('/');
|
||||
entry.thumbnail = await getSignedUrl(
|
||||
presignClient,
|
||||
new GetObjectCommand({ Bucket: bucket, Key: key }),
|
||||
{ expiresIn: 604800 },
|
||||
);
|
||||
} else if (thumb.startsWith('data')) {
|
||||
// Inline data-URL migration: upload to S3 and update the DB entry.
|
||||
const key = mintThumbnailKey();
|
||||
const key = crypto.randomUUID();
|
||||
const { mimeType, data } = base64ParseDataUrl(thumb);
|
||||
const newUrl = `s3://${deps.bucketName}/${key}`;
|
||||
|
||||
@@ -281,72 +232,15 @@ export const handleThumbnailRead = async (
|
||||
}
|
||||
};
|
||||
|
||||
export const handleFsCopyNodeThumbnail = async (
|
||||
payload: { copy?: { thumbnail?: string | null; uuid?: string } | null },
|
||||
deps: {
|
||||
s3: S3Client;
|
||||
bucketName: string;
|
||||
db: { write: (sql: string, params: unknown[]) => Promise<unknown> };
|
||||
},
|
||||
): Promise<void> => {
|
||||
const copy = payload.copy;
|
||||
const thumbnailUrl = copy?.thumbnail;
|
||||
if (!copy || !copy.uuid || typeof thumbnailUrl !== 'string') return;
|
||||
|
||||
// Same trust rule as the read and remove paths: only touch objects this
|
||||
// extension minted.
|
||||
const sourceKey = resolveThumbnailKey(thumbnailUrl);
|
||||
if (!sourceKey) return;
|
||||
|
||||
// The copied row points at the SAME S3 object as its source, and
|
||||
// fs.remove.node deletes the pointed-to object — so the first removal
|
||||
// among the sharers (an overwrite, a trash purge) would break every
|
||||
// other sharer's thumbnail. Give the copy an object of its own.
|
||||
const newKey = mintThumbnailKey();
|
||||
try {
|
||||
await deps.s3.send(
|
||||
new CopyObjectCommand({
|
||||
Bucket: deps.bucketName,
|
||||
CopySource: `${deps.bucketName}/${sourceKey}`,
|
||||
Key: newKey,
|
||||
}),
|
||||
);
|
||||
} catch (err) {
|
||||
// The shared object is already gone (e.g. a sharer was removed
|
||||
// before this fix existed) — the pointer is dead either way, so
|
||||
// drop it rather than leave the row advertising a thumbnail it
|
||||
// doesn't have.
|
||||
await deps.db.write(
|
||||
'UPDATE `fsentries` SET `thumbnail` = NULL WHERE `uuid` = ?',
|
||||
[copy.uuid],
|
||||
);
|
||||
console.warn('[thumbnails] failed to duplicate thumbnail on copy', err);
|
||||
return;
|
||||
}
|
||||
|
||||
await deps.db.write(
|
||||
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
|
||||
[`s3://${deps.bucketName}/${newKey}`, copy.uuid],
|
||||
);
|
||||
};
|
||||
|
||||
export const handleFsRemoveNodeThumbnail = async (
|
||||
payload: { target: { thumbnail?: string | null } },
|
||||
deps: { s3: S3Client; bucketName: string },
|
||||
payload: { target: Record<string, unknown> },
|
||||
deps: { s3: S3Client },
|
||||
): Promise<void> => {
|
||||
const thumbnailUrl = payload.target.thumbnail;
|
||||
if (!thumbnailUrl) return;
|
||||
const thumbnailUrl = payload.target.thumbnail as string | undefined;
|
||||
if (!thumbnailUrl || !thumbnailUrl.startsWith('s3://')) return;
|
||||
|
||||
// Same trust rule as the read path, and load-bearing for the same reason:
|
||||
// the pointer decides which object gets deleted, so a key we didn't mint
|
||||
// would let the owner of one file destroy an object belonging to someone
|
||||
// else just by naming it here.
|
||||
const key = resolveThumbnailKey(thumbnailUrl);
|
||||
if (!key) return;
|
||||
|
||||
await deps.s3.send(
|
||||
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
|
||||
);
|
||||
const [bucket, key] = thumbnailUrl.slice(5).split('/');
|
||||
await deps.s3.send(new DeleteObjectCommand({ Bucket: bucket, Key: key }));
|
||||
};
|
||||
|
||||
extension.on(
|
||||
@@ -385,29 +279,12 @@ extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
|
||||
});
|
||||
});
|
||||
|
||||
// -- fs.copy.node ----------------------------------------------------
|
||||
// A copied entry initially shares its source's thumbnail object; duplicate
|
||||
// it so removing either entry can't break the other's thumbnail.
|
||||
|
||||
extension.on('fs.copy.node', async (_key, payload) => {
|
||||
await handleFsCopyNodeThumbnail(
|
||||
payload as {
|
||||
copy?: { thumbnail?: string | null; uuid?: string } | null;
|
||||
},
|
||||
{
|
||||
s3: getClient(),
|
||||
bucketName: thumbnailBucketName,
|
||||
db: clients.db,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
// -- fs.remove.node --------------------------------------------------
|
||||
// Delete S3 thumbnail when the file is removed.
|
||||
|
||||
extension.on('fs.remove.node', async (_key, payload) => {
|
||||
await handleFsRemoveNodeThumbnail(payload, {
|
||||
s3: getClient(),
|
||||
bucketName: thumbnailBucketName,
|
||||
});
|
||||
});
|
||||
extension.on(
|
||||
'fs.remove.node',
|
||||
async (_key, payload: { target: Record<string, unknown> }) => {
|
||||
await handleFsRemoveNodeThumbnail(payload, { s3: getClient() });
|
||||
},
|
||||
);
|
||||
@@ -51,7 +51,7 @@ beforeAll(async () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
const seedUser = async () => {
|
||||
|
||||
+12
-15
@@ -11,7 +11,7 @@
|
||||
# What this does, in order:
|
||||
# 1. Checks that docker (with the compose plugin) exists.
|
||||
# 2. Creates ./puter-selfhosted/ (override with $env:PUTER_DIR).
|
||||
# 3. Downloads docker-compose.yml + caddy/Caddyfile from the OSS repo.
|
||||
# 3. Downloads docker-compose.yml + nginx.conf from the OSS repo.
|
||||
# 4. Generates fresh secrets and writes .env + puter/config/config.json.
|
||||
# 5. Runs `docker compose up -d` and prints how to find the admin password.
|
||||
#
|
||||
@@ -23,7 +23,7 @@
|
||||
# PUTER_DIR install directory (default: ./puter-selfhosted)
|
||||
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
|
||||
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
|
||||
# PUTER_PORT HTTP port for Caddy (default: 80)
|
||||
# PUTER_PORT HTTP port for nginx (default: 80)
|
||||
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
|
||||
|
||||
[CmdletBinding()]
|
||||
@@ -77,10 +77,10 @@ if ($LASTEXITCODE -ne 0) {
|
||||
# ── Step 2: install dir ─────────────────────────────────────────────
|
||||
$null = New-Item -ItemType Directory -Force -Path $PuterDir
|
||||
Set-Location $PuterDir
|
||||
$null = New-Item -ItemType Directory -Force -Path 'puter/config', 'puter/data', 'puter/data/caddy', 'puter/tls'
|
||||
$null = New-Item -ItemType Directory -Force -Path 'puter/config', 'puter/data', 'puter/tls'
|
||||
Write-Log "install dir: $((Get-Location).Path)"
|
||||
|
||||
# ── Step 3: docker-compose.yml + Caddy config ──────────────────────
|
||||
# ── Step 3: docker-compose.yml + nginx config ──────────────────────
|
||||
Write-Log "downloading docker-compose.yml from $PuterUrl"
|
||||
try {
|
||||
Invoke-WebRequest -Uri "$PuterUrl/docker-compose.yml" -OutFile 'docker-compose.yml' -UseBasicParsing
|
||||
@@ -88,19 +88,17 @@ try {
|
||||
Die "could not fetch $PuterUrl/docker-compose.yml — $_"
|
||||
}
|
||||
|
||||
# The Caddyfile is domain-agnostic — it answers on every Host and leaves
|
||||
# the subdomain routing to Puter — so there's nothing to template in.
|
||||
Write-Log "downloading caddy/Caddyfile from $PuterUrl"
|
||||
$null = New-Item -ItemType Directory -Force -Path 'caddy'
|
||||
Write-Log "downloading nginx/nginx.conf from $PuterUrl"
|
||||
$null = New-Item -ItemType Directory -Force -Path 'nginx'
|
||||
# If the path was previously auto-created as a directory by a failed
|
||||
# `compose up`, remove it so we can write the file there.
|
||||
if (Test-Path 'caddy/Caddyfile' -PathType Container) {
|
||||
Remove-Item 'caddy/Caddyfile' -Recurse -Force
|
||||
if (Test-Path 'nginx/nginx.conf' -PathType Container) {
|
||||
Remove-Item 'nginx/nginx.conf' -Recurse -Force
|
||||
}
|
||||
try {
|
||||
Invoke-WebRequest -Uri "$PuterUrl/caddy/Caddyfile" -OutFile 'caddy/Caddyfile' -UseBasicParsing
|
||||
Invoke-WebRequest -Uri "$PuterUrl/nginx/nginx.conf" -OutFile 'nginx/nginx.conf' -UseBasicParsing
|
||||
} catch {
|
||||
Die "could not fetch $PuterUrl/caddy/Caddyfile — $_"
|
||||
Die "could not fetch $PuterUrl/nginx/nginx.conf — $_"
|
||||
}
|
||||
|
||||
# ── Step 4: secrets, .env, config.json ──────────────────────────────
|
||||
@@ -123,8 +121,7 @@ if ($writeConfig) {
|
||||
|
||||
$envContent = @"
|
||||
HTTP_PORT=$PuterPort
|
||||
# HTTPS_PORT=443 # uncomment after enabling TLS in caddy/Caddyfile
|
||||
# # (see "Step 3 — TLS" in doc/self-hosting.md)
|
||||
# HTTPS_PORT=443 # uncomment after enabling TLS (see doc/selfhosting/full-stack.md)
|
||||
|
||||
MARIADB_ROOT_PASSWORD=$mariadbRootPw
|
||||
MARIADB_DATABASE=puter
|
||||
@@ -208,4 +205,4 @@ Write-Log " cd $PuterDir; docker compose logs -f puter"
|
||||
Write-Log ''
|
||||
Write-Log "open http://${PuterDomain}:${PuterPort} once the puter container is healthy."
|
||||
Write-Log 'first-boot admin password is logged once — grab it with:'
|
||||
Write-Log " cd $PuterDir; docker compose logs puter | Select-String password"
|
||||
Write-Log " cd $PuterDir; docker compose logs puter | Select-String password"
|
||||
+18
-36
@@ -7,8 +7,7 @@
|
||||
# What this does, in order:
|
||||
# 1. Checks that docker (with the compose plugin), curl, and openssl exist.
|
||||
# 2. Creates ./puter-selfhosted/ (override with PUTER_DIR=...).
|
||||
# 3. Downloads docker-compose.yml + caddy/Caddyfile from the OSS repo
|
||||
# (raw.githubusercontent.com).
|
||||
# 3. Downloads docker-compose.yml from the OSS repo (raw.githubusercontent.com).
|
||||
# 4. Generates fresh secrets and writes .env + puter/config/config.json.
|
||||
# 5. Runs `docker compose up -d` and prints the first-boot admin password.
|
||||
#
|
||||
@@ -20,17 +19,15 @@
|
||||
# PUTER_DIR install directory (default: ./puter-selfhosted)
|
||||
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
|
||||
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
|
||||
# PUTER_PORT HTTP port for Caddy (default: 80)
|
||||
# PUTER_PORT HTTP port for nginx (default: 80)
|
||||
# PUTER_PROTOCOL public scheme: http | https (default: http)
|
||||
# Set https once TLS is terminating in front of Puter —
|
||||
# either the bundled Caddy with your certs (see "Step 3 —
|
||||
# TLS" in doc/self-hosting.md) or your own proxy (Traefik,
|
||||
# a cloud LB). Puter then builds https:// origins, S3
|
||||
# URLs, and redirects. Leave it http and the installer
|
||||
# serves plain HTTP on PUTER_PORT.
|
||||
# Set https when a TLS-terminating reverse proxy (Caddy,
|
||||
# Traefik, nginx, a cloud LB) sits in front — Puter then
|
||||
# builds https:// origins, S3 URLs, and redirects. See
|
||||
# "Running behind your own reverse proxy" in doc/self-hosting.md.
|
||||
# PUTER_TRUST_PROXY number of reverse-proxy hops in front (default: 1)
|
||||
# 1 = the bundled Caddy (or a single external proxy);
|
||||
# 2 = two hops (e.g. Cloudflare → Caddy → Puter).
|
||||
# 1 = the bundled nginx (or a single external proxy);
|
||||
# 2 = two hops (e.g. Cloudflare → your proxy → Puter).
|
||||
# PUTER_ENV prod | dev (default: prod)
|
||||
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
|
||||
|
||||
@@ -75,27 +72,25 @@ mkdir -p puter/config puter/data puter/tls
|
||||
# bind-mount roots sidesteps the mismatch without guessing each image's
|
||||
# internal UID. (Docker Desktop on macOS/Windows papers over this with
|
||||
# its VM layer; native Linux docker on Debian/Alpine doesn't.)
|
||||
mkdir -p puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter puter/data/caddy
|
||||
chmod 0777 puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter puter/data/caddy
|
||||
mkdir -p puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter
|
||||
chmod 0777 puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter
|
||||
log "install dir: $(pwd)"
|
||||
|
||||
# ── Step 3: docker-compose.yml + Caddy config ──────────────────────
|
||||
# ── Step 3: docker-compose.yml + nginx config ──────────────────────
|
||||
log "downloading docker-compose.yml from $PUTER_URL"
|
||||
curl -fsSL "$PUTER_URL/docker-compose.yml" -o docker-compose.yml \
|
||||
|| die "could not fetch $PUTER_URL/docker-compose.yml"
|
||||
|
||||
# Caddy is mounted as `./caddy/Caddyfile:/etc/caddy/Caddyfile:ro,z` — if
|
||||
# nginx is mounted as `./nginx/nginx.conf:/etc/nginx/nginx.conf:ro` — if
|
||||
# the host file is missing, docker silently creates a directory at that
|
||||
# path and the mount fails with "not a directory" at container start.
|
||||
# The Caddyfile is domain-agnostic — it answers on every Host and leaves
|
||||
# the subdomain routing to Puter — so there's nothing to template in.
|
||||
log "downloading caddy/Caddyfile from $PUTER_URL"
|
||||
mkdir -p caddy
|
||||
log "downloading nginx/nginx.conf from $PUTER_URL"
|
||||
mkdir -p nginx
|
||||
# If the path was previously auto-created as a dir by a failed `compose up`,
|
||||
# remove it so curl can write the file.
|
||||
[ -d caddy/Caddyfile ] && rmdir caddy/Caddyfile 2>/dev/null || true
|
||||
curl -fsSL "$PUTER_URL/caddy/Caddyfile" -o caddy/Caddyfile \
|
||||
|| die "could not fetch $PUTER_URL/caddy/Caddyfile"
|
||||
[ -d nginx/nginx.conf ] && rmdir nginx/nginx.conf 2>/dev/null || true
|
||||
curl -fsSL "$PUTER_URL/nginx/nginx.conf" -o nginx/nginx.conf \
|
||||
|| die "could not fetch $PUTER_URL/nginx/nginx.conf"
|
||||
|
||||
# ── Step 4: secrets, .env, config.json ──────────────────────────────
|
||||
write_config=1
|
||||
@@ -118,8 +113,7 @@ if [ "$write_config" = "1" ]; then
|
||||
|
||||
cat > .env <<EOF
|
||||
HTTP_PORT=$PUTER_PORT
|
||||
# HTTPS_PORT=443 # uncomment after enabling TLS in caddy/Caddyfile
|
||||
# # (see "Step 3 — TLS" in doc/self-hosting.md)
|
||||
# HTTPS_PORT=443 # uncomment after enabling TLS (see doc/selfhosting/full-stack.md)
|
||||
|
||||
MARIADB_ROOT_PASSWORD=$MARIADB_ROOT_PASSWORD
|
||||
MARIADB_DATABASE=puter
|
||||
@@ -196,18 +190,6 @@ EOF
|
||||
EOF
|
||||
fi
|
||||
|
||||
# `https` only works if something in front is actually terminating it. The
|
||||
# bundled Caddy does that from ./puter/tls once you enable its `:443` block
|
||||
# (see "Step 3 — TLS" in doc/self-hosting.md); your own edge proxy is fine
|
||||
# too. Neither in place means Puter hands out https:// URLs for an endpoint
|
||||
# that only speaks HTTP — broken logins and mixed content.
|
||||
if [ "$PUTER_PROTOCOL" = "https" ] && [ ! -f puter/tls/fullchain.pem ]; then
|
||||
warn "PUTER_PROTOCOL=https but ./puter/tls/fullchain.pem is missing."
|
||||
warn "The bundled Caddy is still serving plain HTTP on port $PUTER_PORT."
|
||||
warn "Either drop a wildcard cert into ./puter/tls and uncomment the :443"
|
||||
warn "block in caddy/Caddyfile, or terminate TLS in your own proxy."
|
||||
fi
|
||||
|
||||
# ── Step 5: bring it up ─────────────────────────────────────────────
|
||||
log "docker compose up -d"
|
||||
docker compose up -d
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
# Reverse proxy in front of Puter — mirrors what the prod ALB does:
|
||||
# accepts every Host header, forwards to the Puter container, and lets
|
||||
# the Puter app handle subdomain-based routing internally (api.*,
|
||||
# site.*, app.*, etc).
|
||||
#
|
||||
# To enable TLS:
|
||||
# 1. Drop your fullchain.pem + privkey.pem into ./puter/tls/.
|
||||
# 2. Uncomment the 443 server{} block below.
|
||||
# 3. Update server_name to your domain (and wildcard subdomains).
|
||||
|
||||
worker_processes auto;
|
||||
events {
|
||||
worker_connections 4096;
|
||||
}
|
||||
|
||||
http {
|
||||
# Required for Puter's WebSocket / socket.io upgrades.
|
||||
map $http_upgrade $connection_upgrade {
|
||||
default upgrade;
|
||||
'' close;
|
||||
}
|
||||
|
||||
# Rough size cap that mirrors prod ALB defaults; tune for your
|
||||
# uploads. Puter chunks large uploads, so 1 GiB per request is plenty.
|
||||
client_max_body_size 1024m;
|
||||
proxy_read_timeout 600s;
|
||||
proxy_send_timeout 600s;
|
||||
proxy_buffering off;
|
||||
server_tokens off;
|
||||
|
||||
upstream puter_backend {
|
||||
server puter:4100;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
upstream s3_backend {
|
||||
# RustFS — see `s3` service in docker-compose.full.yml. Browsers
|
||||
# PUT/GET here for presigned-URL uploads / downloads. Routed via
|
||||
# the `s3.<domain>` subdomain so signature verification works
|
||||
# (Host header preserved end-to-end) and so HTTPS stays clean
|
||||
# (no mixed-content from a port-9000 host publish).
|
||||
server s3:9000;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
# ── HTTP (port 80) ─────────────────────────────────────────────
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name ~^s3\.;
|
||||
|
||||
location / {
|
||||
proxy_pass http://s3_backend;
|
||||
proxy_http_version 1.1;
|
||||
# Critical: preserve the original Host so RustFS validates
|
||||
# the request against the same host the URL was signed for.
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80 default_server;
|
||||
listen [::]:80 default_server;
|
||||
server_name _;
|
||||
|
||||
# Note: when you enable TLS, replace this block with a redirect:
|
||||
# return 301 https://$host$request_uri;
|
||||
location / {
|
||||
proxy_pass http://puter_backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-Port $server_port;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
}
|
||||
}
|
||||
|
||||
# ── HTTPS (port 443) — uncomment after dropping certs in ./puter/tls/ ─
|
||||
# server {
|
||||
# listen 443 ssl;
|
||||
# listen [::]:443 ssl;
|
||||
# http2 on;
|
||||
# server_name ~^s3\.;
|
||||
#
|
||||
# ssl_certificate /etc/nginx/tls/fullchain.pem;
|
||||
# ssl_certificate_key /etc/nginx/tls/privkey.pem;
|
||||
# ssl_protocols TLSv1.2 TLSv1.3;
|
||||
# ssl_ciphers HIGH:!aNULL:!MD5;
|
||||
# ssl_session_cache shared:SSL:10m;
|
||||
# ssl_session_timeout 10m;
|
||||
#
|
||||
# location / {
|
||||
# proxy_pass http://s3_backend;
|
||||
# proxy_http_version 1.1;
|
||||
# proxy_set_header Host $host;
|
||||
# proxy_set_header X-Real-IP $remote_addr;
|
||||
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
# proxy_set_header X-Forwarded-Proto $scheme;
|
||||
# }
|
||||
# }
|
||||
#
|
||||
# server {
|
||||
# listen 443 ssl default_server;
|
||||
# listen [::]:443 ssl default_server;
|
||||
# http2 on;
|
||||
# server_name _;
|
||||
#
|
||||
# ssl_certificate /etc/nginx/tls/fullchain.pem;
|
||||
# ssl_certificate_key /etc/nginx/tls/privkey.pem;
|
||||
# ssl_protocols TLSv1.2 TLSv1.3;
|
||||
# ssl_ciphers HIGH:!aNULL:!MD5;
|
||||
# ssl_session_cache shared:SSL:10m;
|
||||
# ssl_session_timeout 10m;
|
||||
#
|
||||
# location / {
|
||||
# proxy_pass http://puter_backend;
|
||||
# proxy_http_version 1.1;
|
||||
# proxy_set_header Host $host;
|
||||
# proxy_set_header X-Real-IP $remote_addr;
|
||||
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
# proxy_set_header X-Forwarded-Proto $scheme;
|
||||
# proxy_set_header X-Forwarded-Host $host;
|
||||
# proxy_set_header X-Forwarded-Port $server_port;
|
||||
# proxy_set_header Upgrade $http_upgrade;
|
||||
# proxy_set_header Connection $connection_upgrade;
|
||||
# }
|
||||
# }
|
||||
}
|
||||
Generated
+2
-66
@@ -1304,6 +1304,7 @@
|
||||
},
|
||||
"node_modules/@clack/prompts/node_modules/is-unicode-supported": {
|
||||
"version": "1.3.0",
|
||||
"extraneous": true,
|
||||
"inBundle": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
@@ -9932,40 +9933,6 @@
|
||||
"node": ">=10.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/engine.io-client": {
|
||||
"version": "6.5.4",
|
||||
"resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-6.5.4.tgz",
|
||||
"integrity": "sha512-GeZeeRjpD2qf49cZQ0Wvh/8NJNfeXkXXcoGh+F77oEAgo9gUHwT1fCRxSNU+YEEaysOJTnsFHmM5oAcPy4ntvQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@socket.io/component-emitter": "~3.1.0",
|
||||
"debug": "~4.3.1",
|
||||
"engine.io-parser": "~5.2.1",
|
||||
"ws": "~8.17.1",
|
||||
"xmlhttprequest-ssl": "~2.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/engine.io-client/node_modules/ws": {
|
||||
"version": "8.17.1",
|
||||
"resolved": "https://registry.npmjs.org/ws/-/ws-8.17.1.tgz",
|
||||
"integrity": "sha512-6XQFvXTkbfUOZOKKILFG1PDK2NDQs4azKQl26T0YS5CxqWLgXajbPZ+h4gZekJyRqFU8pvnbAbbs/3TgRPy+GQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10.0.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"bufferutil": "^4.0.1",
|
||||
"utf-8-validate": ">=5.0.2"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"bufferutil": {
|
||||
"optional": true
|
||||
},
|
||||
"utf-8-validate": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/engine.io-parser": {
|
||||
"version": "5.2.3",
|
||||
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.2.3.tgz",
|
||||
@@ -15501,12 +15468,6 @@
|
||||
"tslib": "^2.0.3"
|
||||
}
|
||||
},
|
||||
"node_modules/path-browserify": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/path-browserify/-/path-browserify-1.0.1.tgz",
|
||||
"integrity": "sha512-b7uo2UCUOYZcnF/3ID0lulOJi/bafxa1xPe7ZPsammBSpjSWQkjNxlt635YGS2MiR9GjvuXCtz2emr3jbsz98g==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/path-exists": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz",
|
||||
@@ -17338,21 +17299,6 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/socket.io-client": {
|
||||
"version": "4.7.2",
|
||||
"resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-4.7.2.tgz",
|
||||
"integrity": "sha512-vtA0uD4ibrYD793SOIAwlo8cj6haOeMHrGvwPxJsxH7CeIksqJ+3Zc06RvWTIFgiSqx4A3sOnTXpfAEE2Zyz6w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@socket.io/component-emitter": "~3.1.0",
|
||||
"debug": "~4.3.2",
|
||||
"engine.io-client": "~6.5.2",
|
||||
"socket.io-parser": "~4.2.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/socket.io-parser": {
|
||||
"version": "4.2.6",
|
||||
"resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-4.2.6.tgz",
|
||||
@@ -19244,14 +19190,6 @@
|
||||
"integrity": "sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/xmlhttprequest-ssl": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-2.0.0.tgz",
|
||||
"integrity": "sha512-QKxVRxiRACQcVuQEYFsI1hhkrMlrXHPegbbd1yn9UHOmRxY+si12nQYzri3vbzt8VdTTRviqcKxcyllFas5z2A==",
|
||||
"engines": {
|
||||
"node": ">=0.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/xtend": {
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
|
||||
@@ -20256,9 +20194,7 @@
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@heyputer/kv.js": "^0.2.1",
|
||||
"open": "^10.2.0",
|
||||
"path-browserify": "1.0.1",
|
||||
"socket.io-client": "4.7.2"
|
||||
"open": "^10.2.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "^1.49.0",
|
||||
|
||||
+5
-4
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "puter.com",
|
||||
"version": "26.07",
|
||||
"version": "2.5.1",
|
||||
"author": "Puter Technologies Inc.",
|
||||
"license": "AGPL-3.0-only",
|
||||
"description": "Desktop environment in the browser!",
|
||||
@@ -51,8 +51,8 @@
|
||||
"yaml": "^2.8.1"
|
||||
},
|
||||
"scripts": {
|
||||
"test:backend": "npm run setupExtensions && npm run build:workerLib && vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:backend:postgres": "npm run setupExtensions && npm run build:workerLib && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:backend": "npm run setupExtensions && vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:backend:postgres": "npm run setupExtensions && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
|
||||
"test:puterjs": "npm run setupExtensions && vitest run --config src/puter-js/tests/api/vitest.config.ts",
|
||||
"test:puterjs:node": "npm run test:puterjs -- src/puter-js/tests/api/runners/node.test.ts",
|
||||
"test:puterjs:browser": "npm run test:puterjs -- src/puter-js/tests/api/runners/browser.test.ts",
|
||||
@@ -60,7 +60,8 @@
|
||||
"test:puterjs:coverage": "npm run build:workerLib:coverage && npm run setupExtensions && rm -rf src/puter-js/coverage && PUTER_COVERAGE=1 vitest run --config src/puter-js/tests/api/vitest.config.ts && node ./tools/puterjsCoverageReport.mjs",
|
||||
"build:workerLib:coverage": "cd src/puter-js && npm run build:coverage && cd ../worker && npm run build",
|
||||
"start:gui": "nodemon --exec \"node dev-server.js\" ",
|
||||
"start": "node ./tools/start.mjs",
|
||||
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
|
||||
"prestart": "npm run setupExtensions && npm run build:ts",
|
||||
"dev": "npm start",
|
||||
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
|
||||
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
|
||||
|
||||
@@ -1,503 +0,0 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { AlarmClient } from './AlarmClient';
|
||||
import type { IConfig, IPagerConfig } from '../../types';
|
||||
import type { AlertPayload } from './types';
|
||||
|
||||
const pdEvent = vi.hoisted(() => vi.fn(async () => ({})));
|
||||
vi.mock('@pagerduty/pdjs', () => ({ event: pdEvent }));
|
||||
|
||||
const makeClient = (pager: IPagerConfig = {}) =>
|
||||
new AlarmClient({ serverId: 'test-node', pager } as unknown as IConfig);
|
||||
|
||||
/** Register a capturing handler in place of a real transport. */
|
||||
const capture = (
|
||||
client: AlarmClient,
|
||||
minSeverity?: 'critical' | 'error' | 'warning' | 'info',
|
||||
maxSeverity?: 'critical' | 'error' | 'warning' | 'info',
|
||||
) => {
|
||||
const seen: AlertPayload[] = [];
|
||||
client.addAlertHandler(
|
||||
async (alert) => {
|
||||
seen.push(alert);
|
||||
},
|
||||
{ name: 'capture', minSeverity, maxSeverity },
|
||||
);
|
||||
return seen;
|
||||
};
|
||||
|
||||
describe('AlarmClient severity routing', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
pdEvent.mockClear();
|
||||
});
|
||||
|
||||
it('defaults to critical when the call site says nothing', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('boom', 'everything is on fire');
|
||||
|
||||
expect(seen).toHaveLength(1);
|
||||
expect(seen[0].severity).toBe('critical');
|
||||
});
|
||||
|
||||
it('honours a configured default severity', () => {
|
||||
const client = makeClient({ defaultSeverity: 'info' });
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('boom', 'everything is on fire');
|
||||
|
||||
expect(seen[0].severity).toBe('info');
|
||||
});
|
||||
|
||||
it('skips handlers whose floor the alarm does not reach', () => {
|
||||
const client = makeClient();
|
||||
const paging = capture(client, 'warning');
|
||||
const chat = capture(client, 'info');
|
||||
|
||||
client.create('rate-limit', 'a user hit a limit', {}, 'info');
|
||||
|
||||
expect(paging).toHaveLength(0);
|
||||
expect(chat).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('skips handlers whose ceiling the alarm exceeds', () => {
|
||||
const client = makeClient();
|
||||
const chat = capture(client, 'info', 'info');
|
||||
|
||||
client.create('rate-limit', 'a user hit a limit', {}, 'info');
|
||||
client.create('outage', 'everything is on fire', {}, 'critical');
|
||||
|
||||
expect(chat.map((alert) => alert.id)).toEqual(['rate-limit']);
|
||||
});
|
||||
|
||||
it('retiers an alarm from config', () => {
|
||||
const client = makeClient({
|
||||
severityOverrides: { 'noisy:*': 'info' },
|
||||
});
|
||||
const paging = capture(client, 'warning');
|
||||
const chat = capture(client, 'info');
|
||||
|
||||
client.create('noisy:thing', 'used to page', {}, 'critical');
|
||||
|
||||
expect(paging).toHaveLength(0);
|
||||
expect(chat[0].severity).toBe('info');
|
||||
});
|
||||
|
||||
it('mutes an alarm from config', () => {
|
||||
const client = makeClient({
|
||||
severityOverrides: { 'noisy:thing': 'mute' },
|
||||
});
|
||||
const chat = capture(client, 'info');
|
||||
|
||||
client.create('noisy:thing', 'not worth reporting');
|
||||
client.create('noisy:thing', 'still not worth reporting');
|
||||
|
||||
expect(chat).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('lets config override a known-error rule', () => {
|
||||
const client = makeClient({
|
||||
severityOverrides: { 'known:thing': 'critical' },
|
||||
});
|
||||
client.setKnownErrors([
|
||||
{
|
||||
match: { id: 'known:thing' },
|
||||
action: { type: 'severity', value: 'info' },
|
||||
},
|
||||
]);
|
||||
const paging = capture(client, 'warning');
|
||||
|
||||
client.create('known:thing', 'known but escalated');
|
||||
|
||||
expect(paging[0].severity).toBe('critical');
|
||||
});
|
||||
|
||||
it('still respects a no-alert known-error rule', () => {
|
||||
const client = makeClient();
|
||||
client.setKnownErrors([
|
||||
{ match: { id: 'known:quiet' }, action: { type: 'no-alert' } },
|
||||
]);
|
||||
const chat = capture(client, 'info');
|
||||
|
||||
client.create('known:quiet', 'suppressed');
|
||||
|
||||
expect(chat).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AlarmClient alert payload', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
it('reports occurrence counts across repeats', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('flap', 'first');
|
||||
client.create('flap', 'second');
|
||||
|
||||
expect(seen[0]).toMatchObject({ repeatCount: 1, isRepeat: false });
|
||||
expect(seen[1]).toMatchObject({ repeatCount: 2, isRepeat: true });
|
||||
});
|
||||
|
||||
it('renders fields as strings and lifts the stack out of the error', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
const error = new Error('kaboom');
|
||||
|
||||
client.create('boom', 'failed', { error, status: 500 }, 'critical');
|
||||
|
||||
expect(seen[0].fields.status).toBe('500');
|
||||
expect(seen[0].trace).toBe(error.stack);
|
||||
expect(seen[0].shortId).toMatch(/^[a-z]+-[a-z]+-[a-z]+$/);
|
||||
});
|
||||
|
||||
it('keeps one failing handler from starving the others', async () => {
|
||||
const client = makeClient();
|
||||
client.addAlertHandler(
|
||||
async () => {
|
||||
throw new Error('transport down');
|
||||
},
|
||||
{ name: 'broken' },
|
||||
);
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('boom', 'failed');
|
||||
await Promise.resolve();
|
||||
|
||||
expect(seen).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('suppresses alarms once draining', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.onServerPrepareShutdown();
|
||||
client.create('boom', 'too late');
|
||||
|
||||
expect(seen).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AlarmClient transport registration', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
pdEvent.mockClear();
|
||||
});
|
||||
|
||||
it('splits info to Slack and everything above it to PagerDuty', async () => {
|
||||
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
|
||||
const client = makeClient({
|
||||
pagerduty: { enabled: true, routingKey: 'rk' },
|
||||
slack: { enabled: true, webhookUrl: 'https://hooks.example/abc' },
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
client.create('quiet:thing', 'informational', {}, 'info');
|
||||
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
|
||||
expect(pdEvent).not.toHaveBeenCalled();
|
||||
|
||||
client.create('loud:thing', 'paging', {}, 'critical');
|
||||
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(1));
|
||||
// The pager has it; chat doesn't repeat it.
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('sends every severity to Slack when there is no pager', async () => {
|
||||
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
|
||||
const client = makeClient({
|
||||
slack: { enabled: true, webhookUrl: 'https://hooks.example/abc' },
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
client.create(
|
||||
'loud:thing',
|
||||
'nowhere else to send this',
|
||||
{},
|
||||
'critical',
|
||||
);
|
||||
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
|
||||
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('lets config widen the Slack ceiling back out', async () => {
|
||||
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
|
||||
const client = makeClient({
|
||||
pagerduty: { enabled: true, routingKey: 'rk' },
|
||||
slack: {
|
||||
enabled: true,
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
maxSeverity: 'critical',
|
||||
},
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
client.create('loud:thing', 'paging', {}, 'critical');
|
||||
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
|
||||
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('gives each occurrence its own PagerDuty incident by default', async () => {
|
||||
const client = makeClient({
|
||||
pagerduty: { enabled: true, routingKey: 'rk' },
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
client.create('scan:failed', 'first', {}, 'warning');
|
||||
client.create('scan:failed', 'second', {}, 'warning');
|
||||
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
|
||||
|
||||
const keys = pdEvent.mock.calls.map(
|
||||
([arg]: [{ data: { dedup_key: string } }]) => arg.data.dedup_key,
|
||||
);
|
||||
expect(keys[0]).not.toBe(keys[1]);
|
||||
});
|
||||
|
||||
it('collapses repeats of a dedup alarm onto one incident', async () => {
|
||||
const client = makeClient({
|
||||
pagerduty: { enabled: true, routingKey: 'rk' },
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
const raise = () =>
|
||||
client.create(
|
||||
'http_500:POST:/notif/mark-ack:deadlock',
|
||||
'HTTP 500 on POST /notif/mark-ack: deadlock',
|
||||
{},
|
||||
'critical',
|
||||
{ dedup: true },
|
||||
);
|
||||
raise();
|
||||
raise();
|
||||
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
|
||||
|
||||
const keys = pdEvent.mock.calls.map(
|
||||
([arg]: [{ data: { dedup_key: string } }]) => arg.data.dedup_key,
|
||||
);
|
||||
expect(keys[0]).toBe('http_500:POST:/notif/mark-ack:deadlock');
|
||||
expect(keys[1]).toBe(keys[0]);
|
||||
});
|
||||
|
||||
it('skips transports that are enabled but not configured', async () => {
|
||||
const client = makeClient({
|
||||
pagerduty: { enabled: true },
|
||||
slack: { enabled: true },
|
||||
});
|
||||
await client.onServerStart();
|
||||
|
||||
const seen = capture(client);
|
||||
client.create('boom', 'nowhere to send this');
|
||||
|
||||
// Only the capturing handler is registered.
|
||||
expect(seen).toHaveLength(1);
|
||||
expect(pdEvent).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('AlarmClient alarm registry', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
it('finds an alarm by its full id or its short id', () => {
|
||||
const client = makeClient();
|
||||
client.create('disk-pressure', 'nearly full');
|
||||
|
||||
const alarm = client.get('disk-pressure');
|
||||
expect(alarm?.message).toBe('nearly full');
|
||||
expect(client.get(alarm!.shortId)).toBe(alarm);
|
||||
});
|
||||
|
||||
it('returns nothing for an id it has never seen', () => {
|
||||
expect(makeClient().get('never-raised')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('forgets an alarm once cleared, under both ids', () => {
|
||||
const client = makeClient();
|
||||
client.create('disk-pressure', 'nearly full');
|
||||
const shortId = client.get('disk-pressure')!.shortId;
|
||||
|
||||
client.clear('disk-pressure');
|
||||
|
||||
expect(client.get('disk-pressure')).toBeUndefined();
|
||||
expect(client.get(shortId)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ignores a clear for an alarm that is not active', () => {
|
||||
const client = makeClient();
|
||||
expect(() => client.clear('never-raised')).not.toThrow();
|
||||
});
|
||||
|
||||
it('abbreviates long ids in the log line but keeps the short id usable', () => {
|
||||
const log = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
const client = makeClient();
|
||||
const longId = 'a-very-long-alarm-identifier-that-wraps';
|
||||
|
||||
client.create(longId, 'noisy');
|
||||
|
||||
const shortId = client.get(longId)!.shortId;
|
||||
expect(log).toHaveBeenCalledWith(
|
||||
`[alarm] ACTIVE ${shortId} (${longId.slice(0, 20)}...) :: noisy`,
|
||||
);
|
||||
});
|
||||
|
||||
it('accumulates fields across repeats of the same alarm', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('flap', 'first', { a: 1 });
|
||||
client.create('flap', 'second', { b: 2 });
|
||||
|
||||
expect(seen[1].fields).toEqual({ a: '1', b: '2' });
|
||||
expect(client.get('flap')?.occurrences).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('names anonymous handlers by their registration order', () => {
|
||||
const client = makeClient();
|
||||
client.addAlertHandler(async () => {
|
||||
throw new Error('down');
|
||||
});
|
||||
expect(() => client.create('boom', 'x')).not.toThrow();
|
||||
});
|
||||
|
||||
it('only logs the drain notice once', () => {
|
||||
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
log.mockClear();
|
||||
const client = makeClient();
|
||||
|
||||
client.onServerPrepareShutdown();
|
||||
// A second prepare must not restart the drain.
|
||||
client.onServerPrepareShutdown();
|
||||
client.create('a', 'x');
|
||||
client.create('b', 'y');
|
||||
|
||||
const drainLogs = log.mock.calls.filter(
|
||||
([message]) =>
|
||||
message === '[alarm] suppressing alarm while draining',
|
||||
);
|
||||
expect(drainLogs).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('substitutes placeholders for an empty id and message', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
|
||||
client.create('', '');
|
||||
|
||||
expect(seen[0]).toMatchObject({
|
||||
id: 'something-bad',
|
||||
message: 'something bad happened',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('AlarmClient known-error rules', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
it('suppresses alerts for a matching rule', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
client.setKnownErrors([
|
||||
{ match: { id: 'noisy' }, action: { type: 'no-alert' } },
|
||||
]);
|
||||
|
||||
client.create('noisy', 'ignore me');
|
||||
client.create('noisy', 'ignore me again');
|
||||
|
||||
expect(seen).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('leaves alarms with a different id alone', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
client.setKnownErrors([
|
||||
{ match: { id: 'noisy' }, action: { type: 'no-alert' } },
|
||||
]);
|
||||
|
||||
client.create('real', 'page me');
|
||||
|
||||
expect(seen).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('only matches when the message matches too', () => {
|
||||
const rules = [
|
||||
{
|
||||
match: { id: 'timeout', message: 'upstream timed out' },
|
||||
action: { type: 'no-alert' as const },
|
||||
},
|
||||
];
|
||||
|
||||
// Separate clients: suppression is recorded on the alarm, so a
|
||||
// second occurrence of the same id would inherit it.
|
||||
const matching = makeClient();
|
||||
const matchingSeen = capture(matching);
|
||||
matching.setKnownErrors(rules);
|
||||
matching.create('timeout', 'upstream timed out');
|
||||
expect(matchingSeen).toHaveLength(0);
|
||||
|
||||
const differing = makeClient();
|
||||
const differingSeen = capture(differing);
|
||||
differing.setKnownErrors(rules);
|
||||
differing.create('timeout', 'something else entirely');
|
||||
expect(differingSeen.map((alert) => alert.message)).toEqual([
|
||||
'something else entirely',
|
||||
]);
|
||||
});
|
||||
|
||||
it('only matches when every named field matches', () => {
|
||||
const client = makeClient();
|
||||
const seen = capture(client);
|
||||
client.setKnownErrors([
|
||||
{
|
||||
match: { id: 'http', fields: { status: 404 } },
|
||||
action: { type: 'no-alert' },
|
||||
},
|
||||
]);
|
||||
|
||||
client.create('http', 'not found', { status: 404 });
|
||||
client.create('http-2', 'server error', { status: 500 });
|
||||
|
||||
expect(seen.map((alert) => alert.id)).toEqual(['http-2']);
|
||||
});
|
||||
|
||||
it('retiers a matching alarm to a lower severity', () => {
|
||||
const client = makeClient();
|
||||
const paging = capture(client, 'error');
|
||||
const chat = capture(client, 'info');
|
||||
client.setKnownErrors([
|
||||
{
|
||||
match: { id: 'flaky' },
|
||||
action: { type: 'severity', value: 'info' },
|
||||
},
|
||||
]);
|
||||
|
||||
client.create('flaky', 'transient', {}, 'critical');
|
||||
|
||||
expect(paging).toHaveLength(0);
|
||||
expect(chat).toHaveLength(1);
|
||||
expect(chat[0].severity).toBe('info');
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -20,56 +20,64 @@
|
||||
import { event as pdEvent } from '@pagerduty/pdjs';
|
||||
import { inspect } from 'node:util';
|
||||
import { createHash } from 'node:crypto';
|
||||
import type { IConfig, PagerSeverity, SeverityRule } from '../../types';
|
||||
import type { IConfig } from '../../types';
|
||||
import { PuterClient } from '../types';
|
||||
import {
|
||||
meetsMinSeverity,
|
||||
resolveSeverityOverride,
|
||||
withinMaxSeverity,
|
||||
} from './severity';
|
||||
import { createSlackAlertHandler } from './slack';
|
||||
import type {
|
||||
Alarm,
|
||||
AlarmFields,
|
||||
AlarmOptions,
|
||||
AlertHandler,
|
||||
AlertPayload,
|
||||
KnownErrorRule,
|
||||
} from './types';
|
||||
|
||||
export type {
|
||||
Alarm,
|
||||
AlarmFields,
|
||||
AlarmOptions,
|
||||
AlertHandler,
|
||||
AlertPayload,
|
||||
KnownErrorRule,
|
||||
} from './types';
|
||||
export type { PagerSeverity } from '../../types';
|
||||
|
||||
// -- Types ------------------------------------------------------------
|
||||
|
||||
interface RegisteredHandler {
|
||||
name: string;
|
||||
/** Lowest severity this transport accepts. */
|
||||
minSeverity: PagerSeverity;
|
||||
/** Highest severity this transport accepts. */
|
||||
maxSeverity: PagerSeverity;
|
||||
handler: AlertHandler;
|
||||
export interface AlarmFields {
|
||||
error?: Error;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
/** Severity used when neither the call site nor config picks one. */
|
||||
const FALLBACK_SEVERITY: PagerSeverity = 'critical';
|
||||
/** Keeps `info` alarms out of the paging system unless config says otherwise. */
|
||||
const DEFAULT_PAGERDUTY_MIN_SEVERITY: PagerSeverity = 'warning';
|
||||
/** Slack's ceiling once a pager exists: chat gets what doesn't page. */
|
||||
const DEFAULT_SLACK_MAX_SEVERITY_WITH_PAGER: PagerSeverity = 'info';
|
||||
interface AlarmOccurrence {
|
||||
message: string;
|
||||
fields: AlarmFields;
|
||||
timestamp: number;
|
||||
}
|
||||
|
||||
interface Alarm {
|
||||
id: string;
|
||||
shortId: string;
|
||||
message: string;
|
||||
fields: AlarmFields;
|
||||
error?: Error;
|
||||
started: number;
|
||||
timestamps: number[];
|
||||
occurrences: AlarmOccurrence[];
|
||||
severity?: PagerSeverity;
|
||||
noAlert?: boolean;
|
||||
}
|
||||
|
||||
export type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
|
||||
|
||||
export interface AlertPayload {
|
||||
id: string;
|
||||
message: string;
|
||||
source: string;
|
||||
severity: PagerSeverity;
|
||||
custom?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
type AlertHandler = (alert: AlertPayload) => Promise<void>;
|
||||
|
||||
interface KnownErrorRule {
|
||||
match: {
|
||||
id: string;
|
||||
message?: string;
|
||||
fields?: Record<string, unknown>;
|
||||
};
|
||||
action: {
|
||||
type: 'no-alert' | 'severity';
|
||||
value?: PagerSeverity;
|
||||
};
|
||||
}
|
||||
|
||||
// -- Helpers ----------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Deterministic short identifier derived from an alarm ID. Produces a readable
|
||||
* 3-word slug like "amber-delta-fox".
|
||||
* Deterministic short identifier derived from an alarm ID.
|
||||
* Produces a readable 3-word slug like "amber-delta-fox".
|
||||
*/
|
||||
const WORD_POOL = [
|
||||
'alpha',
|
||||
@@ -161,17 +169,13 @@ function cleanFields(fields: AlarmFields): Record<string, string> {
|
||||
// -- AlarmClient ------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Manages system alarms and routes them to alert transports by severity.
|
||||
*
|
||||
* Severity is the routing decision: each transport declares the severity window
|
||||
* it accepts, so `critical` pages on-call while `info` only reaches the chat
|
||||
* channel. Config can retier or mute any alarm id after the fact — see
|
||||
* `pager.severityOverrides` in {@link IConfig}.
|
||||
* Manages system alarms and dispatches alerts to external paging
|
||||
* services (PagerDuty, or any registered handler).
|
||||
*/
|
||||
export class AlarmClient extends PuterClient {
|
||||
private alarms = new Map<string, Alarm>();
|
||||
private aliases = new Map<string, Alarm>();
|
||||
private alertHandlers: RegisteredHandler[] = [];
|
||||
private alertHandlers: AlertHandler[] = [];
|
||||
private knownErrors: KnownErrorRule[] = [];
|
||||
private draining = false;
|
||||
private drainLogged = false;
|
||||
@@ -183,8 +187,39 @@ export class AlarmClient extends PuterClient {
|
||||
// -- Lifecycle ----------------------------------------------------
|
||||
|
||||
override async onServerStart(): Promise<void> {
|
||||
const paging = this.registerPagerDuty();
|
||||
this.registerSlack({ paging });
|
||||
const pagerConf = this.config.pager;
|
||||
if (!pagerConf?.pagerduty?.enabled) return;
|
||||
|
||||
const routingKey = pagerConf.pagerduty.routingKey;
|
||||
if (!routingKey) {
|
||||
console.warn(
|
||||
'[alarm] PagerDuty enabled but no routingKey configured',
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
const serverId = this.config.serverId;
|
||||
|
||||
this.alertHandlers.push(async (alert) => {
|
||||
await pdEvent({
|
||||
data: {
|
||||
routing_key: routingKey,
|
||||
event_action: 'trigger',
|
||||
dedup_key: alert.id,
|
||||
payload: {
|
||||
summary: alert.message,
|
||||
source: alert.source,
|
||||
severity: alert.severity,
|
||||
custom_details: {
|
||||
...alert.custom,
|
||||
server_id: serverId,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
console.log('[alarm] PagerDuty handler registered');
|
||||
}
|
||||
|
||||
override onServerPrepareShutdown(): void {
|
||||
@@ -193,107 +228,22 @@ export class AlarmClient extends PuterClient {
|
||||
console.log('[alarm] entering drain mode — suppressing new alarms');
|
||||
}
|
||||
|
||||
/** @returns Whether a working PagerDuty transport was registered. */
|
||||
private registerPagerDuty(): boolean {
|
||||
const pagerDutyConf = this.config.pager?.pagerduty;
|
||||
if (!pagerDutyConf?.enabled) return false;
|
||||
|
||||
const routingKey = pagerDutyConf.routingKey;
|
||||
if (!routingKey) {
|
||||
console.warn(
|
||||
'[alarm] PagerDuty enabled but no routingKey configured',
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
const serverId = this.config.serverId;
|
||||
const minSeverity =
|
||||
pagerDutyConf.minSeverity ?? DEFAULT_PAGERDUTY_MIN_SEVERITY;
|
||||
|
||||
this.addAlertHandler(
|
||||
async (alert) => {
|
||||
await pdEvent({
|
||||
data: {
|
||||
routing_key: routingKey,
|
||||
event_action: 'trigger',
|
||||
dedup_key: alert.dedupKey,
|
||||
payload: {
|
||||
summary: alert.message,
|
||||
source: alert.source,
|
||||
severity: alert.severity,
|
||||
custom_details: {
|
||||
...alert.custom,
|
||||
server_id: serverId,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
},
|
||||
{ name: 'pagerduty', minSeverity },
|
||||
);
|
||||
|
||||
console.log(
|
||||
`[alarm] PagerDuty handler registered (min severity: ${minSeverity})`,
|
||||
);
|
||||
return true;
|
||||
}
|
||||
|
||||
private registerSlack({ paging }: { paging: boolean }): void {
|
||||
const slackConf = this.config.pager?.slack;
|
||||
if (!slackConf?.enabled) return;
|
||||
|
||||
if (!slackConf.webhookUrl) {
|
||||
console.warn('[alarm] Slack enabled but no webhookUrl configured');
|
||||
return;
|
||||
}
|
||||
|
||||
const minSeverity = slackConf.minSeverity ?? 'info';
|
||||
// With a pager taking everything from `warning` up, chat is where the
|
||||
// rest is recorded — reposting the paging tiers there only trains
|
||||
// people to skim the channel. Without one, Slack is the only place an
|
||||
// alarm can land, so it takes all of them.
|
||||
const maxSeverity =
|
||||
slackConf.maxSeverity ??
|
||||
(paging ? DEFAULT_SLACK_MAX_SEVERITY_WITH_PAGER : 'critical');
|
||||
|
||||
this.addAlertHandler(
|
||||
createSlackAlertHandler(slackConf, {
|
||||
serverId: this.config.serverId,
|
||||
}),
|
||||
{ name: 'slack', minSeverity, maxSeverity },
|
||||
);
|
||||
|
||||
console.log(
|
||||
`[alarm] Slack handler registered (severity ${minSeverity}..${maxSeverity})`,
|
||||
);
|
||||
}
|
||||
|
||||
// -- Public API ---------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create or update an alarm. If the alarm ID already exists, the occurrence
|
||||
* count is incremented and a repeat alert is dispatched.
|
||||
* Create or update an alarm. If the alarm ID already exists, the
|
||||
* occurrence count is incremented and a repeat alert is dispatched.
|
||||
*
|
||||
* `severity` decides where the alarm lands:
|
||||
*
|
||||
* Critical — a real outage; pages on-call. Reserve it for unhandled server
|
||||
* errors. error — pages on-call as well; prefer `critical` or `warning`.
|
||||
* warning — worth a look soon, but nobody gets woken up. info — a record in
|
||||
* the chat channel; never pages.
|
||||
*
|
||||
* Omit it to take `pager.defaultSeverity` (itself defaulting to
|
||||
* 'critical'). Operators can retier or mute any alarm id from config
|
||||
* afterwards, so the value here is the starting point, not the last word.
|
||||
*
|
||||
* Pass `{ dedup: true }` when repeats of this id are one recurring fault
|
||||
* that should collapse into a single incident — see {@link AlarmOptions}.
|
||||
* `severity` is the PagerDuty severity for this alarm; omit for the
|
||||
* default 'critical'. Use 'info' / 'warning' for expected-but-worth-
|
||||
* tracking signals (e.g. a user hitting a rate limit) so they record
|
||||
* and de-dupe like any other alarm but don't page on-call.
|
||||
*/
|
||||
create(
|
||||
id: string,
|
||||
message: string,
|
||||
fields: AlarmFields = {},
|
||||
severity?: PagerSeverity,
|
||||
opts: AlarmOptions = {},
|
||||
): void {
|
||||
if (this.draining) {
|
||||
if (!this.drainLogged) {
|
||||
@@ -317,11 +267,8 @@ export class AlarmClient extends PuterClient {
|
||||
message,
|
||||
fields,
|
||||
severity,
|
||||
dedup: opts.dedup,
|
||||
started: Date.now(),
|
||||
// `recordOccurrence` below stamps the first occurrence; seeding one
|
||||
// here too would report every alarm as one occurrence ahead.
|
||||
timestamps: [],
|
||||
timestamps: [Date.now()],
|
||||
occurrences: [],
|
||||
};
|
||||
if (fields.error) alarm.error = fields.error;
|
||||
@@ -348,28 +295,16 @@ export class AlarmClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Register an additional alert handler. Handlers are called for every alarm
|
||||
* that isn't suppressed by a known-error rule or muted by config, and whose
|
||||
* severity falls inside the handler's own `minSeverity`..`maxSeverity`
|
||||
* window (default: everything).
|
||||
* Register an additional alert handler. Handlers are called for
|
||||
* every alarm that isn't suppressed by a known-error rule.
|
||||
*/
|
||||
addAlertHandler(
|
||||
handler: AlertHandler,
|
||||
opts: {
|
||||
name?: string;
|
||||
minSeverity?: PagerSeverity;
|
||||
maxSeverity?: PagerSeverity;
|
||||
} = {},
|
||||
): void {
|
||||
this.alertHandlers.push({
|
||||
name: opts.name ?? `handler-${this.alertHandlers.length}`,
|
||||
minSeverity: opts.minSeverity ?? 'info',
|
||||
maxSeverity: opts.maxSeverity ?? 'critical',
|
||||
handler,
|
||||
});
|
||||
addAlertHandler(handler: AlertHandler): void {
|
||||
this.alertHandlers.push(handler);
|
||||
}
|
||||
|
||||
/** Add rules that can suppress or adjust severity of known errors. */
|
||||
/**
|
||||
* Add rules that can suppress or adjust severity of known errors.
|
||||
*/
|
||||
setKnownErrors(rules: KnownErrorRule[]): void {
|
||||
this.knownErrors = rules;
|
||||
}
|
||||
@@ -446,72 +381,25 @@ export class AlarmClient extends PuterClient {
|
||||
this.dispatchAlert(alarm);
|
||||
}
|
||||
|
||||
/**
|
||||
* Call-site severity, then any known-error rule (both already on the
|
||||
* alarm), then the config override — so an operator always has the last
|
||||
* word over what the code asked for.
|
||||
*/
|
||||
private resolveSeverity(alarm: Alarm): SeverityRule {
|
||||
const base =
|
||||
alarm.severity ??
|
||||
this.config.pager?.defaultSeverity ??
|
||||
FALLBACK_SEVERITY;
|
||||
return (
|
||||
resolveSeverityOverride(
|
||||
alarm.id,
|
||||
this.config.pager?.severityOverrides,
|
||||
) ?? base
|
||||
);
|
||||
}
|
||||
|
||||
private dispatchAlert(alarm: Alarm): void {
|
||||
const resolved = this.resolveSeverity(alarm);
|
||||
if (resolved === 'mute') {
|
||||
if (!alarm.muteLogged) {
|
||||
alarm.muteLogged = true;
|
||||
console.log(`[alarm] MUTED by config ${displayId(alarm)}`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
alarm.severity = resolved;
|
||||
|
||||
const severity = alarm.severity ?? 'critical';
|
||||
const fieldsClean = cleanFields(alarm.fields);
|
||||
const repeatCount = alarm.timestamps.length;
|
||||
|
||||
const id = alarm.id || 'something-bad';
|
||||
|
||||
const payload: AlertPayload = {
|
||||
id,
|
||||
// A de-duplicating transport should fold repeats of a recurring
|
||||
// fault into one incident, but everything else is a fresh event
|
||||
// each time it fires — the start time keeps occurrence numbering
|
||||
// from colliding with an incident left open by an earlier boot.
|
||||
dedupKey: alarm.dedup
|
||||
? id
|
||||
: `${id}#${alarm.started}.${repeatCount}`,
|
||||
shortId: alarm.shortId,
|
||||
id: alarm.id || 'something-bad',
|
||||
message: alarm.message || alarm.id || 'something bad happened',
|
||||
source: 'alarm',
|
||||
severity: resolved,
|
||||
fields: fieldsClean,
|
||||
trace: alarm.error?.stack,
|
||||
repeatCount,
|
||||
isRepeat: repeatCount > 1,
|
||||
severity,
|
||||
custom: {
|
||||
fields: fieldsClean,
|
||||
trace: alarm.error?.stack,
|
||||
repeat_count: repeatCount,
|
||||
repeat_count: alarm.timestamps.length,
|
||||
},
|
||||
};
|
||||
|
||||
for (const { name, minSeverity, maxSeverity, handler } of this
|
||||
.alertHandlers) {
|
||||
if (!meetsMinSeverity(resolved, minSeverity)) continue;
|
||||
if (!withinMaxSeverity(resolved, maxSeverity)) continue;
|
||||
for (const handler of this.alertHandlers) {
|
||||
handler(payload).catch((err) => {
|
||||
console.error(
|
||||
`[alarm] ${name} alert handler failed: ${err?.message}`,
|
||||
);
|
||||
console.error(`[alarm] alert handler failed: ${err?.message}`);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { meetsMinSeverity, resolveSeverityOverride } from './severity';
|
||||
|
||||
describe('meetsMinSeverity', () => {
|
||||
it('accepts everything at or above the floor', () => {
|
||||
expect(meetsMinSeverity('critical', 'warning')).toBe(true);
|
||||
expect(meetsMinSeverity('error', 'warning')).toBe(true);
|
||||
expect(meetsMinSeverity('warning', 'warning')).toBe(true);
|
||||
expect(meetsMinSeverity('info', 'warning')).toBe(false);
|
||||
});
|
||||
|
||||
it('lets an info floor take every severity', () => {
|
||||
for (const severity of [
|
||||
'critical',
|
||||
'error',
|
||||
'warning',
|
||||
'info',
|
||||
] as const) {
|
||||
expect(meetsMinSeverity(severity, 'info')).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSeverityOverride', () => {
|
||||
it('returns undefined without overrides or on no match', () => {
|
||||
expect(resolveSeverityOverride('a:b', undefined)).toBeUndefined();
|
||||
expect(
|
||||
resolveSeverityOverride('a:b', { 'c:*': 'info' }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('matches an exact id', () => {
|
||||
expect(
|
||||
resolveSeverityOverride('cronMonitor:lowSignupRate', {
|
||||
'cronMonitor:lowSignupRate': 'info',
|
||||
}),
|
||||
).toBe('info');
|
||||
});
|
||||
|
||||
it('matches a prefix pattern', () => {
|
||||
expect(
|
||||
resolveSeverityOverride('cronMonitor:high_aiLogEntries', {
|
||||
'cronMonitor:*': 'warning',
|
||||
}),
|
||||
).toBe('warning');
|
||||
});
|
||||
|
||||
it('prefers the exact id over a prefix pattern', () => {
|
||||
expect(
|
||||
resolveSeverityOverride('cronMonitor:lowSignupRate', {
|
||||
'cronMonitor:*': 'warning',
|
||||
'cronMonitor:lowSignupRate': 'mute',
|
||||
}),
|
||||
).toBe('mute');
|
||||
});
|
||||
|
||||
it('prefers the longest matching prefix', () => {
|
||||
expect(
|
||||
resolveSeverityOverride('abuse:card-verification:setup-failed', {
|
||||
'abuse:*': 'info',
|
||||
'abuse:card-verification:*': 'mute',
|
||||
}),
|
||||
).toBe('mute');
|
||||
});
|
||||
|
||||
it('drops an unrecognized rule rather than guessing', () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
expect(
|
||||
resolveSeverityOverride('a:b', {
|
||||
'a:b': 'silent' as unknown as 'mute',
|
||||
}),
|
||||
).toBeUndefined();
|
||||
expect(warn).toHaveBeenCalled();
|
||||
warn.mockRestore();
|
||||
});
|
||||
});
|
||||
@@ -1,94 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { PagerSeverity, SeverityRule } from '../../types';
|
||||
|
||||
/** Ascending urgency. A transport takes everything at or above its floor. */
|
||||
const RANK: Record<PagerSeverity, number> = {
|
||||
info: 0,
|
||||
warning: 1,
|
||||
error: 2,
|
||||
critical: 3,
|
||||
};
|
||||
|
||||
const SEVERITIES = Object.keys(RANK) as PagerSeverity[];
|
||||
|
||||
export function isPagerSeverity(value: unknown): value is PagerSeverity {
|
||||
return typeof value === 'string' && value in RANK;
|
||||
}
|
||||
|
||||
/** True when `severity` is urgent enough for a transport whose floor is `min`. */
|
||||
export function meetsMinSeverity(
|
||||
severity: PagerSeverity,
|
||||
min: PagerSeverity,
|
||||
): boolean {
|
||||
return RANK[severity] >= RANK[min];
|
||||
}
|
||||
|
||||
/**
|
||||
* True when `severity` is quiet enough for a transport whose ceiling is `max`.
|
||||
* A ceiling is what keeps a chat transport from repeating everything the pager
|
||||
* already delivered.
|
||||
*/
|
||||
export function withinMaxSeverity(
|
||||
severity: PagerSeverity,
|
||||
max: PagerSeverity,
|
||||
): boolean {
|
||||
return RANK[severity] <= RANK[max];
|
||||
}
|
||||
|
||||
/**
|
||||
* Look up the operator override for an alarm id. Exact ids win over prefix
|
||||
* patterns (`cronMonitor:*`); among patterns the longest prefix wins, so a
|
||||
* specific rule can carve an exception out of a broad one.
|
||||
*/
|
||||
export function resolveSeverityOverride(
|
||||
id: string,
|
||||
overrides: Record<string, SeverityRule> | undefined,
|
||||
): SeverityRule | undefined {
|
||||
if (!overrides) return undefined;
|
||||
|
||||
const exact = overrides[id];
|
||||
if (exact !== undefined) return validRule(id, exact);
|
||||
|
||||
let bestLength = -1;
|
||||
let best: SeverityRule | undefined;
|
||||
for (const [pattern, rule] of Object.entries(overrides)) {
|
||||
if (!pattern.endsWith('*')) continue;
|
||||
const prefix = pattern.slice(0, -1);
|
||||
if (!id.startsWith(prefix)) continue;
|
||||
if (prefix.length <= bestLength) continue;
|
||||
bestLength = prefix.length;
|
||||
best = rule;
|
||||
}
|
||||
return best === undefined ? undefined : validRule(id, best);
|
||||
}
|
||||
|
||||
/**
|
||||
* A typo in the override map would otherwise silently mute or escalate an
|
||||
* alarm, so an unrecognized value is dropped with a warning instead.
|
||||
*/
|
||||
function validRule(id: string, rule: SeverityRule): SeverityRule | undefined {
|
||||
if (rule === 'mute' || isPagerSeverity(rule)) return rule;
|
||||
console.warn(
|
||||
`[alarm] ignoring invalid severity override "${rule}" for ${id} ` +
|
||||
`(expected mute or one of ${SEVERITIES.join(', ')})`,
|
||||
);
|
||||
return undefined;
|
||||
}
|
||||
@@ -1,206 +0,0 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { buildSlackMessage, createSlackAlertHandler } from './slack';
|
||||
import type { AlertPayload } from './types';
|
||||
|
||||
const alert = (over: Partial<AlertPayload> = {}): AlertPayload => ({
|
||||
id: 'cronMonitor:high_aiLogEntries',
|
||||
dedupKey: 'cronMonitor:high_aiLogEntries#1.1',
|
||||
shortId: 'amber-delta-fox',
|
||||
message: 'High AI log entries: 1200 in the last 10 minutes',
|
||||
source: 'alarm',
|
||||
severity: 'warning',
|
||||
fields: { count: '1200', threshold: '1000' },
|
||||
repeatCount: 1,
|
||||
isRepeat: false,
|
||||
...over,
|
||||
});
|
||||
|
||||
describe('buildSlackMessage', () => {
|
||||
it('renders severity, message and fields', () => {
|
||||
const msg = buildSlackMessage(alert(), {
|
||||
channel: '#alerts',
|
||||
username: 'puter-alarms',
|
||||
serverId: 'oregon',
|
||||
});
|
||||
|
||||
expect(msg.text).toContain('[WARNING]');
|
||||
expect(msg.text).toContain('High AI log entries');
|
||||
expect(msg.channel).toBe('#alerts');
|
||||
expect(msg.username).toBe('puter-alarms');
|
||||
expect(msg.attachments[0].fields).toEqual([
|
||||
{ title: 'count', value: '1200', short: true },
|
||||
{ title: 'threshold', value: '1000', short: true },
|
||||
]);
|
||||
expect(msg.attachments[0].footer).toBe(
|
||||
'amber-delta-fox • cronMonitor:high_aiLogEntries • oregon',
|
||||
);
|
||||
});
|
||||
|
||||
it('marks repeats with an occurrence count', () => {
|
||||
const msg = buildSlackMessage(
|
||||
alert({ repeatCount: 7, isRepeat: true }),
|
||||
);
|
||||
expect(msg.text).toContain('(x7)');
|
||||
});
|
||||
|
||||
it('colours each severity differently', () => {
|
||||
const colors = (['critical', 'error', 'warning', 'info'] as const).map(
|
||||
(severity) =>
|
||||
buildSlackMessage(alert({ severity })).attachments[0].color,
|
||||
);
|
||||
expect(new Set(colors).size).toBe(4);
|
||||
});
|
||||
|
||||
it('puts the stack in a code block and keeps it out of the fields', () => {
|
||||
const msg = buildSlackMessage(
|
||||
alert({
|
||||
fields: { error: 'Error: boom', path: '/api/x' },
|
||||
trace: 'Error: boom\n at handler',
|
||||
}),
|
||||
);
|
||||
expect(msg.attachments[0].text).toBe(
|
||||
'```Error: boom\n at handler```',
|
||||
);
|
||||
expect(msg.attachments[0].fields.map((f) => f.title)).toEqual(['path']);
|
||||
});
|
||||
|
||||
it('truncates long values and caps the field count', () => {
|
||||
const fields: Record<string, string> = { long: 'x'.repeat(1000) };
|
||||
for (let i = 0; i < 30; i++) fields[`f${i}`] = String(i);
|
||||
|
||||
const msg = buildSlackMessage(alert({ fields }));
|
||||
expect(msg.attachments[0].fields.length).toBe(12);
|
||||
expect(msg.attachments[0].fields[0].value).toHaveLength(400);
|
||||
expect(msg.attachments[0].fields[0].value.endsWith('…')).toBe(true);
|
||||
});
|
||||
|
||||
it('omits channel and username when unset', () => {
|
||||
const msg = buildSlackMessage(alert());
|
||||
expect(msg.channel).toBeUndefined();
|
||||
expect(msg.username).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('createSlackAlertHandler', () => {
|
||||
let fetchMock: ReturnType<typeof vi.fn>;
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('posts the payload to the webhook', async () => {
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
channel: '#alerts',
|
||||
});
|
||||
await handler(alert());
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
const [url, init] = fetchMock.mock.calls[0];
|
||||
expect(url).toBe('https://hooks.example/abc');
|
||||
expect(init.method).toBe('POST');
|
||||
expect(JSON.parse(init.body).channel).toBe('#alerts');
|
||||
});
|
||||
|
||||
it('throttles repeats of the same alarm and lets other ids through', async () => {
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
repeatThrottleMs: 60_000,
|
||||
});
|
||||
|
||||
await handler(alert());
|
||||
await handler(alert({ repeatCount: 2, isRepeat: true }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
await handler(alert({ id: 'other:alarm' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
|
||||
vi.advanceTimersByTime(60_000);
|
||||
await handler(alert({ repeatCount: 3, isRepeat: true }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it('posts every occurrence when throttling is disabled', async () => {
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
repeatThrottleMs: 0,
|
||||
});
|
||||
|
||||
await handler(alert());
|
||||
await handler(alert({ repeatCount: 2, isRepeat: true }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('does not let a failed post consume the throttle slot', async () => {
|
||||
fetchMock.mockResolvedValueOnce({ ok: false, status: 500 });
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
repeatThrottleMs: 60_000,
|
||||
});
|
||||
|
||||
await expect(handler(alert())).rejects.toThrow('500');
|
||||
await handler(alert());
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('releases the throttle slot when the post itself errors', async () => {
|
||||
fetchMock.mockRejectedValueOnce(new Error('socket hang up'));
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
repeatThrottleMs: 60_000,
|
||||
});
|
||||
|
||||
await expect(handler(alert())).rejects.toThrow('socket hang up');
|
||||
await handler(alert());
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('evicts throttle entries that have aged out once the map fills', async () => {
|
||||
const throttleMs = 60_000;
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
repeatThrottleMs: throttleMs,
|
||||
});
|
||||
|
||||
// Fill to the high-water mark, then age every entry past the window.
|
||||
for (let i = 0; i < 5000; i++) {
|
||||
await handler(alert({ id: `aged:${i}` }));
|
||||
}
|
||||
vi.advanceTimersByTime(throttleMs);
|
||||
|
||||
await handler(alert({ id: 'fresh' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(5001);
|
||||
|
||||
// The aged ids were pruned, so they post again immediately.
|
||||
await handler(alert({ id: 'aged:0' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(5002);
|
||||
});
|
||||
|
||||
it('drops the oldest entry when the map is full of live ones', async () => {
|
||||
const handler = createSlackAlertHandler({
|
||||
webhookUrl: 'https://hooks.example/abc',
|
||||
repeatThrottleMs: 60_000,
|
||||
});
|
||||
|
||||
for (let i = 0; i < 5000; i++) {
|
||||
await handler(alert({ id: `live:${i}` }));
|
||||
}
|
||||
|
||||
// Nothing has aged out, so the oldest id makes room for the new one.
|
||||
await handler(alert({ id: 'newcomer' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(5001);
|
||||
|
||||
await handler(alert({ id: 'live:0' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(5002);
|
||||
// The newcomer is still throttled.
|
||||
await handler(alert({ id: 'newcomer' }));
|
||||
expect(fetchMock).toHaveBeenCalledTimes(5002);
|
||||
});
|
||||
});
|
||||
@@ -1,178 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { ISlackAlertConfig, PagerSeverity } from '../../types';
|
||||
import type { AlertHandler, AlertPayload } from './types';
|
||||
|
||||
const REQUEST_TIMEOUT_MS = 5000;
|
||||
export const DEFAULT_REPEAT_THROTTLE_MS = 15 * 60 * 1000;
|
||||
|
||||
/** Beyond this many tracked alarm ids, throttle state is pruned. */
|
||||
const MAX_THROTTLE_ENTRIES = 5000;
|
||||
|
||||
const MAX_FIELDS = 12;
|
||||
const MAX_FIELD_VALUE = 400;
|
||||
const MAX_TRACE = 1500;
|
||||
/** Values under this length sit two-per-row in the Slack attachment. */
|
||||
const SHORT_FIELD_LENGTH = 40;
|
||||
|
||||
const STYLE: Record<PagerSeverity, { emoji: string; color: string }> = {
|
||||
critical: { emoji: ':rotating_light:', color: '#d64545' },
|
||||
error: { emoji: ':red_circle:', color: '#e08d4c' },
|
||||
warning: { emoji: ':warning:', color: '#e0b84c' },
|
||||
info: { emoji: ':information_source:', color: '#4c8de0' },
|
||||
};
|
||||
|
||||
interface SlackField {
|
||||
title: string;
|
||||
value: string;
|
||||
short: boolean;
|
||||
}
|
||||
|
||||
export interface SlackMessage {
|
||||
text: string;
|
||||
channel?: string;
|
||||
username?: string;
|
||||
icon_emoji?: string;
|
||||
attachments: Array<{
|
||||
color: string;
|
||||
fallback: string;
|
||||
fields: SlackField[];
|
||||
footer: string;
|
||||
mrkdwn_in: string[];
|
||||
text?: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
function truncate(value: string, max: number): string {
|
||||
return value.length <= max ? value : `${value.slice(0, max - 1)}…`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render an alert as an incoming-webhook payload: severity-coloured attachment,
|
||||
* the alarm's fields as a table, and the stack (when there is one) as a code
|
||||
* block. Split out from the handler so the formatting is testable on its own.
|
||||
*/
|
||||
export function buildSlackMessage(
|
||||
alert: AlertPayload,
|
||||
opts: { channel?: string; username?: string; serverId?: string } = {},
|
||||
): SlackMessage {
|
||||
const style = STYLE[alert.severity] ?? STYLE.info;
|
||||
const repeat = alert.isRepeat ? ` (x${alert.repeatCount})` : '';
|
||||
const headline = `${style.emoji} *[${alert.severity.toUpperCase()}]* ${alert.message}${repeat}`;
|
||||
|
||||
const fields: SlackField[] = [];
|
||||
for (const [key, value] of Object.entries(alert.fields)) {
|
||||
if (fields.length >= MAX_FIELDS) break;
|
||||
// `error` is already rendered as the trace block below.
|
||||
if (key === 'error') continue;
|
||||
const rendered = truncate(value, MAX_FIELD_VALUE);
|
||||
fields.push({
|
||||
title: key,
|
||||
value: rendered,
|
||||
short: rendered.length <= SHORT_FIELD_LENGTH,
|
||||
});
|
||||
}
|
||||
|
||||
const footerParts = [alert.shortId, alert.id];
|
||||
if (opts.serverId) footerParts.push(opts.serverId);
|
||||
|
||||
return {
|
||||
text: headline,
|
||||
...(opts.channel ? { channel: opts.channel } : {}),
|
||||
...(opts.username ? { username: opts.username } : {}),
|
||||
attachments: [
|
||||
{
|
||||
color: style.color,
|
||||
fallback: `[${alert.severity}] ${alert.message}`,
|
||||
fields,
|
||||
footer: footerParts.join(' • '),
|
||||
mrkdwn_in: ['text'],
|
||||
...(alert.trace
|
||||
? { text: '```' + truncate(alert.trace, MAX_TRACE) + '```' }
|
||||
: {}),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Post alerts to a Slack incoming webhook. Repeats of the same alarm id are
|
||||
* throttled so a hot loop doesn't flood the channel — the occurrence count on
|
||||
* the next post that gets through tells the reader what they missed.
|
||||
*/
|
||||
export function createSlackAlertHandler(
|
||||
conf: ISlackAlertConfig,
|
||||
opts: { serverId?: string } = {},
|
||||
): AlertHandler {
|
||||
const webhookUrl = conf.webhookUrl as string;
|
||||
const throttleMs = conf.repeatThrottleMs ?? DEFAULT_REPEAT_THROTTLE_MS;
|
||||
const lastPosted = new Map<string, number>();
|
||||
|
||||
const shouldPost = (alert: AlertPayload): boolean => {
|
||||
if (throttleMs <= 0) return true;
|
||||
const now = Date.now();
|
||||
const previous = lastPosted.get(alert.id);
|
||||
if (previous !== undefined && now - previous < throttleMs) return false;
|
||||
|
||||
if (lastPosted.size >= MAX_THROTTLE_ENTRIES) {
|
||||
for (const [id, at] of lastPosted) {
|
||||
if (now - at >= throttleMs) lastPosted.delete(id);
|
||||
}
|
||||
// Still full of live entries — drop the oldest to stay bounded.
|
||||
if (lastPosted.size >= MAX_THROTTLE_ENTRIES) {
|
||||
const oldest = lastPosted.keys().next().value;
|
||||
if (oldest !== undefined) lastPosted.delete(oldest);
|
||||
}
|
||||
}
|
||||
lastPosted.set(alert.id, now);
|
||||
return true;
|
||||
};
|
||||
|
||||
return async (alert) => {
|
||||
if (!shouldPost(alert)) return;
|
||||
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
try {
|
||||
const res = await fetch(webhookUrl, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(
|
||||
buildSlackMessage(alert, {
|
||||
channel: conf.channel,
|
||||
username: conf.username,
|
||||
serverId: opts.serverId,
|
||||
}),
|
||||
),
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (!res.ok) {
|
||||
// A rejected post shouldn't leave the id marked as delivered.
|
||||
lastPosted.delete(alert.id);
|
||||
throw new Error(`Slack webhook returned ${res.status}`);
|
||||
}
|
||||
} catch (err) {
|
||||
lastPosted.delete(alert.id);
|
||||
throw err;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -1,95 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { PagerSeverity } from '../../types';
|
||||
|
||||
export interface AlarmFields {
|
||||
error?: Error;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface AlarmOccurrence {
|
||||
message: string;
|
||||
fields: AlarmFields;
|
||||
timestamp: number;
|
||||
}
|
||||
|
||||
export interface AlarmOptions {
|
||||
/**
|
||||
* Collapse repeats of this alarm id into a single incident on transports
|
||||
* that de-duplicate (PagerDuty). Only right when the id already pins down
|
||||
* one recurring fault — an uncaught error on a route, say — so that N
|
||||
* occurrences really are one thing to fix. Off by default: every other
|
||||
* alarm raises its own incident per occurrence.
|
||||
*/
|
||||
dedup?: boolean;
|
||||
}
|
||||
|
||||
export interface Alarm extends AlarmOptions {
|
||||
id: string;
|
||||
shortId: string;
|
||||
message: string;
|
||||
fields: AlarmFields;
|
||||
error?: Error;
|
||||
started: number;
|
||||
timestamps: number[];
|
||||
occurrences: AlarmOccurrence[];
|
||||
severity?: PagerSeverity;
|
||||
noAlert?: boolean;
|
||||
/** Set once a config mute has been logged, so it's reported only once. */
|
||||
muteLogged?: boolean;
|
||||
}
|
||||
|
||||
export interface AlertPayload {
|
||||
id: string;
|
||||
/**
|
||||
* Key a de-duplicating transport groups by. Equal to `id` for alarms raised
|
||||
* with `dedup`, and unique per occurrence for everything else.
|
||||
*/
|
||||
dedupKey: string;
|
||||
/** Readable slug for the same alarm, for humans quoting it back. */
|
||||
shortId: string;
|
||||
message: string;
|
||||
source: string;
|
||||
severity: PagerSeverity;
|
||||
/** Field values rendered as strings, ready to display. */
|
||||
fields: Record<string, string>;
|
||||
/** Stack of the attached error, when the alarm carried one. */
|
||||
trace?: string;
|
||||
/** Total occurrences of this alarm so far, including this one. */
|
||||
repeatCount: number;
|
||||
/** False for the first occurrence of an alarm id. */
|
||||
isRepeat: boolean;
|
||||
/** PagerDuty `custom_details` payload. */
|
||||
custom?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export type AlertHandler = (alert: AlertPayload) => Promise<void>;
|
||||
|
||||
export interface KnownErrorRule {
|
||||
match: {
|
||||
id: string;
|
||||
message?: string;
|
||||
fields?: Record<string, unknown>;
|
||||
};
|
||||
action: {
|
||||
type: 'no-alert' | 'severity';
|
||||
value?: PagerSeverity;
|
||||
};
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,247 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
|
||||
import { DatabaseClientFactory } from './index.js';
|
||||
import { MySQLDatabaseClient } from './MySQLDatabaseClient.js';
|
||||
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
|
||||
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
|
||||
|
||||
const config = (engine: string): IConfig =>
|
||||
({ port: 0, extensions: [], database: { engine } }) as IConfig;
|
||||
|
||||
const sqlite = () => new SqliteDatabaseClient(config('sqlite'));
|
||||
const mysql = () => new MySQLDatabaseClient(config('mysql'));
|
||||
const postgres = () => new PostgresDatabaseClient(config('postgres'));
|
||||
|
||||
/**
|
||||
* Minimal concrete client backed by fixture rows — exercises the shared helpers
|
||||
* on `AbstractDatabaseClient` that the real engines inherit.
|
||||
*/
|
||||
class FixtureDatabaseClient extends AbstractDatabaseClient {
|
||||
override readonly engineName = 'fixture';
|
||||
readonly reads: string[] = [];
|
||||
readonly preads: string[] = [];
|
||||
readonly writes: { query: string; params: unknown[] }[] = [];
|
||||
|
||||
constructor(
|
||||
private readonly replicaRows: Record<string, unknown>[] | Error,
|
||||
private readonly primaryRows: Record<string, unknown>[] = [],
|
||||
) {
|
||||
super(config('fixture'));
|
||||
}
|
||||
|
||||
override async read(query: string): Promise<Record<string, unknown>[]> {
|
||||
this.reads.push(query);
|
||||
if (this.replicaRows instanceof Error) throw this.replicaRows;
|
||||
return this.replicaRows;
|
||||
}
|
||||
|
||||
override async pread(query: string): Promise<Record<string, unknown>[]> {
|
||||
this.preads.push(query);
|
||||
return this.primaryRows;
|
||||
}
|
||||
|
||||
override async write(
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<WriteResult> {
|
||||
this.writes.push({ query, params });
|
||||
return { insertId: 1, affectedRows: 1, anyRowsAffected: true };
|
||||
}
|
||||
}
|
||||
|
||||
describe('DatabaseClientFactory', () => {
|
||||
it('picks sqlite when no engine is configured', () => {
|
||||
expect(
|
||||
new DatabaseClientFactory({ port: 0, extensions: [] } as IConfig),
|
||||
).toBeInstanceOf(SqliteDatabaseClient);
|
||||
});
|
||||
|
||||
it('refuses an unknown engine name', () => {
|
||||
expect(() => new DatabaseClientFactory(config('cassandra'))).toThrow(
|
||||
'Unknown database engine: cassandra',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AbstractDatabaseClient — unimplemented surface', () => {
|
||||
const bare = () => new AbstractDatabaseClient(config('none'));
|
||||
|
||||
it('refuses to run queries until a subclass implements them', async () => {
|
||||
await expect(bare().read('SELECT 1')).rejects.toThrow(
|
||||
'DatabaseClient.read() not implemented',
|
||||
);
|
||||
await expect(bare().pread('SELECT 1')).rejects.toThrow(
|
||||
'DatabaseClient.pread() not implemented',
|
||||
);
|
||||
await expect(bare().write('DELETE FROM x')).rejects.toThrow(
|
||||
'DatabaseClient.write() not implemented',
|
||||
);
|
||||
await expect(bare().batchWrite([])).rejects.toThrow(
|
||||
'DatabaseClient.batchWrite() not implemented',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AbstractDatabaseClient — replica-aware reads', () => {
|
||||
it('returns replica rows without waiting on the primary', async () => {
|
||||
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ id: 1 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('falls back to the primary when the replica has not caught up', async () => {
|
||||
const client = new FixtureDatabaseClient([], [{ id: 2 }]);
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ id: 2 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('falls back to the primary when the replica read throws', async () => {
|
||||
const client = new FixtureDatabaseClient(
|
||||
new Error('replica unavailable'),
|
||||
[{ id: 3 }],
|
||||
);
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ id: 3 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('names the failing query when a required read finds nothing', async () => {
|
||||
const client = new FixtureDatabaseClient([], []);
|
||||
await expect(
|
||||
client.requireRead('SELECT * FROM `user` WHERE `id` = ?'),
|
||||
).rejects.toThrow(
|
||||
'required read returned no rows: SELECT * FROM `user` WHERE `id` = ?',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AbstractDatabaseClient — SQL generation', () => {
|
||||
it('generates an INSERT with quoted columns and placeholders', async () => {
|
||||
const client = new FixtureDatabaseClient([]);
|
||||
await client.insert('user', { username: 'ada', email: null });
|
||||
|
||||
expect(client.writes).toEqual([
|
||||
{
|
||||
query: 'INSERT INTO `user` (`username`, `email`) VALUES (?, ?)',
|
||||
params: ['ada', null],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('quotes each dotted segment and escapes embedded backticks', () => {
|
||||
const client = sqlite();
|
||||
expect(client.quoteIdentifier('user')).toBe('`user`');
|
||||
expect(client.quoteIdentifier('db.user.id')).toBe('`db`.`user`.`id`');
|
||||
expect(client.quoteIdentifier('user.*')).toBe('`user`.*');
|
||||
expect(client.quoteIdentifier('we`ird')).toBe('`we``ird`');
|
||||
});
|
||||
|
||||
it('quotes postgres identifiers with double quotes', () => {
|
||||
const client = postgres();
|
||||
expect(client.quoteIdentifier('db.user')).toBe('"db"."user"');
|
||||
expect(client.quoteIdentifier('we"ird')).toBe('"we""ird"');
|
||||
expect(client.quoteIdentifier('user.*')).toBe('"user".*');
|
||||
});
|
||||
|
||||
it('renders booleans the way each engine expects', () => {
|
||||
expect(sqlite().booleanLiteral(true)).toBe('1');
|
||||
expect(sqlite().booleanValue(false)).toBe(0);
|
||||
expect(postgres().booleanLiteral(true)).toBe('TRUE');
|
||||
expect(postgres().booleanLiteral(false)).toBe('FALSE');
|
||||
expect(postgres().booleanValue(true)).toBe(true);
|
||||
});
|
||||
|
||||
it('picks the engine-specific ignore-conflict syntax', () => {
|
||||
expect(sqlite().insertIgnoreInto('kv')).toBe(
|
||||
'INSERT OR IGNORE INTO `kv`',
|
||||
);
|
||||
expect(sqlite().insertIgnoreSuffix()).toBe('');
|
||||
expect(postgres().insertIgnoreInto('kv')).toBe('INSERT INTO "kv"');
|
||||
expect(postgres().insertIgnoreSuffix()).toBe(' ON CONFLICT DO NOTHING');
|
||||
expect(mysql().insertIgnoreInto('kv')).toBe('INSERT IGNORE INTO `kv`');
|
||||
expect(mysql().insertIgnoreSuffix()).toBe('');
|
||||
});
|
||||
|
||||
it('builds an upsert clause per engine', () => {
|
||||
expect(mysql().upsertClause(['user_id'], ['value', 'dt'])).toBe(
|
||||
'ON DUPLICATE KEY UPDATE `value` = ?, `dt` = ?',
|
||||
);
|
||||
expect(sqlite().upsertClause(['user_id', 'app'], ['value'])).toBe(
|
||||
'ON CONFLICT(`user_id`, `app`) DO UPDATE SET `value` = ?',
|
||||
);
|
||||
expect(postgres().upsertClause(['user_id'], ['value'])).toBe(
|
||||
'ON CONFLICT("user_id") DO UPDATE SET "value" = ?',
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects an upsert with nothing to update', () => {
|
||||
expect(() => sqlite().upsertClause(['user_id'], [])).toThrow(
|
||||
'upsertClause requires at least one update column',
|
||||
);
|
||||
});
|
||||
|
||||
it('extracts JSON text using each engine dialect', () => {
|
||||
expect(sqlite().jsonTextExtract('`metadata`', ['a', 'b'])).toBe(
|
||||
"json_extract(`metadata`, '$.a.b')",
|
||||
);
|
||||
expect(mysql().jsonTextExtract('`metadata`', ['a'])).toBe(
|
||||
"JSON_UNQUOTE(JSON_EXTRACT(`metadata`, '$.a'))",
|
||||
);
|
||||
expect(postgres().jsonTextExtract('"metadata"', ['a', 'b'])).toBe(
|
||||
`"metadata" #>> ARRAY['a', 'b']`,
|
||||
);
|
||||
});
|
||||
|
||||
it('escapes quotes inside JSON path segments', () => {
|
||||
expect(sqlite().jsonTextExtract('`m`', ["it's"])).toBe(
|
||||
"json_extract(`m`, '$.it''s')",
|
||||
);
|
||||
});
|
||||
|
||||
it('coalesces expressions and rejects an empty list', () => {
|
||||
expect(sqlite().nullCoalesce('`a`', '`b`', '0')).toBe(
|
||||
'COALESCE(`a`, `b`, 0)',
|
||||
);
|
||||
expect(() => sqlite().nullCoalesce()).toThrow(
|
||||
'nullCoalesce requires at least one expression',
|
||||
);
|
||||
});
|
||||
|
||||
it('only postgres needs a RETURNING clause to learn the insert id', () => {
|
||||
expect(postgres().returningIdClause()).toBe(' RETURNING id');
|
||||
expect(sqlite().returningIdClause()).toBe('');
|
||||
expect(mysql().returningIdClause()).toBe('');
|
||||
});
|
||||
|
||||
it('falls back to `otherwise` for an engine with no explicit choice', () => {
|
||||
const client = new FixtureDatabaseClient([]);
|
||||
expect(
|
||||
client.case({ sqlite: 'a', mysql: 'b', otherwise: 'fallback' }),
|
||||
).toBe('fallback');
|
||||
expect(client.case({ fixture: 'exact', otherwise: 'fallback' })).toBe(
|
||||
'exact',
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -37,8 +37,8 @@ type SqlJsonPath = readonly [string, ...string[]];
|
||||
* Base database client. Subclasses must override every method that throws here.
|
||||
*
|
||||
* Do not instantiate directly — use the factory exported from
|
||||
* `clients/database/index.ts` which picks the right implementation based on
|
||||
* `config.database.engine`.
|
||||
* `clients/database/index.ts` which picks the right implementation
|
||||
* based on `config.database.engine`.
|
||||
*/
|
||||
export class AbstractDatabaseClient extends PuterClient {
|
||||
/** Short name used by `case()` to pick engine-specific values. */
|
||||
@@ -52,7 +52,9 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
// Abstract interface — subclasses MUST override
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
/** Execute a read query. Returns an array of row objects. */
|
||||
/**
|
||||
* Execute a read query. Returns an array of row objects.
|
||||
*/
|
||||
async read(
|
||||
_query: string,
|
||||
_params: unknown[] = [],
|
||||
@@ -61,9 +63,9 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Read that prefers the primary database (useful when read-replicas may
|
||||
* have replication lag). In single-node setups this is identical to
|
||||
* `read()`.
|
||||
* Read that prefers the primary database (useful when read-replicas
|
||||
* may have replication lag). In single-node setups this is identical
|
||||
* to `read()`.
|
||||
*/
|
||||
async pread(
|
||||
_query: string,
|
||||
@@ -72,12 +74,16 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
throw new Error('DatabaseClient.pread() not implemented');
|
||||
}
|
||||
|
||||
/** Execute a write query (INSERT / UPDATE / DELETE). */
|
||||
/**
|
||||
* Execute a write query (INSERT / UPDATE / DELETE).
|
||||
*/
|
||||
async write(_query: string, _params: unknown[] = []): Promise<WriteResult> {
|
||||
throw new Error('DatabaseClient.write() not implemented');
|
||||
}
|
||||
|
||||
/** Execute multiple write statements in a single transaction. */
|
||||
/**
|
||||
* Execute multiple write statements in a single transaction.
|
||||
*/
|
||||
async batchWrite(_entries: BatchEntry[]): Promise<void> {
|
||||
throw new Error('DatabaseClient.batchWrite() not implemented');
|
||||
}
|
||||
@@ -105,9 +111,9 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Like `read()` but falls back to the primary when read-replicas are in
|
||||
* use. Subclasses may override with replica-aware logic; the default
|
||||
* delegates to `pread()`.
|
||||
* Like `read()` but falls back to the primary when read-replicas are
|
||||
* in use. Subclasses may override with replica-aware logic; the
|
||||
* default delegates to `pread()`.
|
||||
*/
|
||||
async tryHardRead(
|
||||
query: string,
|
||||
@@ -127,7 +133,9 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
return primary;
|
||||
}
|
||||
|
||||
/** Like `tryHardRead()` but throws when the result set is empty. */
|
||||
/**
|
||||
* Like `tryHardRead()` but throws when the result set is empty.
|
||||
*/
|
||||
async requireRead(
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
@@ -143,12 +151,9 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
* Return the value from `choices` that matches the current engine.
|
||||
*
|
||||
* Usage:
|
||||
*
|
||||
* db.case({
|
||||
* sqlite: "datetime('now')",
|
||||
* mysql: 'NOW()',
|
||||
* otherwise: 'NOW()',
|
||||
* });
|
||||
* ```
|
||||
* db.case({ sqlite: "datetime('now')", mysql: 'NOW()', otherwise: 'NOW()' })
|
||||
* ```
|
||||
*
|
||||
* If the engine name isn't present in `choices`, falls back to
|
||||
* `choices.otherwise`.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -17,95 +17,13 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import {
|
||||
MySQLDatabaseClient,
|
||||
compareMigrationFilenames,
|
||||
} from './MySQLDatabaseClient.js';
|
||||
|
||||
interface FakeConnection {
|
||||
query: ReturnType<typeof vi.fn>;
|
||||
execute: ReturnType<typeof vi.fn>;
|
||||
beginTransaction: ReturnType<typeof vi.fn>;
|
||||
commit: ReturnType<typeof vi.fn>;
|
||||
rollback: ReturnType<typeof vi.fn>;
|
||||
release: ReturnType<typeof vi.fn>;
|
||||
}
|
||||
|
||||
interface FakePool {
|
||||
poolConfig: Record<string, unknown>;
|
||||
connectionHandlers: ((conn: unknown) => void)[];
|
||||
connections: FakeConnection[];
|
||||
calls: { sql: string; values?: unknown[] }[];
|
||||
ended: boolean;
|
||||
endError: unknown;
|
||||
endThrows: unknown;
|
||||
respond: (sql: string, values?: unknown[]) => unknown;
|
||||
on: (event: string, handler: (conn: unknown) => void) => FakePool;
|
||||
promise: () => { getConnection: () => Promise<FakeConnection> };
|
||||
end: (cb: (err?: unknown) => void) => void;
|
||||
}
|
||||
|
||||
// mysql2 is the wire boundary: stub the driver's pool so we can assert on
|
||||
// the exact SQL, parameters and connection choreography the adapter emits.
|
||||
const { createPoolMock, createdPools } = vi.hoisted(() => {
|
||||
const createdPools: FakePool[] = [];
|
||||
|
||||
const createPoolMock = vi.fn((poolConfig: Record<string, unknown>) => {
|
||||
const pool: FakePool = {
|
||||
poolConfig,
|
||||
connectionHandlers: [],
|
||||
connections: [],
|
||||
calls: [],
|
||||
ended: false,
|
||||
endError: null,
|
||||
endThrows: null,
|
||||
respond: () => [[], undefined],
|
||||
on(event, handler) {
|
||||
if (event === 'connection') {
|
||||
pool.connectionHandlers.push(handler);
|
||||
}
|
||||
return pool;
|
||||
},
|
||||
promise() {
|
||||
return {
|
||||
getConnection: async () => {
|
||||
const run = async (sql: string, values?: unknown[]) => {
|
||||
pool.calls.push({ sql, values });
|
||||
return pool.respond(sql, values);
|
||||
};
|
||||
const conn: FakeConnection = {
|
||||
query: vi.fn(run),
|
||||
execute: vi.fn(run),
|
||||
beginTransaction: vi.fn(async () => {}),
|
||||
commit: vi.fn(async () => {}),
|
||||
rollback: vi.fn(async () => {}),
|
||||
release: vi.fn(),
|
||||
};
|
||||
pool.connections.push(conn);
|
||||
return conn;
|
||||
},
|
||||
};
|
||||
},
|
||||
end(cb) {
|
||||
if (pool.endThrows) throw pool.endThrows;
|
||||
pool.ended = true;
|
||||
cb(pool.endError ?? undefined);
|
||||
},
|
||||
};
|
||||
createdPools.push(pool);
|
||||
return pool;
|
||||
});
|
||||
|
||||
return { createPoolMock, createdPools };
|
||||
});
|
||||
|
||||
vi.mock('mysql2', () => ({ createPool: createPoolMock }));
|
||||
|
||||
describe('compareMigrationFilenames', () => {
|
||||
it('orders numbered migrations numerically, not lexically', () => {
|
||||
const input = [
|
||||
@@ -214,9 +132,7 @@ const codedError = (code: string) => {
|
||||
|
||||
describe('MySQLDatabaseClient.read — replica failover', () => {
|
||||
it('fails over to the primary on batcher load-shed errors', async () => {
|
||||
const replica = {
|
||||
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
|
||||
};
|
||||
const replica = { execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')) };
|
||||
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
|
||||
const client = makeClient({ replica, primary });
|
||||
|
||||
@@ -225,9 +141,7 @@ describe('MySQLDatabaseClient.read — replica failover', () => {
|
||||
});
|
||||
|
||||
it('fails over on transient connection errors', async () => {
|
||||
const replica = {
|
||||
execute: vi.fn().mockRejectedValue(codedError('ECONNRESET')),
|
||||
};
|
||||
const replica = { execute: vi.fn().mockRejectedValue(codedError('ECONNRESET')) };
|
||||
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
|
||||
const client = makeClient({ replica, primary });
|
||||
|
||||
@@ -235,9 +149,7 @@ describe('MySQLDatabaseClient.read — replica failover', () => {
|
||||
});
|
||||
|
||||
it('rethrows deterministic SQL errors without touching the primary', async () => {
|
||||
const replica = {
|
||||
execute: vi.fn().mockRejectedValue(codedError('ER_PARSE_ERROR')),
|
||||
};
|
||||
const replica = { execute: vi.fn().mockRejectedValue(codedError('ER_PARSE_ERROR')) };
|
||||
const primary = { execute: vi.fn() };
|
||||
const client = makeClient({ replica, primary });
|
||||
|
||||
@@ -248,9 +160,7 @@ describe('MySQLDatabaseClient.read — replica failover', () => {
|
||||
});
|
||||
|
||||
it('does not fail over in single-node configuration', async () => {
|
||||
const replica = {
|
||||
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
|
||||
};
|
||||
const replica = { execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')) };
|
||||
const primary = { execute: vi.fn() };
|
||||
const client = makeClient({ replica, primary, multiNode: false });
|
||||
|
||||
@@ -260,588 +170,3 @@ describe('MySQLDatabaseClient.read — replica failover', () => {
|
||||
expect(primary.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// -- Driver-level behaviour (stubbed mysql2 pool) ---------------------
|
||||
|
||||
const mysqlConfig = (
|
||||
database: Partial<NonNullable<IConfig['database']>> = {},
|
||||
): IConfig =>
|
||||
({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
database: { engine: 'mysql', ...database },
|
||||
}) as IConfig;
|
||||
|
||||
const startClient = async (
|
||||
database: Partial<NonNullable<IConfig['database']>> = {},
|
||||
): Promise<MySQLDatabaseClient> => {
|
||||
const client = new MySQLDatabaseClient(mysqlConfig(database));
|
||||
await client.onServerStart();
|
||||
return client;
|
||||
};
|
||||
|
||||
// The batcher coalesces a batch into `stmt1;stmt2; SELECT 1`, so the driver
|
||||
// must answer with one row-set per statement plus one for the SELECT 1.
|
||||
const rowSets = (...sets: unknown[][]) => [[...sets, [{ 1: 1 }]], undefined];
|
||||
|
||||
beforeEach(() => {
|
||||
createdPools.length = 0;
|
||||
createPoolMock.mockClear();
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — pool construction', () => {
|
||||
it('builds the primary pool from config with multi-statement support', async () => {
|
||||
await startClient({
|
||||
host: 'db.internal',
|
||||
port: 3307,
|
||||
user: 'puter',
|
||||
password: 'hunter2',
|
||||
database: 'puterdb',
|
||||
});
|
||||
|
||||
expect(createPoolMock).toHaveBeenCalledTimes(1);
|
||||
expect(createdPools[0].poolConfig).toEqual({
|
||||
maxPreparedStatements: 900,
|
||||
connectionLimit: 30,
|
||||
enableKeepAlive: true,
|
||||
host: 'db.internal',
|
||||
port: 3307,
|
||||
user: 'puter',
|
||||
password: 'hunter2',
|
||||
database: 'puterdb',
|
||||
multipleStatements: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('falls back to loopback defaults when the endpoint is unspecified', async () => {
|
||||
await startClient();
|
||||
|
||||
expect(createdPools[0].poolConfig).toMatchObject({
|
||||
host: '127.0.0.1',
|
||||
port: 3306,
|
||||
user: 'root',
|
||||
password: '',
|
||||
database: 'puter',
|
||||
});
|
||||
});
|
||||
|
||||
it('arms a server-side statement timeout on every new connection', async () => {
|
||||
await startClient({ selectTimeoutMs: 12_000 });
|
||||
|
||||
const pool = createdPools[0];
|
||||
expect(pool.connectionHandlers).toHaveLength(1);
|
||||
|
||||
const conn = { query: vi.fn() };
|
||||
pool.connectionHandlers[0](conn);
|
||||
expect(conn.query).toHaveBeenCalledWith(
|
||||
'SET SESSION max_execution_time = 12000',
|
||||
);
|
||||
});
|
||||
|
||||
it('omits the statement timeout when configured to 0', async () => {
|
||||
await startClient({ selectTimeoutMs: 0 });
|
||||
expect(createdPools[0].connectionHandlers).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('shares one pool between reads and writes without a replica', async () => {
|
||||
await startClient();
|
||||
expect(createPoolMock).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('creates a second pool when a read-replica is configured', async () => {
|
||||
await startClient({
|
||||
replica: { host: 'replica.internal', port: 3306 },
|
||||
});
|
||||
|
||||
expect(createPoolMock).toHaveBeenCalledTimes(2);
|
||||
expect(createdPools[1].poolConfig).toMatchObject({
|
||||
host: 'replica.internal',
|
||||
multipleStatements: true,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — query interface', () => {
|
||||
it('sends the query and its parameters through to the driver', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
pool.respond = () => rowSets([{ id: 1, username: 'ada' }]);
|
||||
|
||||
await expect(
|
||||
client.read('SELECT * FROM `user` WHERE `id` = ?', [1]),
|
||||
).resolves.toEqual([{ id: 1, username: 'ada' }]);
|
||||
|
||||
expect(pool.calls).toEqual([
|
||||
{
|
||||
sql: 'SELECT * FROM `user` WHERE `id` = ?; SELECT 1',
|
||||
values: [1],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('returns an empty array when the driver yields no row-set', async () => {
|
||||
const client = await startClient();
|
||||
createdPools[0].respond = () => [[], undefined];
|
||||
|
||||
await expect(client.read('SELECT 1')).resolves.toEqual([]);
|
||||
await expect(client.pread('SELECT 1')).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it('reads from the replica pool and primary-reads from the primary', async () => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
const [primary, replica] = createdPools;
|
||||
primary.respond = () => rowSets([{ from: 'primary' }]);
|
||||
replica.respond = () => rowSets([{ from: 'replica' }]);
|
||||
|
||||
await expect(client.read('SELECT 1')).resolves.toEqual([
|
||||
{ from: 'replica' },
|
||||
]);
|
||||
await expect(client.pread('SELECT 1')).resolves.toEqual([
|
||||
{ from: 'primary' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('maps the mysql result header onto the shared write result', async () => {
|
||||
const client = await startClient();
|
||||
createdPools[0].respond = () =>
|
||||
rowSets({ insertId: 42, affectedRows: 2 } as unknown as unknown[]);
|
||||
|
||||
await expect(
|
||||
client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
|
||||
'ada',
|
||||
1,
|
||||
]),
|
||||
).resolves.toEqual({
|
||||
insertId: 42,
|
||||
affectedRows: 2,
|
||||
anyRowsAffected: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('reports no rows affected when the header omits the counters', async () => {
|
||||
const client = await startClient();
|
||||
createdPools[0].respond = () => rowSets({} as unknown as unknown[]);
|
||||
|
||||
await expect(client.write('DELETE FROM `user`')).resolves.toEqual({
|
||||
insertId: 0,
|
||||
affectedRows: 0,
|
||||
anyRowsAffected: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('builds an INSERT with quoted identifiers and positional params', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
pool.respond = () =>
|
||||
rowSets({ insertId: 7, affectedRows: 1 } as unknown as unknown[]);
|
||||
|
||||
await expect(
|
||||
client.insert('user', { username: 'ada', email: null }),
|
||||
).resolves.toMatchObject({ insertId: 7 });
|
||||
|
||||
expect(pool.calls[0]).toEqual({
|
||||
sql:
|
||||
'INSERT INTO `user` (`username`, `email`) VALUES (?, ?); ' +
|
||||
'SELECT 1',
|
||||
values: ['ada', null],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — batchWrite transactions', () => {
|
||||
it('runs every statement on one connection inside a transaction', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
|
||||
await client.batchWrite([
|
||||
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
|
||||
{
|
||||
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
|
||||
values: ['s'],
|
||||
},
|
||||
]);
|
||||
|
||||
expect(pool.connections).toHaveLength(1);
|
||||
const conn = pool.connections[0];
|
||||
expect(conn.beginTransaction).toHaveBeenCalledTimes(1);
|
||||
expect(conn.execute.mock.calls).toEqual([
|
||||
['UPDATE `user` SET `x` = ?', [1]],
|
||||
['DELETE FROM `sessions` WHERE `uuid` = ?', ['s']],
|
||||
]);
|
||||
expect(conn.commit).toHaveBeenCalledTimes(1);
|
||||
expect(conn.rollback).not.toHaveBeenCalled();
|
||||
expect(conn.release).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('rolls back and releases the connection when a statement fails', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
pool.respond = (sql) => {
|
||||
if (sql.startsWith('DELETE')) throw new Error('constraint blew up');
|
||||
return [[], undefined];
|
||||
};
|
||||
|
||||
await expect(
|
||||
client.batchWrite([
|
||||
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
|
||||
{ statement: 'DELETE FROM `user`', values: [] },
|
||||
]),
|
||||
).rejects.toThrow('constraint blew up');
|
||||
|
||||
const conn = pool.connections[0];
|
||||
expect(conn.commit).not.toHaveBeenCalled();
|
||||
expect(conn.rollback).toHaveBeenCalledTimes(1);
|
||||
expect(conn.release).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('still rethrows the original failure when the rollback also fails', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
pool.respond = () => {
|
||||
throw new Error('statement blew up');
|
||||
};
|
||||
|
||||
const original = client.batchWrite([
|
||||
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
|
||||
]);
|
||||
// The rollback is attached after the connection is handed out, so
|
||||
// patch it once the adapter has acquired one.
|
||||
await Promise.resolve();
|
||||
pool.connections[0]?.rollback.mockRejectedValue(
|
||||
new Error('rollback blew up'),
|
||||
);
|
||||
|
||||
await expect(original).rejects.toThrow('statement blew up');
|
||||
expect(pool.connections[0].release).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('never acquires a connection for an empty batch', async () => {
|
||||
const client = await startClient();
|
||||
await client.batchWrite([]);
|
||||
expect(createdPools[0].connections).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — tryHardRead', () => {
|
||||
it('issues exactly one query in single-node configuration', async () => {
|
||||
const client = await startClient();
|
||||
const pool = createdPools[0];
|
||||
pool.respond = () => rowSets([{ id: 1 }]);
|
||||
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ id: 1 },
|
||||
]);
|
||||
expect(pool.calls).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('prefers replica rows when the replica has them', async () => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
const [primary, replica] = createdPools;
|
||||
primary.respond = () => rowSets([{ from: 'primary' }]);
|
||||
replica.respond = () => rowSets([{ from: 'replica' }]);
|
||||
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ from: 'replica' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('falls back to the primary when the replica is behind', async () => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
const [primary, replica] = createdPools;
|
||||
primary.respond = () => rowSets([{ from: 'primary' }]);
|
||||
replica.respond = () => rowSets([]);
|
||||
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ from: 'primary' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('falls back to the primary when the replica query throws', async () => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
const [primary, replica] = createdPools;
|
||||
primary.respond = () => rowSets([{ from: 'primary' }]);
|
||||
replica.respond = () => {
|
||||
throw new Error('replica down');
|
||||
};
|
||||
|
||||
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
|
||||
{ from: 'primary' },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — migrations', () => {
|
||||
let dir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(join(tmpdir(), 'puter-mysql-mig-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('applies mysql migrations in numeric order, one statement at a time', async () => {
|
||||
writeFileSync(
|
||||
join(dir, 'mysql_mig_2.sql'),
|
||||
'CREATE TABLE b (id INT);\nCREATE TABLE c (id INT);\n',
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, 'mysql_mig_10.sql'),
|
||||
'CREATE TABLE d (id INT);',
|
||||
);
|
||||
// Other engines' files and non-SQL noise must be ignored.
|
||||
writeFileSync(join(dir, 'postgres_mig_1.sql'), 'CREATE TABLE nope ();');
|
||||
writeFileSync(join(dir, 'mysql_notes.txt'), 'not sql');
|
||||
|
||||
await startClient({ migrationPaths: [dir] });
|
||||
|
||||
const pool = createdPools[0];
|
||||
expect(pool.calls.map((c) => c.sql)).toEqual([
|
||||
'CREATE TABLE b (id INT)',
|
||||
'CREATE TABLE c (id INT)',
|
||||
'CREATE TABLE d (id INT)',
|
||||
]);
|
||||
expect(pool.connections[0].release).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('reports which file and statement index failed', async () => {
|
||||
writeFileSync(
|
||||
join(dir, 'mysql_mig_1.sql'),
|
||||
'CREATE TABLE ok (id INT);\nCREATE TABLE bad (id INT);',
|
||||
);
|
||||
|
||||
const client = new MySQLDatabaseClient(
|
||||
mysqlConfig({ migrationPaths: [dir] }),
|
||||
);
|
||||
const cause = new Error('ER_PARSE_ERROR');
|
||||
// The pool only exists once onServerStart runs, so arm the failure
|
||||
// through the factory.
|
||||
createPoolMock.mockImplementationOnce((poolConfig) => {
|
||||
const pool = createPoolMock.getMockImplementation()!(poolConfig);
|
||||
pool.respond = (sql: string) => {
|
||||
if (sql.includes('bad')) throw cause;
|
||||
return [[], undefined];
|
||||
};
|
||||
return pool;
|
||||
});
|
||||
|
||||
await expect(client.onServerStart()).rejects.toThrow(
|
||||
'[mysql] failed to apply mysql_mig_1.sql at statement 1',
|
||||
);
|
||||
});
|
||||
|
||||
it('fails loudly when a configured migration path is unreadable', async () => {
|
||||
await expect(
|
||||
startClient({ migrationPaths: [join(dir, 'does-not-exist')] }),
|
||||
).rejects.toThrow('[mysql] migration path is unreadable');
|
||||
});
|
||||
|
||||
it('skips a configured directory that holds no mysql migrations', async () => {
|
||||
writeFileSync(join(dir, 'readme.md'), '# nothing here');
|
||||
await startClient({ migrationPaths: [dir] });
|
||||
expect(createdPools[0].calls).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient — pool lifecycle', () => {
|
||||
it('replaces and closes the previous primary pool on reinit', async () => {
|
||||
const client = await startClient();
|
||||
const original = createdPools[0];
|
||||
|
||||
client.reinitPrimary();
|
||||
|
||||
expect(createdPools).toHaveLength(2);
|
||||
expect(original.ended).toBe(true);
|
||||
// Single-node: the replica batcher must follow the new primary.
|
||||
createdPools[1].respond = () => rowSets([{ from: 'new-primary' }]);
|
||||
await expect(client.read('SELECT 1')).resolves.toEqual([
|
||||
{ from: 'new-primary' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('replaces only the replica pool on replica reinit', async () => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
const [primary, originalReplica] = createdPools;
|
||||
|
||||
client.reinitReplica();
|
||||
|
||||
expect(createdPools).toHaveLength(3);
|
||||
expect(originalReplica.ended).toBe(true);
|
||||
expect(primary.ended).toBe(false);
|
||||
});
|
||||
|
||||
it('ignores a replica reinit when no replica is configured', async () => {
|
||||
const client = await startClient();
|
||||
client.reinitReplica();
|
||||
expect(createdPools).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('ignores reinit once shutdown has started', async () => {
|
||||
const client = await startClient();
|
||||
await client.onServerPrepareShutdown();
|
||||
client.reinitPrimary();
|
||||
expect(createdPools).toHaveLength(1);
|
||||
await client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('closes both pools on shutdown when a replica is configured', async () => {
|
||||
const client = await startClient({ replica: { host: 'replica' } });
|
||||
await client.onServerShutdown();
|
||||
|
||||
expect(createdPools[0].ended).toBe(true);
|
||||
expect(createdPools[1].ended).toBe(true);
|
||||
});
|
||||
|
||||
it('closes the pools once the drain window elapses', async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const client = await startClient();
|
||||
await client.onServerPrepareShutdown();
|
||||
expect(createdPools[0].ended).toBe(false);
|
||||
|
||||
await vi.advanceTimersByTimeAsync(60_000);
|
||||
expect(createdPools[0].ended).toBe(true);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('cancels the drain timer when shutdown arrives first', async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const client = await startClient();
|
||||
await client.onServerPrepareShutdown();
|
||||
// A second prepare is a no-op — only one drain timer may exist.
|
||||
await client.onServerPrepareShutdown();
|
||||
expect(vi.getTimerCount()).toBe(1);
|
||||
|
||||
await client.onServerShutdown();
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
expect(createdPools[0].ended).toBe(true);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('surfaces a driver error raised while closing the pool', async () => {
|
||||
const client = await startClient();
|
||||
createdPools[0].endError = new Error('pool refused to close');
|
||||
|
||||
await expect(client.onServerShutdown()).rejects.toThrow(
|
||||
'pool refused to close',
|
||||
);
|
||||
});
|
||||
|
||||
it('surfaces a driver error thrown synchronously by end()', async () => {
|
||||
const client = await startClient();
|
||||
createdPools[0].endThrows = new Error('end exploded');
|
||||
|
||||
await expect(client.onServerShutdown()).rejects.toThrow('end exploded');
|
||||
});
|
||||
});
|
||||
|
||||
describe('MySQLDatabaseClient.readWithRetry', () => {
|
||||
const codedFailure = (code: string) => {
|
||||
const err = new Error(code) as Error & { code: string };
|
||||
err.code = code;
|
||||
return err;
|
||||
};
|
||||
|
||||
it('classifies transient connection failures as retriable', () => {
|
||||
expect(
|
||||
MySQLDatabaseClient.isRetriableError(codedFailure('ECONNRESET')),
|
||||
).toBe(true);
|
||||
expect(
|
||||
MySQLDatabaseClient.isRetriableError(codedFailure('ER_DUP_ENTRY')),
|
||||
).toBe(false);
|
||||
expect(
|
||||
MySQLDatabaseClient.isRetriableError(new Error('Connection lost')),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('retries a transient failure and returns the eventual result', async () => {
|
||||
const client = await startClient();
|
||||
let attempts = 0;
|
||||
const operation = vi.fn(async () => {
|
||||
attempts += 1;
|
||||
if (attempts < 3) throw codedFailure('PROTOCOL_CONNECTION_LOST');
|
||||
return [{ ok: 1 }];
|
||||
});
|
||||
|
||||
await expect(
|
||||
client.readWithRetry('health', operation, { baseBackoffMs: 0 }),
|
||||
).resolves.toEqual([{ ok: 1 }]);
|
||||
expect(operation).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it('gives up after the attempt budget is spent', async () => {
|
||||
const client = await startClient();
|
||||
const operation = vi.fn(async () => {
|
||||
throw codedFailure('ETIMEDOUT');
|
||||
});
|
||||
|
||||
await expect(
|
||||
client.readWithRetry('health', operation, {
|
||||
maxAttempts: 2,
|
||||
baseBackoffMs: 0,
|
||||
}),
|
||||
).rejects.toMatchObject({ code: 'ETIMEDOUT' });
|
||||
expect(operation).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('never retries a deterministic SQL error', async () => {
|
||||
const client = await startClient();
|
||||
const operation = vi.fn(async () => {
|
||||
throw codedFailure('ER_PARSE_ERROR');
|
||||
});
|
||||
|
||||
await expect(
|
||||
client.readWithRetry('health', operation, { baseBackoffMs: 0 }),
|
||||
).rejects.toMatchObject({ code: 'ER_PARSE_ERROR' });
|
||||
expect(operation).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('stops retrying once shutdown has started', async () => {
|
||||
const client = await startClient();
|
||||
await client.onServerPrepareShutdown();
|
||||
const operation = vi.fn(async () => {
|
||||
throw codedFailure('ECONNRESET');
|
||||
});
|
||||
|
||||
await expect(
|
||||
client.readWithRetry('health', operation, { baseBackoffMs: 0 }),
|
||||
).rejects.toMatchObject({ code: 'ECONNRESET' });
|
||||
expect(operation).toHaveBeenCalledTimes(1);
|
||||
await client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('applies jittered exponential backoff capped at maxBackoffMs', async () => {
|
||||
const client = await startClient();
|
||||
const delays: number[] = [];
|
||||
const realSetTimeout = globalThis.setTimeout;
|
||||
const timeoutSpy = vi
|
||||
.spyOn(globalThis, 'setTimeout')
|
||||
.mockImplementation(((fn: () => void, ms?: number) => {
|
||||
delays.push(ms ?? 0);
|
||||
return realSetTimeout(fn, 0);
|
||||
}) as typeof setTimeout);
|
||||
|
||||
try {
|
||||
const operation = vi.fn(async () => {
|
||||
throw codedFailure('ECONNRESET');
|
||||
});
|
||||
await expect(
|
||||
client.readWithRetry('health', operation, {
|
||||
maxAttempts: 4,
|
||||
baseBackoffMs: 100,
|
||||
maxBackoffMs: 150,
|
||||
jitterRatio: 0,
|
||||
}),
|
||||
).rejects.toMatchObject({ code: 'ECONNRESET' });
|
||||
expect(delays).toEqual([100, 150, 150]);
|
||||
} finally {
|
||||
timeoutSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -248,9 +248,9 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
|
||||
/**
|
||||
* Apply `.sql` files from each configured migration directory in order.
|
||||
* Files within a directory are sorted lexically. Files MUST be idempotent —
|
||||
* there is no per-file applied-state tracking. Failures abort startup so
|
||||
* operators see schema problems loud.
|
||||
* Files within a directory are sorted lexically. Files MUST be
|
||||
* idempotent — there is no per-file applied-state tracking. Failures
|
||||
* abort startup so operators see schema problems loud.
|
||||
*/
|
||||
private async runMigrations(): Promise<void> {
|
||||
const paths = this.config.database?.migrationPaths;
|
||||
@@ -414,10 +414,8 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
|
||||
return isRetriableError(error);
|
||||
}
|
||||
|
||||
/**
|
||||
* Replica failures worth retrying on the primary: batcher load-shed or
|
||||
* transient connection errors — never deterministic SQL errors.
|
||||
*/
|
||||
/** Replica failures worth retrying on the primary: batcher load-shed
|
||||
* or transient connection errors — never deterministic SQL errors. */
|
||||
private static isFailoverWorthy(error: unknown): boolean {
|
||||
const code = (error as { code?: string })?.code;
|
||||
return code === 'dbBatchFailed' || isRetriableError(error);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -89,22 +89,20 @@ export class SQLBatcher {
|
||||
#metricAttrs;
|
||||
|
||||
/**
|
||||
* @param {object} dbPool Mysql2 pool
|
||||
* @param {object} dbPool mysql2 pool
|
||||
* @param {object} [opts]
|
||||
* @param {number} [opts.maxTimeInQueue] Ms an item may wait before flush
|
||||
* @param {number} [opts.maxBatchSize] Items coalesced per flush
|
||||
* @param {number} [opts.maxQueueSize] Drop-oldest high-water mark
|
||||
* @param {number} [opts.failureThreshold] Consecutive failures to open the
|
||||
* breaker
|
||||
* @param {number} [opts.cooldownMs] Breaker open duration after last
|
||||
* failure
|
||||
* @param {'primary' | 'replica'} [opts.poolLabel] Role label on metrics; in
|
||||
* @param {number} [opts.maxTimeInQueue] ms an item may wait before flush
|
||||
* @param {number} [opts.maxBatchSize] items coalesced per flush
|
||||
* @param {number} [opts.maxQueueSize] drop-oldest high-water mark
|
||||
* @param {number} [opts.failureThreshold] consecutive failures to open the breaker
|
||||
* @param {number} [opts.cooldownMs] breaker open duration after last failure
|
||||
* @param {'primary'|'replica'} [opts.poolLabel] role label on metrics; in
|
||||
* single-node setups the 'replica' batcher shares the primary pool, so
|
||||
* this reflects the read/write role rather than a physical instance
|
||||
* @param {boolean} [opts.readOnly] This batcher only ever carries SELECTs,
|
||||
* @param {boolean} [opts.readOnly] this batcher only ever carries SELECTs,
|
||||
* so any transient failure is safe to retry
|
||||
* @param {number} [opts.acquireTimeoutMs] Max wait for a pooled connection;
|
||||
* 0 disables the bound
|
||||
* @param {number} [opts.acquireTimeoutMs] max wait for a pooled
|
||||
* connection; 0 disables the bound
|
||||
*/
|
||||
constructor(
|
||||
dbPool,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,437 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import Database from 'better-sqlite3';
|
||||
import { existsSync, mkdtempSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import { DatabaseClientFactory } from './index.js';
|
||||
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
|
||||
|
||||
/** Highest schema version the migration table can reach. */
|
||||
const CURRENT_SCHEMA_VERSION = 60;
|
||||
const SYSTEM_USER_UUID = '5d4adce0-a381-4982-9c02-6e2540026238';
|
||||
|
||||
const sqliteConfig = (
|
||||
database: Partial<NonNullable<IConfig['database']>> = {},
|
||||
): IConfig =>
|
||||
({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
database: { engine: 'sqlite', inMemory: true, ...database },
|
||||
}) as IConfig;
|
||||
|
||||
const bootClient = async (
|
||||
database: Partial<NonNullable<IConfig['database']>> = {},
|
||||
): Promise<SqliteDatabaseClient> => {
|
||||
const client = new SqliteDatabaseClient(sqliteConfig(database));
|
||||
await client.onServerStart();
|
||||
return client;
|
||||
};
|
||||
|
||||
const userVersionOf = async (client: SqliteDatabaseClient): Promise<number> => {
|
||||
const [row] = await client.read('PRAGMA user_version');
|
||||
return row.user_version as number;
|
||||
};
|
||||
|
||||
describe('SqliteDatabaseClient — boot and migrations', () => {
|
||||
let client: SqliteDatabaseClient;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await bootClient();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('is what the factory picks for the sqlite engine', () => {
|
||||
expect(new DatabaseClientFactory(sqliteConfig())).toBeInstanceOf(
|
||||
SqliteDatabaseClient,
|
||||
);
|
||||
});
|
||||
|
||||
it('migrates a fresh database all the way to the current version', async () => {
|
||||
expect(await userVersionOf(client)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
});
|
||||
|
||||
it('runs the javascript migrations, not just the .sql ones', async () => {
|
||||
// The `system` user only exists because 0025 (a .dbmig.js file) ran
|
||||
// inside the migration VM.
|
||||
const rows = await client.read(
|
||||
'SELECT `username` FROM `user` WHERE `uuid` = ?',
|
||||
[SYSTEM_USER_UUID],
|
||||
);
|
||||
expect(rows).toEqual([{ username: 'system' }]);
|
||||
});
|
||||
|
||||
it('leaves an already-migrated database untouched on a second boot', async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
|
||||
const path = join(dir, 'nested', 'puter.sqlite');
|
||||
try {
|
||||
const first = new SqliteDatabaseClient(
|
||||
sqliteConfig({ inMemory: false, path }),
|
||||
);
|
||||
await first.onServerStart();
|
||||
await first.write(
|
||||
'INSERT INTO `kv` (`user_id`, `kkey_hash`, `kkey`, `value`) ' +
|
||||
'VALUES (?, ?, ?, ?)',
|
||||
[1, 1, 'boot-marker', '"kept"'],
|
||||
);
|
||||
first.onServerShutdown();
|
||||
|
||||
expect(existsSync(path)).toBe(true);
|
||||
|
||||
const second = new SqliteDatabaseClient(
|
||||
sqliteConfig({ inMemory: false, path }),
|
||||
);
|
||||
await second.onServerStart();
|
||||
await expect(
|
||||
second.read('SELECT `value` FROM `kv` WHERE `kkey` = ?', [
|
||||
'boot-marker',
|
||||
]),
|
||||
).resolves.toEqual([{ value: '"kept"' }]);
|
||||
expect(await userVersionOf(second)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
second.onServerShutdown();
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('stops early at a configured target version', async () => {
|
||||
const partial = await bootClient({ targetVersion: 5 });
|
||||
try {
|
||||
expect(await userVersionOf(partial)).toBe(5);
|
||||
// 0005 landed (apps.background); 0011 (notification) did not.
|
||||
await expect(
|
||||
partial.read(
|
||||
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'notification'",
|
||||
),
|
||||
).resolves.toEqual([]);
|
||||
await expect(
|
||||
partial.read(
|
||||
"SELECT 1 FROM pragma_table_info('apps') WHERE name = 'background'",
|
||||
),
|
||||
).resolves.toHaveLength(1);
|
||||
} finally {
|
||||
partial.onServerShutdown();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('SqliteDatabaseClient — legacy version inference', () => {
|
||||
let dir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-legacy-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('treats user_version=0 with no bootstrap tables as uninitialized', async () => {
|
||||
const path = join(dir, 'blank.sqlite');
|
||||
new Database(path).close();
|
||||
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const client = new SqliteDatabaseClient(
|
||||
sqliteConfig({ inMemory: false, path }),
|
||||
);
|
||||
try {
|
||||
await client.onServerStart();
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining('bootstrap tables are missing'),
|
||||
);
|
||||
// A full migration run followed, so the schema is current.
|
||||
expect(await userVersionOf(client)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
} finally {
|
||||
client.onServerShutdown();
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('infers the schema version from table and column markers', async () => {
|
||||
const path = join(dir, 'legacy.sqlite');
|
||||
const seed = new Database(path);
|
||||
seed.exec(`
|
||||
CREATE TABLE user (
|
||||
id INTEGER PRIMARY KEY,
|
||||
username TEXT,
|
||||
otp_secret TEXT,
|
||||
otp_enabled INTEGER,
|
||||
otp_recovery_codes TEXT
|
||||
);
|
||||
CREATE TABLE apps (
|
||||
id INTEGER PRIMARY KEY,
|
||||
uid TEXT,
|
||||
background INTEGER,
|
||||
metadata TEXT
|
||||
);
|
||||
CREATE TABLE user_to_user_permissions (id INTEGER PRIMARY KEY);
|
||||
CREATE TABLE audit_user_to_user_permissions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
issuer_user_id INTEGER,
|
||||
holder_user_id INTEGER
|
||||
);
|
||||
CREATE TABLE sessions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
created_at INTEGER,
|
||||
last_activity INTEGER
|
||||
);
|
||||
CREATE TABLE kv (id INTEGER PRIMARY KEY, value JSON);
|
||||
INSERT INTO apps (uid)
|
||||
VALUES ('app-e3ac5486-da8c-42ad-8377-8728086e0980');
|
||||
`);
|
||||
seed.close();
|
||||
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
// targetVersion just past the inferred version keeps the run a
|
||||
// no-op, so the assertion is about inference alone.
|
||||
const client = new SqliteDatabaseClient(
|
||||
sqliteConfig({ inMemory: false, path, targetVersion: 22 }),
|
||||
);
|
||||
try {
|
||||
await client.onServerStart();
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
'[sqlite] user_version=0; inferred legacy schema version 21',
|
||||
);
|
||||
// Nothing was applied, so user_version stays where it was.
|
||||
expect(await userVersionOf(client)).toBe(0);
|
||||
} finally {
|
||||
client.onServerShutdown();
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('reports version 0 when only the bootstrap tables exist', async () => {
|
||||
const path = join(dir, 'bootstrap-only.sqlite');
|
||||
const seed = new Database(path);
|
||||
seed.exec(`
|
||||
CREATE TABLE user (id INTEGER PRIMARY KEY);
|
||||
CREATE TABLE apps (id INTEGER PRIMARY KEY);
|
||||
`);
|
||||
seed.close();
|
||||
|
||||
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
const client = new SqliteDatabaseClient(
|
||||
sqliteConfig({ inMemory: false, path, targetVersion: 1 }),
|
||||
);
|
||||
try {
|
||||
await client.onServerStart();
|
||||
expect(log).toHaveBeenCalledWith('[sqlite] database version: 0');
|
||||
} finally {
|
||||
client.onServerShutdown();
|
||||
log.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('SqliteDatabaseClient — query interface', () => {
|
||||
let client: SqliteDatabaseClient;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await bootClient();
|
||||
await client.write(
|
||||
'CREATE TABLE `widget` (`id` INTEGER PRIMARY KEY, ' +
|
||||
'`name` TEXT, `enabled` INTEGER, `seen_at` TEXT)',
|
||||
);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('reports lastInsertRowid and change counts on write', async () => {
|
||||
const inserted = await client.write(
|
||||
'INSERT INTO `widget` (`name`) VALUES (?)',
|
||||
['spanner'],
|
||||
);
|
||||
expect(inserted).toEqual({
|
||||
insertId: 1,
|
||||
affectedRows: 1,
|
||||
anyRowsAffected: true,
|
||||
});
|
||||
|
||||
const missed = await client.write(
|
||||
'UPDATE `widget` SET `name` = ? WHERE `id` = ?',
|
||||
['nope', 999],
|
||||
);
|
||||
expect(missed).toMatchObject({
|
||||
affectedRows: 0,
|
||||
anyRowsAffected: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('binds booleans as sqlite integers', async () => {
|
||||
await client.write(
|
||||
'INSERT INTO `widget` (`name`, `enabled`) VALUES (?, ?)',
|
||||
['toggled', true],
|
||||
);
|
||||
await client.write(
|
||||
'INSERT INTO `widget` (`name`, `enabled`) VALUES (?, ?)',
|
||||
['untoggled', false],
|
||||
);
|
||||
|
||||
await expect(
|
||||
client.read('SELECT `name`, `enabled` FROM `widget` ORDER BY `id`'),
|
||||
).resolves.toEqual([
|
||||
{ name: 'toggled', enabled: 1 },
|
||||
{ name: 'untoggled', enabled: 0 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('rewrites now() into the sqlite equivalent', async () => {
|
||||
await client.write(
|
||||
'INSERT INTO `widget` (`name`, `seen_at`) VALUES (?, NOW())',
|
||||
['stamped'],
|
||||
);
|
||||
const [row] = await client.read(
|
||||
'SELECT `seen_at` FROM `widget` WHERE `name` = ?',
|
||||
['stamped'],
|
||||
);
|
||||
expect(row.seen_at).toMatch(/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/);
|
||||
});
|
||||
|
||||
it('generates an INSERT from a data object', async () => {
|
||||
await client.insert('widget', { name: 'generated', enabled: true });
|
||||
await expect(
|
||||
client.read('SELECT `name`, `enabled` FROM `widget`'),
|
||||
).resolves.toEqual([{ name: 'generated', enabled: 1 }]);
|
||||
});
|
||||
|
||||
it('primary-reads through the same single-node connection', async () => {
|
||||
await client.write('INSERT INTO `widget` (`name`) VALUES (?)', ['p']);
|
||||
await expect(
|
||||
client.pread('SELECT `name` FROM `widget`'),
|
||||
).resolves.toEqual([{ name: 'p' }]);
|
||||
});
|
||||
|
||||
it('applies a batch write atomically', async () => {
|
||||
await client.batchWrite([
|
||||
{
|
||||
statement:
|
||||
'INSERT INTO `widget` (`name`, `enabled`) VALUES (?, ?)',
|
||||
values: ['one', true],
|
||||
},
|
||||
{
|
||||
statement: 'INSERT INTO `widget` (`name`) VALUES (?)',
|
||||
values: ['two'],
|
||||
},
|
||||
]);
|
||||
|
||||
await expect(
|
||||
client.read('SELECT `name` FROM `widget` ORDER BY `id`'),
|
||||
).resolves.toEqual([{ name: 'one' }, { name: 'two' }]);
|
||||
});
|
||||
|
||||
it('rolls the whole batch back when one statement fails', async () => {
|
||||
await expect(
|
||||
client.batchWrite([
|
||||
{
|
||||
statement: 'INSERT INTO `widget` (`name`) VALUES (?)',
|
||||
values: ['kept-if-broken'],
|
||||
},
|
||||
{
|
||||
statement: 'INSERT INTO `nonexistent` (`name`) VALUES (?)',
|
||||
values: ['boom'],
|
||||
},
|
||||
]),
|
||||
).rejects.toThrow(/no such table/i);
|
||||
|
||||
await expect(
|
||||
client.read('SELECT `name` FROM `widget`'),
|
||||
).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it('surfaces sqlite errors for malformed SQL', async () => {
|
||||
await expect(client.read('SELEC oops')).rejects.toThrow(
|
||||
/syntax error/i,
|
||||
);
|
||||
});
|
||||
|
||||
it('closes the database on shutdown', async () => {
|
||||
const closable = await bootClient();
|
||||
closable.onServerShutdown();
|
||||
await expect(closable.read('SELECT 1')).rejects.toThrow(
|
||||
'The database connection is not open',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// Single-node engines have no replica to race, so `tryHardRead` must not
|
||||
// issue the query twice — the base-class default fires `pread()` and
|
||||
// `read()` in parallel, which on sqlite is the same connection.
|
||||
describe('SqliteDatabaseClient — tryHardRead', () => {
|
||||
let client: SqliteDatabaseClient;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await bootClient();
|
||||
await client.write(
|
||||
'CREATE TABLE `widget` (`id` INTEGER PRIMARY KEY, `name` TEXT)',
|
||||
);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('executes the statement exactly once', async () => {
|
||||
await client.write('INSERT INTO `widget` (`name`) VALUES (?)', ['one']);
|
||||
const readSpy = vi.spyOn(client, 'read');
|
||||
const preadSpy = vi.spyOn(client, 'pread');
|
||||
|
||||
await expect(
|
||||
client.tryHardRead('SELECT `name` FROM `widget`'),
|
||||
).resolves.toEqual([{ name: 'one' }]);
|
||||
|
||||
expect(readSpy).toHaveBeenCalledTimes(1);
|
||||
expect(preadSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('executes the statement exactly once when it matches no rows', async () => {
|
||||
const readSpy = vi.spyOn(client, 'read');
|
||||
const preadSpy = vi.spyOn(client, 'pread');
|
||||
|
||||
await expect(
|
||||
client.tryHardRead('SELECT `name` FROM `widget`'),
|
||||
).resolves.toEqual([]);
|
||||
|
||||
expect(readSpy).toHaveBeenCalledTimes(1);
|
||||
expect(preadSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws from requireRead when nothing matches', async () => {
|
||||
await expect(
|
||||
client.requireRead(
|
||||
'SELECT `name` FROM `widget` WHERE `id` = ?',
|
||||
[42],
|
||||
),
|
||||
).rejects.toThrow('required read returned no rows');
|
||||
});
|
||||
|
||||
it('returns the rows from requireRead when something matches', async () => {
|
||||
await client.write('INSERT INTO `widget` (`name`) VALUES (?)', ['ok']);
|
||||
await expect(
|
||||
client.requireRead('SELECT `name` FROM `widget`'),
|
||||
).resolves.toEqual([{ name: 'ok' }]);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -27,9 +27,9 @@ import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
|
||||
const MIGRATIONS_DIR = resolve(__dirname, './migrations/sqlite');
|
||||
|
||||
/**
|
||||
* Ordered list of [threshold_version, files[]] pairs. A database whose
|
||||
* `user_version` is <= threshold_version will have the corresponding files
|
||||
* applied.
|
||||
* Ordered list of [threshold_version, files[]] pairs.
|
||||
* A database whose `user_version` is <= threshold_version will have
|
||||
* the corresponding files applied.
|
||||
*/
|
||||
const AVAILABLE_MIGRATIONS: [number, string[]][] = [
|
||||
[-1, ['0001_create-tables.sql', '0002_add-default-apps.sql']],
|
||||
@@ -158,16 +158,6 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
return this.read(query, params);
|
||||
}
|
||||
|
||||
override async tryHardRead(
|
||||
query: string,
|
||||
params: unknown[] = [],
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
// No replica to race, so the base class's parallel
|
||||
// primary-plus-replica read would run the same statement twice on
|
||||
// the one connection.
|
||||
return this.read(query, params);
|
||||
}
|
||||
|
||||
@Span('db.write', (query: string) => ({ 'db.statement': query }))
|
||||
override async write(
|
||||
query: string,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -33,9 +33,9 @@ import { PostgresDatabaseClient } from './PostgresDatabaseClient';
|
||||
import { SqliteDatabaseClient } from './SqliteDatabaseClient';
|
||||
|
||||
/**
|
||||
* Factory class registered in `puterClients`. PuterServer calls `new
|
||||
* DatabaseClientFactory(config)` — the constructor returns the concrete
|
||||
* subclass selected by `config.database.engine`.
|
||||
* Factory class registered in `puterClients`. PuterServer calls
|
||||
* `new DatabaseClientFactory(config)` — the constructor returns the
|
||||
* concrete subclass selected by `config.database.engine`.
|
||||
*/
|
||||
export const DatabaseClientFactory = class DatabaseClientFactory {
|
||||
constructor(config: IConfig) {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -41,10 +41,11 @@ const RETRIABLE_ERROR_MESSAGES = [
|
||||
];
|
||||
|
||||
/**
|
||||
* Failures where the statement provably never reached the server, so a retry
|
||||
* can never double-apply it — safe even for writes. Anything that can occur
|
||||
* after the statement was sent (resets, protocol drops) is deliberately
|
||||
* excluded: the server may have committed before the connection died.
|
||||
* Failures where the statement provably never reached the server, so a
|
||||
* retry can never double-apply it — safe even for writes. Anything that
|
||||
* can occur after the statement was sent (resets, protocol drops) is
|
||||
* deliberately excluded: the server may have committed before the
|
||||
* connection died.
|
||||
*/
|
||||
const NEVER_SENT_ERROR_CODES = new Set([
|
||||
'ECONNREFUSED',
|
||||
@@ -58,9 +59,10 @@ const errorCode = (error: unknown): string | undefined =>
|
||||
(error as { code?: string } | null)?.code;
|
||||
|
||||
/**
|
||||
* Transient connection-level failures worth retrying — but only for statements
|
||||
* that are safe to run twice (reads). Row-level errors (duplicate key,
|
||||
* constraint violations) are deterministic and never match.
|
||||
* Transient connection-level failures worth retrying — but only for
|
||||
* statements that are safe to run twice (reads). Row-level errors
|
||||
* (duplicate key, constraint violations) are deterministic and never
|
||||
* match.
|
||||
*/
|
||||
export const isRetriableError = (error: unknown): boolean => {
|
||||
const code = errorCode(error);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -20,11 +20,12 @@
|
||||
/**
|
||||
* DELIMITER-aware splitter for MySQL dump / migration files.
|
||||
*
|
||||
* Splits `sql` into individual statements using the active statement delimiter
|
||||
* (default `;`). Recognises `DELIMITER X` lines, single-quoted strings,
|
||||
* double-quoted strings, backtick-quoted identifiers, line comments (`-- `,
|
||||
* `#`) and block comments (`/* ... *\/`). DELIMITER directives are stripped
|
||||
* from the output (they're a client-side concept, not server SQL).
|
||||
* Splits `sql` into individual statements using the active statement
|
||||
* delimiter (default `;`). Recognises `DELIMITER X` lines, single-quoted
|
||||
* strings, double-quoted strings, backtick-quoted identifiers, line
|
||||
* comments (`-- `, `#`) and block comments (`/* ... *\/`). DELIMITER
|
||||
* directives are stripped from the output (they're a client-side concept,
|
||||
* not server SQL).
|
||||
*
|
||||
* Returns trimmed, non-empty statements without the trailing delimiter.
|
||||
*/
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { splitPostgresStatements } from './splitPostgresStatements';
|
||||
|
||||
describe('splitPostgresStatements', () => {
|
||||
it('splits on semicolons and trims each statement', () => {
|
||||
expect(
|
||||
splitPostgresStatements('SELECT 1;\n SELECT 2 ;\nSELECT 3'),
|
||||
).toEqual(['SELECT 1', 'SELECT 2', 'SELECT 3']);
|
||||
});
|
||||
|
||||
it('drops empty statements from stray semicolons', () => {
|
||||
expect(splitPostgresStatements(';;SELECT 1;;')).toEqual(['SELECT 1']);
|
||||
});
|
||||
|
||||
it('returns nothing for whitespace-only input', () => {
|
||||
expect(splitPostgresStatements(' \n\t ')).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps semicolons inside single-quoted literals', () => {
|
||||
expect(
|
||||
splitPostgresStatements("INSERT INTO t VALUES ('a;b'); SELECT 1"),
|
||||
).toEqual(["INSERT INTO t VALUES ('a;b')", 'SELECT 1']);
|
||||
});
|
||||
|
||||
it('handles doubled and backslash-escaped quotes in literals', () => {
|
||||
expect(
|
||||
splitPostgresStatements("SELECT 'it''s; fine'; SELECT 2"),
|
||||
).toEqual(["SELECT 'it''s; fine'", 'SELECT 2']);
|
||||
expect(splitPostgresStatements("SELECT E'a\\'; b'; SELECT 2")).toEqual([
|
||||
"SELECT E'a\\'; b'",
|
||||
'SELECT 2',
|
||||
]);
|
||||
});
|
||||
|
||||
it('keeps semicolons inside quoted identifiers', () => {
|
||||
expect(
|
||||
splitPostgresStatements('SELECT "we;ird" FROM t; SELECT 2'),
|
||||
).toEqual(['SELECT "we;ird" FROM t', 'SELECT 2']);
|
||||
});
|
||||
|
||||
it('handles a doubled quote inside a quoted identifier', () => {
|
||||
expect(
|
||||
splitPostgresStatements('SELECT "we""ird;x" FROM t; SELECT 2'),
|
||||
).toEqual(['SELECT "we""ird;x" FROM t', 'SELECT 2']);
|
||||
});
|
||||
|
||||
it('ignores semicolons inside line comments', () => {
|
||||
expect(
|
||||
splitPostgresStatements('SELECT 1 -- trailing; note\n; SELECT 2'),
|
||||
).toEqual(['SELECT 1 -- trailing; note', 'SELECT 2']);
|
||||
});
|
||||
|
||||
it('ignores semicolons inside block comments', () => {
|
||||
expect(
|
||||
splitPostgresStatements('SELECT /* a; b */ 1; SELECT 2'),
|
||||
).toEqual(['SELECT /* a; b */ 1', 'SELECT 2']);
|
||||
});
|
||||
|
||||
it('keeps a whole dollar-quoted function body together', () => {
|
||||
const sql = [
|
||||
'CREATE FUNCTION bump() RETURNS trigger AS $$',
|
||||
'BEGIN',
|
||||
' NEW.updated_at := now();',
|
||||
' RETURN NEW;',
|
||||
'END;',
|
||||
'$$ LANGUAGE plpgsql;',
|
||||
'SELECT 1',
|
||||
].join('\n');
|
||||
|
||||
const statements = splitPostgresStatements(sql);
|
||||
expect(statements).toHaveLength(2);
|
||||
expect(statements[0]).toContain('RETURN NEW;');
|
||||
expect(statements[0].endsWith('$$ LANGUAGE plpgsql')).toBe(true);
|
||||
expect(statements[1]).toBe('SELECT 1');
|
||||
});
|
||||
|
||||
it('respects a tagged dollar quote and its nested untagged pair', () => {
|
||||
const sql = 'SELECT $body$ inner $$ still; inside $body$; SELECT 2';
|
||||
expect(splitPostgresStatements(sql)).toEqual([
|
||||
'SELECT $body$ inner $$ still; inside $body$',
|
||||
'SELECT 2',
|
||||
]);
|
||||
});
|
||||
|
||||
it('treats a bare dollar sign as ordinary text', () => {
|
||||
expect(splitPostgresStatements('SELECT 1 $ 2; SELECT 3')).toEqual([
|
||||
'SELECT 1 $ 2',
|
||||
'SELECT 3',
|
||||
]);
|
||||
});
|
||||
|
||||
it('emits an unterminated final statement', () => {
|
||||
expect(
|
||||
splitPostgresStatements('SELECT 1; SELECT 2 -- no newline'),
|
||||
).toEqual(['SELECT 1', 'SELECT 2 -- no newline']);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,531 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { createServer, type Server } from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import { DDBClient } from './DDBClient';
|
||||
|
||||
const TABLE = 'kv-items';
|
||||
const GSI = 'by-kind';
|
||||
|
||||
const localConfig = (): IConfig =>
|
||||
({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: { inMemory: true, bootstrapTables: true },
|
||||
}) as unknown as IConfig;
|
||||
|
||||
/**
|
||||
* The emulator reports a table as CREATING for a tick after `CreateTable`
|
||||
* returns, so wait until it actually answers a read before using it.
|
||||
*/
|
||||
const awaitTable = async (
|
||||
client: DDBClient,
|
||||
table: string,
|
||||
key: Record<string, unknown>,
|
||||
): Promise<void> => {
|
||||
for (let attempt = 0; attempt < 100; attempt++) {
|
||||
try {
|
||||
await client.get(table, key);
|
||||
return;
|
||||
} catch (error) {
|
||||
if ((error as Error).name !== 'ResourceNotFoundException')
|
||||
throw error;
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
}
|
||||
}
|
||||
throw new Error(`table ${table} never became queryable`);
|
||||
};
|
||||
|
||||
const tableSchema = {
|
||||
TableName: TABLE,
|
||||
KeySchema: [
|
||||
{ AttributeName: 'pk', KeyType: 'HASH' as const },
|
||||
{ AttributeName: 'sk', KeyType: 'RANGE' as const },
|
||||
],
|
||||
AttributeDefinitions: [
|
||||
{ AttributeName: 'pk', AttributeType: 'S' as const },
|
||||
{ AttributeName: 'sk', AttributeType: 'S' as const },
|
||||
{ AttributeName: 'kind', AttributeType: 'S' as const },
|
||||
],
|
||||
GlobalSecondaryIndexes: [
|
||||
{
|
||||
IndexName: GSI,
|
||||
KeySchema: [{ AttributeName: 'kind', KeyType: 'HASH' as const }],
|
||||
Projection: { ProjectionType: 'ALL' as const },
|
||||
ProvisionedThroughput: {
|
||||
ReadCapacityUnits: 5,
|
||||
WriteCapacityUnits: 5,
|
||||
},
|
||||
},
|
||||
],
|
||||
ProvisionedThroughput: { ReadCapacityUnits: 5, WriteCapacityUnits: 5 },
|
||||
};
|
||||
|
||||
describe('DDBClient — item operations', () => {
|
||||
let client: DDBClient;
|
||||
|
||||
beforeAll(async () => {
|
||||
client = new DDBClient(localConfig());
|
||||
await client.createTableIfNotExists(tableSchema);
|
||||
await awaitTable(client, TABLE, { pk: 'ready', sk: 'ready' });
|
||||
});
|
||||
|
||||
it('round-trips an item and reports consumed capacity', async () => {
|
||||
await client.put(TABLE, { pk: 'u1', sk: 'profile', name: 'ada' });
|
||||
|
||||
const result = await client.get(TABLE, { pk: 'u1', sk: 'profile' });
|
||||
expect(result.Item).toEqual({ pk: 'u1', sk: 'profile', name: 'ada' });
|
||||
expect(result.ConsumedCapacity?.TableName).toBe(TABLE);
|
||||
});
|
||||
|
||||
it('returns no Item for a key that does not exist', async () => {
|
||||
const result = await client.get(TABLE, { pk: 'nobody', sk: 'here' });
|
||||
expect(result.Item).toBeUndefined();
|
||||
});
|
||||
|
||||
it('honours a strongly consistent read', async () => {
|
||||
await client.put(TABLE, { pk: 'u2', sk: 'profile', name: 'grace' });
|
||||
const result = await client.get(
|
||||
TABLE,
|
||||
{ pk: 'u2', sk: 'profile' },
|
||||
true,
|
||||
);
|
||||
expect(result.Item?.name).toBe('grace');
|
||||
});
|
||||
|
||||
it('applies an update expression and returns the new item', async () => {
|
||||
await client.put(TABLE, { pk: 'u3', sk: 'counter', hits: 1 });
|
||||
const updated = await client.update(
|
||||
TABLE,
|
||||
{ pk: 'u3', sk: 'counter' },
|
||||
'SET #h = :next',
|
||||
{ ':next': 5 },
|
||||
{ '#h': 'hits' },
|
||||
);
|
||||
expect(updated.Attributes).toMatchObject({ hits: 5 });
|
||||
});
|
||||
|
||||
it('supports an update expression with no values or names', async () => {
|
||||
await client.put(TABLE, { pk: 'u4', sk: 'doc', stale: true });
|
||||
const updated = await client.update(
|
||||
TABLE,
|
||||
{ pk: 'u4', sk: 'doc' },
|
||||
'REMOVE stale',
|
||||
);
|
||||
expect(updated.Attributes).toEqual({ pk: 'u4', sk: 'doc' });
|
||||
});
|
||||
|
||||
it('deletes an item', async () => {
|
||||
await client.put(TABLE, { pk: 'u5', sk: 'temp' });
|
||||
await client.del(TABLE, { pk: 'u5', sk: 'temp' });
|
||||
const result = await client.get(TABLE, { pk: 'u5', sk: 'temp' });
|
||||
expect(result.Item).toBeUndefined();
|
||||
});
|
||||
|
||||
it('fetches keys from several tables in one batch', async () => {
|
||||
await client.put(TABLE, { pk: 'b1', sk: 'a', v: 1 });
|
||||
await client.put(TABLE, { pk: 'b1', sk: 'b', v: 2 });
|
||||
|
||||
const result = await client.batchGet([
|
||||
{ table: TABLE, items: { pk: 'b1', sk: 'a' } },
|
||||
{ table: TABLE, items: { pk: 'b1', sk: 'b' } },
|
||||
]);
|
||||
|
||||
const values = (result.Responses?.[TABLE] ?? [])
|
||||
.map((item) => item.v as number)
|
||||
.sort();
|
||||
expect(values).toEqual([1, 2]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — queries', () => {
|
||||
let client: DDBClient;
|
||||
|
||||
beforeAll(async () => {
|
||||
client = new DDBClient(localConfig());
|
||||
await client.createTableIfNotExists(tableSchema);
|
||||
await awaitTable(client, TABLE, { pk: 'ready', sk: 'ready' });
|
||||
for (const sk of ['post#1', 'post#2', 'post#3', 'note#1']) {
|
||||
await client.put(TABLE, {
|
||||
pk: 'q1',
|
||||
sk,
|
||||
kind: sk.split('#')[0],
|
||||
size: sk === 'post#2' ? 10 : 1,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('queries by partition key', async () => {
|
||||
const result = await client.query(TABLE, { pk: 'q1' });
|
||||
expect(result.Items).toHaveLength(4);
|
||||
expect(result.ConsumedCapacity?.TableName).toBe(TABLE);
|
||||
});
|
||||
|
||||
it('narrows the sort key with begins_with', async () => {
|
||||
const result = await client.query(
|
||||
TABLE,
|
||||
{ pk: 'q1' },
|
||||
0,
|
||||
undefined,
|
||||
'',
|
||||
false,
|
||||
{ beginsWith: { key: 'sk', value: 'post#' } },
|
||||
);
|
||||
expect(result.Items?.map((item) => item.sk)).toEqual([
|
||||
'post#1',
|
||||
'post#2',
|
||||
'post#3',
|
||||
]);
|
||||
});
|
||||
|
||||
it('ignores an empty begins_with value', async () => {
|
||||
const result = await client.query(
|
||||
TABLE,
|
||||
{ pk: 'q1' },
|
||||
0,
|
||||
undefined,
|
||||
'',
|
||||
false,
|
||||
{ beginsWith: { key: 'sk', value: '' } },
|
||||
);
|
||||
expect(result.Items).toHaveLength(4);
|
||||
});
|
||||
|
||||
it('applies a filter expression with its own names and values', async () => {
|
||||
const result = await client.query(
|
||||
TABLE,
|
||||
{ pk: 'q1' },
|
||||
0,
|
||||
undefined,
|
||||
'',
|
||||
false,
|
||||
{
|
||||
filter: {
|
||||
expression: '#s > :min',
|
||||
names: { '#s': 'size' },
|
||||
values: { ':min': 5 },
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(result.Items?.map((item) => item.sk)).toEqual(['post#2']);
|
||||
});
|
||||
|
||||
it('counts without returning items', async () => {
|
||||
const result = await client.query(
|
||||
TABLE,
|
||||
{ pk: 'q1' },
|
||||
0,
|
||||
undefined,
|
||||
'',
|
||||
false,
|
||||
{ select: 'COUNT' },
|
||||
);
|
||||
expect(result.Count).toBe(4);
|
||||
expect(result.Items).toBeUndefined();
|
||||
});
|
||||
|
||||
it('pages through results with the last evaluated key', async () => {
|
||||
const seen: string[] = [];
|
||||
let pageKey: Record<string, unknown> | undefined;
|
||||
let pages = 0;
|
||||
|
||||
do {
|
||||
const page = await client.query(TABLE, { pk: 'q1' }, 2, pageKey);
|
||||
pages += 1;
|
||||
expect(page.Items?.length).toBeLessThanOrEqual(2);
|
||||
for (const item of page.Items ?? []) seen.push(item.sk as string);
|
||||
pageKey = page.LastEvaluatedKey;
|
||||
} while (pageKey && pages < 10);
|
||||
|
||||
expect(pages).toBeGreaterThan(1);
|
||||
expect(new Set(seen).size).toBe(4);
|
||||
});
|
||||
|
||||
it('queries a secondary index', async () => {
|
||||
const result = await client.query(
|
||||
TABLE,
|
||||
{ kind: 'note' },
|
||||
0,
|
||||
undefined,
|
||||
GSI,
|
||||
);
|
||||
expect(result.Items?.map((item) => item.sk)).toEqual(['note#1']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — batch writes', () => {
|
||||
let client: DDBClient;
|
||||
|
||||
beforeAll(async () => {
|
||||
client = new DDBClient(localConfig());
|
||||
await client.createTableIfNotExists(tableSchema);
|
||||
await awaitTable(client, TABLE, { pk: 'ready', sk: 'ready' });
|
||||
});
|
||||
|
||||
it('reports no consumed capacity for an empty batch', async () => {
|
||||
await expect(client.batchPut([])).resolves.toEqual({
|
||||
ConsumedCapacity: [],
|
||||
});
|
||||
});
|
||||
|
||||
it('splits a batch larger than the service limit into chunks', async () => {
|
||||
const params = Array.from({ length: 60 }, (_, index) => ({
|
||||
table: TABLE,
|
||||
item: { pk: 'bulk', sk: `item-${index}`, index },
|
||||
}));
|
||||
|
||||
const result = await client.batchPut(params);
|
||||
|
||||
expect(result.ConsumedCapacity).toHaveLength(1);
|
||||
expect(result.ConsumedCapacity[0].TableName).toBe(TABLE);
|
||||
expect(result.ConsumedCapacity[0].CapacityUnits).toBeGreaterThan(0);
|
||||
|
||||
const stored = await client.query(TABLE, { pk: 'bulk' });
|
||||
expect(stored.Items).toHaveLength(60);
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — expired item sweep', () => {
|
||||
it('deletes only the items past their ttl when the table already existed', async () => {
|
||||
const client = new DDBClient(localConfig());
|
||||
const table = 'ttl-items';
|
||||
const schema = {
|
||||
TableName: table,
|
||||
KeySchema: [{ AttributeName: 'pk', KeyType: 'HASH' as const }],
|
||||
AttributeDefinitions: [
|
||||
{ AttributeName: 'pk', AttributeType: 'S' as const },
|
||||
],
|
||||
ProvisionedThroughput: {
|
||||
ReadCapacityUnits: 5,
|
||||
WriteCapacityUnits: 5,
|
||||
},
|
||||
};
|
||||
|
||||
await client.createTableIfNotExists(schema, 'expireAt');
|
||||
await awaitTable(client, table, { pk: 'ready' });
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
await client.put(table, { pk: 'stale', expireAt: now - 60 });
|
||||
await client.put(table, { pk: 'fresh', expireAt: now + 3600 });
|
||||
await client.put(table, { pk: 'eternal' });
|
||||
|
||||
// Second call finds the table in use, so the sweep runs.
|
||||
await client.createTableIfNotExists(schema, 'expireAt');
|
||||
|
||||
expect((await client.get(table, { pk: 'stale' })).Item).toBeUndefined();
|
||||
expect((await client.get(table, { pk: 'fresh' })).Item).toBeDefined();
|
||||
expect((await client.get(table, { pk: 'eternal' })).Item).toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — expired item sweep on an empty table', () => {
|
||||
it('completes without deleting anything', async () => {
|
||||
const client = new DDBClient(localConfig());
|
||||
const table = 'ttl-empty';
|
||||
const schema = {
|
||||
TableName: table,
|
||||
KeySchema: [{ AttributeName: 'pk', KeyType: 'HASH' as const }],
|
||||
AttributeDefinitions: [
|
||||
{ AttributeName: 'pk', AttributeType: 'S' as const },
|
||||
],
|
||||
ProvisionedThroughput: {
|
||||
ReadCapacityUnits: 5,
|
||||
WriteCapacityUnits: 5,
|
||||
},
|
||||
};
|
||||
|
||||
await client.createTableIfNotExists(schema, 'expireAt');
|
||||
await awaitTable(client, table, { pk: 'ready' });
|
||||
await expect(
|
||||
client.createTableIfNotExists(schema, 'expireAt'),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('DDBClient — credentialled configuration', () => {
|
||||
it('refuses an aws config that is missing a key', () => {
|
||||
expect(
|
||||
() =>
|
||||
new DDBClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: { aws: { access_key: 'only-one' } },
|
||||
} as unknown as IConfig),
|
||||
).toThrow('requires both `access_key` and `secret_key`');
|
||||
});
|
||||
|
||||
it('skips table creation on a managed deployment', async () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const client = new DDBClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: {
|
||||
aws: { access_key: 'a', secret_key: 'b', region: 'us-west-2' },
|
||||
endpoint: 'http://127.0.0.1:1',
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
|
||||
// Would fail against the unreachable endpoint if it actually tried.
|
||||
await client.createTableIfNotExists(tableSchema);
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining(
|
||||
'Creating DynamoDB tables is disabled by default',
|
||||
),
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it('rebinds an aws-credentialled client on recreate', async () => {
|
||||
const client = new DDBClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: {
|
||||
aws: { access_key: 'a', secret_key: 'b' },
|
||||
endpoint: 'http://127.0.0.1:1',
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
|
||||
await expect(client.recreateClient()).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it('rebinds the underlying client on recreate', async () => {
|
||||
const client = new DDBClient(localConfig());
|
||||
await client.createTableIfNotExists(tableSchema);
|
||||
await awaitTable(client, TABLE, { pk: 'ready', sk: 'ready' });
|
||||
await client.put(TABLE, { pk: 'recreate', sk: 'a', v: 1 });
|
||||
|
||||
await client.recreateClient();
|
||||
|
||||
// Same in-memory store: the recreated client sees earlier writes.
|
||||
const result = await client.get(TABLE, { pk: 'recreate', sk: 'a' });
|
||||
expect(result.Item?.v).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// Unprocessed items are a throttling signal the emulator never produces, so
|
||||
// the retry loop is driven against a stub speaking the DynamoDB wire format.
|
||||
describe('DDBClient — unprocessed batch items', () => {
|
||||
let server: Server;
|
||||
let endpoint: string;
|
||||
let responses: unknown[];
|
||||
let requestCount = 0;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = createServer((req, res) => {
|
||||
let body = '';
|
||||
req.on('data', (chunk) => {
|
||||
body += chunk;
|
||||
});
|
||||
req.on('end', () => {
|
||||
const next = responses.shift() ?? {};
|
||||
requestCount += 1;
|
||||
res.writeHead(200, {
|
||||
'content-type': 'application/x-amz-json-1.0',
|
||||
});
|
||||
res.end(JSON.stringify(next));
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', resolve),
|
||||
);
|
||||
endpoint = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
});
|
||||
|
||||
const stubClient = () =>
|
||||
new DDBClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: {
|
||||
aws: { access_key: 'a', secret_key: 'b', region: 'us-west-2' },
|
||||
endpoint,
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
|
||||
const unprocessed = (pk: string) => ({
|
||||
UnprocessedItems: {
|
||||
[TABLE]: [{ PutRequest: { Item: { pk: { S: pk } } } }],
|
||||
},
|
||||
ConsumedCapacity: [{ TableName: TABLE, CapacityUnits: 1 }],
|
||||
});
|
||||
|
||||
it('retries the leftover items and sums capacity across attempts', async () => {
|
||||
requestCount = 0;
|
||||
responses = [
|
||||
unprocessed('retry-me'),
|
||||
{
|
||||
UnprocessedItems: {},
|
||||
ConsumedCapacity: [{ TableName: TABLE, CapacityUnits: 2 }],
|
||||
},
|
||||
];
|
||||
|
||||
const result = await stubClient().batchPut([
|
||||
{ table: TABLE, item: { pk: 'retry-me' } },
|
||||
]);
|
||||
|
||||
expect(requestCount).toBe(2);
|
||||
expect(result.ConsumedCapacity).toEqual([
|
||||
{ TableName: TABLE, CapacityUnits: 3 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('gives up with a bad-request error when items never drain', async () => {
|
||||
requestCount = 0;
|
||||
responses = Array.from({ length: 12 }, () => unprocessed('stuck'));
|
||||
|
||||
await expect(
|
||||
stubClient().batchPut([{ table: TABLE, item: { pk: 'stuck' } }]),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'bad_request',
|
||||
message: 'Failed to batch write all items to DynamoDB',
|
||||
});
|
||||
// One initial attempt plus the full retry budget.
|
||||
expect(requestCount).toBe(9);
|
||||
}, 30_000);
|
||||
|
||||
it('tolerates a response that reports no capacity at all', async () => {
|
||||
requestCount = 0;
|
||||
responses = [{ UnprocessedItems: {} }];
|
||||
|
||||
await expect(
|
||||
stubClient().batchPut([{ table: TABLE, item: { pk: 'quiet' } }]),
|
||||
).resolves.toEqual({ ConsumedCapacity: [] });
|
||||
expect(requestCount).toBe(1);
|
||||
});
|
||||
|
||||
it('ignores capacity entries with no table name', async () => {
|
||||
requestCount = 0;
|
||||
responses = [
|
||||
{
|
||||
UnprocessedItems: {},
|
||||
ConsumedCapacity: [{ CapacityUnits: 7 }],
|
||||
},
|
||||
];
|
||||
|
||||
await expect(
|
||||
stubClient().batchPut([{ table: TABLE, item: { pk: 'anon' } }]),
|
||||
).resolves.toEqual({ ConsumedCapacity: [] });
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -453,7 +453,6 @@ export class DDBClient extends PuterClient {
|
||||
return;
|
||||
}
|
||||
|
||||
let alreadyExisted = false;
|
||||
try {
|
||||
const client = await this.#getDocumentClient();
|
||||
await client.send(new CreateTableCommand(params));
|
||||
@@ -461,33 +460,14 @@ export class DDBClient extends PuterClient {
|
||||
if ((error as Error)?.name !== 'ResourceInUseException') {
|
||||
throw error;
|
||||
}
|
||||
alreadyExisted = true;
|
||||
}
|
||||
|
||||
// Only sweep a table that was already there. `CreateTable` returns
|
||||
// while the table is still CREATING, so scanning it right away races
|
||||
// its transition to ACTIVE and intermittently throws
|
||||
// `ResourceNotFoundException` — and a table we just created holds no
|
||||
// items to expire anyway.
|
||||
if (ttlAttribute && alreadyExisted) {
|
||||
try {
|
||||
await this.#deleteExpiredItems(
|
||||
params.TableName!,
|
||||
params.KeySchema!,
|
||||
ttlAttribute,
|
||||
);
|
||||
} catch (error) {
|
||||
// The sweep is opportunistic cleanup, so a table that isn't
|
||||
// queryable yet must never fail boot. Emulators differ on
|
||||
// whether `CreateTable` reports an existing table as in-use,
|
||||
// which can still land us here on a CREATING table.
|
||||
if ((error as Error)?.name !== 'ResourceNotFoundException') {
|
||||
throw error;
|
||||
}
|
||||
console.warn(
|
||||
`[ddb] skipped TTL sweep for ${params.TableName}: table not queryable yet`,
|
||||
);
|
||||
}
|
||||
if (ttlAttribute) {
|
||||
await this.#deleteExpiredItems(
|
||||
params.TableName!,
|
||||
params.KeySchema!,
|
||||
ttlAttribute,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,299 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import { EmailClient } from './EmailClient';
|
||||
|
||||
const FROM = '"Puter" <[email protected]>';
|
||||
|
||||
/**
|
||||
* Nodemailer's JSON transport is a real transport that serializes the message
|
||||
* instead of talking SMTP — the send path stays genuine while the wire stays
|
||||
* local.
|
||||
*/
|
||||
const jsonTransportConfig = (overrides: Partial<IConfig> = {}): IConfig =>
|
||||
({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
env: 'prod',
|
||||
email: { jsonTransport: true, from: FROM },
|
||||
...overrides,
|
||||
}) as unknown as IConfig;
|
||||
|
||||
const startClient = (overrides: Partial<IConfig> = {}): EmailClient => {
|
||||
const client = new EmailClient(jsonTransportConfig(overrides));
|
||||
client.onServerStart();
|
||||
return client;
|
||||
};
|
||||
|
||||
/** The JSON transport reports the serialized message on `.message`. */
|
||||
const sentMessage = (info: unknown) =>
|
||||
JSON.parse((info as { message: string }).message) as {
|
||||
from: { address: string; name: string };
|
||||
to: { address: string }[];
|
||||
subject: string;
|
||||
html?: string;
|
||||
headers?: Record<string, string>;
|
||||
};
|
||||
|
||||
describe('EmailClient — transport lifecycle', () => {
|
||||
it('reports as configured once a transport is wired up', () => {
|
||||
const client = startClient();
|
||||
expect(client.isConfigured).toBe(true);
|
||||
client.onServerShutdown();
|
||||
expect(client.isConfigured).toBe(false);
|
||||
});
|
||||
|
||||
it('warns and stays unconfigured when no transport is set', () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const client = new EmailClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
} as unknown as IConfig);
|
||||
client.onServerStart();
|
||||
|
||||
expect(client.isConfigured).toBe(false);
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining('no email transport configured'),
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it('drops the send and returns null when no transport exists', async () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const client = new EmailClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
} as unknown as IConfig);
|
||||
client.onServerStart();
|
||||
|
||||
await expect(
|
||||
client.sendRaw({ to: '[email protected]', subject: 'hi', text: 'hi' }),
|
||||
).resolves.toBeNull();
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining('without transport'),
|
||||
expect.objectContaining({ to: '[email protected]' }),
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('EmailClient — sending', () => {
|
||||
it('sends a rendered template through the transport', async () => {
|
||||
const client = startClient();
|
||||
const info = await client.sendRaw({
|
||||
to: '[email protected]',
|
||||
subject: 'raw subject',
|
||||
html: '<p>hello</p>',
|
||||
headers: { 'List-Unsubscribe': '<https://puter.test/u>' },
|
||||
});
|
||||
|
||||
const message = sentMessage(info);
|
||||
expect(message.subject).toBe('raw subject');
|
||||
expect(message.to[0].address).toBe('[email protected]');
|
||||
expect(message.from.address).toBe('[email protected]');
|
||||
expect(message.headers?.['List-Unsubscribe']).toBe(
|
||||
'<https://puter.test/u>',
|
||||
);
|
||||
});
|
||||
|
||||
it('lets the caller override the from address', async () => {
|
||||
const client = startClient();
|
||||
const info = await client.sendRaw({
|
||||
from: '[email protected]',
|
||||
to: '[email protected]',
|
||||
subject: 's',
|
||||
text: 't',
|
||||
});
|
||||
expect(sentMessage(info).from.address).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('falls back to the built-in sender when config has none', async () => {
|
||||
const client = startClient({
|
||||
email: { jsonTransport: true },
|
||||
} as unknown as Partial<IConfig>);
|
||||
const info = await client.sendRaw({
|
||||
to: '[email protected]',
|
||||
subject: 's',
|
||||
text: 't',
|
||||
});
|
||||
expect(sentMessage(info).from.address).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('refuses an unknown template name', async () => {
|
||||
const client = startClient();
|
||||
await expect(
|
||||
client.send('[email protected]', 'not-a-template' as never),
|
||||
).rejects.toThrow('Unknown email template: not-a-template');
|
||||
});
|
||||
|
||||
it('surfaces transport failures to the caller', async () => {
|
||||
// Port 1 on loopback is never listening, so the send really fails.
|
||||
const client = startClient({
|
||||
email: { host: '127.0.0.1', port: 1, from: FROM },
|
||||
} as unknown as Partial<IConfig>);
|
||||
|
||||
await expect(
|
||||
client.sendRaw({
|
||||
to: '[email protected]',
|
||||
subject: 's',
|
||||
text: 't',
|
||||
}),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('EmailClient — template rendering', () => {
|
||||
it('renders the code into both subject and body', async () => {
|
||||
const client = startClient();
|
||||
const captured: { subject?: string; html?: string } = {};
|
||||
const original = client.sendRaw.bind(client);
|
||||
vi.spyOn(client, 'sendRaw').mockImplementation(async (options) => {
|
||||
captured.subject = options.subject;
|
||||
captured.html = options.html;
|
||||
return original(options);
|
||||
});
|
||||
|
||||
await client.send('[email protected]', 'email_verification_code', {
|
||||
code: '424242',
|
||||
});
|
||||
|
||||
expect(captured.subject).toBe('424242 is your confirmation code');
|
||||
expect(captured.html).toContain('<strong>424242</strong>');
|
||||
});
|
||||
|
||||
it('escapes html and converts newlines in nl2br values', async () => {
|
||||
const client = startClient();
|
||||
let html = '';
|
||||
vi.spyOn(client, 'sendRaw').mockImplementation(async (options) => {
|
||||
html = options.html ?? '';
|
||||
return null;
|
||||
});
|
||||
|
||||
await client.send('[email protected]', 'listing-rejected', {
|
||||
app_name: 'demo',
|
||||
app_title: 'Demo',
|
||||
reason: 'first & "only"\n<b>line</b>',
|
||||
});
|
||||
|
||||
expect(html).toContain(
|
||||
'first & "only"<br /><b>line</b>',
|
||||
);
|
||||
});
|
||||
|
||||
it('renders an empty string for a missing nl2br value', async () => {
|
||||
const client = startClient();
|
||||
let html = '';
|
||||
vi.spyOn(client, 'sendRaw').mockImplementation(async (options) => {
|
||||
html = options.html ?? '';
|
||||
return null;
|
||||
});
|
||||
|
||||
await client.send('[email protected]', 'listing-rejected', {
|
||||
app_name: 'demo',
|
||||
app_title: 'Demo',
|
||||
});
|
||||
|
||||
expect(html).toContain('<blockquote></blockquote>');
|
||||
});
|
||||
});
|
||||
|
||||
describe('EmailClient.clean', () => {
|
||||
const clean = (email: string) => startClient().clean(email);
|
||||
|
||||
it('strips subaddressing on ordinary domains', () => {
|
||||
expect(clean('[email protected]')).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('ignores dots and subaddressing for gmail', () => {
|
||||
expect(clean('[email protected]')).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('canonicalizes googlemail onto gmail', () => {
|
||||
expect(clean('[email protected]')).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('keeps yahoo subaddressing, which yahoo treats as distinct', () => {
|
||||
expect(clean('[email protected]')).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('ignores dots for icloud aliases', () => {
|
||||
expect(clean('[email protected]')).toBe('[email protected]');
|
||||
});
|
||||
|
||||
it('returns anything that is not an address unchanged', () => {
|
||||
expect(clean('not-an-email')).toBe('not-an-email');
|
||||
expect(clean('@nolocal.test')).toBe('@nolocal.test');
|
||||
});
|
||||
});
|
||||
|
||||
describe('EmailClient.validate', () => {
|
||||
it('waves everything through in a dev environment', async () => {
|
||||
const client = startClient({ env: 'dev' } as Partial<IConfig>);
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects a blocked domain suffix', async () => {
|
||||
const client = startClient({
|
||||
blockedEmailDomains: ['blocked.test'],
|
||||
} as unknown as Partial<IConfig>);
|
||||
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(
|
||||
false,
|
||||
);
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it('matches the blocklist against the cleaned address', async () => {
|
||||
const client = startClient({
|
||||
blockedEmailDomains: ['@gmail.com'],
|
||||
} as unknown as Partial<IConfig>);
|
||||
await expect(
|
||||
client.validate('[email protected]'),
|
||||
).resolves.toBe(false);
|
||||
});
|
||||
|
||||
it('lets a registered validator veto an address', async () => {
|
||||
const client = startClient();
|
||||
const seen: string[] = [];
|
||||
client.addValidator((email) => {
|
||||
seen.push(email);
|
||||
return !email.startsWith('spam');
|
||||
});
|
||||
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(
|
||||
false,
|
||||
);
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(true);
|
||||
// Validators always see the canonical form.
|
||||
expect(seen).toEqual(['[email protected]', '[email protected]']);
|
||||
});
|
||||
|
||||
it('supports asynchronous validators', async () => {
|
||||
const client = startClient();
|
||||
client.addValidator(async (email) => email !== '[email protected]');
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(false);
|
||||
await expect(client.validate('[email protected]')).resolves.toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -110,11 +110,10 @@ const DOMAIN_ALIASES: Record<string, string> = {
|
||||
|
||||
/**
|
||||
* Unified email client. Handles:
|
||||
*
|
||||
* - Template-based outbound mail (via `send`)
|
||||
* - Raw nodemailer passthrough (via `sendRaw`)
|
||||
* - Canonical-form normalization for dedup (via `clean`)
|
||||
* - Policy + extensible validation (via `validate`)
|
||||
* - Template-based outbound mail (via `send`)
|
||||
* - Raw nodemailer passthrough (via `sendRaw`)
|
||||
* - Canonical-form normalization for dedup (via `clean`)
|
||||
* - Policy + extensible validation (via `validate`)
|
||||
*/
|
||||
export class EmailClient extends PuterClient {
|
||||
private transport: ReturnType<typeof nodemailer.createTransport> | null =
|
||||
@@ -152,7 +151,9 @@ export class EmailClient extends PuterClient {
|
||||
|
||||
// -- Public API: sending ------------------------------------------
|
||||
|
||||
/** Render a template and send it to `to`. */
|
||||
/**
|
||||
* Render a template and send it to `to`.
|
||||
*/
|
||||
async send<T extends EmailTemplateName>(
|
||||
to: string,
|
||||
template: T,
|
||||
@@ -172,12 +173,12 @@ export class EmailClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw send — bypasses the template system. Useful for one-off admin emails
|
||||
* that don't warrant a named template.
|
||||
* Raw send — bypasses the template system. Useful for one-off
|
||||
* admin emails that don't warrant a named template.
|
||||
*
|
||||
* Returns the transport's send result, or `null` when no transport is
|
||||
* configured (the send is a no-op in that case — callers that must not
|
||||
* silently drop mail should check `isConfigured` first).
|
||||
* Returns the transport's send result, or `null` when no transport
|
||||
* is configured (the send is a no-op in that case — callers that
|
||||
* must not silently drop mail should check `isConfigured` first).
|
||||
*/
|
||||
async sendRaw(options: SendMailOptions) {
|
||||
if (!this.transport) {
|
||||
@@ -201,9 +202,9 @@ export class EmailClient extends PuterClient {
|
||||
// -- Public API: clean / validate ---------------------------------
|
||||
|
||||
/**
|
||||
* Normalize an email to its canonical form for dedup comparisons. Applies
|
||||
* provider-specific rules (e.g. Gmail ignores dots in the local part) plus
|
||||
* generic subaddressing removal.
|
||||
* Normalize an email to its canonical form for dedup comparisons.
|
||||
* Applies provider-specific rules (e.g. Gmail ignores dots in
|
||||
* the local part) plus generic subaddressing removal.
|
||||
*/
|
||||
clean(email: string): string {
|
||||
let [local, domain] = email.split('@');
|
||||
@@ -232,9 +233,9 @@ export class EmailClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether an email is allowed to be used. Checks domain blocklist
|
||||
* plus any registered validators (services can call `addValidator()` to
|
||||
* register custom policy hooks).
|
||||
* Check whether an email is allowed to be used. Checks domain
|
||||
* blocklist plus any registered validators (services can call
|
||||
* `addValidator()` to register custom policy hooks).
|
||||
*/
|
||||
async validate(email: string): Promise<boolean> {
|
||||
if (this.config.env === 'dev') return true;
|
||||
@@ -257,8 +258,9 @@ export class EmailClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Register a custom validation hook. Services can call this during their
|
||||
* startup to veto specific emails (e.g. a disposable-email service).
|
||||
* Register a custom validation hook. Services can call this
|
||||
* during their startup to veto specific emails (e.g. a
|
||||
* disposable-email service).
|
||||
*/
|
||||
addValidator(fn: EmailValidator): void {
|
||||
this.validators.push(fn);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -18,12 +18,12 @@
|
||||
*/
|
||||
|
||||
/**
|
||||
* Email template definitions. Keys are the template names; values are the
|
||||
* Handlebars-compilable `subject` and `html` strings.
|
||||
* Email template definitions. Keys are the template names; values are
|
||||
* the Handlebars-compilable `subject` and `html` strings.
|
||||
*
|
||||
* Rendered values are supplied by callers of `EmailClient.send()`. Variables
|
||||
* use standard Handlebars syntax: `{{var}}`, `{{#if cond}}…{{/if}}`, and the
|
||||
* custom helper `{{{nl2br text}}}` for HTML-safe newline conversion.
|
||||
* Rendered values are supplied by callers of `EmailClient.send()`.
|
||||
* Variables use standard Handlebars syntax: `{{var}}`, `{{#if cond}}…{{/if}}`,
|
||||
* and the custom helper `{{{nl2br text}}}` for HTML-safe newline conversion.
|
||||
*/
|
||||
|
||||
export interface EmailTemplate {
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
@@ -8,7 +7,6 @@ import {
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import { extensionStore } from '../../extensions.ts';
|
||||
import { PuterServer } from '../../server.ts';
|
||||
import { setupTestServer } from '../../testUtil.ts';
|
||||
import type { EventClient } from './EventClient.js';
|
||||
@@ -23,11 +21,12 @@ describe('EventClient', () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
// Each test gets a fresh key so listeners registered by earlier tests
|
||||
// never collide with later ones.
|
||||
// never collide with later ones (the client has no public way to
|
||||
// unsubscribe).
|
||||
let key: string;
|
||||
beforeEach(() => {
|
||||
key = `test.${Math.random().toString(36).slice(2)}`;
|
||||
@@ -185,74 +184,6 @@ describe('EventClient', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('off', () => {
|
||||
it('stops delivering to a removed listener', () => {
|
||||
const kept = vi.fn();
|
||||
const removed = vi.fn();
|
||||
target.on(key, kept);
|
||||
target.on(key, removed);
|
||||
|
||||
target.off(key, removed);
|
||||
target.emit(key, {}, {});
|
||||
|
||||
expect(kept).toHaveBeenCalledTimes(1);
|
||||
expect(removed).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('ignores a key nobody ever subscribed to', () => {
|
||||
expect(() => target.off(key, vi.fn())).not.toThrow();
|
||||
});
|
||||
|
||||
it('leaves the list alone when the listener was never registered', () => {
|
||||
const listener = vi.fn();
|
||||
target.on(key, listener);
|
||||
|
||||
target.off(key, vi.fn());
|
||||
target.emit(key, {}, {});
|
||||
|
||||
expect(listener).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extension-registered listeners', () => {
|
||||
afterEach(() => {
|
||||
delete extensionStore.events[key];
|
||||
});
|
||||
|
||||
it('fires alongside listeners registered on the client', () => {
|
||||
const fromExtension = vi.fn();
|
||||
const fromClient = vi.fn();
|
||||
extensionStore.events[key] = [fromExtension];
|
||||
target.on(key, fromClient);
|
||||
|
||||
target.emit(key, { n: 1 }, { source: 'test' });
|
||||
|
||||
expect(fromExtension).toHaveBeenCalledWith(
|
||||
key,
|
||||
{ n: 1 },
|
||||
{ source: 'test' },
|
||||
);
|
||||
expect(fromClient).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('is awaited by emitAndWait', async () => {
|
||||
const order: string[] = [];
|
||||
extensionStore.events[key] = [
|
||||
async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||
order.push('extension');
|
||||
},
|
||||
];
|
||||
target.on(key, () => {
|
||||
order.push('client');
|
||||
});
|
||||
|
||||
await target.emitAndWait(key, {}, {});
|
||||
|
||||
expect(order).toEqual(['client', 'extension']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('lifecycle hooks', () => {
|
||||
it('onServerStart emits a serverStart event', () => {
|
||||
const listener = vi.fn();
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -44,19 +44,20 @@ export class EventClient extends PuterClient {
|
||||
/**
|
||||
* Dispatch an event to every matching subscriber.
|
||||
*
|
||||
* Match semantics: emit walks every dot-separated prefix of `key`, looking
|
||||
* up `<prefix>.*` listeners for prefixes shorter than the full key, and
|
||||
* exact-key listeners on the final iteration. So emitting
|
||||
* Match semantics: emit walks every dot-separated prefix of `key`,
|
||||
* looking up `<prefix>.*` listeners for prefixes shorter than the full
|
||||
* key, and exact-key listeners on the final iteration. So emitting
|
||||
* `outer.gui.item.removed` fires subscribers on:
|
||||
*
|
||||
* - `outer.*`
|
||||
* - `outer.gui.*`
|
||||
* - `outer.gui.item.*`
|
||||
* - `outer.gui.item.removed`
|
||||
* - `outer.*`
|
||||
* - `outer.gui.*`
|
||||
* - `outer.gui.item.*`
|
||||
* - `outer.gui.item.removed`
|
||||
*
|
||||
* Subscribers are still keyed in a single map — wildcards just live
|
||||
* under their literal `<prefix>.*` string. No regex, no per-emit
|
||||
* scan of every listener.
|
||||
*
|
||||
* Subscribers are still keyed in a single map — wildcards just live under
|
||||
* their literal `<prefix>.*` string. No regex, no per-emit scan of every
|
||||
* listener.
|
||||
*/
|
||||
emit<T extends keyof EventMap>(
|
||||
key: T,
|
||||
@@ -84,15 +85,15 @@ export class EventClient extends PuterClient {
|
||||
/**
|
||||
* Like `emit`, but awaits every matched listener before resolving.
|
||||
*
|
||||
* Use this when the emitter needs to act on mutations the handlers made to
|
||||
* `data` — e.g. validation hooks where a listener can set `data.allow =
|
||||
* false` to reject, or pre-commit pipelines where every stage must complete
|
||||
* before the next step runs. Regular `emit` is fire-and-forget and can't
|
||||
* observe handler state changes.
|
||||
* Use this when the emitter needs to act on mutations the handlers made
|
||||
* to `data` — e.g. validation hooks where a listener can set
|
||||
* `data.allow = false` to reject, or pre-commit pipelines where every
|
||||
* stage must complete before the next step runs. Regular `emit` is
|
||||
* fire-and-forget and can't observe handler state changes.
|
||||
*
|
||||
* Listeners run sequentially in the order they're registered so an earlier
|
||||
* handler's mutation is visible to later ones. A listener that throws is
|
||||
* logged (same as `emit`) and the chain continues.
|
||||
* Listeners run sequentially in the order they're registered so an
|
||||
* earlier handler's mutation is visible to later ones. A listener that
|
||||
* throws is logged (same as `emit`) and the chain continues.
|
||||
*/
|
||||
async emitAndWait<T extends keyof EventMap>(
|
||||
key: T,
|
||||
@@ -132,13 +133,15 @@ export class EventClient extends PuterClient {
|
||||
/**
|
||||
* Subscribe to an event by exact key OR a wildcard prefix.
|
||||
*
|
||||
* Wildcards: a key ending in `.*` matches every event whose name starts
|
||||
* with the prefix. `outer.*` matches `outer.gui.item.removed`,
|
||||
* `outer.fs.write-hash`, and any other dot-extended descendant. Exact keys
|
||||
* still match exactly. See `emit()` for the dispatch order.
|
||||
* Wildcards: a key ending in `.*` matches every event whose name
|
||||
* starts with the prefix. `outer.*` matches `outer.gui.item.removed`,
|
||||
* `outer.fs.write-hash`, and any other dot-extended descendant.
|
||||
* Exact keys still match exactly. See `emit()` for the dispatch order.
|
||||
*
|
||||
* Callback receives the full `(key, data, meta)` tuple as passed
|
||||
* to `emit()` — wildcard subscribers can branch on the triggering
|
||||
* event name.
|
||||
*
|
||||
* Callback receives the full `(key, data, meta)` tuple as passed to
|
||||
* `emit()` — wildcard subscribers can branch on the triggering event name.
|
||||
*/
|
||||
on<P extends ListenKey>(
|
||||
key: P,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -74,11 +74,11 @@ export type PreludeCheckStatus =
|
||||
| 'transaction_mismatch';
|
||||
|
||||
/**
|
||||
* Prelude Verify v2 client (https://docs.prelude.so/verify/v2). Sends and
|
||||
* checks SMS one-time codes — Prelude generates, delivers, and validates the
|
||||
* code, so we never store one ourselves. No-ops with a warning when no API key
|
||||
* is configured (so dev environments don't crash); `isConfigured()` lets
|
||||
* callers surface a clean "phone verification unavailable" error instead.
|
||||
* Prelude Verify v2 client (https://docs.prelude.so/verify/v2). Sends and checks
|
||||
* SMS one-time codes — Prelude generates, delivers, and validates the code, so
|
||||
* we never store one ourselves. No-ops with a warning when no API key is
|
||||
* configured (so dev environments don't crash); `isConfigured()` lets callers
|
||||
* surface a clean "phone verification unavailable" error instead.
|
||||
*/
|
||||
export class PreludeClient extends PuterClient {
|
||||
private apiKey: string | null = null;
|
||||
@@ -113,9 +113,7 @@ export class PreludeClient extends PuterClient {
|
||||
|
||||
/**
|
||||
* Whether SMS verification should be offered for a country. False when the
|
||||
* country is unknown, has no SMS channel, or its rate exceeds the cost
|
||||
* cap.
|
||||
*
|
||||
* country is unknown, has no SMS channel, or its rate exceeds the cost cap.
|
||||
* @param iso ISO-3166 alpha-2 (e.g. 'US') — from the parsed phone number.
|
||||
*/
|
||||
isCountrySupported(iso: string | undefined): boolean {
|
||||
@@ -176,9 +174,7 @@ export class PreludeClient extends PuterClient {
|
||||
}
|
||||
|
||||
/**
|
||||
* Check a code the user entered against the active verification for
|
||||
* `target`.
|
||||
*
|
||||
* Check a code the user entered against the active verification for `target`.
|
||||
* @returns `{ status }` — `'success'` means verified.
|
||||
*/
|
||||
async checkVerification(
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -17,13 +17,11 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Per-country Prelude SMS pricing, hardcoded from the Prelude global price
|
||||
/** Per-country Prelude SMS pricing, hardcoded from the Prelude global price
|
||||
* sheet. `sms` is the average cost in EUR, or null when SMS is unavailable
|
||||
* there. Used to cap phone verification to affordable countries (see
|
||||
* PreludeClient.isCountrySupported). Regenerate from the price sheet when
|
||||
* Prelude updates rates.
|
||||
*/
|
||||
* Prelude updates rates. */
|
||||
export interface CountryPrice {
|
||||
name: string;
|
||||
/** Prelude SMS average in EUR, or null when SMS is not available. */
|
||||
|
||||
@@ -1,171 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import { RedisClient } from './RedisClient';
|
||||
|
||||
const config = (redis: Record<string, unknown>): IConfig =>
|
||||
({ port: 0, extensions: [], redis }) as unknown as IConfig;
|
||||
|
||||
type ClusterOptions = {
|
||||
redisOptions?: { tls?: unknown; connectTimeout?: number };
|
||||
clusterRetryStrategy?: (attempts: number) => number;
|
||||
};
|
||||
|
||||
const optionsOf = (client: RedisClient): ClusterOptions =>
|
||||
(client as unknown as { options: ClusterOptions }).options;
|
||||
|
||||
describe('RedisClient — in-process mock', () => {
|
||||
let client: RedisClient | null = null;
|
||||
|
||||
afterEach(async () => {
|
||||
if (client) await client.onServerShutdown();
|
||||
client = null;
|
||||
});
|
||||
|
||||
it('falls back to the mock when no startup nodes are configured', async () => {
|
||||
client = new RedisClient(config({}));
|
||||
await client.set('greeting', 'hello');
|
||||
await expect(client.get('greeting')).resolves.toBe('hello');
|
||||
});
|
||||
|
||||
it('honours an explicit mock request even with startup nodes present', async () => {
|
||||
client = new RedisClient(
|
||||
config({
|
||||
useMock: true,
|
||||
startupNodes: [{ host: 'redis.internal', port: 6379 }],
|
||||
}),
|
||||
);
|
||||
await client.set('k', 'v');
|
||||
await expect(client.get('k')).resolves.toBe('v');
|
||||
});
|
||||
|
||||
it('exposes the shutdown hook directly on the cluster instance', () => {
|
||||
client = new RedisClient(config({}));
|
||||
expect(typeof client.onServerShutdown).toBe('function');
|
||||
});
|
||||
|
||||
it('disconnects when a clean quit fails', async () => {
|
||||
const instance = new RedisClient(config({}));
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
vi.spyOn(instance, 'quit').mockRejectedValue(new Error('quit refused'));
|
||||
const disconnect = vi.spyOn(instance, 'disconnect');
|
||||
|
||||
await instance.onServerShutdown();
|
||||
|
||||
expect(disconnect).toHaveBeenCalledTimes(1);
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
'[redis] failed to quit redis client cleanly',
|
||||
expect.any(Error),
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('RedisClient — real cluster wiring', () => {
|
||||
let client: RedisClient | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
client?.disconnect();
|
||||
client = null;
|
||||
});
|
||||
|
||||
const startupNodes = [{ host: '127.0.0.1', port: 7001 }];
|
||||
|
||||
it('enables TLS by default and bounds the connect timeout', () => {
|
||||
client = new RedisClient(config({ startupNodes }));
|
||||
expect(optionsOf(client).redisOptions?.tls).toEqual({});
|
||||
expect(optionsOf(client).redisOptions?.connectTimeout).toBe(10000);
|
||||
});
|
||||
|
||||
it('lets a plain-TCP deployment opt out of TLS', () => {
|
||||
client = new RedisClient(config({ startupNodes, tls: false }));
|
||||
expect(optionsOf(client).redisOptions?.tls).toBeUndefined();
|
||||
});
|
||||
|
||||
it('backs off on cluster retries up to a ceiling', () => {
|
||||
client = new RedisClient(config({ startupNodes, tls: false }));
|
||||
const retry = optionsOf(client).clusterRetryStrategy!;
|
||||
expect(retry(0)).toBe(100);
|
||||
expect(retry(5)).toBe(600);
|
||||
expect(retry(100)).toBe(2000);
|
||||
});
|
||||
|
||||
it('treats startup churn as a warning and everything else as an error', () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const error = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
client = new RedisClient(config({ startupNodes, tls: false }));
|
||||
|
||||
client.emit('error', new Error('None of startup nodes is available'));
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining('startup issue while connecting'),
|
||||
);
|
||||
|
||||
const fatal = new Error('WRONGPASS invalid credentials');
|
||||
client.emit('error', fatal);
|
||||
expect(error).toHaveBeenCalledWith('[redis] cluster error', fatal);
|
||||
|
||||
warn.mockRestore();
|
||||
error.mockRestore();
|
||||
});
|
||||
|
||||
it('applies the same triage to per-node errors', () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const error = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
client = new RedisClient(config({ startupNodes, tls: false }));
|
||||
|
||||
const transient = new Error('boom');
|
||||
transient.name = 'ClusterAllFailedError';
|
||||
client.emit('node error', transient, '127.0.0.1:7001');
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining(
|
||||
'startup issue for cluster node 127.0.0.1:7001',
|
||||
),
|
||||
);
|
||||
|
||||
const fatal = new Error('node exploded');
|
||||
client.emit('node error', fatal, '127.0.0.1:7002');
|
||||
expect(error).toHaveBeenCalledWith(
|
||||
'[redis] cluster node error (127.0.0.1:7002)',
|
||||
fatal,
|
||||
);
|
||||
|
||||
// Non-Error payloads are stringified rather than crashing the handler.
|
||||
client.emit('node error', 'None of startup nodes is available', 'n3');
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining('startup issue for cluster node n3'),
|
||||
);
|
||||
|
||||
warn.mockRestore();
|
||||
error.mockRestore();
|
||||
});
|
||||
|
||||
it('logs the transport lifecycle once each', () => {
|
||||
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
client = new RedisClient(config({ startupNodes, tls: false }));
|
||||
|
||||
client.emit('connect');
|
||||
client.emit('ready');
|
||||
|
||||
expect(log).toHaveBeenCalledWith('[redis] cluster transport connected');
|
||||
expect(log).toHaveBeenCalledWith('[redis] cluster ready');
|
||||
log.mockRestore();
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -108,13 +108,13 @@ const buildCluster = (config: IConfig): Cluster => {
|
||||
|
||||
/**
|
||||
* `RedisClient` IS the ioredis `Cluster` instance — consumers call
|
||||
* `this.clients.redis.get(...)` / `.set(...)` directly rather than going
|
||||
* through an inner `.client` field. Lifecycle methods (`onServerShutdown`) are
|
||||
* attached onto the cluster instance itself.
|
||||
* `this.clients.redis.get(...)` / `.set(...)` directly rather than
|
||||
* going through an inner `.client` field. Lifecycle methods
|
||||
* (`onServerShutdown`) are attached onto the cluster instance itself.
|
||||
*
|
||||
* Type-wise, `RedisClient` is `Cluster & WithLifecycle`; the registry- facing
|
||||
* value below is a constructor that returns that shape. Mirrors the
|
||||
* `DatabaseClientFactory` pattern.
|
||||
* Type-wise, `RedisClient` is `Cluster & WithLifecycle`; the registry-
|
||||
* facing value below is a constructor that returns that shape. Mirrors
|
||||
* the `DatabaseClientFactory` pattern.
|
||||
*/
|
||||
export type RedisClient = Cluster & WithLifecycle;
|
||||
|
||||
|
||||
@@ -1,489 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { GetObjectCommand } from '@aws-sdk/client-s3';
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types';
|
||||
import { S3Client } from './S3Client';
|
||||
|
||||
const LOCAL_BUCKET = 'puter-local';
|
||||
|
||||
/**
|
||||
* The multipart helper takes its command sender as a parameter, so the failure
|
||||
* paths can be driven without a live endpoint.
|
||||
*/
|
||||
type S3Internals = {
|
||||
awsConfig: { endpoint?: string };
|
||||
migrateLegacyStorage(opts?: {
|
||||
bucket?: string;
|
||||
legacyPath?: string;
|
||||
}): Promise<{ migratedFileCount: number; scannedEntryCount: number }>;
|
||||
uploadMultipart(args: {
|
||||
bucket: string;
|
||||
client: { send: (command: unknown) => Promise<unknown> };
|
||||
filePath: string;
|
||||
fileSize: number;
|
||||
key: string;
|
||||
}): Promise<void>;
|
||||
};
|
||||
|
||||
const internals = (client: S3Client) => client as unknown as S3Internals;
|
||||
|
||||
const localConfig = (): IConfig =>
|
||||
({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
s3: { localConfig: { inMemory: true } },
|
||||
}) as unknown as IConfig;
|
||||
|
||||
const startLocal = async (): Promise<S3Client> => {
|
||||
const client = new S3Client(localConfig());
|
||||
await client.onServerStart();
|
||||
return client;
|
||||
};
|
||||
|
||||
const objectBody = async (
|
||||
client: S3Client,
|
||||
key: string,
|
||||
): Promise<string | undefined> => {
|
||||
const result = await client
|
||||
.get()
|
||||
.send(new GetObjectCommand({ Bucket: LOCAL_BUCKET, Key: key }));
|
||||
return result.Body?.transformToString();
|
||||
};
|
||||
|
||||
describe('S3Client — local development endpoint', () => {
|
||||
let client: S3Client;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await startLocal();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('points clients at the in-process object store', async () => {
|
||||
await client
|
||||
.get()
|
||||
.send(
|
||||
new GetObjectCommand({
|
||||
Bucket: LOCAL_BUCKET,
|
||||
Key: 'definitely-absent',
|
||||
}),
|
||||
)
|
||||
.then(
|
||||
() => {
|
||||
throw new Error('expected a missing-key failure');
|
||||
},
|
||||
(error: { name?: string }) => {
|
||||
expect(error.name).toBe('NoSuchKey');
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('reuses one client per region and mints a new one per region', () => {
|
||||
const first = client.get('us-west-2');
|
||||
expect(client.get('us-west-2')).toBe(first);
|
||||
expect(client.get('eu-central-1')).not.toBe(first);
|
||||
});
|
||||
|
||||
it('serves presigning from the same client when no public endpoint is set', () => {
|
||||
expect(client.getForPresign('us-west-2')).toBe(client.get('us-west-2'));
|
||||
});
|
||||
|
||||
it('keeps modest upload thresholds so the local store is not overwhelmed', () => {
|
||||
expect(client.maxSingleUploadSize).toBe(10 * 1024 * 1024);
|
||||
expect(client.partSize).toBe(5 * 1024 * 1024);
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — remote endpoint configuration', () => {
|
||||
let local: S3Client;
|
||||
let remote: S3Client | null = null;
|
||||
|
||||
beforeEach(async () => {
|
||||
local = await startLocal();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (remote) await remote.onServerShutdown();
|
||||
remote = null;
|
||||
await local.onServerShutdown();
|
||||
});
|
||||
|
||||
const remoteConfig = (extra: Record<string, unknown>): IConfig =>
|
||||
({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
s3: {
|
||||
s3Config: {
|
||||
endpoint: internals(local).awsConfig.endpoint,
|
||||
accessKeyId: 'fakeAccessKeyId',
|
||||
secretAccessKey: 'fakeSecretAccessKey',
|
||||
region: 'us-west-2',
|
||||
forcePathStyle: true,
|
||||
...extra,
|
||||
},
|
||||
},
|
||||
}) as unknown as IConfig;
|
||||
|
||||
it('talks to a configured S3-compatible endpoint', async () => {
|
||||
remote = new S3Client(remoteConfig({}));
|
||||
await remote.onServerStart();
|
||||
|
||||
await remote
|
||||
.get()
|
||||
.send(new GetObjectCommand({ Bucket: LOCAL_BUCKET, Key: 'nope' }))
|
||||
.then(
|
||||
() => {
|
||||
throw new Error('expected a missing-key failure');
|
||||
},
|
||||
(error: { name?: string }) => {
|
||||
expect(error.name).toBe('NoSuchKey');
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('signs browser-facing URLs against the public endpoint', async () => {
|
||||
remote = new S3Client(
|
||||
remoteConfig({ publicEndpoint: 'https://cdn.example.test' }),
|
||||
);
|
||||
await remote.onServerStart();
|
||||
|
||||
const presign = remote.getForPresign('us-west-2');
|
||||
expect(presign).not.toBe(remote.get('us-west-2'));
|
||||
// Cached per region like the regular client map.
|
||||
expect(remote.getForPresign('us-west-2')).toBe(presign);
|
||||
expect(remote.getForPresign('eu-central-1')).not.toBe(presign);
|
||||
});
|
||||
|
||||
it('shares one client when the public endpoint matches the private one', async () => {
|
||||
const endpoint = internals(local).awsConfig.endpoint;
|
||||
remote = new S3Client(remoteConfig({ publicEndpoint: endpoint }));
|
||||
await remote.onServerStart();
|
||||
|
||||
expect(remote.getForPresign('us-west-2')).toBe(remote.get('us-west-2'));
|
||||
});
|
||||
|
||||
it('raises the upload thresholds when using the ambient credential chain', async () => {
|
||||
remote = new S3Client({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
s3: { s3Config: { useCredentialChain: true } },
|
||||
} as unknown as IConfig);
|
||||
await remote.onServerStart();
|
||||
|
||||
expect(remote.partSize).toBe(64 * 1024 * 1024);
|
||||
expect(remote.maxSingleUploadSize).toBe(128 * 1024 * 1024);
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — legacy storage migration', () => {
|
||||
let client: S3Client;
|
||||
let dir: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await startLocal();
|
||||
dir = mkdtempSync(join(tmpdir(), 'puter-s3-legacy-'));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
await client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('does nothing when there is no legacy directory', async () => {
|
||||
await expect(
|
||||
internals(client).migrateLegacyStorage({
|
||||
legacyPath: join(dir, 'absent'),
|
||||
}),
|
||||
).resolves.toEqual({ migratedFileCount: 0, scannedEntryCount: 0 });
|
||||
});
|
||||
|
||||
it('uploads each legacy file, skips directories, then removes the tree', async () => {
|
||||
const small = 'small-file-contents';
|
||||
writeFileSync(join(dir, 'small.txt'), small);
|
||||
writeFileSync(join(dir, 'other.txt'), 'other');
|
||||
mkdirSync(join(dir, 'a-subdirectory'));
|
||||
|
||||
await expect(
|
||||
internals(client).migrateLegacyStorage({ legacyPath: dir }),
|
||||
).resolves.toEqual({ migratedFileCount: 2, scannedEntryCount: 3 });
|
||||
|
||||
await expect(objectBody(client, 'small.txt')).resolves.toBe(small);
|
||||
await expect(objectBody(client, 'other.txt')).resolves.toBe('other');
|
||||
expect(existsSync(dir)).toBe(false);
|
||||
});
|
||||
|
||||
it('switches to a multipart upload for a file over the single-put limit', async () => {
|
||||
// S3 requires every part but the last to be at least 5 MiB, so the
|
||||
// fixture has to straddle that for real.
|
||||
client.maxSingleUploadSize = 1024 * 1024;
|
||||
client.partSize = 5 * 1024 * 1024;
|
||||
const body = 'X'.repeat(6 * 1024 * 1024);
|
||||
writeFileSync(join(dir, 'large.bin'), body);
|
||||
|
||||
await expect(
|
||||
internals(client).migrateLegacyStorage({ legacyPath: dir }),
|
||||
).resolves.toEqual({ migratedFileCount: 1, scannedEntryCount: 1 });
|
||||
await expect(objectBody(client, 'large.bin')).resolves.toBe(body);
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — multipart failure handling', () => {
|
||||
let client: S3Client;
|
||||
let dir: string;
|
||||
let filePath: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await startLocal();
|
||||
dir = mkdtempSync(join(tmpdir(), 'puter-s3-multipart-'));
|
||||
filePath = join(dir, 'part-source.bin');
|
||||
writeFileSync(filePath, 'Y'.repeat(30));
|
||||
client.partSize = 10;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
await client.onServerShutdown();
|
||||
});
|
||||
|
||||
const commandName = (command: unknown) =>
|
||||
(command as { constructor: { name: string } }).constructor.name;
|
||||
|
||||
it('fails when the store will not open a multipart upload', async () => {
|
||||
const send = vi.fn(async () => ({}));
|
||||
|
||||
await expect(
|
||||
internals(client).uploadMultipart({
|
||||
bucket: LOCAL_BUCKET,
|
||||
client: { send },
|
||||
filePath,
|
||||
fileSize: 30,
|
||||
key: 'no-upload-id',
|
||||
}),
|
||||
).rejects.toThrow('Failed to start multipart upload');
|
||||
});
|
||||
|
||||
it('aborts the upload when a part comes back without an ETag', async () => {
|
||||
const sent: string[] = [];
|
||||
const send = vi.fn(async (command: unknown) => {
|
||||
const name = commandName(command);
|
||||
sent.push(name);
|
||||
if (name === 'CreateMultipartUploadCommand') {
|
||||
return { UploadId: 'upload-1' };
|
||||
}
|
||||
if (name === 'UploadPartCommand') return {};
|
||||
return {};
|
||||
});
|
||||
|
||||
await expect(
|
||||
internals(client).uploadMultipart({
|
||||
bucket: LOCAL_BUCKET,
|
||||
client: { send },
|
||||
filePath,
|
||||
fileSize: 30,
|
||||
key: 'etag-less',
|
||||
}),
|
||||
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
|
||||
|
||||
expect(sent).toEqual([
|
||||
'CreateMultipartUploadCommand',
|
||||
'UploadPartCommand',
|
||||
'AbortMultipartUploadCommand',
|
||||
]);
|
||||
});
|
||||
|
||||
it('sends one part per chunk and completes with every ETag', async () => {
|
||||
const parts: { PartNumber: number; ContentLength: number }[] = [];
|
||||
let completed: { ETag: string; PartNumber: number }[] | undefined;
|
||||
const send = vi.fn(async (command: unknown) => {
|
||||
const name = commandName(command);
|
||||
const input = (command as { input: Record<string, unknown> }).input;
|
||||
if (name === 'CreateMultipartUploadCommand') {
|
||||
return { UploadId: 'upload-1' };
|
||||
}
|
||||
if (name === 'UploadPartCommand') {
|
||||
parts.push({
|
||||
PartNumber: input.PartNumber as number,
|
||||
ContentLength: input.ContentLength as number,
|
||||
});
|
||||
return { ETag: `etag-${input.PartNumber}` };
|
||||
}
|
||||
if (name === 'CompleteMultipartUploadCommand') {
|
||||
completed = (
|
||||
input.MultipartUpload as {
|
||||
Parts: { ETag: string; PartNumber: number }[];
|
||||
}
|
||||
).Parts;
|
||||
}
|
||||
return {};
|
||||
});
|
||||
|
||||
client.partSize = 12;
|
||||
await internals(client).uploadMultipart({
|
||||
bucket: LOCAL_BUCKET,
|
||||
client: { send },
|
||||
filePath,
|
||||
fileSize: 30,
|
||||
key: 'chunked',
|
||||
});
|
||||
|
||||
expect(parts).toEqual([
|
||||
{ PartNumber: 1, ContentLength: 12 },
|
||||
{ PartNumber: 2, ContentLength: 12 },
|
||||
{ PartNumber: 3, ContentLength: 6 },
|
||||
]);
|
||||
expect(completed).toEqual([
|
||||
{ ETag: 'etag-1', PartNumber: 1 },
|
||||
{ ETag: 'etag-2', PartNumber: 2 },
|
||||
{ ETag: 'etag-3', PartNumber: 3 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('aborts and rethrows when completing the upload fails', async () => {
|
||||
const sent: string[] = [];
|
||||
const send = vi.fn(async (command: unknown) => {
|
||||
const name = commandName(command);
|
||||
sent.push(name);
|
||||
if (name === 'CreateMultipartUploadCommand') {
|
||||
return { UploadId: 'upload-1' };
|
||||
}
|
||||
if (name === 'UploadPartCommand') return { ETag: 'etag' };
|
||||
if (name === 'CompleteMultipartUploadCommand') {
|
||||
throw new Error('complete rejected');
|
||||
}
|
||||
return {};
|
||||
});
|
||||
|
||||
await expect(
|
||||
internals(client).uploadMultipart({
|
||||
bucket: LOCAL_BUCKET,
|
||||
client: { send },
|
||||
filePath,
|
||||
fileSize: 30,
|
||||
key: 'incomplete',
|
||||
}),
|
||||
).rejects.toThrow('complete rejected');
|
||||
|
||||
expect(sent.at(-1)).toBe('AbortMultipartUploadCommand');
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — disk-backed local store', () => {
|
||||
let cwd: string;
|
||||
let dir: string;
|
||||
let client: S3Client | null = null;
|
||||
|
||||
beforeEach(() => {
|
||||
cwd = process.cwd();
|
||||
dir = mkdtempSync(join(tmpdir(), 'puter-s3-disk-'));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (client) await client.onServerShutdown();
|
||||
client = null;
|
||||
process.chdir(cwd);
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('persists to the configured directories and drains legacy storage', async () => {
|
||||
// The legacy sweep resolves `storage/` against the working
|
||||
// directory, so the whole fixture lives in a throwaway cwd.
|
||||
process.chdir(dir);
|
||||
mkdirSync(join(dir, 'storage'));
|
||||
writeFileSync(join(dir, 'storage', 'legacy.txt'), 'from-disk');
|
||||
|
||||
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
client = new S3Client({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
s3: {
|
||||
localConfig: {
|
||||
host: '127.0.0.1',
|
||||
port: 0,
|
||||
dataDir: join(dir, 'fauxqs-data'),
|
||||
s3StorageDir: join(dir, 'fauxqs-s3'),
|
||||
},
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
await client.onServerStart();
|
||||
|
||||
expect(log).toHaveBeenCalledWith(
|
||||
'[s3] migrated 1 file(s) from legacy storage',
|
||||
);
|
||||
await expect(objectBody(client, 'legacy.txt')).resolves.toBe(
|
||||
'from-disk',
|
||||
);
|
||||
expect(existsSync(join(dir, 'storage'))).toBe(false);
|
||||
log.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — short reads during multipart', () => {
|
||||
let client: S3Client;
|
||||
let dir: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await startLocal();
|
||||
dir = mkdtempSync(join(tmpdir(), 'puter-s3-short-'));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
await client.onServerShutdown();
|
||||
});
|
||||
|
||||
it('stops uploading parts once the file runs out early', async () => {
|
||||
const filePath = join(dir, 'truncated.bin');
|
||||
writeFileSync(filePath, 'Z'.repeat(20));
|
||||
client.partSize = 10;
|
||||
|
||||
const parts: number[] = [];
|
||||
const send = vi.fn(async (command: unknown) => {
|
||||
const name = (command as { constructor: { name: string } })
|
||||
.constructor.name;
|
||||
const input = (command as { input: Record<string, unknown> }).input;
|
||||
if (name === 'CreateMultipartUploadCommand') {
|
||||
return { UploadId: 'upload-1' };
|
||||
}
|
||||
if (name === 'UploadPartCommand') {
|
||||
parts.push(input.ContentLength as number);
|
||||
return { ETag: `etag-${input.PartNumber}` };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
|
||||
// The caller's size is stale — the file is shorter than advertised.
|
||||
await internals(client).uploadMultipart({
|
||||
bucket: LOCAL_BUCKET,
|
||||
client: { send },
|
||||
filePath,
|
||||
fileSize: 50,
|
||||
key: 'truncated',
|
||||
});
|
||||
|
||||
expect(parts).toEqual([10, 10]);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -184,8 +184,8 @@ export class S3Client extends PuterClient {
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Get (or create) an S3Client for the given region. Clients are cached
|
||||
* per-region for connection reuse.
|
||||
* Get (or create) an S3Client for the given region.
|
||||
* Clients are cached per-region for connection reuse.
|
||||
*/
|
||||
get(
|
||||
region = this.config.s3_region || this.config.region || 'us-west-2',
|
||||
@@ -213,10 +213,10 @@ export class S3Client extends PuterClient {
|
||||
|
||||
/**
|
||||
* Client used to generate browser-facing presigned URLs. When
|
||||
* `s3Config.publicEndpoint` is set, this returns a client bound to that
|
||||
* endpoint — its signatures resolve against the public host the browser
|
||||
* will actually hit. When unset, falls back to the regular client (prod
|
||||
* behavior: one public endpoint everywhere).
|
||||
* `s3Config.publicEndpoint` is set, this returns a client bound to
|
||||
* that endpoint — its signatures resolve against the public host
|
||||
* the browser will actually hit. When unset, falls back to the
|
||||
* regular client (prod behavior: one public endpoint everywhere).
|
||||
*/
|
||||
getForPresign(
|
||||
region = this.config.s3_region || this.config.region || 'us-west-2',
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -45,8 +45,8 @@ export interface IExtensionClientInstances {
|
||||
/**
|
||||
* Optional ClickHouse analytics client. Absent by default — a production
|
||||
* deployment registers it via an extension to speed up the app-stats path
|
||||
* at scale (see {@link ClickhouseClient}). Always branch on its presence and
|
||||
* fall back to SQL.
|
||||
* at scale (see {@link ClickhouseClient}). Always branch on its presence
|
||||
* and fall back to SQL.
|
||||
*/
|
||||
clickhouse?: ClickhouseClient;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -31,66 +31,15 @@ import DEFAULT_APP_ICON from './default-app-icon.js';
|
||||
/**
|
||||
* REST endpoints for app management.
|
||||
*
|
||||
* Delegates to AppDriver for the actual CRUD + permission logic — these routes
|
||||
* are just thin shape adapters that translate REST conventions into driver
|
||||
* calls.
|
||||
* Delegates to AppDriver for the actual CRUD + permission logic —
|
||||
* these routes are just thin shape adapters that translate REST
|
||||
* conventions into driver calls.
|
||||
*/
|
||||
export class AppController extends PuterController {
|
||||
get appStore() {
|
||||
return this.stores.app;
|
||||
}
|
||||
|
||||
// In-flight background app-open writes. Tracked only so tests and
|
||||
// shutdown can wait for them — the request path never does.
|
||||
#pendingOpenWrites = new Set();
|
||||
|
||||
/**
|
||||
* Record an app open. `app_opens` is analytics: it backs the recent-apps
|
||||
* list and the open counters, and no response field is derived from it. The
|
||||
* client posts this without awaiting and updates its own recent list
|
||||
* optimistically, so holding a response open for a primary write only added
|
||||
* latency to the launch that write is measuring.
|
||||
*
|
||||
* Failures are logged, never surfaced — a dropped stat must not turn into a
|
||||
* failed app open.
|
||||
*
|
||||
* @param {string} appUid
|
||||
* @param {number} userId
|
||||
* @returns {Promise<void>} Settles when the write and event emit finish
|
||||
*/
|
||||
#recordAppOpen(appUid, userId) {
|
||||
const ts = Math.floor(Date.now() / 1000);
|
||||
const work = (async () => {
|
||||
try {
|
||||
await this.clients.db.write(
|
||||
'INSERT INTO `app_opens` (`app_uid`, `user_id`, `ts`) VALUES (?, ?, ?)',
|
||||
[appUid, userId, ts],
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[rao] insert failed:', e);
|
||||
}
|
||||
|
||||
try {
|
||||
this.clients.event?.emitAndWait(
|
||||
'app.opened',
|
||||
{ app_uid: appUid, user_id: userId, ts },
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
// event emission best-effort
|
||||
}
|
||||
})();
|
||||
|
||||
this.#pendingOpenWrites.add(work);
|
||||
work.finally(() => this.#pendingOpenWrites.delete(work));
|
||||
return work;
|
||||
}
|
||||
|
||||
/** Await every in-flight app-open write. */
|
||||
async drainPendingAppOpens() {
|
||||
await Promise.allSettled([...this.#pendingOpenWrites]);
|
||||
}
|
||||
|
||||
get appDriver() {
|
||||
// Drivers are wired into the shared driversContainers export by
|
||||
// PuterServer at boot. Controllers get them lazily via this getter
|
||||
@@ -199,10 +148,33 @@ export class AppController extends PuterController {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
|
||||
// Validation and authorization are settled by this point, so
|
||||
// the caller learns the outcome now and the stats write lands
|
||||
// on its own. See `#recordAppOpen`.
|
||||
this.#recordAppOpen(app_uid, req.actor.user.id);
|
||||
// Persist open record
|
||||
try {
|
||||
await this.clients.db.write(
|
||||
'INSERT INTO `app_opens` (`app_uid`, `user_id`, `ts`) VALUES (?, ?, ?)',
|
||||
[
|
||||
app_uid,
|
||||
req.actor.user.id,
|
||||
Math.floor(Date.now() / 1000),
|
||||
],
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[rao] insert failed:', e);
|
||||
}
|
||||
|
||||
try {
|
||||
this.clients.event?.emitAndWait(
|
||||
'app.opened',
|
||||
{
|
||||
app_uid,
|
||||
user_id: req.actor.user.id,
|
||||
ts: Math.floor(Date.now() / 1000),
|
||||
},
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
// event emission best-effort
|
||||
}
|
||||
|
||||
res.json({});
|
||||
},
|
||||
@@ -242,13 +214,7 @@ export class AppController extends PuterController {
|
||||
source: 'appsRoute',
|
||||
args: req.query ?? {},
|
||||
});
|
||||
return {
|
||||
...shaped,
|
||||
privateAccess:
|
||||
shaped.privateAccess?.hasAccess === false
|
||||
? shaped.privateAccess
|
||||
: privateAccess,
|
||||
};
|
||||
return { ...shaped, privateAccess };
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -471,11 +437,7 @@ export class AppController extends PuterController {
|
||||
}
|
||||
setIconSecurityHeaders(res);
|
||||
res.set('Content-Type', mime);
|
||||
// Same freshness as the redirect path above — this is the
|
||||
// same resource, just served inline because no CDN file
|
||||
// exists yet. A 60s TTL made every app launch re-fetch the
|
||||
// icon over a connection the launch itself is competing for.
|
||||
res.set('Cache-Control', 'public, max-age=900');
|
||||
res.set('Cache-Control', 'public, max-age=60');
|
||||
res.send(Buffer.from(icon.slice(commaIdx + 1), 'base64'));
|
||||
|
||||
// Trigger background generation so next request hits the CDN
|
||||
@@ -506,14 +468,6 @@ export class AppController extends PuterController {
|
||||
}
|
||||
|
||||
onServerStart() {}
|
||||
/**
|
||||
* Let outstanding stats writes finish before the process goes away —
|
||||
* otherwise a deploy silently drops every open recorded in the seconds
|
||||
* before it.
|
||||
*/
|
||||
async onServerPrepareShutdown() {
|
||||
await this.drainPendingAppOpens();
|
||||
}
|
||||
|
||||
onServerPrepareShutdown() {}
|
||||
onServerShutdown() {}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -91,11 +91,6 @@ const uniqueName = (prefix: string) =>
|
||||
const uniqueIndexUrl = () =>
|
||||
`https://example-${Math.random().toString(36).slice(2, 10)}.test/`;
|
||||
|
||||
// 1x1 transparent PNG. The `icon` write path validates the decoded payload,
|
||||
// not just the declared MIME, so icon fixtures must be real images.
|
||||
const MINIMAL_PNG_BASE64 =
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
|
||||
|
||||
const createApp = async (
|
||||
actor: Actor,
|
||||
overrides: Record<string, unknown> = {},
|
||||
@@ -290,7 +285,12 @@ describe('AppController POST /rao', () => {
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
callRoute('post', '/rao', makeReq({ body: {}, actor }), res),
|
||||
callRoute(
|
||||
'post',
|
||||
'/rao',
|
||||
makeReq({ body: {}, actor }),
|
||||
res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
@@ -328,9 +328,6 @@ describe('AppController POST /rao', () => {
|
||||
);
|
||||
expect(captured.body).toEqual({});
|
||||
|
||||
// The stats write is deliberately not awaited by the request.
|
||||
await server.controllers.apps.drainPendingAppOpens();
|
||||
|
||||
const rows = (await server.clients.db.read(
|
||||
'SELECT `app_uid`, `user_id` FROM `app_opens` WHERE `app_uid` = ? AND `user_id` = ?',
|
||||
[app.uid, owner.userId],
|
||||
@@ -338,74 +335,6 @@ describe('AppController POST /rao', () => {
|
||||
expect(rows).toHaveLength(1);
|
||||
});
|
||||
|
||||
// `app_opens` is analytics: the response carries nothing derived from it
|
||||
// and the client never awaits the post. Holding the response open for a
|
||||
// primary write put that write's latency inside the app launch it was
|
||||
// measuring, on a connection the launch was competing for.
|
||||
it('responds without waiting for the stats write', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
|
||||
let releaseWrite = () => {};
|
||||
const writeSpy = vi
|
||||
.spyOn(server.clients.db, 'write')
|
||||
.mockImplementation(
|
||||
() =>
|
||||
new Promise((resolve) => {
|
||||
releaseWrite = () => resolve(undefined as never);
|
||||
}),
|
||||
);
|
||||
|
||||
try {
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(owner.actor, () =>
|
||||
callRoute(
|
||||
'post',
|
||||
'/rao',
|
||||
makeReq({ body: { app_uid: app.uid }, actor: owner.actor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
|
||||
// The handler returned while the INSERT is still outstanding.
|
||||
expect(captured.body).toEqual({});
|
||||
expect(writeSpy).toHaveBeenCalled();
|
||||
} finally {
|
||||
releaseWrite();
|
||||
await server.controllers.apps.drainPendingAppOpens();
|
||||
writeSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
// A stats failure must never turn into a failed app open — and since the
|
||||
// write is now unawaited, it must not surface as an unhandled rejection.
|
||||
it('swallows a failing stats write without rejecting the request', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
|
||||
const writeSpy = vi
|
||||
.spyOn(server.clients.db, 'write')
|
||||
.mockRejectedValue(new Error('primary is down'));
|
||||
|
||||
try {
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(owner.actor, () =>
|
||||
callRoute(
|
||||
'post',
|
||||
'/rao',
|
||||
makeReq({ body: { app_uid: app.uid }, actor: owner.actor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
expect(captured.body).toEqual({});
|
||||
await expect(
|
||||
server.controllers.apps.drainPendingAppOpens(),
|
||||
).resolves.toBeUndefined();
|
||||
} finally {
|
||||
writeSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('falls back to actor.app.uid when the body omits app_uid', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
@@ -425,8 +354,6 @@ describe('AppController POST /rao', () => {
|
||||
);
|
||||
expect(captured.body).toEqual({});
|
||||
|
||||
await server.controllers.apps.drainPendingAppOpens();
|
||||
|
||||
const rows = (await server.clients.db.read(
|
||||
'SELECT `app_uid` FROM `app_opens` WHERE `app_uid` = ? AND `user_id` = ?',
|
||||
[app.uid, owner.userId],
|
||||
@@ -500,73 +427,6 @@ describe('AppController GET /apps/:name', () => {
|
||||
expect((arr[0] as Record<string, unknown>).uid).toBe(app.uid);
|
||||
expect(arr[1]).toBeNull();
|
||||
});
|
||||
|
||||
// The driver is the authoritative producer of launch metadata: it runs
|
||||
// the hosted-subdomain guard, which this route knows nothing about. The
|
||||
// route's own entitlement re-check must never turn that denial into an
|
||||
// affirmative verdict — the GUI launcher appends `puter.auth.token` to
|
||||
// whatever `index_url` it is handed.
|
||||
|
||||
it('preserves the hosted-backing denial for the app owner', async () => {
|
||||
const owner = await makeUser();
|
||||
const sub = uniqueName('gone');
|
||||
const row = await server.stores.subdomain.create({
|
||||
userId: owner.userId,
|
||||
subdomain: sub,
|
||||
});
|
||||
const app = await createApp(owner.actor, {
|
||||
index_url: `https://${sub}.site.puter.localhost/`,
|
||||
});
|
||||
await server.stores.subdomain.deleteByUuid(
|
||||
String((row as { uuid: string }).uuid),
|
||||
{ userId: owner.userId },
|
||||
);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(owner.actor, () =>
|
||||
callRoute(
|
||||
'get',
|
||||
'/apps/:name',
|
||||
makeReq({ params: { name: app.name }, actor: owner.actor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
const body = captured.body as Record<string, unknown>;
|
||||
expect(body.privateAccess).toMatchObject({
|
||||
hasAccess: false,
|
||||
reason: 'hosted_backing_unavailable',
|
||||
});
|
||||
});
|
||||
|
||||
it('omits the stale index_url for a non-owner', async () => {
|
||||
const owner = await makeUser();
|
||||
const stranger = await makeUser();
|
||||
const sub = uniqueName('gone');
|
||||
const row = await server.stores.subdomain.create({
|
||||
userId: owner.userId,
|
||||
subdomain: sub,
|
||||
});
|
||||
const app = await createApp(owner.actor, {
|
||||
index_url: `https://${sub}.site.puter.localhost/`,
|
||||
});
|
||||
await server.stores.subdomain.deleteByUuid(
|
||||
String((row as { uuid: string }).uuid),
|
||||
{ userId: owner.userId },
|
||||
);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(stranger.actor, () =>
|
||||
callRoute(
|
||||
'get',
|
||||
'/apps/:name',
|
||||
makeReq({ params: { name: app.name }, actor: stranger.actor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
const body = captured.body as Record<string, unknown>;
|
||||
expect(body.index_url).toBeUndefined();
|
||||
expect(body.privateAccess).toMatchObject({ hasAccess: false });
|
||||
});
|
||||
});
|
||||
|
||||
// ── POST /query/app ─────────────────────────────────────────────────
|
||||
@@ -789,9 +649,7 @@ describe('AppController GET /app-icon/:app_uid', () => {
|
||||
|
||||
it('decodes a data URL icon and serves the declared MIME', async () => {
|
||||
const owner = await makeUser();
|
||||
// Real PNG bytes: the write path sniffs the decoded payload and
|
||||
// rejects anything that isn't the image type it claims to be.
|
||||
const png = Buffer.from(MINIMAL_PNG_BASE64, 'base64');
|
||||
const png = Buffer.from('mock-png-bytes');
|
||||
const dataUrl = `data:image/png;base64,${png.toString('base64')}`;
|
||||
const app = await createApp(owner.actor, { icon: dataUrl });
|
||||
|
||||
@@ -808,25 +666,6 @@ describe('AppController GET /app-icon/:app_uid', () => {
|
||||
expect((captured.body as Buffer).equals(png)).toBe(true);
|
||||
});
|
||||
|
||||
// Icons are fetched on every app launch, over a connection the launch is
|
||||
// already competing for. The inline path must not be cached more briefly
|
||||
// than the redirect path that serves the same resource.
|
||||
it('caches an inline data-URL icon as long as the redirect path', async () => {
|
||||
const owner = await makeUser();
|
||||
const dataUrl = `data:image/png;base64,${MINIMAL_PNG_BASE64}`;
|
||||
const app = await createApp(owner.actor, { icon: dataUrl });
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app-icon/:app_uid',
|
||||
makeReq({ params: { app_uid: app.uid } }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(captured.headers['cache-control']).toBe('public, max-age=900');
|
||||
});
|
||||
|
||||
it('falls back to the default icon when the data-URL MIME is not allowlisted', async () => {
|
||||
const owner = await makeUser();
|
||||
// text/html is NOT in the icon allowlist — must NOT be echoed back.
|
||||
@@ -868,153 +707,3 @@ describe('AppController GET /app-icon/:app_uid', () => {
|
||||
expect(captured.statusCode).toBe(200);
|
||||
});
|
||||
});
|
||||
|
||||
// ── /rao actor gating ───────────────────────────────────────────────
|
||||
|
||||
describe('AppController POST /rao actor gating', () => {
|
||||
it('refuses to record an open for a scoped access token', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const app = await createApp(actor);
|
||||
// Shared / scoped credentials must not drive analytics counters.
|
||||
const tokenActor = {
|
||||
...actor,
|
||||
accessToken: { uuid: uuidv4(), permissions: [] },
|
||||
} as unknown as Actor;
|
||||
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
withActor(tokenActor, () =>
|
||||
callRoute(
|
||||
'post',
|
||||
'/rao',
|
||||
makeReq({
|
||||
body: { app_uid: app.uid },
|
||||
actor: tokenActor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403, legacyCode: 'forbidden' });
|
||||
});
|
||||
|
||||
it("refuses to let an app actor record another app's open", async () => {
|
||||
const { actor } = await makeUser();
|
||||
const own = await createApp(actor);
|
||||
const other = await createApp(actor);
|
||||
const appActor = {
|
||||
...actor,
|
||||
app: { uid: own.uid as string },
|
||||
} as unknown as Actor;
|
||||
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
withActor(appActor, () =>
|
||||
callRoute(
|
||||
'post',
|
||||
'/rao',
|
||||
makeReq({
|
||||
body: { app_uid: other.uid },
|
||||
actor: appActor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 403, legacyCode: 'forbidden' });
|
||||
});
|
||||
});
|
||||
|
||||
// ── app-icon redirect + fallback paths ──────────────────────────────
|
||||
|
||||
describe('AppController GET /app-icon remote icons', () => {
|
||||
// AppDriver.create validates the `icon` column, so these fixtures write
|
||||
// it straight to the row. AppStore caches by uid — drop that entry or the
|
||||
// handler reads the pre-update app back.
|
||||
const setIcon = async (uid: string, icon: string) => {
|
||||
await server.clients.db.write(
|
||||
'UPDATE `apps` SET `icon` = ? WHERE `uid` = ?',
|
||||
[icon, uid],
|
||||
);
|
||||
await server.stores.app.invalidateByUid(uid);
|
||||
};
|
||||
|
||||
it('redirects to an icon hosted on a trusted host', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
const { static_hosting_domain: hostingDomain } = (
|
||||
server.controllers.apps as unknown as {
|
||||
config: { static_hosting_domain: string };
|
||||
}
|
||||
).config;
|
||||
const trusted = `https://puter-app-icons.${hostingDomain}/${app.uid}-128.png`;
|
||||
await setIcon(String(app.uid), trusted);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app-icon/:app_uid',
|
||||
makeReq({ params: { app_uid: app.uid } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.redirectStatus).toBe(302);
|
||||
expect(captured.redirectUrl).toBe(trusted);
|
||||
expect(captured.headers['cache-control']).toBe('public, max-age=900');
|
||||
});
|
||||
|
||||
it('never redirects to an icon URL on an untrusted host', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
// An open redirect here would let an app row point browsers anywhere.
|
||||
await setIcon(String(app.uid), 'https://evil.example.com/icon.png');
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app-icon/:app_uid',
|
||||
makeReq({ params: { app_uid: app.uid } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.redirectUrl).toBeUndefined();
|
||||
expect(captured.headers['content-type']).toContain('image/svg+xml');
|
||||
});
|
||||
|
||||
it('serves the default icon for a malformed data URL with no comma', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
await setIcon(String(app.uid), 'data:image/png;base64');
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app-icon/:app_uid',
|
||||
makeReq({ params: { app_uid: app.uid } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.headers['content-type']).toContain('image/svg+xml');
|
||||
});
|
||||
|
||||
it('serves the default icon for an unknown app uid', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app-icon/:app_uid',
|
||||
makeReq({ params: { app_uid: `app-${uuidv4()}` } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.headers['content-type']).toContain('image/svg+xml');
|
||||
});
|
||||
|
||||
it('serves the default icon for a bare (non-URL, non-data) icon value', async () => {
|
||||
const owner = await makeUser();
|
||||
const app = await createApp(owner.actor);
|
||||
await setIcon(String(app.uid), 'legacy-icon-name.png');
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app-icon/:app_uid/:size',
|
||||
makeReq({ params: { app_uid: app.uid, size: '64' } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.headers['content-type']).toContain('image/svg+xml');
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,20 +1,21 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import bcrypt from 'bcrypt';
|
||||
@@ -132,31 +133,6 @@ export class AuthController extends PuterController {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
|
||||
// Browser-only gate. The session id is client-chosen and travels in a
|
||||
// link, so it is not a secret — the `Origin` header is what actually
|
||||
// says who is asking, and only a browser is prevented from lying about
|
||||
// it. A caller with no `Origin` (curl, a server-side fetch) could
|
||||
// otherwise collect a token minted for someone else's app just by
|
||||
// knowing the id.
|
||||
//
|
||||
// `"null"` is rejected too: sandboxed iframes and `file://` documents
|
||||
// serialise their opaque origin that way, and two *unrelated* opaque
|
||||
// origins would compare equal to each other.
|
||||
const reqOrigin = req.headers.origin;
|
||||
if (!reqOrigin || reqOrigin === 'null') {
|
||||
throw new HttpError(403, 'Origin not allowed', {
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
}
|
||||
|
||||
// The app identity this caller is allowed to collect a token for,
|
||||
// derived from the browser-attested header rather than anything in
|
||||
// the request body — so no client, honest or not, can influence the
|
||||
// comparison made after the token arrives.
|
||||
const expectedAppUid =
|
||||
await this.services.auth.appUidFromOrigin(reqOrigin);
|
||||
|
||||
const { resolve, promise } = Promise.withResolvers<void>();
|
||||
|
||||
let token: string | null = null;
|
||||
@@ -177,56 +153,12 @@ export class AuthController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
// Audience check. The postMessage hand-off this relay stands in for
|
||||
// is origin-bound for free — it posts with `targetOrigin`, so a page
|
||||
// can only ever receive a token minted for *itself*. Delivering
|
||||
// server-side dropped that binding; this restores it. Without it a
|
||||
// popup talked into minting for app X (see `trustsOpenerOriginParam`
|
||||
// in the GUI) hands X's token to whoever holds the session id.
|
||||
if (!this.#tokenIsForApp(token, expectedAppUid)) {
|
||||
// Deliberately the same 408 the no-token path returns: a caller
|
||||
// learns only that nothing arrived for them, not that a token
|
||||
// for a different app went past.
|
||||
throw new HttpError(408, 'Request timeout.', {
|
||||
legacyCode: 'request_timeout',
|
||||
});
|
||||
}
|
||||
|
||||
res.json({
|
||||
auth_token: token,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a relayed token is an app-under-user token minted for
|
||||
* `expectedAppUid`. Verifies the signature — an unverified decode would let
|
||||
* a caller relay a token whose claims it wrote itself.
|
||||
*/
|
||||
#tokenIsForApp(token: string, expectedAppUid: string): boolean {
|
||||
try {
|
||||
const payload = this.services.token.verify<{
|
||||
type?: string;
|
||||
app_uid?: string;
|
||||
}>('auth', token);
|
||||
return (
|
||||
payload.type === 'app-under-user' &&
|
||||
!!payload.app_uid &&
|
||||
payload.app_uid === expectedAppUid
|
||||
);
|
||||
} catch {
|
||||
// Malformed, expired, or signed with a key we don't hold.
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@Post('/login/set', {
|
||||
subdomain: ['api'],
|
||||
// Unauthenticated fan-out to every `/login/wait` listener on the
|
||||
// session id. A legitimate popup posts here exactly once per sign-in,
|
||||
// so a generous per-IP cap costs honest traffic nothing while denying
|
||||
// an attacker unbounded attempts to land a token on a guessed id.
|
||||
rateLimit: [
|
||||
{ scope: 'login-set', limit: 60, window: 15 * 60_000, key: 'ip' },
|
||||
],
|
||||
})
|
||||
async loginSet(req: Request, res: Response) {
|
||||
const { session, auth_token } = req.body;
|
||||
@@ -250,11 +182,6 @@ export class AuthController extends PuterController {
|
||||
// -- Login -------------------------------------------------------
|
||||
|
||||
@Post('/login', {
|
||||
// Returns a full session token in the response body. Reflected CORS
|
||||
// would otherwise let any page trade a password for that token and
|
||||
// read it — third-party sign-in goes through `puter.auth.signIn()`,
|
||||
// whose popup runs on this origin and yields an app-scoped token.
|
||||
guiOriginOnly: true,
|
||||
captcha: true,
|
||||
// Two limits: per-fingerprint keeps users behind a shared IP
|
||||
// (offices, campuses) from throttling each other, while the
|
||||
@@ -371,8 +298,6 @@ export class AuthController extends PuterController {
|
||||
// -- Login: OTP verification -------------------------------------
|
||||
|
||||
@Post('/login/otp', {
|
||||
// Second leg of `/login` — also completes into `#completeLogin`.
|
||||
guiOriginOnly: true,
|
||||
captcha: true,
|
||||
rateLimit: [
|
||||
{ scope: 'login-otp', limit: 15, window: 30 * 60_000 },
|
||||
@@ -437,8 +362,6 @@ export class AuthController extends PuterController {
|
||||
// -- Login: recovery code ----------------------------------------
|
||||
|
||||
@Post('/login/recovery-code', {
|
||||
// Second leg of `/login` — also completes into `#completeLogin`.
|
||||
guiOriginOnly: true,
|
||||
captcha: true,
|
||||
rateLimit: [
|
||||
{ scope: 'login-recovery', limit: 10, window: 60 * 60_000 },
|
||||
@@ -516,9 +439,6 @@ export class AuthController extends PuterController {
|
||||
// -- Signup ------------------------------------------------------
|
||||
|
||||
@Post('/signup', {
|
||||
// Completes into `#completeLogin`, so it hands back a session token
|
||||
// exactly like `/login`. Same reasoning.
|
||||
guiOriginOnly: true,
|
||||
captcha: true,
|
||||
rateLimit: [
|
||||
{ scope: 'signup', limit: 10, window: 15 * 60_000 },
|
||||
@@ -1042,10 +962,10 @@ export class AuthController extends PuterController {
|
||||
antiCsrf: true,
|
||||
})
|
||||
async handleLogout(req: Request, res: Response): Promise<void> {
|
||||
// Clear the session cookie + `puter_token_v2`. Nothing issues the
|
||||
// latter any more (it came from the retired token migration), but
|
||||
// authProbe still reads it as a fallback, so a value left in a
|
||||
// browser would re-authenticate the next request.
|
||||
// Clear the session cookie + the v2 migrate-token companion
|
||||
// cookie (set after `/auth/migrate-token` for app-under-user
|
||||
// iframes). authProbe reads `puter_token_v2` as a fallback, so
|
||||
// a stale value would re-authenticate the next request.
|
||||
res.clearCookie(this.config.cookie_name ?? 'puter_token');
|
||||
res.clearCookie('puter_token_v2');
|
||||
// Drop any step-up elevation too, so it can't reactivate on a shared
|
||||
@@ -2338,18 +2258,15 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
await this.#validateEmail(new_email);
|
||||
|
||||
// Block if any OTHER confirmed account (password or OIDC) already
|
||||
// Block if any confirmed account (password or OIDC) already
|
||||
// owns that email. Match raw + canonical to collapse gmail
|
||||
// aliases — which is also why the caller has to be excluded: an
|
||||
// alias of your own current address resolves back to you, and
|
||||
// "already in use" about yourself is nonsense.
|
||||
// aliases.
|
||||
const canonical = cleanEmail(new_email);
|
||||
const existing =
|
||||
(await this.stores.user.getByEmail(new_email)) ??
|
||||
(await this.stores.user.getByCleanEmail(canonical));
|
||||
if (
|
||||
existing &&
|
||||
existing.id !== req.actor!.user.id &&
|
||||
(existing.email_confirmed || existing.password !== null)
|
||||
) {
|
||||
throw new HttpError(400, 'This email is already in use.', {
|
||||
@@ -2755,95 +2672,12 @@ export class AuthController extends PuterController {
|
||||
res.json({});
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared input validation for the user-app grant/revoke handlers, which
|
||||
* accept a caller-supplied `origin` as an alternative to `app_uid`. All
|
||||
* parameters are optional-but-typed: presence is enforced by the handlers'
|
||||
* own `app_uid`/`permission` checks after origin resolution.
|
||||
*
|
||||
* These are bounded only against absurd input. The width of the column a
|
||||
* permission lands in is enforced by the permission service instead, on the
|
||||
* _rewritten_ string: `fs:/path:mode` is rewritten to `fs:<uuid>:mode`
|
||||
* before it is stored, so a deep path is a ~45-character row and must not
|
||||
* be rejected for the length of the path the caller typed.
|
||||
*/
|
||||
#validateAppPermissionParams(params: {
|
||||
app_uid?: unknown;
|
||||
origin?: unknown;
|
||||
permission?: unknown;
|
||||
extra?: unknown;
|
||||
meta?: unknown;
|
||||
}): void {
|
||||
const MAX_LEN = 4096;
|
||||
for (const key of ['app_uid', 'origin', 'permission'] as const) {
|
||||
const value = params[key];
|
||||
if (value === undefined || value === null) continue;
|
||||
if (typeof value !== 'string' || value.length > MAX_LEN) {
|
||||
throw new HttpError(400, `Invalid \`${key}\``, {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
}
|
||||
// `extra` and `meta` are forwarded into the audit row and read as
|
||||
// objects downstream. A non-object would fault *after* the grant is
|
||||
// committed, so reject it up front. `null` is treated as absent, the
|
||||
// same as the string parameters above.
|
||||
for (const key of ['extra', 'meta'] as const) {
|
||||
const value = params[key];
|
||||
if (value === undefined || value === null) continue;
|
||||
if (typeof value !== 'object' || Array.isArray(value)) {
|
||||
throw new HttpError(400, `Invalid \`${key}\``, {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves a caller-supplied `origin` to the uid of a _registered_ app.
|
||||
*
|
||||
* `appUidFromOrigin` synthesises a deterministic `app-<uuidv5>` uid for an
|
||||
* origin that has no app row yet, and the permission services resolve their
|
||||
* identifier as uid-_or-name_. Passing a synthetic uid straight through
|
||||
* would therefore let whoever registered an app under that literal name
|
||||
* collect a grant the user made to the origin — the uid is derived from a
|
||||
* published namespace constant, so it can be computed and squatted offline.
|
||||
* Only a uid that names an existing app row is accepted.
|
||||
*
|
||||
* An `origin` supplied alongside an `app_uid` takes precedence over it (see
|
||||
* the grant/revoke handlers). The origin is what a consent prompt shows the
|
||||
* user, so it — not a uid travelling beside it — has to decide who receives
|
||||
* the grant; otherwise a caller could name one app on screen and grant to
|
||||
* another. No caller sends both with different intent.
|
||||
*/
|
||||
async #registeredAppUidFromOrigin(origin: string): Promise<string> {
|
||||
const uid = await this.services.auth.appUidFromOrigin(origin);
|
||||
const app = await this.stores.app.getByUid(uid);
|
||||
if (!app) {
|
||||
throw new HttpError(404, `entity_not_found: app:${uid}`, {
|
||||
legacyCode: 'subject_does_not_exist',
|
||||
});
|
||||
}
|
||||
return app.uid;
|
||||
}
|
||||
|
||||
@Post('/auth/grant-user-app', {
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
})
|
||||
async handleGrantUserApp(req: Request, res: Response): Promise<void> {
|
||||
let { app_uid } = req.body;
|
||||
const { origin, permission, extra, meta } = req.body;
|
||||
this.#validateAppPermissionParams({
|
||||
app_uid,
|
||||
origin,
|
||||
permission,
|
||||
extra,
|
||||
meta,
|
||||
});
|
||||
if (origin) {
|
||||
app_uid = await this.#registeredAppUidFromOrigin(origin);
|
||||
}
|
||||
const { app_uid, permission, extra, meta } = req.body;
|
||||
if (!app_uid || !permission) {
|
||||
throw new HttpError(400, 'Missing `app_uid` or `permission`', {
|
||||
legacyCode: 'bad_request',
|
||||
@@ -2853,8 +2687,8 @@ export class AuthController extends PuterController {
|
||||
req.actor!,
|
||||
app_uid,
|
||||
permission,
|
||||
extra ?? undefined,
|
||||
meta ?? undefined,
|
||||
extra,
|
||||
meta,
|
||||
);
|
||||
res.json({});
|
||||
}
|
||||
@@ -2914,17 +2748,7 @@ export class AuthController extends PuterController {
|
||||
requireUserActor: true,
|
||||
})
|
||||
async handleRevokeUserApp(req: Request, res: Response): Promise<void> {
|
||||
let { app_uid } = req.body;
|
||||
const { origin, permission, meta } = req.body;
|
||||
this.#validateAppPermissionParams({
|
||||
app_uid,
|
||||
origin,
|
||||
permission,
|
||||
meta,
|
||||
});
|
||||
if (origin) {
|
||||
app_uid = await this.#registeredAppUidFromOrigin(origin);
|
||||
}
|
||||
const { app_uid, permission, meta } = req.body;
|
||||
if (!app_uid || !permission) {
|
||||
throw new HttpError(400, 'Missing `app_uid` or `permission`', {
|
||||
legacyCode: 'bad_request',
|
||||
@@ -2934,14 +2758,14 @@ export class AuthController extends PuterController {
|
||||
await this.services.permission.revokeUserAppAll(
|
||||
req.actor!,
|
||||
app_uid,
|
||||
meta ?? undefined,
|
||||
meta,
|
||||
);
|
||||
} else {
|
||||
await this.services.permission.revokeUserAppPermission(
|
||||
req.actor!,
|
||||
app_uid,
|
||||
permission,
|
||||
meta ?? undefined,
|
||||
meta,
|
||||
);
|
||||
}
|
||||
res.json({});
|
||||
@@ -3084,15 +2908,7 @@ export class AuthController extends PuterController {
|
||||
let { app_uid } = req.body;
|
||||
const { origin, permission, extra, meta } = req.body;
|
||||
if (origin && !app_uid) {
|
||||
// Registered apps only, for the same reason the user-app handlers
|
||||
// insist on it: a synthesised `app-<uuidv5(origin)>` is resolved
|
||||
// downstream as uid-*or-name*, so it would land on whoever
|
||||
// registered an app under that literal name. A dev-app grant is
|
||||
// scanned with the issuer's authority for anyone running as that
|
||||
// app, so that hands this user's permission to the squatter.
|
||||
// Without a squatter the synthetic uid resolves to nothing and
|
||||
// this 404s regardless, so nothing legitimate changes.
|
||||
app_uid = await this.#registeredAppUidFromOrigin(origin);
|
||||
app_uid = await this.services.auth.appUidFromOrigin(origin);
|
||||
}
|
||||
if (!app_uid || !permission) {
|
||||
throw new HttpError(400, 'Missing `app_uid` or `permission`', {
|
||||
@@ -3117,8 +2933,7 @@ export class AuthController extends PuterController {
|
||||
let { app_uid } = req.body;
|
||||
const { origin, permission, meta } = req.body;
|
||||
if (origin && !app_uid) {
|
||||
// Registered apps only — see handleGrantDevApp.
|
||||
app_uid = await this.#registeredAppUidFromOrigin(origin);
|
||||
app_uid = await this.services.auth.appUidFromOrigin(origin);
|
||||
}
|
||||
if (!app_uid || !permission) {
|
||||
throw new HttpError(400, 'Missing `app_uid` or `permission`', {
|
||||
@@ -3131,14 +2946,13 @@ export class AuthController extends PuterController {
|
||||
app_uid,
|
||||
meta,
|
||||
);
|
||||
} else {
|
||||
await this.services.permission.revokeDevAppPermission(
|
||||
req.actor!,
|
||||
app_uid,
|
||||
permission,
|
||||
meta,
|
||||
);
|
||||
}
|
||||
await this.services.permission.revokeDevAppPermission(
|
||||
req.actor!,
|
||||
app_uid,
|
||||
permission,
|
||||
meta,
|
||||
);
|
||||
res.json({});
|
||||
}
|
||||
|
||||
@@ -3154,22 +2968,17 @@ export class AuthController extends PuterController {
|
||||
|
||||
const [appPerms, userPermsOut, userPermsIn] = await Promise.all([
|
||||
db.read(
|
||||
// The permissions table stores the numeric `app_id` FK; the
|
||||
// public shape exposes the app's `uid`.
|
||||
'SELECT a.`uid` AS app_uid, p.`permission`, p.`extra` ' +
|
||||
'FROM `user_to_app_permissions` p ' +
|
||||
'JOIN `apps` a ON a.`id` = p.`app_id` ' +
|
||||
'WHERE p.`user_id` = ?',
|
||||
'SELECT `app_uid`, `permission`, `extra` FROM `user_to_app_permissions` WHERE `user_id` = ?',
|
||||
[userId],
|
||||
),
|
||||
db.read(
|
||||
'SELECT u.`username`, p.`permission`, p.`extra` FROM `user_to_user_permissions` p ' +
|
||||
'JOIN `user` u ON u.`id` = p.`holder_user_id` WHERE p.`issuer_user_id` = ?',
|
||||
'JOIN `user` u ON u.`id` = p.`target_user_id` WHERE p.`issuer_user_id` = ?',
|
||||
[userId],
|
||||
),
|
||||
db.read(
|
||||
'SELECT u.`username`, p.`permission`, p.`extra` FROM `user_to_user_permissions` p ' +
|
||||
'JOIN `user` u ON u.`id` = p.`issuer_user_id` WHERE p.`holder_user_id` = ?',
|
||||
'JOIN `user` u ON u.`id` = p.`issuer_user_id` WHERE p.`target_user_id` = ?',
|
||||
[userId],
|
||||
),
|
||||
]);
|
||||
@@ -3256,9 +3065,9 @@ export class AuthController extends PuterController {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
}
|
||||
|
||||
// Grant the app-is-authenticated flag
|
||||
const userPermGrantPromise =
|
||||
this.services.permission.grantUserAppPermission(
|
||||
await this.services.permission.grantUserAppPermission(
|
||||
req.actor!,
|
||||
app_uid,
|
||||
'flag:app-is-authenticated',
|
||||
@@ -3266,7 +3075,7 @@ export class AuthController extends PuterController {
|
||||
{},
|
||||
);
|
||||
|
||||
const tokenPromise = this.services.auth.getUserAppToken(
|
||||
const token = await this.services.auth.getUserAppToken(
|
||||
req.actor!,
|
||||
app_uid,
|
||||
);
|
||||
@@ -3293,11 +3102,7 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
})();
|
||||
|
||||
const [, token] = await Promise.all([
|
||||
userPermGrantPromise,
|
||||
tokenPromise,
|
||||
missingFSPathPromise,
|
||||
]);
|
||||
await Promise.all([userPermGrantPromise, missingFSPathPromise]);
|
||||
|
||||
try {
|
||||
const a = app as {
|
||||
@@ -3365,6 +3170,84 @@ export class AuthController extends PuterController {
|
||||
|
||||
// -- Access tokens -----------------------------------------------
|
||||
|
||||
async handleMigrateToken(req: Request, res: Response): Promise<void> {
|
||||
// 1. Browser-only gate. No Origin header → reject (server-side
|
||||
// callers should re-auth properly). The exchange itself is open
|
||||
// to any browser origin: puter.js apps live on arbitrary
|
||||
// third-party domains, the v1 bearer token is the credential,
|
||||
// and cookies are never read here — so a hostile page learns
|
||||
// nothing it doesn't already hold. Origin *trust* only decides
|
||||
// whether the `puter_token_v2` companion cookie is set below.
|
||||
const reqOrigin = req.headers.origin;
|
||||
if (!reqOrigin) {
|
||||
throw new HttpError(403, 'Origin not allowed', {
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Extract token. Header takes precedence so body-capture logs
|
||||
// (if any) never see the credential.
|
||||
const authHeader = req.headers.authorization;
|
||||
const headerToken =
|
||||
typeof authHeader === 'string' && authHeader.startsWith('Bearer ')
|
||||
? authHeader.slice('Bearer '.length).trim()
|
||||
: null;
|
||||
const bodyToken =
|
||||
typeof req.body?.token === 'string' ? req.body.token.trim() : null;
|
||||
const v1Token = headerToken || bodyToken;
|
||||
if (!v1Token) {
|
||||
throw new HttpError(400, 'Missing token', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
|
||||
const result = await this.services.auth.migrateLegacyToken(v1Token, {
|
||||
ip: req.ip,
|
||||
userAgent:
|
||||
typeof req.headers['user-agent'] === 'string'
|
||||
? req.headers['user-agent']
|
||||
: undefined,
|
||||
});
|
||||
// `puter_token_v2` is the cookie companion to v2 app tokens —
|
||||
// it lets the GUI's cookie-only middleware authenticate
|
||||
// subsequent same-origin calls without the client forwarding
|
||||
// Authorization headers. Issuing it is gated on the trusted
|
||||
// origin allowlist: an arbitrary attacker page must not be able
|
||||
// to plant a session cookie on the GUI origin (login CSRF).
|
||||
// Untrusted origins still get the token in the JSON body — the
|
||||
// cookie is only consumed same-origin (see authProbe) so they
|
||||
// lose nothing. Access tokens are programmatic (no browser
|
||||
// cookie surface) so we deliberately skip them here.
|
||||
if (
|
||||
result.kind === 'app' &&
|
||||
this.#isMigrateTokenOriginAllowed(reqOrigin)
|
||||
) {
|
||||
res.cookie('puter_token_v2', result.token, {
|
||||
...sessionCookieFlags(this.config),
|
||||
httpOnly: true,
|
||||
});
|
||||
}
|
||||
res.json(result);
|
||||
}
|
||||
|
||||
#isMigrateTokenOriginAllowed(origin: string): boolean {
|
||||
// Origin headers and config values both reach us in inconsistent
|
||||
// shapes (trailing slash, mixed case from misconfigured deploys,
|
||||
// stray whitespace from JSON config edits). Normalize both sides
|
||||
// before equality so a `config.origin` with a trailing slash
|
||||
// doesn't reject every same-origin browser call.
|
||||
const normalize = (raw: string | undefined): string =>
|
||||
(raw ?? '').trim().replace(/\/+$/, '').toLowerCase();
|
||||
const incoming = normalize(origin);
|
||||
if (!incoming) return false;
|
||||
if (incoming === normalize(this.config.origin)) return true;
|
||||
const allowlist = (
|
||||
this.config as { allow_migrate_token_origins?: string[] }
|
||||
).allow_migrate_token_origins;
|
||||
if (!Array.isArray(allowlist)) return false;
|
||||
return allowlist.some((entry) => normalize(entry) === incoming);
|
||||
}
|
||||
|
||||
@Post('/auth/create-access-token', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
@@ -3726,24 +3609,10 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
|
||||
@Get('/session/sync-cookie', {
|
||||
// Installs the session cookie. Only page script on our own origin
|
||||
// should be able to ask for that (the `tokenSource` check below is the
|
||||
// companion rule: the token has to come from an Authorization header,
|
||||
// not a URL).
|
||||
guiOriginOnly: true,
|
||||
requireUserActor: true,
|
||||
allowUnconfirmed: true,
|
||||
})
|
||||
async handleSessionSyncCookie(req: Request, res: Response): Promise<void> {
|
||||
// This route installs a session cookie, so the token has to come from
|
||||
// page script on our own origin rather than from the URL.
|
||||
if (req.tokenSource !== 'header') {
|
||||
throw new HttpError(
|
||||
401,
|
||||
'This endpoint requires an Authorization header.',
|
||||
{ legacyCode: 'token_auth_failed' },
|
||||
);
|
||||
}
|
||||
if (!req.actor?.session?.uid) {
|
||||
res.status(400).end();
|
||||
return;
|
||||
@@ -4095,6 +3964,19 @@ export class AuthController extends PuterController {
|
||||
},
|
||||
(req, res) => this.handleRenameSession(req, res),
|
||||
);
|
||||
|
||||
router.post(
|
||||
'/auth/migrate-token',
|
||||
{
|
||||
rateLimit: {
|
||||
scope: 'migrate-token',
|
||||
limit: 20,
|
||||
window: 15 * 60_000,
|
||||
key: 'ip',
|
||||
},
|
||||
},
|
||||
(req, res) => this.handleMigrateToken(req, res),
|
||||
);
|
||||
}
|
||||
|
||||
// -- Private helpers ----------------------------------------------
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Wiring test for `guiOriginOnly`.
|
||||
*
|
||||
* `originGate.test.ts` covers what the gate decides. This covers that the
|
||||
* routes which hand a session credential back to the caller actually opt into
|
||||
* it — a correct gate wired to nothing protects nothing, and the failure is
|
||||
* invisible (the route keeps working, just for everybody).
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { AuthController } from './AuthController.js';
|
||||
import {
|
||||
ROUTES_METADATA_KEY,
|
||||
type CollectedRoute,
|
||||
} from '../../core/http/types.js';
|
||||
|
||||
// The decorators register on the prototype via `addInitializer`, which only
|
||||
// runs once an instance exists. We never start this one — constructing it is
|
||||
// enough to populate the route metadata.
|
||||
const collectRoutes = (): CollectedRoute[] => {
|
||||
const proto = AuthController.prototype as unknown as Record<
|
||||
string,
|
||||
CollectedRoute[] | undefined
|
||||
>;
|
||||
if (!proto[ROUTES_METADATA_KEY]) {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
new (AuthController as any)({}, {}, {}, {}, {});
|
||||
}
|
||||
return proto[ROUTES_METADATA_KEY] ?? [];
|
||||
};
|
||||
|
||||
/**
|
||||
* Every route that answers with a usable session credential. Adding one here
|
||||
* without `guiOriginOnly` is the mistake this test exists to catch: reflected
|
||||
* CORS would let any page read the credential out of the response.
|
||||
*/
|
||||
const CREDENTIAL_ROUTES: Array<[method: string, path: string]> = [
|
||||
['post', '/login'],
|
||||
['post', '/login/otp'],
|
||||
['post', '/login/recovery-code'],
|
||||
['post', '/signup'],
|
||||
['get', '/session/sync-cookie'],
|
||||
];
|
||||
|
||||
describe('guiOriginOnly wiring on AuthController', () => {
|
||||
const routes = collectRoutes();
|
||||
|
||||
it('registers the routes under test at all (guards against a rename)', () => {
|
||||
for (const [method, path] of CREDENTIAL_ROUTES) {
|
||||
const found = routes.find(
|
||||
(r) => r.method === method && r.path === path,
|
||||
);
|
||||
expect(found, `${method.toUpperCase()} ${path} not registered`)
|
||||
.toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it.each(CREDENTIAL_ROUTES)(
|
||||
'gates %s %s to the GUI origin',
|
||||
(method, path) => {
|
||||
const route = routes.find(
|
||||
(r) => r.method === method && r.path === path,
|
||||
);
|
||||
expect(route?.options.guiOriginOnly).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
// The popup relay is the third-party sign-in path: `puter.auth.signIn()`
|
||||
// polls `/login/wait` from whatever origin the app is served from, and
|
||||
// `/login/set` is posted by the popup. Gating either to our own origin
|
||||
// would break every third-party app, and neither needs it — the token
|
||||
// they move is app-scoped, not a session.
|
||||
it.each([
|
||||
['post', '/login/wait'],
|
||||
['post', '/login/set'],
|
||||
])('leaves %s %s open cross-origin', (method, path) => {
|
||||
const route = routes.find(
|
||||
(r) => r.method === method && r.path === path,
|
||||
);
|
||||
expect(route, `${method.toUpperCase()} ${path} not registered`)
|
||||
.toBeDefined();
|
||||
expect(route?.options.guiOriginOnly).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -27,19 +27,20 @@ import { PuterController } from '../types.js';
|
||||
*
|
||||
* The route is intentionally a thin shell: parse the four custom
|
||||
* `X-Broadcast-*` headers, hand the raw body + parsed body off to
|
||||
* `BroadcastService.verifyAndEmit()`, and translate its structured result into
|
||||
* HTTP. All the cryptography, replay protection, and event-bus dispatch live in
|
||||
* the service so they're reusable from tests / direct callers.
|
||||
* `BroadcastService.verifyAndEmit()`, and translate its structured
|
||||
* result into HTTP. All the cryptography, replay protection, and
|
||||
* event-bus dispatch live in the service so they're reusable from
|
||||
* tests / direct callers.
|
||||
*
|
||||
* Mounted with `subdomain: '*'` (any host) because peers reach the webhook
|
||||
* through the ALB DNS, not the public `api.<domain>` subdomain, so the host
|
||||
* header can be an internal ALB hostname rather than `api.<domain>` or
|
||||
* `<domain>`. Authentication is via the HMAC + peer-id + nonce triple, not the
|
||||
* host.
|
||||
* Mounted with `subdomain: '*'` (any host) because peers reach the
|
||||
* webhook through the ALB DNS, not the public `api.<domain>` subdomain,
|
||||
* so the host header can be an internal ALB hostname rather than
|
||||
* `api.<domain>` or `<domain>`. Authentication is via the HMAC +
|
||||
* peer-id + nonce triple, not the host.
|
||||
*
|
||||
* `req.rawBody` is captured by the global JSON parser and is what the HMAC
|
||||
* verifies against — do NOT switch this route to a custom body parser without
|
||||
* preserving the raw bytes.
|
||||
* `req.rawBody` is captured by the global JSON parser and is what the
|
||||
* HMAC verifies against — do NOT switch this route to a custom body
|
||||
* parser without preserving the raw bytes.
|
||||
*/
|
||||
@Controller('/broadcast')
|
||||
export class BroadcastController extends PuterController {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -28,9 +28,8 @@ const ALLOWED_SORT_ORDER = ['asc', 'desc'];
|
||||
* Desktop/UI preference routes.
|
||||
*
|
||||
* Two categories:
|
||||
*
|
||||
* - User-level: desktop background, taskbar items (UserStore)
|
||||
* - Folder-level: layout, sort_by/sort_order (fsentries table)
|
||||
* - User-level: desktop background, taskbar items (UserStore)
|
||||
* - Folder-level: layout, sort_by/sort_order (fsentries table)
|
||||
*/
|
||||
export class DesktopController extends PuterController {
|
||||
constructor(config, clients, stores, services) {
|
||||
@@ -196,15 +195,16 @@ export class DesktopController extends PuterController {
|
||||
// -- Helpers ------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Update columns on an actor-owned fsentry. Accepts either `item_uid` (fast
|
||||
* path) or `item_path` (path lookups in FSEntryStore now have a
|
||||
* recursive-CTE fallback for legacy rows with a NULL `path` column, so this
|
||||
* works for old accounts too).
|
||||
* Update columns on an actor-owned fsentry. Accepts either `item_uid`
|
||||
* (fast path) or `item_path` (path lookups in FSEntryStore now have
|
||||
* a recursive-CTE fallback for legacy rows with a NULL `path` column,
|
||||
* so this works for old accounts too).
|
||||
*
|
||||
* Ownership: `entry.user_id === actor.user.id`. Kept as added validation
|
||||
* against bad paths — the previous "drop user_id entirely" theory turned
|
||||
* out to be wrong (the actual legacy issue was NULL paths, not user_id
|
||||
* drift), so this filter doesn't lock out old accounts in practice.
|
||||
* Ownership: `entry.user_id === actor.user.id`. Kept as added
|
||||
* validation against bad paths — the previous "drop user_id entirely"
|
||||
* theory turned out to be wrong (the actual legacy issue was NULL
|
||||
* paths, not user_id drift), so this filter doesn't lock out old
|
||||
* accounts in practice.
|
||||
*/
|
||||
async #updateFSEntry(actor, { item_uid, item_path }, patch) {
|
||||
if (!item_uid && !item_path) {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,438 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Upstream-error translation and rate-limit rejection in `/drivers/call`.
|
||||
*
|
||||
* When a driver's upstream provider fails, the caller must see a stable Puter
|
||||
* error code rather than a raw vendor payload. Each SDK reports its status
|
||||
* differently (`status`, `response.status`, an AWS `$metadata` block, or only a
|
||||
* message), so the controller sniffs all four — this suite pins the mapping for
|
||||
* every shape and asserts the resulting `legacyCode` / `statusCode`, plus the
|
||||
* `upstreamStatus` / `upstreamCode` diagnostic fields.
|
||||
*
|
||||
* The synthetic driver stands in for a provider-backed one: it is the input to
|
||||
* the translation under test, and it lets a single controller instance cover
|
||||
* every failure shape without real provider credentials.
|
||||
*/
|
||||
|
||||
/* eslint-disable @typescript-eslint/no-explicit-any */
|
||||
import { Readable, Writable } from 'node:stream';
|
||||
import type { Request, RequestHandler, Response } from 'express';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { DriverMethodLifecycleEvent } from '../../clients/event/types.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { configureRateLimit } from '../../core/http/middleware/rateLimit.js';
|
||||
import { DriverController } from './DriverController.js';
|
||||
|
||||
// -- Harness ---------------------------------------------------------
|
||||
|
||||
const captureCallHandler = (controller: DriverController): RequestHandler => {
|
||||
let handler: RequestHandler | undefined;
|
||||
const fakeRouter = {
|
||||
post: (path: string, _opts: unknown, h: RequestHandler) => {
|
||||
if (path === '/call') handler = h;
|
||||
return fakeRouter;
|
||||
},
|
||||
get: () => fakeRouter,
|
||||
use: () => fakeRouter,
|
||||
};
|
||||
controller.registerRoutes(fakeRouter as any);
|
||||
if (!handler) throw new Error('failed to capture POST /call handler');
|
||||
return handler;
|
||||
};
|
||||
|
||||
// A real Writable so `result.stream.pipe(res)` behaves like the express
|
||||
// response it stands in for.
|
||||
class MockRes extends Writable {
|
||||
statusCode = 200;
|
||||
body: unknown;
|
||||
headers: Record<string, string> = {};
|
||||
chunks: Buffer[] = [];
|
||||
override _write(
|
||||
chunk: Buffer,
|
||||
_enc: BufferEncoding,
|
||||
cb: (e?: Error) => void,
|
||||
) {
|
||||
this.chunks.push(Buffer.from(chunk));
|
||||
cb();
|
||||
}
|
||||
status(code: number) {
|
||||
this.statusCode = code;
|
||||
return this;
|
||||
}
|
||||
json(body: unknown) {
|
||||
this.body = body;
|
||||
return this;
|
||||
}
|
||||
setHeader(k: string, v: string) {
|
||||
this.headers[k.toLowerCase()] = v;
|
||||
return this;
|
||||
}
|
||||
}
|
||||
|
||||
const makeReq = (body: Record<string, unknown>): Request =>
|
||||
({
|
||||
body,
|
||||
headers: {},
|
||||
query: {},
|
||||
ip: '127.0.0.1',
|
||||
socket: { remoteAddress: '127.0.0.1' },
|
||||
}) as unknown as Request;
|
||||
|
||||
interface BuildOptions {
|
||||
run?: () => unknown;
|
||||
rateLimit?: unknown;
|
||||
/**
|
||||
* Rate-limit buckets are keyed by iface+method, so give tests that exercise
|
||||
* limits their own namespace.
|
||||
*/
|
||||
iface?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a controller around one synthetic driver. No actor is attached, so the
|
||||
* permission scan is skipped and the call reaches the driver method.
|
||||
*/
|
||||
const build = (opts: BuildOptions = {}) => {
|
||||
const events: DriverMethodLifecycleEvent[] = [];
|
||||
const eventClient = {
|
||||
emitAndWait: vi.fn(async () => {}),
|
||||
emit: vi.fn((_key: string, payload: unknown) => {
|
||||
events.push(payload as DriverMethodLifecycleEvent);
|
||||
}),
|
||||
on: vi.fn(),
|
||||
};
|
||||
const alarms: Array<{ id: string; severity: string }> = [];
|
||||
const iface = opts.iface ?? 'test-iface';
|
||||
const driver = {
|
||||
driverInterface: iface,
|
||||
driverName: 'test-driver',
|
||||
isDefault: true,
|
||||
...(opts.rateLimit ? { rateLimit: opts.rateLimit } : {}),
|
||||
run:
|
||||
opts.run ??
|
||||
(() => {
|
||||
throw new Error('no run configured');
|
||||
}),
|
||||
};
|
||||
const controller = new DriverController(
|
||||
{} as any,
|
||||
{
|
||||
event: eventClient,
|
||||
alarm: {
|
||||
create: (id: string, _t: string, _f: unknown, sev: string) => {
|
||||
alarms.push({ id, severity: sev });
|
||||
},
|
||||
},
|
||||
} as any,
|
||||
{} as any,
|
||||
{} as any,
|
||||
{ testDriver: driver } as any,
|
||||
);
|
||||
return {
|
||||
handler: captureCallHandler(controller),
|
||||
events,
|
||||
alarms,
|
||||
driver,
|
||||
iface,
|
||||
};
|
||||
};
|
||||
|
||||
const callWith = async (run: () => unknown) => {
|
||||
const { handler, events } = build({ run });
|
||||
const res = new MockRes();
|
||||
const err = await runWithContext({}, () =>
|
||||
handler(
|
||||
makeReq({ interface: 'test-iface', method: 'run' }),
|
||||
res as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
).then(
|
||||
() => null,
|
||||
(e: unknown) => e,
|
||||
);
|
||||
return { err, res, events };
|
||||
};
|
||||
|
||||
const throwing = (payload: unknown) => () => {
|
||||
throw payload;
|
||||
};
|
||||
|
||||
// -- Upstream status extraction --------------------------------------
|
||||
|
||||
describe('DriverController upstream error translation', () => {
|
||||
it('maps an upstream 429 to a Puter 429 with upstream_rate_limited', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing(
|
||||
Object.assign(new Error('slow down'), {
|
||||
status: 429,
|
||||
code: 'rate_limit_exceeded',
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 429,
|
||||
legacyCode: 'upstream_rate_limited',
|
||||
message: 'slow down',
|
||||
fields: {
|
||||
upstreamStatus: 429,
|
||||
upstreamCode: 'rate_limit_exceeded',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('maps upstream 401 and 403 to a 500 upstream_auth_failed — never leaking auth state to the caller', async () => {
|
||||
for (const status of [401, 403]) {
|
||||
const { err } = await callWith(
|
||||
throwing(Object.assign(new Error('bad key'), { status })),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 500,
|
||||
legacyCode: 'upstream_auth_failed',
|
||||
fields: { upstreamStatus: status },
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('maps any upstream 5xx to a 400 upstream_provider_unavailable with a generic message', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing(
|
||||
Object.assign(new Error('internal provider stack trace'), {
|
||||
status: 503,
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'upstream_provider_unavailable',
|
||||
message: 'AI provider unavailable',
|
||||
fields: { upstreamStatus: 503 },
|
||||
});
|
||||
});
|
||||
|
||||
it('maps a generic upstream 4xx to a 400 upstream_bad_request', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing(
|
||||
Object.assign(new Error('unsupported parameter'), {
|
||||
status: 422,
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'upstream_bad_request',
|
||||
message: 'unsupported parameter',
|
||||
fields: { upstreamStatus: 422 },
|
||||
});
|
||||
});
|
||||
|
||||
it('prefers the nested error.message and error.code over the top-level ones', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing({
|
||||
status: 400,
|
||||
message: 'outer',
|
||||
code: 'outer_code',
|
||||
error: { message: 'inner detail', code: 'inner_code' },
|
||||
}),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
legacyCode: 'upstream_bad_request',
|
||||
message: 'inner detail',
|
||||
fields: { upstreamStatus: 400, upstreamCode: 'inner_code' },
|
||||
});
|
||||
});
|
||||
|
||||
it('reads the status from statusCode when `status` is absent', async () => {
|
||||
const { err } = await callWith(throwing({ statusCode: 429 }));
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 429,
|
||||
legacyCode: 'upstream_rate_limited',
|
||||
});
|
||||
});
|
||||
|
||||
it('reads the status from a nested response object (axios-style)', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing({ response: { status: 429 }, message: 'axios rejected' }),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 429,
|
||||
legacyCode: 'upstream_rate_limited',
|
||||
fields: { upstreamStatus: 429 },
|
||||
});
|
||||
});
|
||||
|
||||
it('reads the status from an AWS $metadata block', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing({
|
||||
$metadata: { httpStatusCode: 400 },
|
||||
message: 'ValidationException',
|
||||
}),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'upstream_bad_request',
|
||||
fields: { upstreamStatus: 400 },
|
||||
});
|
||||
});
|
||||
|
||||
it('sniffs a status out of the message when nothing else carries one', async () => {
|
||||
const { err } = await callWith(
|
||||
throwing(new Error('Request failed with status code 422')),
|
||||
);
|
||||
expect(err).toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'upstream_bad_request',
|
||||
fields: { upstreamStatus: 422 },
|
||||
});
|
||||
});
|
||||
|
||||
it('does not treat a bare 4xx-looking number in the message as a status', async () => {
|
||||
const raw = new Error('the answer contained 404 rows');
|
||||
const { err } = await callWith(throwing(raw));
|
||||
// No status could be derived, so the original error passes through
|
||||
// untranslated rather than being mislabelled.
|
||||
expect(err).toBe(raw);
|
||||
});
|
||||
|
||||
it('passes an HttpError from the driver straight through', async () => {
|
||||
const { HttpError } = await import('../../core/http/HttpError.js');
|
||||
const raw = new HttpError(404, 'no such key', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
const { err } = await callWith(throwing(raw));
|
||||
expect(err).toBe(raw);
|
||||
});
|
||||
|
||||
it('passes non-object throwables through untouched', async () => {
|
||||
const { err } = await callWith(throwing('a bare string'));
|
||||
expect(err).toBe('a bare string');
|
||||
});
|
||||
|
||||
it('passes an error with a sub-400 status through untranslated', async () => {
|
||||
const raw = { status: 302, message: 'redirected' };
|
||||
const { err } = await callWith(throwing(raw));
|
||||
expect(err).toBe(raw);
|
||||
});
|
||||
|
||||
it('emits the error lifecycle event with the original (untranslated) error', async () => {
|
||||
const raw = Object.assign(new Error('provider down'), { status: 500 });
|
||||
const { events, err } = await callWith(throwing(raw));
|
||||
|
||||
const errorEvent = events.find(
|
||||
(e) => (e as { phase?: string }).phase === 'error',
|
||||
) as unknown as Record<string, unknown>;
|
||||
expect(errorEvent).toBeDefined();
|
||||
expect(errorEvent.iface).toBe('test-iface');
|
||||
expect(errorEvent.method).toBe('run');
|
||||
expect(errorEvent.driver).toBe('test-driver');
|
||||
expect(errorEvent.error).toBe(raw);
|
||||
expect(typeof errorEvent.durationMs).toBe('number');
|
||||
// The caller still sees the translated error.
|
||||
expect(err).toMatchObject({
|
||||
legacyCode: 'upstream_provider_unavailable',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// -- Stream results --------------------------------------------------
|
||||
|
||||
describe('DriverController stream responses', () => {
|
||||
it('sets Transfer-Encoding: chunked for a chunked stream result', async () => {
|
||||
const { handler } = build({
|
||||
run: () => ({
|
||||
dataType: 'stream',
|
||||
content_type: 'audio/mpeg',
|
||||
chunked: true,
|
||||
stream: Readable.from(['a', 'b']),
|
||||
}),
|
||||
});
|
||||
const res = new MockRes();
|
||||
|
||||
await runWithContext({}, () =>
|
||||
handler(
|
||||
makeReq({ interface: 'test-iface', method: 'run' }),
|
||||
res as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
);
|
||||
|
||||
expect(res.headers['content-type']).toBe('audio/mpeg');
|
||||
expect(res.headers['transfer-encoding']).toBe('chunked');
|
||||
// A piped stream never produces a JSON body.
|
||||
expect(res.body).toBeUndefined();
|
||||
});
|
||||
|
||||
it('omits Transfer-Encoding for a non-chunked stream result', async () => {
|
||||
const { handler } = build({
|
||||
run: () => ({
|
||||
dataType: 'stream',
|
||||
content_type: 'audio/mpeg',
|
||||
stream: Readable.from(['a']),
|
||||
}),
|
||||
});
|
||||
const res = new MockRes();
|
||||
|
||||
await runWithContext({}, () =>
|
||||
handler(
|
||||
makeReq({ interface: 'test-iface', method: 'run' }),
|
||||
res as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
);
|
||||
|
||||
expect(res.headers['content-type']).toBe('audio/mpeg');
|
||||
expect('transfer-encoding' in res.headers).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// -- Rate limiting ---------------------------------------------------
|
||||
|
||||
describe('DriverController per-method rate limiting', () => {
|
||||
beforeEach(() => {
|
||||
configureRateLimit({ disabled: false } as never);
|
||||
});
|
||||
|
||||
it('answers 429 and raises a de-duped info alarm once the per-method budget is spent', async () => {
|
||||
const { handler, alarms, iface } = build({
|
||||
run: () => ({ ok: true }),
|
||||
rateLimit: { default: { limit: 1, window: 60_000 } },
|
||||
iface: 'rate-limited-iface',
|
||||
});
|
||||
const call = () =>
|
||||
runWithContext({}, () =>
|
||||
handler(
|
||||
makeReq({ interface: iface, method: 'run' }),
|
||||
new MockRes() as unknown as Response,
|
||||
() => {},
|
||||
),
|
||||
);
|
||||
|
||||
await call();
|
||||
await expect(call()).rejects.toMatchObject({
|
||||
statusCode: 429,
|
||||
legacyCode: 'too_many_requests',
|
||||
});
|
||||
|
||||
expect(alarms).toContainEqual({
|
||||
id: 'driver_rate_limit_hit:rate-limited-iface:run',
|
||||
severity: 'info',
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,23 +1,23 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import { metrics } from '@opentelemetry/api';
|
||||
import type { Request, Response } from 'express';
|
||||
import { actorUid } from '../../core/actor.js';
|
||||
import { Context } from '../../core/context.js';
|
||||
@@ -45,17 +45,6 @@ import { PuterController } from '../types.js';
|
||||
|
||||
type DriverInstance = WithLifecycle & Record<string, unknown>;
|
||||
|
||||
// Every driver call is timed here already, for the lifecycle events below.
|
||||
// Recording the same number as a histogram makes the per-interface latency
|
||||
// distribution available downstream; which interfaces are worth keeping is a
|
||||
// collector-side decision, not one made here, so this deliberately records
|
||||
// everything and lets the export pipeline drop what it doesn't want.
|
||||
const meter = metrics.getMeter('puter-backend');
|
||||
const driverCallDuration = meter.createHistogram('driver.call.duration', {
|
||||
description: 'Wall time of a driver method call',
|
||||
unit: 'ms',
|
||||
});
|
||||
|
||||
const extractUpstreamStatus = (e: {
|
||||
status?: number;
|
||||
statusCode?: number;
|
||||
@@ -440,11 +429,6 @@ export class DriverController extends PuterController {
|
||||
() => (fn as (...x: unknown[]) => any).call(driver, args),
|
||||
);
|
||||
} catch (e) {
|
||||
driverCallDuration.record(Date.now() - startedAt, {
|
||||
driver: ifaceName,
|
||||
'driver.method': method,
|
||||
outcome: 'error',
|
||||
});
|
||||
this.clients.event?.emit(
|
||||
`driver.${ifaceName}.${method}.error`,
|
||||
{
|
||||
@@ -462,15 +446,6 @@ export class DriverController extends PuterController {
|
||||
);
|
||||
throw translateProviderError(e);
|
||||
}
|
||||
|
||||
// Same window the span and the lifecycle events measure: for streamed
|
||||
// results this is stream start, not stream drain. Worth remembering
|
||||
// when reading AI latency — it is time-to-first-token, not total.
|
||||
driverCallDuration.record(Date.now() - startedAt, {
|
||||
driver: ifaceName,
|
||||
'driver.method': method,
|
||||
outcome: 'ok',
|
||||
});
|
||||
this.clients.event?.emit(
|
||||
`driver.${ifaceName}.${method}.after`,
|
||||
{
|
||||
@@ -558,10 +533,10 @@ export class DriverController extends PuterController {
|
||||
// the live instance, so lifecycle hooks / inherited framework
|
||||
// methods can never be dispatched.
|
||||
this.#callableMethods.set(instance, resolveCallableMethods(instance));
|
||||
// Register each alias pointing at the same instance. Calls that pass
|
||||
// a provider id in the `driver` slot (e.g. `aws-polly` or
|
||||
// `openai-tts` instead of the unified `ai-tts`, as SDK bundles
|
||||
// predating the unified drivers do) resolve here; the handler sets
|
||||
// Register each alias pointing at the same instance. Legacy puter-js
|
||||
// calls that pass a provider id in the `driver` slot (e.g. the TTS
|
||||
// module sends `aws-polly` / `openai-tts` / `elevenlabs-tts` instead
|
||||
// of the unified `ai-tts`) resolve here; the handler sets
|
||||
// Context.driverName to the alias so the method can route to the
|
||||
// right internal provider.
|
||||
for (const alias of meta.aliases) {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import Busboy from 'busboy';
|
||||
@@ -438,11 +439,6 @@ export class FSController extends PuterController {
|
||||
let parseFailure: Error | null = null;
|
||||
const uploadPromises: Promise<UploadedBatchWriteItem | null>[] = [];
|
||||
const uploadedIndexes = new Set<number>();
|
||||
// Manifest indexes are client-chosen and may skip values (an
|
||||
// ignored `.DS_Store`, or a sparse numbering). `prepareBatchWrites`
|
||||
// re-indexes by array position, so the two spaces have to be
|
||||
// bridged explicitly or file parts get paired with the wrong item.
|
||||
const preparedIndexByManifestIndex = new Map<number, number>();
|
||||
let fileOrderIndex = 0;
|
||||
|
||||
const failParse = (error: unknown) => {
|
||||
@@ -553,12 +549,6 @@ export class FSController extends PuterController {
|
||||
{ pathAlreadyNormalized: true },
|
||||
);
|
||||
|
||||
activeManifestItems.forEach((item, position) => {
|
||||
preparedIndexByManifestIndex.set(
|
||||
item.index,
|
||||
position,
|
||||
);
|
||||
});
|
||||
preparedBatch =
|
||||
await this.services.fs.prepareBatchWrites(
|
||||
userId,
|
||||
@@ -631,13 +621,9 @@ export class FSController extends PuterController {
|
||||
}
|
||||
uploadedIndexes.add(itemIndex);
|
||||
|
||||
const preparedIndex =
|
||||
preparedIndexByManifestIndex.get(itemIndex);
|
||||
const preparedItem =
|
||||
preparedIndex === undefined
|
||||
? undefined
|
||||
: preparedBatch.itemsByIndex.get(preparedIndex);
|
||||
if (preparedIndex === undefined || !preparedItem) {
|
||||
preparedBatch.itemsByIndex.get(itemIndex);
|
||||
if (!preparedItem) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
`Batch write metadata was not found for index ${itemIndex}`,
|
||||
@@ -655,7 +641,7 @@ export class FSController extends PuterController {
|
||||
|
||||
return await this.services.fs.uploadPreparedBatchItem({
|
||||
preparedBatch,
|
||||
itemIndex: preparedIndex,
|
||||
itemIndex,
|
||||
fileContent: stream,
|
||||
uploadTracker,
|
||||
});
|
||||
@@ -667,12 +653,6 @@ export class FSController extends PuterController {
|
||||
}
|
||||
})();
|
||||
uploadPromises.push(uploadPromise);
|
||||
// Failures are collected with `Promise.allSettled` once
|
||||
// parsing finishes, which is many ticks away — attach an
|
||||
// inert handler now so an immediate rejection (e.g. a file
|
||||
// part arriving before the manifest) isn't reported as an
|
||||
// unhandled rejection in the meantime.
|
||||
void uploadPromise.catch(() => undefined);
|
||||
});
|
||||
|
||||
const parsingComplete = new Promise<void>((resolve, reject) => {
|
||||
@@ -682,18 +662,6 @@ export class FSController extends PuterController {
|
||||
|
||||
req.pipe(busboy);
|
||||
await parsingComplete;
|
||||
// A manifest that failed to parse never sets
|
||||
// `manifestPreparationPromise`, so surface the real reason
|
||||
// (invalid JSON, bad item index, duplicate index → 409) before
|
||||
// falling back to "no manifest was sent at all". Scoped to that
|
||||
// case on purpose: with no prepared batch nothing can have been
|
||||
// uploaded, so there is nothing to clean up. A parse failure that
|
||||
// arrives *after* a manifest was accepted has to fall through to
|
||||
// the cleanup path below, which sweeps objects already in storage.
|
||||
if (parseFailure && !manifestPreparationPromise) {
|
||||
await Promise.allSettled(uploadPromises);
|
||||
throw parseFailure;
|
||||
}
|
||||
if (!manifestPreparationPromise) {
|
||||
await Promise.allSettled(uploadPromises);
|
||||
throw new HttpError(400, 'Batch write manifest is required', {
|
||||
@@ -2091,8 +2059,7 @@ export class FSController extends PuterController {
|
||||
// the ActorUser type. Access via the escape hatch until a proper
|
||||
// storage-quota mechanism is in place.
|
||||
const actorUser = req.actor?.user as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
Record<string, unknown> | undefined;
|
||||
|
||||
const candidates = [
|
||||
this.#toStorageCapacityCandidate(actorUser?.free_storage),
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -928,87 +928,6 @@ describe('LegacyFSController.copy', () => {
|
||||
// The entry must still exist — the ghost handler must NOT have run.
|
||||
expect(await server.stores.fsEntry.getEntryByPath(src)).not.toBeNull();
|
||||
});
|
||||
|
||||
it('surfaces a name collision, then reports and removes the replaced entry on overwrite', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const username = actor.user!.username!;
|
||||
const src = `/${username}/Documents/dup.txt`;
|
||||
const existing = `/${username}/Pictures/dup.txt`;
|
||||
for ( const p of [src, existing] ) {
|
||||
await withActor(actor, () =>
|
||||
controller.touch(
|
||||
makeReq({ body: { path: p }, actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
// Without overwrite: the v1 conflict contract the GUI's
|
||||
// replace/skip prompts key on.
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
controller.copy(
|
||||
makeReq({
|
||||
body: {
|
||||
source: src,
|
||||
destination: `/${username}/Pictures`,
|
||||
},
|
||||
actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 409,
|
||||
legacyCode: 'item_with_same_name_exists',
|
||||
fields: { entry_name: 'dup.txt' },
|
||||
});
|
||||
|
||||
const replaced = (await server.stores.fsEntry.getEntryByPath(
|
||||
existing,
|
||||
))!;
|
||||
|
||||
// With overwrite: the replaced entry rides along in the response
|
||||
// (so the caller can drop its row) and item.removed tells every
|
||||
// other client to do the same — without it they keep a ghost row
|
||||
// until the directory is re-listed.
|
||||
const emitSpy = vi.spyOn(server.clients.event, 'emit');
|
||||
let body: Array<{
|
||||
copied: { path: string };
|
||||
overwritten?: { id: string };
|
||||
}>;
|
||||
let removedCall: (typeof emitSpy.mock.calls)[number] | undefined;
|
||||
try {
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(actor, () =>
|
||||
controller.copy(
|
||||
makeReq({
|
||||
body: {
|
||||
source: src,
|
||||
destination: `/${username}/Pictures`,
|
||||
overwrite: true,
|
||||
},
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
);
|
||||
body = captured.body as typeof body;
|
||||
removedCall = emitSpy.mock.calls.find(
|
||||
([eventName]) => eventName === 'outer.gui.item.removed',
|
||||
);
|
||||
} finally {
|
||||
emitSpy.mockRestore();
|
||||
}
|
||||
|
||||
expect(body[0].copied.path).toBe(existing);
|
||||
expect(body[0].overwritten?.id).toBe(replaced.uuid);
|
||||
expect(removedCall).toBeTruthy();
|
||||
const removedPayload = removedCall?.[1] as {
|
||||
response?: { uid?: string };
|
||||
};
|
||||
expect(removedPayload.response?.uid).toBe(replaced.uuid);
|
||||
});
|
||||
});
|
||||
|
||||
// ── move ────────────────────────────────────────────────────────────
|
||||
@@ -1085,89 +1004,6 @@ describe('LegacyFSController.move', () => {
|
||||
const body = captured.body as { moved: { path: string } };
|
||||
expect(body.moved.path).toBe(`/${username}/Pictures/bar`);
|
||||
});
|
||||
|
||||
it('surfaces a name collision, then reports and removes the replaced entry on overwrite', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const username = actor.user!.username!;
|
||||
const src = `/${username}/Documents/clash.txt`;
|
||||
const existing = `/${username}/Pictures/clash.txt`;
|
||||
for ( const p of [src, existing] ) {
|
||||
await withActor(actor, () =>
|
||||
controller.touch(
|
||||
makeReq({ body: { path: p }, actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
// Without overwrite: the v1 conflict contract the GUI's
|
||||
// replace/skip prompts key on.
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
controller.move(
|
||||
makeReq({
|
||||
body: {
|
||||
source: src,
|
||||
destination: `/${username}/Pictures`,
|
||||
},
|
||||
actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 409,
|
||||
legacyCode: 'item_with_same_name_exists',
|
||||
fields: { entry_name: 'clash.txt' },
|
||||
});
|
||||
|
||||
const replaced = (await server.stores.fsEntry.getEntryByPath(
|
||||
existing,
|
||||
))!;
|
||||
|
||||
// With overwrite: the replaced entry rides along in the response
|
||||
// (so the caller can drop its row) and item.removed tells every
|
||||
// other client to do the same — without it they keep a ghost row
|
||||
// until the directory is re-listed.
|
||||
const emitSpy = vi.spyOn(server.clients.event, 'emit');
|
||||
let body: {
|
||||
moved: { path: string };
|
||||
old_path: string;
|
||||
overwritten?: { id: string };
|
||||
};
|
||||
let removedCall: (typeof emitSpy.mock.calls)[number] | undefined;
|
||||
try {
|
||||
const { res, captured } = makeRes();
|
||||
await withActor(actor, () =>
|
||||
controller.move(
|
||||
makeReq({
|
||||
body: {
|
||||
source: src,
|
||||
destination: `/${username}/Pictures`,
|
||||
overwrite: true,
|
||||
},
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
);
|
||||
body = captured.body as typeof body;
|
||||
removedCall = emitSpy.mock.calls.find(
|
||||
([eventName]) => eventName === 'outer.gui.item.removed',
|
||||
);
|
||||
} finally {
|
||||
emitSpy.mockRestore();
|
||||
}
|
||||
|
||||
expect(body.old_path).toBe(src);
|
||||
expect(body.moved.path).toBe(existing);
|
||||
expect(body.overwritten?.id).toBe(replaced.uuid);
|
||||
expect(removedCall).toBeTruthy();
|
||||
const removedPayload = removedCall?.[1] as {
|
||||
response?: { uid?: string };
|
||||
};
|
||||
expect(removedPayload.response?.uid).toBe(replaced.uuid);
|
||||
});
|
||||
});
|
||||
|
||||
// ── search ──────────────────────────────────────────────────────────
|
||||
@@ -2907,48 +2743,6 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
|
||||
const body = captured.body as { thumbnail: string };
|
||||
expect(typeof body.thumbnail).toBe('string');
|
||||
});
|
||||
|
||||
// The write ACL here covers the entry being annotated, not whatever the
|
||||
// thumbnail string points at. A storage pointer stored verbatim is later
|
||||
// presigned (and deleted) by the thumbnails extension using the server's
|
||||
// own credentials, so the owner of one file could name another user's
|
||||
// object — an fs object's key is its fsentry uuid — and have the server
|
||||
// read or destroy it. Only inline image data is accepted.
|
||||
it.each([
|
||||
['an s3:// pointer', 's3://puter-local/00000000-0000-4000-8000-000000000000'],
|
||||
['an https URL', 'https://cdn.example.com/x.png'],
|
||||
['a bare object key', 'thumbnails/whatever'],
|
||||
])('rejects %s instead of storing it verbatim', async (_label, thumbnail) => {
|
||||
const { actor } = await makeUser();
|
||||
const username = actor.user!.username!;
|
||||
const target = `/${username}/Documents/thumbme3-${uuidv4()}.txt`;
|
||||
await withActor(actor, () =>
|
||||
controller.touch(
|
||||
makeReq({
|
||||
body: { path: target, set_modified_to_now: true },
|
||||
actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
);
|
||||
const entry = await server.stores.fsEntry.getEntryByPath(target);
|
||||
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
controller.updateFsentryThumbnail(
|
||||
makeReq({
|
||||
body: { uid: entry!.uuid, thumbnail },
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
|
||||
const after = await server.stores.fsEntry.getEntryByUuid(entry!.uuid);
|
||||
expect(after?.thumbnail ?? null).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
// ── GET /get-launch-apps ────────────────────────────────────────────
|
||||
@@ -3116,24 +2910,4 @@ describe('LegacyFSController GET /get-launch-apps', () => {
|
||||
expect(entry?.index_url).toBe('https://dev-owned-domain.example/');
|
||||
expect(entry?.privateAccess).toBeUndefined();
|
||||
});
|
||||
|
||||
// The rows are fetched in one batched lookup, which returns a map — the
|
||||
// handler has to re-impose the recency order the uid list carries.
|
||||
it('preserves the most-recent-first order of the underlying uid list', async () => {
|
||||
const { actor, userId } = await makeUser();
|
||||
const first = await makeHostedApp(userId, 'unregistered-a');
|
||||
const second = await makeHostedApp(userId, 'unregistered-b');
|
||||
const third = await makeHostedApp(userId, 'unregistered-c');
|
||||
|
||||
await recordOpen(userId, first.uid);
|
||||
await recordOpen(userId, second.uid);
|
||||
await recordOpen(userId, third.uid);
|
||||
|
||||
const recentUids = await server.stores.app.getRecentAppOpens(userId, {
|
||||
limit: 10,
|
||||
});
|
||||
const returned = (await fetchRecent(actor)).map((a) => a.uuid);
|
||||
|
||||
expect(returned).toEqual(recentUids);
|
||||
});
|
||||
});
|
||||
@@ -1,20 +1,21 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import Busboy from 'busboy';
|
||||
@@ -183,60 +184,48 @@ export class LegacyFSController extends PuterController {
|
||||
) => Promise<string[]>;
|
||||
}
|
||||
).getRecentAppOpens?.(userId, { limit: 10 })) ?? [];
|
||||
// One batched read for the rows, then the backing checks
|
||||
// concurrently. Serially awaiting a lookup per uid put ~2
|
||||
// round trips of latency on every desktop boot.
|
||||
const appsByUid = await (
|
||||
this.stores.app as unknown as {
|
||||
getByUids: (
|
||||
uids: string[],
|
||||
) => Promise<Map<string, Record<string, unknown>>>;
|
||||
const apps: unknown[] = [];
|
||||
for (const uid of recentUids) {
|
||||
const app = await (
|
||||
this.stores.app as unknown as {
|
||||
getByUid: (
|
||||
uid: string,
|
||||
) => Promise<Record<string, unknown> | null>;
|
||||
}
|
||||
).getByUid(uid);
|
||||
if (app) {
|
||||
// Don't hand out an index_url whose puter-hosted
|
||||
// backing is gone or reclaimed. The taskbar launches
|
||||
// recents by name (so AppDriver's guard applies), but
|
||||
// this list is a launch-metadata producer like any
|
||||
// other — a future consumer reading index_url
|
||||
// straight off it shouldn't inherit a stale origin.
|
||||
const backingGone =
|
||||
await hostedIndexUrlBackingIsUnavailable({
|
||||
app,
|
||||
subdomainStore: this.stores.subdomain,
|
||||
config: this.config,
|
||||
}).catch(() => true);
|
||||
apps.push({
|
||||
uuid: app.uid,
|
||||
name: app.name,
|
||||
title: app.title,
|
||||
icon: app.icon ?? null,
|
||||
godmode: Boolean(app.godmode),
|
||||
maximize_on_start: Boolean(app.maximize_on_start),
|
||||
index_url: backingGone ? null : app.index_url,
|
||||
...(backingGone
|
||||
? { privateAccess: buildHostedBackingDenial() }
|
||||
: {}),
|
||||
// An app with no owner isn't owned by a Puter user —
|
||||
// it's an "external" (origin-bootstrapped) app.
|
||||
external:
|
||||
app.owner_user_id == null ||
|
||||
app.owner_user_id === '',
|
||||
});
|
||||
}
|
||||
).getByUids(recentUids);
|
||||
|
||||
// `recentUids` is ordered most-recent-first; preserve it.
|
||||
const orderedApps = recentUids
|
||||
.map((uid) => appsByUid.get(uid))
|
||||
.filter((app): app is Record<string, unknown> =>
|
||||
Boolean(app),
|
||||
);
|
||||
|
||||
// Don't hand out an index_url whose puter-hosted backing is
|
||||
// gone or reclaimed. The taskbar launches recents by name (so
|
||||
// AppDriver's guard applies), but this list is a
|
||||
// launch-metadata producer like any other — a future consumer
|
||||
// reading index_url straight off it shouldn't inherit a stale
|
||||
// origin.
|
||||
const backingGoneFlags = await Promise.all(
|
||||
orderedApps.map((app) =>
|
||||
hostedIndexUrlBackingIsUnavailable({
|
||||
app,
|
||||
subdomainStore: this.stores.subdomain,
|
||||
config: this.config,
|
||||
}).catch(() => true),
|
||||
),
|
||||
);
|
||||
|
||||
recent = orderedApps.map((app, index) => {
|
||||
const backingGone = backingGoneFlags[index];
|
||||
return {
|
||||
uuid: app.uid,
|
||||
name: app.name,
|
||||
title: app.title,
|
||||
icon: app.icon ?? null,
|
||||
godmode: Boolean(app.godmode),
|
||||
maximize_on_start: Boolean(app.maximize_on_start),
|
||||
index_url: backingGone ? null : app.index_url,
|
||||
...(backingGone
|
||||
? { privateAccess: buildHostedBackingDenial() }
|
||||
: {}),
|
||||
// An app with no owner isn't owned by a Puter user —
|
||||
// it's an "external" (origin-bootstrapped) app.
|
||||
external:
|
||||
app.owner_user_id == null ||
|
||||
app.owner_user_id === '',
|
||||
};
|
||||
});
|
||||
}
|
||||
recent = apps;
|
||||
}
|
||||
|
||||
res.json({ recommended, recent });
|
||||
@@ -588,61 +577,26 @@ export class LegacyFSController extends PuterController {
|
||||
'write',
|
||||
);
|
||||
|
||||
// The v1 wire contract reports the entry an overwrite replaced
|
||||
// (`overwritten`) so clients can drop its stale row/icon. The copy
|
||||
// deletes that entry, so resolve it beforehand.
|
||||
const overwriteRequested = getBoolean(body, 'overwrite') ?? false;
|
||||
let overwrittenEntry = null;
|
||||
if (overwriteRequested) {
|
||||
const targetName = getString(body, 'new_name') ?? source.name;
|
||||
const targetPath =
|
||||
destinationParent.path === '/'
|
||||
? `/${targetName}`
|
||||
: `${destinationParent.path}/${targetName}`;
|
||||
overwrittenEntry =
|
||||
await this.stores.fsEntry.getEntryByPath(targetPath);
|
||||
}
|
||||
|
||||
const copy = await this.services.fs.copy(userId, {
|
||||
source,
|
||||
destinationParent,
|
||||
newName: getString(body, 'new_name'),
|
||||
overwrite: overwriteRequested,
|
||||
overwrite: getBoolean(body, 'overwrite') ?? false,
|
||||
dedupeName: getBoolean(body, 'dedupe_name', 'change_name') ?? false,
|
||||
});
|
||||
await this.#emitGuiEvent('outer.gui.item.added', copy);
|
||||
// Without this, every other client keeps a ghost row for the
|
||||
// replaced entry until the directory is re-listed.
|
||||
if (overwrittenEntry) {
|
||||
await this.#emitGuiEvent(
|
||||
'outer.gui.item.removed',
|
||||
overwrittenEntry,
|
||||
);
|
||||
}
|
||||
|
||||
// Legacy response shape: `[{copied: fsentry, overwritten?}]`.
|
||||
// Array is historical — originally supported bulk copy.
|
||||
const legacyEntryOpts = {
|
||||
const copied = await toLegacyEntry(this.clients.event, copy, {
|
||||
fsEntryStore: this.stores.fsEntry,
|
||||
userStore: this.stores.user as unknown as {
|
||||
getById: (
|
||||
id: number,
|
||||
) => Promise<Record<string, unknown> | null>;
|
||||
},
|
||||
};
|
||||
const copied = await toLegacyEntry(
|
||||
this.clients.event,
|
||||
copy,
|
||||
legacyEntryOpts,
|
||||
);
|
||||
const overwritten = overwrittenEntry
|
||||
? await toLegacyEntry(
|
||||
this.clients.event,
|
||||
overwrittenEntry,
|
||||
legacyEntryOpts,
|
||||
)
|
||||
: undefined;
|
||||
res.json([{ copied, ...(overwritten ? { overwritten } : {}) }]);
|
||||
});
|
||||
res.json([{ copied }]);
|
||||
};
|
||||
|
||||
move = async (req: Request, res: Response): Promise<void> => {
|
||||
@@ -674,30 +628,11 @@ export class LegacyFSController extends PuterController {
|
||||
'write',
|
||||
);
|
||||
|
||||
// The v1 wire contract reports the entry an overwrite replaced
|
||||
// (`overwritten`) so clients can drop its stale row/icon. The move
|
||||
// deletes that entry, so resolve it beforehand.
|
||||
const overwriteRequested = getBoolean(body, 'overwrite') ?? false;
|
||||
let overwrittenEntry = null;
|
||||
if (overwriteRequested) {
|
||||
const targetName = getString(body, 'new_name') ?? source.name;
|
||||
const targetPath =
|
||||
destinationParent.path === '/'
|
||||
? `/${targetName}`
|
||||
: `${destinationParent.path}/${targetName}`;
|
||||
const existing =
|
||||
await this.stores.fsEntry.getEntryByPath(targetPath);
|
||||
// Moving an entry onto its own path is not an overwrite.
|
||||
if (existing && existing.uuid !== source.uuid) {
|
||||
overwrittenEntry = existing;
|
||||
}
|
||||
}
|
||||
|
||||
const moved = await this.services.fs.move(userId, {
|
||||
source,
|
||||
destinationParent,
|
||||
newName: getString(body, 'new_name'),
|
||||
overwrite: overwriteRequested,
|
||||
overwrite: getBoolean(body, 'overwrite') ?? false,
|
||||
dedupeName: getBoolean(body, 'dedupe_name', 'change_name') ?? false,
|
||||
// Trash/restore rides on this: GUI sends
|
||||
// `{ original_name, original_path, trashed_ts }` when moving into
|
||||
@@ -710,41 +645,17 @@ export class LegacyFSController extends PuterController {
|
||||
await this.#emitGuiEvent('outer.gui.item.moved', moved, {
|
||||
old_path: oldPath,
|
||||
});
|
||||
// Without this, every other client keeps a ghost row for the
|
||||
// replaced entry until the directory is re-listed.
|
||||
if (overwrittenEntry) {
|
||||
await this.#emitGuiEvent(
|
||||
'outer.gui.item.removed',
|
||||
overwrittenEntry,
|
||||
);
|
||||
}
|
||||
|
||||
// Legacy response shape: `{moved: fsentry, old_path, overwritten?}`.
|
||||
const legacyEntryOpts = {
|
||||
// Legacy response shape: `{moved: fsentry, old_path}`.
|
||||
const movedEntry = await toLegacyEntry(this.clients.event, moved, {
|
||||
fsEntryStore: this.stores.fsEntry,
|
||||
userStore: this.stores.user as unknown as {
|
||||
getById: (
|
||||
id: number,
|
||||
) => Promise<Record<string, unknown> | null>;
|
||||
},
|
||||
};
|
||||
const movedEntry = await toLegacyEntry(
|
||||
this.clients.event,
|
||||
moved,
|
||||
legacyEntryOpts,
|
||||
);
|
||||
const overwritten = overwrittenEntry
|
||||
? await toLegacyEntry(
|
||||
this.clients.event,
|
||||
overwrittenEntry,
|
||||
legacyEntryOpts,
|
||||
)
|
||||
: undefined;
|
||||
res.json({
|
||||
moved: movedEntry,
|
||||
old_path: oldPath,
|
||||
...(overwritten ? { overwritten } : {}),
|
||||
});
|
||||
res.json({ moved: movedEntry, old_path: oldPath });
|
||||
};
|
||||
|
||||
delete = async (req: Request, res: Response): Promise<void> => {
|
||||
@@ -937,14 +848,6 @@ export class LegacyFSController extends PuterController {
|
||||
throw new HttpError(400, 'Missing `thumbnail`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
// Only inline image data. Clients generate the thumbnail themselves
|
||||
// and the thumbnails extension is what turns it into a storage
|
||||
// pointer; accepting a pointer here would let a caller name an object
|
||||
// the server would then sign reads of, and delete, on their behalf.
|
||||
if (!thumbnail.startsWith('data:'))
|
||||
throw new HttpError(400, '`thumbnail` must be a data: URL', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const entry = await this.stores.fsEntry.getEntryByUuid(uid);
|
||||
if (!entry || !entry.path)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -18,24 +18,13 @@
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
asRecord,
|
||||
fsEntryMimeType,
|
||||
getBoolean,
|
||||
getString,
|
||||
loadLegacyAssociatedApps,
|
||||
signEntryThumbnail,
|
||||
signingConfigFromAppConfig,
|
||||
toLegacyEntry,
|
||||
} from './legacyFsHelpers.js';
|
||||
import { loadLegacyAssociatedApps } from './legacyFsHelpers.js';
|
||||
import type { FSEntry } from '../../stores/fs/FSEntry.js';
|
||||
|
||||
const entryWithApp = (associatedAppId: number): FSEntry =>
|
||||
({ associatedAppId }) as unknown as FSEntry;
|
||||
|
||||
const appRow = (
|
||||
overrides: Record<string, unknown>,
|
||||
): Record<string, unknown> => ({
|
||||
const appRow = (overrides: Record<string, unknown>): Record<string, unknown> => ({
|
||||
id: 1,
|
||||
uid: 'app-1',
|
||||
owner_user_id: 99,
|
||||
@@ -111,336 +100,3 @@ describe('loadLegacyAssociatedApps associated_app redaction', () => {
|
||||
expect(app.godmode).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('asRecord', () => {
|
||||
it.each([
|
||||
['null', null],
|
||||
['undefined', undefined],
|
||||
['a number', 42],
|
||||
['a string', 'nope'],
|
||||
['an array', [1, 2]],
|
||||
])('returns an empty record for %s', (_label, value) => {
|
||||
expect(asRecord(value)).toEqual({});
|
||||
});
|
||||
|
||||
it('passes an object through unchanged', () => {
|
||||
const source = { a: 1 };
|
||||
expect(asRecord(source)).toBe(source);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getString', () => {
|
||||
it('returns the first non-empty string among the candidate keys', () => {
|
||||
expect(getString({ a: '', b: 'second' }, 'a', 'b')).toBe('second');
|
||||
});
|
||||
|
||||
it('ignores non-string values', () => {
|
||||
expect(getString({ a: 5, b: true }, 'a', 'b')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('returns undefined when no key matches', () => {
|
||||
expect(getString({}, 'missing')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('getBoolean', () => {
|
||||
it.each([
|
||||
[true, true],
|
||||
[false, false],
|
||||
[1, true],
|
||||
[0, false],
|
||||
['1', true],
|
||||
['TRUE', true],
|
||||
[' yes ', true],
|
||||
['on', true],
|
||||
['0', false],
|
||||
['false', false],
|
||||
['no', false],
|
||||
['off', false],
|
||||
])('coerces %o to %s', (input, expected) => {
|
||||
expect(getBoolean({ flag: input }, 'flag')).toBe(expected);
|
||||
});
|
||||
|
||||
it.each([[2], ['maybe'], [null], [{}]])(
|
||||
'returns undefined for the uncoercible value %o',
|
||||
(input) => {
|
||||
expect(getBoolean({ flag: input }, 'flag')).toBeUndefined();
|
||||
},
|
||||
);
|
||||
|
||||
it('falls through to a later alias when the first key is absent', () => {
|
||||
expect(
|
||||
getBoolean({ change_name: true }, 'dedupe_name', 'change_name'),
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('fsEntryMimeType', () => {
|
||||
it('reports directories as "folder"', () => {
|
||||
expect(fsEntryMimeType({ isDir: true, name: 'Documents' })).toBe(
|
||||
'folder',
|
||||
);
|
||||
});
|
||||
|
||||
it('derives a MIME type with charset from the file name', () => {
|
||||
expect(fsEntryMimeType({ isDir: false, name: 'a.png' })).toBe(
|
||||
'image/png',
|
||||
);
|
||||
expect(fsEntryMimeType({ isDir: false, name: 'a.txt' })).toContain(
|
||||
'text/plain',
|
||||
);
|
||||
});
|
||||
|
||||
it('returns null for an extensionless name', () => {
|
||||
expect(fsEntryMimeType({ isDir: false, name: 'LICENSE' })).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('signEntryThumbnail', () => {
|
||||
it('returns the input untouched when there is no event client', async () => {
|
||||
expect(
|
||||
await signEntryThumbnail(undefined, 'uuid-1', 's3://bucket/key'),
|
||||
).toBe('s3://bucket/key');
|
||||
});
|
||||
|
||||
it('normalizes a missing thumbnail to null', async () => {
|
||||
expect(await signEntryThumbnail(undefined, 'uuid-1', null)).toBeNull();
|
||||
});
|
||||
|
||||
it('returns the URL the listener rewrote onto the payload', async () => {
|
||||
const eventClient = {
|
||||
emitAndWait: async (
|
||||
_key: string,
|
||||
payload: { thumbnail: string },
|
||||
) => {
|
||||
payload.thumbnail = 'https://signed.test/thumb.png';
|
||||
},
|
||||
} as never;
|
||||
expect(
|
||||
await signEntryThumbnail(eventClient, 'uuid-1', 's3://bucket/key'),
|
||||
).toBe('https://signed.test/thumb.png');
|
||||
});
|
||||
|
||||
it('returns null when the listener blanks the thumbnail', async () => {
|
||||
const eventClient = {
|
||||
emitAndWait: async (
|
||||
_key: string,
|
||||
payload: { thumbnail: string },
|
||||
) => {
|
||||
payload.thumbnail = '';
|
||||
},
|
||||
} as never;
|
||||
expect(
|
||||
await signEntryThumbnail(eventClient, 'uuid-1', 's3://bucket/key'),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('keeps the original value when the listener throws', async () => {
|
||||
const eventClient = {
|
||||
emitAndWait: async () => {
|
||||
throw new Error('extension down');
|
||||
},
|
||||
} as never;
|
||||
expect(
|
||||
await signEntryThumbnail(eventClient, 'uuid-1', 's3://bucket/key'),
|
||||
).toBe('s3://bucket/key');
|
||||
});
|
||||
});
|
||||
|
||||
describe('signingConfigFromAppConfig', () => {
|
||||
it('returns the secret and api base url when both are set', () => {
|
||||
expect(
|
||||
signingConfigFromAppConfig({
|
||||
url_signature_secret: 's3cret',
|
||||
api_base_url: 'https://api.test',
|
||||
} as never),
|
||||
).toEqual({ secret: 's3cret', apiBaseUrl: 'https://api.test' });
|
||||
});
|
||||
|
||||
it('fails loudly when the signing secret is missing', () => {
|
||||
expect(() =>
|
||||
signingConfigFromAppConfig({
|
||||
api_base_url: 'https://api.test',
|
||||
} as never),
|
||||
).toThrowError(/url_signature_secret not set/);
|
||||
});
|
||||
|
||||
it('fails loudly when the api base url is missing', () => {
|
||||
expect(() =>
|
||||
signingConfigFromAppConfig({
|
||||
url_signature_secret: 's3cret',
|
||||
} as never),
|
||||
).toThrowError(/api_base_url not set/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('toLegacyEntry', () => {
|
||||
const baseEntry = (overrides: Partial<FSEntry> = {}): FSEntry =>
|
||||
({
|
||||
uuid: 'uuid-1',
|
||||
parentUid: 'parent-1',
|
||||
path: '/alice/Documents/report.pdf',
|
||||
name: 'report.pdf',
|
||||
isDir: false,
|
||||
isShortcut: false,
|
||||
shortcutTo: null,
|
||||
isSymlink: false,
|
||||
symlinkPath: null,
|
||||
isPublic: false,
|
||||
thumbnail: null,
|
||||
immutable: false,
|
||||
metadata: null,
|
||||
modified: 1000,
|
||||
created: 900,
|
||||
accessed: 950,
|
||||
size: 12,
|
||||
layout: null,
|
||||
subdomains: [],
|
||||
workers: [],
|
||||
suggestedApps: [],
|
||||
associatedAppId: null,
|
||||
userId: 7,
|
||||
...overrides,
|
||||
}) as unknown as FSEntry;
|
||||
|
||||
it('produces the snake_case v1 shape for a file', async () => {
|
||||
const shaped = await toLegacyEntry(undefined, baseEntry());
|
||||
expect(shaped).toMatchObject({
|
||||
id: 'uuid-1',
|
||||
uid: 'uuid-1',
|
||||
uuid: 'uuid-1',
|
||||
parent_id: 'parent-1',
|
||||
parent_uid: 'parent-1',
|
||||
dirname: '/alice/Documents',
|
||||
dirpath: '/alice/Documents',
|
||||
is_dir: false,
|
||||
is_shortcut: 0,
|
||||
is_symlink: 0,
|
||||
has_website: false,
|
||||
is_empty: false,
|
||||
associated_app: null,
|
||||
appdata_app: undefined,
|
||||
});
|
||||
expect(shaped.type).toContain('application/pdf');
|
||||
});
|
||||
|
||||
it('names the owning app for an AppData path', async () => {
|
||||
const shaped = await toLegacyEntry(
|
||||
undefined,
|
||||
baseEntry({ path: '/alice/AppData/app-42/state.json' }),
|
||||
);
|
||||
expect(shaped.appdata_app).toBe('app-42');
|
||||
});
|
||||
|
||||
it('reports has_website when the entry carries a subdomain', async () => {
|
||||
const shaped = await toLegacyEntry(
|
||||
undefined,
|
||||
baseEntry({
|
||||
subdomains: [{ subdomain: 'site' }] as never,
|
||||
}),
|
||||
);
|
||||
expect(shaped.has_website).toBe(true);
|
||||
});
|
||||
|
||||
it('probes for children to set is_empty on a directory', async () => {
|
||||
const fsEntryStore = {
|
||||
listChildren: async () => [{ uuid: 'child' }],
|
||||
} as never;
|
||||
const shaped = await toLegacyEntry(
|
||||
undefined,
|
||||
baseEntry({ isDir: true, name: 'Documents' }),
|
||||
{ fsEntryStore },
|
||||
);
|
||||
expect(shaped.is_empty).toBe(false);
|
||||
expect(shaped.type).toBe('folder');
|
||||
});
|
||||
|
||||
it('treats a failed child probe as non-empty rather than throwing', async () => {
|
||||
const fsEntryStore = {
|
||||
listChildren: async () => {
|
||||
throw new Error('db down');
|
||||
},
|
||||
} as never;
|
||||
const shaped = await toLegacyEntry(
|
||||
undefined,
|
||||
baseEntry({ isDir: true }),
|
||||
{ fsEntryStore },
|
||||
);
|
||||
expect(shaped.is_empty).toBe(false);
|
||||
});
|
||||
|
||||
it('hydrates the owner as a username-only object', async () => {
|
||||
const userStore = {
|
||||
getById: async () => ({ username: 'alice', email: '[email protected]' }),
|
||||
};
|
||||
const shaped = await toLegacyEntry(undefined, baseEntry(), {
|
||||
userStore,
|
||||
});
|
||||
// Username only — the rest of the user row must not ride along.
|
||||
expect(shaped.owner).toEqual({ username: 'alice' });
|
||||
});
|
||||
|
||||
it('omits the owner when the lookup fails', async () => {
|
||||
const userStore = {
|
||||
getById: async () => {
|
||||
throw new Error('db down');
|
||||
},
|
||||
};
|
||||
const shaped = await toLegacyEntry(undefined, baseEntry(), {
|
||||
userStore,
|
||||
});
|
||||
expect(shaped).not.toHaveProperty('owner');
|
||||
});
|
||||
|
||||
it('embeds associated_app from the prebuilt map', async () => {
|
||||
const appsById = new Map([[3, { uid: 'app-3', name: 'Editor' }]]);
|
||||
const shaped = await toLegacyEntry(
|
||||
undefined,
|
||||
baseEntry({ associatedAppId: 3 }),
|
||||
{ appsById },
|
||||
);
|
||||
expect(shaped.associated_app).toEqual({ uid: 'app-3', name: 'Editor' });
|
||||
});
|
||||
|
||||
it('emits a null associated_app when the id is not in the map', async () => {
|
||||
const shaped = await toLegacyEntry(
|
||||
undefined,
|
||||
baseEntry({ associatedAppId: 9 }),
|
||||
{ appsById: new Map() },
|
||||
);
|
||||
expect(shaped.associated_app).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('loadLegacyAssociatedApps short-circuit', () => {
|
||||
it('makes no store call when no entry carries an app id', async () => {
|
||||
let calls = 0;
|
||||
const store = {
|
||||
getByIds: async () => {
|
||||
calls += 1;
|
||||
return new Map();
|
||||
},
|
||||
};
|
||||
const out = await loadLegacyAssociatedApps(store, [
|
||||
{ associatedAppId: null } as unknown as FSEntry,
|
||||
]);
|
||||
expect(out.size).toBe(0);
|
||||
expect(calls).toBe(0);
|
||||
});
|
||||
|
||||
it('dedupes repeated app ids into a single lookup', async () => {
|
||||
const seen: number[][] = [];
|
||||
const store = {
|
||||
getByIds: async (ids: number[]) => {
|
||||
seen.push(ids);
|
||||
return new Map(ids.map((id) => [id, appRow({ id })]));
|
||||
},
|
||||
};
|
||||
await loadLegacyAssociatedApps(store, [
|
||||
entryWithApp(4),
|
||||
entryWithApp(4),
|
||||
entryWithApp(5),
|
||||
]);
|
||||
expect(seen).toEqual([[4, 5]]);
|
||||
});
|
||||
});
|
||||
@@ -1,20 +1,21 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import { posix as pathPosix } from 'node:path';
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import type { Readable } from 'node:stream';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -314,76 +314,6 @@ describe('HomepageController GET /app/:name', () => {
|
||||
expect(html).not.toContain(secretUrl);
|
||||
expect(html).not.toContain('owner_user_id');
|
||||
});
|
||||
|
||||
it('serves the same app shell under /desktop/app/:name', async () => {
|
||||
const { userId } = await makeUser();
|
||||
const name = `desk-${Math.random().toString(36).slice(2, 10)}`;
|
||||
await server.stores.app.create(
|
||||
{
|
||||
name,
|
||||
title: 'Desktop App',
|
||||
description: 'opens on the desktop instead of the dashboard',
|
||||
index_url: `https://example.com/${name}/`,
|
||||
approved_for_listing: 1,
|
||||
},
|
||||
{ ownerUserId: userId },
|
||||
);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/desktop/app/:name',
|
||||
makeReq({ params: { name }, path: `/desktop/app/${name}` }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(captured.statusCode).toBe(200);
|
||||
const html = String(captured.body);
|
||||
expect(html).toMatch(/<!DOCTYPE html>/i);
|
||||
expect(html).toContain('Desktop App');
|
||||
});
|
||||
|
||||
it('returns 404 under /desktop/app/:name when the app is unknown', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/desktop/app/:name',
|
||||
makeReq({
|
||||
params: { name: 'no-such-app' },
|
||||
path: '/desktop/app/no-such-app',
|
||||
}),
|
||||
res,
|
||||
);
|
||||
expect(captured.statusCode).toBe(404);
|
||||
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
|
||||
});
|
||||
|
||||
it('omits index_url even for a public app — the shell is not the launch authority', async () => {
|
||||
const { userId } = await makeUser();
|
||||
const name = `pub-${Math.random().toString(36).slice(2, 10)}`;
|
||||
const indexUrl = `https://public-${name}.example.com/`;
|
||||
await server.stores.app.create(
|
||||
{ name, title: 'Public App', index_url: indexUrl },
|
||||
{ ownerUserId: userId },
|
||||
);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/app/:name',
|
||||
makeReq({ params: { name }, path: `/app/${name}` }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(captured.statusCode).toBe(200);
|
||||
const html = String(captured.body);
|
||||
expect(html).toContain('Public App');
|
||||
// The GUI re-reads the app through the driver before launching, which
|
||||
// is where the entitlement gate and hosted-backing guard run. Baking
|
||||
// a launch URL into server-rendered HTML buys nothing and costs the
|
||||
// lookups those guards require.
|
||||
expect(html).not.toContain(indexUrl);
|
||||
});
|
||||
});
|
||||
|
||||
// ── /show/* ─────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -20,7 +20,6 @@
|
||||
import express from 'express';
|
||||
import path from 'node:path';
|
||||
import { PuterController } from '../types.js';
|
||||
import { toAppShellView } from '../../util/appShellView.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter';
|
||||
import type {
|
||||
PuterHomepageService,
|
||||
@@ -33,8 +32,8 @@ import type {
|
||||
* under `<gui_assets_root>/src` for non-dist/src paths (images, fonts, lib
|
||||
* files referenced from the shell).
|
||||
*
|
||||
* All root-subdomain-only. Registered last in the controller list so the static
|
||||
* catch-all doesn't shadow specific API routes.
|
||||
* All root-subdomain-only. Registered last in the controller list so the
|
||||
* static catch-all doesn't shadow specific API routes.
|
||||
*/
|
||||
export class HomepageController extends PuterController {
|
||||
registerRoutes(router: PuterRouter) {
|
||||
@@ -91,14 +90,8 @@ export class HomepageController extends PuterController {
|
||||
router.get('/@:username', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
// -- /app/:name - app metadata baked into the shell ----------
|
||||
// `/desktop/app/:name` is the same landing booted on the desktop
|
||||
// instead of the dashboard; the GUI strips the prefix and treats the
|
||||
// rest of the path identically, so both render the same shell.
|
||||
|
||||
const sendAppShell = async (
|
||||
req: express.Request,
|
||||
res: express.Response,
|
||||
) => {
|
||||
router.get('/app/:name', {}, async (req, res) => {
|
||||
const name = String(req.params.name ?? '');
|
||||
const app = name ? await this.stores.app.getByName(name) : null;
|
||||
|
||||
@@ -109,11 +102,17 @@ export class HomepageController extends PuterController {
|
||||
: (app.metadata as Record<string, unknown> | null)) ??
|
||||
{};
|
||||
// Never bake the raw store row into the shell — it exposes
|
||||
// `index_url`, `owner_user_id`, and moderation flags. The
|
||||
// shell only needs a preview; the GUI re-reads the app
|
||||
// through the apps driver before launching, and that read is
|
||||
// where the entitlement gate and hosted-backing guard run.
|
||||
const clientApp = toAppShellView(app);
|
||||
// `index_url`, `owner_user_id`, and admin flags. Run it
|
||||
// through the authoritative serializer, which redacts to a
|
||||
// safe field subset and drops `index_url` for a private app
|
||||
// this actor (possibly anonymous) isn't entitled to.
|
||||
const actor =
|
||||
(req as express.Request & { actor?: unknown }).actor ??
|
||||
null;
|
||||
const clientApp = await this.drivers.apps.toClientView(
|
||||
app,
|
||||
actor,
|
||||
);
|
||||
await sendShell(req, res, {
|
||||
title: String(app.title ?? name),
|
||||
description: String(app.description ?? ''),
|
||||
@@ -136,10 +135,7 @@ export class HomepageController extends PuterController {
|
||||
? name.charAt(0).toUpperCase() + name.slice(1)
|
||||
: 'Puter',
|
||||
});
|
||||
};
|
||||
|
||||
router.get('/app/:name', {}, sendAppShell);
|
||||
router.get('/desktop/app/:name', {}, sendAppShell);
|
||||
});
|
||||
|
||||
// -- /show/* - launch explorer with the requested file path --
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -22,10 +22,10 @@ import { PuterController } from '../types.js';
|
||||
|
||||
/**
|
||||
* Site hosting endpoints. Listing and create/update are not exposed as
|
||||
* controller routes — clients use the `puter-subdomains` driver (select /
|
||||
* create / update / read) so they get the v1-shape with uuids and nested
|
||||
* objects (no raw mysql ids). Only `/delete-site` lives here because v1 also
|
||||
* exposed it as a top-level POST.
|
||||
* controller routes — clients use the `puter-subdomains` driver
|
||||
* (select / create / update / read) so they get the v1-shape with
|
||||
* uuids and nested objects (no raw mysql ids). Only `/delete-site`
|
||||
* lives here because v1 also exposed it as a top-level POST.
|
||||
*/
|
||||
export class HostingController extends PuterController {
|
||||
constructor(config, clients, stores, services) {
|
||||
|
||||
@@ -1,184 +0,0 @@
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
|
||||
// `/delete-site` is the only hosting route exposed as a controller endpoint
|
||||
// (everything else goes through the `puter-subdomains` driver), and it is an
|
||||
// ownership-gated destructive operation — so the interesting cases are the
|
||||
// refusals.
|
||||
|
||||
let server;
|
||||
let deleteSite;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
const router = new PuterRouter();
|
||||
server.controllers.hosting.registerRoutes(router);
|
||||
deleteSite = router.routes.find(
|
||||
(route) => route.path === '/delete-site',
|
||||
).handler;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
const makeUser = async () => {
|
||||
const username = `host-${Math.random().toString(36).slice(2, 10)}`;
|
||||
const created = await server.stores.user.create({
|
||||
username,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email: `${username}@test.local`,
|
||||
requires_email_confirmation: false,
|
||||
});
|
||||
return {
|
||||
userId: created.id,
|
||||
username,
|
||||
actor: { user: { id: created.id, uuid: created.uuid, username } },
|
||||
};
|
||||
};
|
||||
|
||||
const makeSite = async (userId) => {
|
||||
const subdomain = `site-${Math.random().toString(36).slice(2, 10)}`;
|
||||
return server.stores.subdomain.create({ userId, subdomain });
|
||||
};
|
||||
|
||||
const makeReq = (init) => ({
|
||||
body: init.body,
|
||||
query: {},
|
||||
headers: {},
|
||||
actor: init.actor,
|
||||
});
|
||||
|
||||
const makeRes = () => {
|
||||
const captured = { statusCode: 200, body: undefined };
|
||||
const res = {
|
||||
json: vi.fn((value) => {
|
||||
captured.body = value;
|
||||
return res;
|
||||
}),
|
||||
status: vi.fn((code) => {
|
||||
captured.statusCode = code;
|
||||
return res;
|
||||
}),
|
||||
};
|
||||
return { res, captured };
|
||||
};
|
||||
|
||||
describe('HostingController /delete-site', () => {
|
||||
it('rejects a missing site_uuid', async () => {
|
||||
const { actor } = await makeUser();
|
||||
await expect(
|
||||
deleteSite(makeReq({ body: {}, actor }), makeRes().res),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects a non-string site_uuid', async () => {
|
||||
const { actor } = await makeUser();
|
||||
await expect(
|
||||
deleteSite(
|
||||
makeReq({ body: { site_uuid: 42 }, actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('rejects a missing body entirely', async () => {
|
||||
const { actor } = await makeUser();
|
||||
await expect(
|
||||
deleteSite(makeReq({ body: undefined, actor }), makeRes().res),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('404s a site uuid that does not exist', async () => {
|
||||
const { actor } = await makeUser();
|
||||
await expect(
|
||||
deleteSite(
|
||||
makeReq({ body: { site_uuid: uuidv4() }, actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 404,
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
});
|
||||
|
||||
it("404s another user's site rather than revealing it exists", async () => {
|
||||
const owner = await makeUser();
|
||||
const stranger = await makeUser();
|
||||
const site = await makeSite(owner.userId);
|
||||
|
||||
await expect(
|
||||
deleteSite(
|
||||
makeReq({
|
||||
body: { site_uuid: site.uuid },
|
||||
actor: stranger.actor,
|
||||
}),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
// Still there.
|
||||
expect(
|
||||
await server.stores.subdomain.getByUuid(site.uuid),
|
||||
).not.toBeNull();
|
||||
});
|
||||
|
||||
it('refuses to delete a protected subdomain', async () => {
|
||||
const { userId, actor } = await makeUser();
|
||||
const site = await makeSite(userId);
|
||||
await server.clients.db.write(
|
||||
'UPDATE `subdomains` SET `protected` = ? WHERE `uuid` = ?',
|
||||
[1, site.uuid],
|
||||
);
|
||||
|
||||
await expect(
|
||||
deleteSite(
|
||||
makeReq({ body: { site_uuid: site.uuid }, actor }),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
expect(
|
||||
await server.stores.subdomain.getByUuid(site.uuid),
|
||||
).not.toBeNull();
|
||||
});
|
||||
|
||||
it('deletes the caller-owned site and answers with an empty object', async () => {
|
||||
const { userId, actor } = await makeUser();
|
||||
const site = await makeSite(userId);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await deleteSite(
|
||||
makeReq({ body: { site_uuid: site.uuid }, actor }),
|
||||
res,
|
||||
);
|
||||
expect(captured.body).toEqual({});
|
||||
expect(await server.stores.subdomain.getByUuid(site.uuid)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -24,18 +24,19 @@ import type { NotificationService } from '../../services/notification/Notificati
|
||||
import { PuterController } from '../types.js';
|
||||
|
||||
/**
|
||||
* GUI-facing notification endpoints. These supplement the `puter-notifications`
|
||||
* driver (which handles CRUD via `/drivers/call`) with two small mutation
|
||||
* routes that the puter desktop client calls directly.
|
||||
* GUI-facing notification endpoints. These supplement the
|
||||
* `puter-notifications` driver (which handles CRUD via `/drivers/call`)
|
||||
* with two small mutation routes that the puter desktop client calls
|
||||
* directly.
|
||||
*
|
||||
* Both routes emit `outer.gui.notif.ack` via the NotificationService so other
|
||||
* open tabs for the same user see the state change immediately.
|
||||
* Both routes emit `outer.gui.notif.ack` via the NotificationService
|
||||
* so other open tabs for the same user see the state change immediately.
|
||||
*/
|
||||
@Controller('/notif')
|
||||
export class NotificationController extends PuterController {
|
||||
/**
|
||||
* POST /notif/mark-ack — user dismissed a notification. Sets `acknowledged`
|
||||
* timestamp; pushes ack event to sockets.
|
||||
* POST /notif/mark-ack — user dismissed a notification.
|
||||
* Sets `acknowledged` timestamp; pushes ack event to sockets.
|
||||
*/
|
||||
@Post('/mark-ack', {
|
||||
subdomain: 'api',
|
||||
@@ -77,8 +78,8 @@ export class NotificationController extends PuterController {
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /notif/mark-read — user saw a notification. Sets `shown` timestamp;
|
||||
* pushes ack event to sockets.
|
||||
* POST /notif/mark-read — user saw a notification.
|
||||
* Sets `shown` timestamp; pushes ack event to sockets.
|
||||
*/
|
||||
@Post('/mark-read', {
|
||||
subdomain: 'api',
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -27,7 +27,6 @@ import {
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import jwt from 'jsonwebtoken';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
@@ -330,77 +329,6 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
|
||||
expect(decoded).toMatchObject({ referrer: 'http://ref.test' });
|
||||
});
|
||||
|
||||
it('bakes a whitelisted return_to (/app/<name>) into the state redirect_uri', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/:provider/start',
|
||||
makeReq({
|
||||
params: { provider: 'custom' },
|
||||
query: { return_to: '/app/my-App_2' },
|
||||
}),
|
||||
res,
|
||||
);
|
||||
const state = new URL(captured.redirectUrl ?? '').searchParams.get(
|
||||
'state',
|
||||
);
|
||||
const decoded = oidc().verifyState(state!);
|
||||
expect(String(decoded?.redirect_uri)).toBe(
|
||||
`${TEST_ORIGIN}/app/my-App_2`,
|
||||
);
|
||||
});
|
||||
|
||||
it('bakes a whitelisted desktop app landing (/desktop/app/<name>) into the state redirect_uri', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/:provider/start',
|
||||
makeReq({
|
||||
params: { provider: 'custom' },
|
||||
query: { return_to: '/desktop/app/my-App_2' },
|
||||
}),
|
||||
res,
|
||||
);
|
||||
const state = new URL(captured.redirectUrl ?? '').searchParams.get(
|
||||
'state',
|
||||
);
|
||||
const decoded = oidc().verifyState(state!);
|
||||
expect(String(decoded?.redirect_uri)).toBe(
|
||||
`${TEST_ORIGIN}/desktop/app/my-App_2`,
|
||||
);
|
||||
});
|
||||
|
||||
it('ignores a non-whitelisted return_to', async () => {
|
||||
const bad_values = [
|
||||
'/app/evil/extra',
|
||||
'/app/',
|
||||
'/app/name?x=1',
|
||||
'//evil.test',
|
||||
'/settings',
|
||||
`/app/${'a'.repeat(101)}`,
|
||||
'/desktop/app/evil/extra',
|
||||
'/desktop/app/',
|
||||
'/dashboard/app/name',
|
||||
];
|
||||
for (const return_to of bad_values) {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/:provider/start',
|
||||
makeReq({
|
||||
params: { provider: 'custom' },
|
||||
query: { return_to },
|
||||
}),
|
||||
res,
|
||||
);
|
||||
const state = new URL(
|
||||
captured.redirectUrl ?? '',
|
||||
).searchParams.get('state');
|
||||
const decoded = oidc().verifyState(state!);
|
||||
expect(String(decoded?.redirect_uri)).toBe(TEST_ORIGIN);
|
||||
}
|
||||
});
|
||||
|
||||
it('signs revalidate-flow state with user_uuid + flow=revalidate', async () => {
|
||||
const userUuid = uuidv4();
|
||||
const { res, captured } = makeRes();
|
||||
@@ -693,76 +621,6 @@ describe('OIDCController login callback', () => {
|
||||
expect(captured.cookies).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('redirects back to an /app/<name> landing after sign-in', async () => {
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: `${TEST_ORIGIN}/app/some-app`,
|
||||
});
|
||||
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
|
||||
access_token: 'access',
|
||||
id_token: 'id',
|
||||
} as never);
|
||||
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
|
||||
sub: `sub-${Math.random().toString(36).slice(2, 8)}`,
|
||||
email: `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`,
|
||||
email_verified: true,
|
||||
} as never);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/login',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(captured.redirectUrl).toBe(`${TEST_ORIGIN}/app/some-app`);
|
||||
});
|
||||
|
||||
it('keeps an /app/<name> landing in the error redirect (suspended user)', async () => {
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const email = `sus-app-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
const created = await runWithContext(
|
||||
{ req: makeReq({}) },
|
||||
() =>
|
||||
oidc().createUserFromOIDC('custom', {
|
||||
sub,
|
||||
email,
|
||||
email_verified: true,
|
||||
}),
|
||||
);
|
||||
expect(created.success).toBe(true);
|
||||
await server.stores.user.update(created.user!.id, { suspended: 1 });
|
||||
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: `${TEST_ORIGIN}/app/some-app`,
|
||||
});
|
||||
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
|
||||
access_token: 'access',
|
||||
id_token: 'id',
|
||||
} as never);
|
||||
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
|
||||
sub,
|
||||
email,
|
||||
email_verified: true,
|
||||
} as never);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/login',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.redirectStatus).toBe(302);
|
||||
const url = new URL(captured.redirectUrl ?? '');
|
||||
expect(url.pathname).toBe('/app/some-app');
|
||||
expect(url.searchParams.get('auth_error')).toBe('1');
|
||||
expect(url.searchParams.get('action')).toBe('login');
|
||||
expect(captured.cookies).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('appends oidc_login=true and uses popup-style URL when state is from a popup flow', async () => {
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const email = `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
@@ -1045,39 +903,6 @@ describe('OIDCController signup veto (abuse harness)', () => {
|
||||
expect(captured.redirectUrl).toContain('message=signup_blocked');
|
||||
expect(captured.redirectUrl).not.toContain('request_code');
|
||||
});
|
||||
|
||||
it('keeps a veto legible when the listener stamps its own code', async () => {
|
||||
const email = `veto-${Math.random().toString(36).slice(2, 8)}@test.local`;
|
||||
signupValidateOverride = (data) => {
|
||||
if (data.email !== email) return;
|
||||
data.allow = false;
|
||||
data.code = 'email_reputation_too_low';
|
||||
data.trail_id = 'trail-custom-code';
|
||||
};
|
||||
stubIdp(`sub-${Math.random().toString(36).slice(2, 8)}`, email);
|
||||
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/signup',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
|
||||
// A listener code isn't one of the display codes, but the failure
|
||||
// is still a blocked signup — collapsing it to `unauthorized`
|
||||
// would tell the user sign-in broke and bury the Request Code
|
||||
// support needs.
|
||||
expect(captured.redirectUrl).toContain('message=signup_blocked');
|
||||
expect(captured.redirectUrl).not.toContain('message=unauthorized');
|
||||
expect(captured.redirectUrl).toContain(
|
||||
'request_code=trail-custom-code',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('OIDCController browser binding', () => {
|
||||
@@ -1651,98 +1476,3 @@ describe('OIDCController GET /auth/revalidate-done', () => {
|
||||
expect(body).toContain(JSON.stringify(TEST_ORIGIN));
|
||||
});
|
||||
});
|
||||
|
||||
// ── POST /auth/oidc/verify-popup-return ─────────────────────────────
|
||||
|
||||
/**
|
||||
* The proof exists because the popup return leg states two things the popup
|
||||
* cannot check — the opener's origin and that a login completed — and a URL
|
||||
* built from a verified `state` is byte-identical to one anybody can type.
|
||||
* The opener's origin picks the app a token is minted for, so it has to be
|
||||
* attested rather than read.
|
||||
*/
|
||||
describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
|
||||
const redeem = async (opener_state: unknown) => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'post',
|
||||
'/auth/oidc/verify-popup-return',
|
||||
makeReq({ body: { opener_state } }),
|
||||
res,
|
||||
);
|
||||
return captured;
|
||||
};
|
||||
|
||||
it('hands back what a genuine proof attests', async () => {
|
||||
const proof = server.services.oidc.signPopupReturn({
|
||||
opener_origin: 'https://opener.test',
|
||||
msg_id: '77',
|
||||
oidc_login: true,
|
||||
});
|
||||
const captured = await redeem(proof);
|
||||
expect(captured.body).toEqual({
|
||||
opener_origin: 'https://opener.test',
|
||||
msg_id: '77',
|
||||
oidc_login: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects a proof signed with someone else’s key', async () => {
|
||||
// The whole point: only the server can mint one of these.
|
||||
const forged = jwt.sign(
|
||||
{ opener_origin: 'https://console.puter.com', oidc_login: true },
|
||||
'not-the-server-secret',
|
||||
{ keyid: 'v2' },
|
||||
);
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/auth/oidc/verify-popup-return',
|
||||
makeReq({ body: { opener_state: forged } }),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('rejects an expired proof', async () => {
|
||||
// Comfortably past `TokenService`'s 30s clock tolerance — the proof is
|
||||
// redeemed on the very next request, so a stale one is never genuine.
|
||||
const stale = server.services.token.sign(
|
||||
'oidc-state',
|
||||
{ opener_origin: 'https://opener.test', oidc_login: true },
|
||||
{ expiresIn: -600 },
|
||||
);
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/auth/oidc/verify-popup-return',
|
||||
makeReq({ body: { opener_state: stale } }),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('rejects a missing or non-string proof', async () => {
|
||||
for (const bad of [undefined, null, '', 42, {}]) {
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/auth/oidc/verify-popup-return',
|
||||
makeReq({ body: { opener_state: bad } }),
|
||||
makeRes().res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
}
|
||||
});
|
||||
|
||||
it('reports oidc_login false when the proof does not claim a login', async () => {
|
||||
// The error leg mints one of these: a real return, but nothing was
|
||||
// signed in on it, so it must not suppress the account picker.
|
||||
const proof = server.services.oidc.signPopupReturn({
|
||||
opener_origin: 'https://opener.test',
|
||||
msg_id: '77',
|
||||
oidc_login: false,
|
||||
});
|
||||
expect((await redeem(proof)).body).toMatchObject({ oidc_login: false });
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
/*
|
||||
/**
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
@@ -47,35 +47,6 @@ const ALLOWED_ERRORS = [
|
||||
'signup_blocked',
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* Pick the display code for a failed user resolution.
|
||||
*
|
||||
* A `code` is only ever set when the signup-validate harness vetoed the signup,
|
||||
* but the code it stamps comes from an abuse listener and is not drawn from
|
||||
* {@link ALLOWED_ERRORS} — so clamping it directly turns every vetoed OIDC
|
||||
* signup into a bare `unauthorized`, which reads as "sign-in broke" and hides
|
||||
* both the real cause and the Request Code that support looks the decision up
|
||||
* by. Anything unrecognized falls back to the veto's own category instead.
|
||||
*/
|
||||
function resolutionErrorCode(code: string | undefined): string {
|
||||
if (!code) return 'unauthorized';
|
||||
return (ALLOWED_ERRORS as readonly string[]).includes(code)
|
||||
? code
|
||||
: 'signup_blocked';
|
||||
}
|
||||
|
||||
// GUI pages an OIDC flow may return to: /desktop, /dashboard, and direct app
|
||||
// landings (/app/<name> and its desktop-booted twin /desktop/app/<name>,
|
||||
// mirroring APP_NAME_REGEX in AppDriver). Strict whitelist — never a
|
||||
// client-supplied URL (no open redirect).
|
||||
function isWhitelistedReturnPath(path: string): boolean {
|
||||
return (
|
||||
path === '/desktop' ||
|
||||
path === '/dashboard' ||
|
||||
/^(\/desktop)?\/app\/[a-zA-Z0-9_-]{1,100}$/.test(path)
|
||||
);
|
||||
}
|
||||
|
||||
function buildErrorRedirectUrl(
|
||||
origin: string,
|
||||
sourceFlow: string,
|
||||
@@ -83,11 +54,6 @@ function buildErrorRedirectUrl(
|
||||
message: string,
|
||||
stateDecoded?: Record<string, unknown>,
|
||||
requestCode?: string,
|
||||
// Signs the popup-return proof. Passed in because this is a module-level
|
||||
// helper with no access to services; omitted by callers that have no
|
||||
// state to attest (the proof is simply absent then, and the popup falls
|
||||
// back to its browser-attested sources).
|
||||
signPopupReturn?: (payload: Record<string, unknown>) => string,
|
||||
): string {
|
||||
const targetFlow =
|
||||
OIDC_ERROR_REDIRECT_MAP[sourceFlow]?.[errorCondition] ?? sourceFlow;
|
||||
@@ -96,20 +62,6 @@ function buildErrorRedirectUrl(
|
||||
? message
|
||||
: 'unauthorized';
|
||||
|
||||
// Land back on the whitelisted page the flow started from (e.g. an
|
||||
// /app/<name> landing) so the retry — and the eventual success — keeps
|
||||
// the user's destination. redirect_uri comes from the signed state and
|
||||
// was built server-side, but re-check the path against the whitelist.
|
||||
let pagePath = '/';
|
||||
if (typeof stateDecoded?.redirect_uri === 'string') {
|
||||
try {
|
||||
const statePath = new URL(stateDecoded.redirect_uri).pathname;
|
||||
if (isWhitelistedReturnPath(statePath)) pagePath = statePath;
|
||||
} catch {
|
||||
// unparsable redirect_uri: fall back to the root page
|
||||
}
|
||||
}
|
||||
|
||||
let params: URLSearchParams;
|
||||
if (stateDecoded?.embedded_in_popup && stateDecoded?.msg_id != null) {
|
||||
params = new URLSearchParams({
|
||||
@@ -122,19 +74,6 @@ function buildErrorRedirectUrl(
|
||||
if (stateDecoded?.opener_origin) {
|
||||
params.set('opener_origin', String(stateDecoded.opener_origin));
|
||||
}
|
||||
// Same reasoning as the success leg: the popup cannot tell a verified
|
||||
// `opener_origin` from a typed one, so attest it. The error leg is a
|
||||
// real return from the provider too — the flow failed, not the hop.
|
||||
if (signPopupReturn) {
|
||||
params.set(
|
||||
'opener_state',
|
||||
signPopupReturn({
|
||||
opener_origin: stateDecoded?.opener_origin ?? null,
|
||||
msg_id: stateDecoded?.msg_id ?? null,
|
||||
oidc_login: false,
|
||||
}),
|
||||
);
|
||||
}
|
||||
} else {
|
||||
params = new URLSearchParams({
|
||||
action: targetFlow,
|
||||
@@ -145,7 +84,7 @@ function buildErrorRedirectUrl(
|
||||
if (requestCode) {
|
||||
params.set('request_code', requestCode);
|
||||
}
|
||||
return `${base}${pagePath}?${params.toString()}`;
|
||||
return `${base}/?${params.toString()}`;
|
||||
}
|
||||
|
||||
function appendQueryParam(url: string, key: string, value: string): string {
|
||||
@@ -162,8 +101,8 @@ function constantTimeEqual(a: string, b: string): boolean {
|
||||
}
|
||||
|
||||
/**
|
||||
* True iff `target` parses as a URL whose origin equals `origin`. Used to clamp
|
||||
* OIDC redirect targets — `startsWith` would accept
|
||||
* True iff `target` parses as a URL whose origin equals `origin`. Used to
|
||||
* clamp OIDC redirect targets — `startsWith` would accept
|
||||
* `https://puter.com.evil.com` against `https://puter.com`.
|
||||
*/
|
||||
function isSameOrigin(target: string, origin: string): boolean {
|
||||
@@ -181,51 +120,6 @@ function isSameOrigin(target: string, origin: string): boolean {
|
||||
*/
|
||||
export class OIDCController extends PuterController {
|
||||
registerRoutes(router: PuterRouter): void {
|
||||
// -- POST /auth/oidc/verify-popup-return ---------------------
|
||||
// Public — hand back the facts a popup-return proof attests to.
|
||||
//
|
||||
// A sign-in popup returning from a provider is told the opener's
|
||||
// origin and that a login completed. It cannot check either: the
|
||||
// values arrive as query parameters, and a URL built from a verified
|
||||
// `state` looks exactly like one an attacker typed. The opener's
|
||||
// origin decides which app a token gets minted for, so the popup
|
||||
// redeems the signed proof here instead of believing the raw
|
||||
// parameters.
|
||||
//
|
||||
// Unauthenticated on purpose — it reveals nothing the caller did not
|
||||
// already hand over, and a forged or expired proof yields nothing.
|
||||
|
||||
router.post(
|
||||
'/auth/oidc/verify-popup-return',
|
||||
{
|
||||
subdomain: 'api',
|
||||
rateLimit: {
|
||||
scope: 'oidc-verify-popup-return',
|
||||
limit: 60,
|
||||
window: 60_000,
|
||||
},
|
||||
},
|
||||
async (req: Request, res: Response) => {
|
||||
const proof = req.body?.opener_state;
|
||||
if (typeof proof !== 'string' || !proof) {
|
||||
throw new HttpError(400, 'Missing `opener_state`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const decoded = this.services.oidc.verifyPopupReturn(proof);
|
||||
if (!decoded) {
|
||||
throw new HttpError(400, 'Invalid `opener_state`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
res.json({
|
||||
opener_origin: decoded.opener_origin ?? null,
|
||||
msg_id: decoded.msg_id ?? null,
|
||||
oidc_login: decoded.oidc_login === true,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
// -- GET /auth/oidc/providers --------------------------------
|
||||
// Public — list enabled provider IDs for the frontend.
|
||||
|
||||
@@ -272,15 +166,15 @@ export class OIDCController extends PuterController {
|
||||
|
||||
let appRedirectUri = flowRedirects[flow] ?? (origin || '/');
|
||||
|
||||
// Optional GUI return path so login started from /desktop,
|
||||
// /dashboard, or an /app/<name> landing lands back there.
|
||||
// Optional GUI return path so login started from /desktop or
|
||||
// /dashboard lands back there. Strict whitelist — never a
|
||||
// client-supplied URL (no open redirect).
|
||||
const rawReturnTo = Array.isArray(req.query.return_to)
|
||||
? req.query.return_to[0]
|
||||
: req.query.return_to;
|
||||
if (
|
||||
(flow === 'login' || flow === 'signup') &&
|
||||
typeof rawReturnTo === 'string' &&
|
||||
isWhitelistedReturnPath(rawReturnTo)
|
||||
(rawReturnTo === '/desktop' || rawReturnTo === '/dashboard')
|
||||
) {
|
||||
appRedirectUri = `${origin}${rawReturnTo}`;
|
||||
}
|
||||
@@ -421,10 +315,9 @@ export class OIDCController extends PuterController {
|
||||
origin,
|
||||
'login',
|
||||
'other',
|
||||
resolutionErrorCode(resolved.code),
|
||||
resolved.code ?? 'unauthorized',
|
||||
stateDecoded,
|
||||
resolved.requestCode,
|
||||
(p) => this.services.oidc.signPopupReturn(p),
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -442,8 +335,6 @@ export class OIDCController extends PuterController {
|
||||
'other',
|
||||
'account_suspended',
|
||||
stateDecoded,
|
||||
undefined,
|
||||
(p) => this.services.oidc.signPopupReturn(p),
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -459,7 +350,6 @@ export class OIDCController extends PuterController {
|
||||
const origin = this.config.origin ?? '';
|
||||
const result = await this.#processCallback(req, res, 'signup');
|
||||
if ('error' in result) {
|
||||
console.warn(`OIDC signup callback error: ${result.error}`);
|
||||
return res.redirect(
|
||||
302,
|
||||
buildErrorRedirectUrl(
|
||||
@@ -488,10 +378,9 @@ export class OIDCController extends PuterController {
|
||||
origin,
|
||||
'signup',
|
||||
'other',
|
||||
resolutionErrorCode(resolved.code),
|
||||
resolved.code ?? 'unauthorized',
|
||||
stateDecoded,
|
||||
resolved.requestCode,
|
||||
(p) => this.services.oidc.signPopupReturn(p),
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -506,8 +395,6 @@ export class OIDCController extends PuterController {
|
||||
'other',
|
||||
'account_suspended',
|
||||
stateDecoded,
|
||||
undefined,
|
||||
(p) => this.services.oidc.signPopupReturn(p),
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -610,14 +497,13 @@ if (window.opener) {
|
||||
|
||||
/**
|
||||
* Resolve an OIDC callback to a Puter user. In order:
|
||||
*
|
||||
* 1. Existing link on (provider, sub) → that user.
|
||||
* 2. Email matches an existing account whose email is CONFIRMED → link
|
||||
* (provider, sub) to that user.
|
||||
* 3. Email matches an account whose email is UNCONFIRMED → refuse. We don't
|
||||
* know who owns an unconfirmed address, so linking would let whoever
|
||||
* controls the OIDC identity hijack a pending signup.
|
||||
* 4. Otherwise create a new user and link.
|
||||
* 1. Existing link on (provider, sub) → that user.
|
||||
* 2. Email matches an existing account whose email is CONFIRMED →
|
||||
* link (provider, sub) to that user.
|
||||
* 3. Email matches an account whose email is UNCONFIRMED → refuse.
|
||||
* We don't know who owns an unconfirmed address, so linking would
|
||||
* let whoever controls the OIDC identity hijack a pending signup.
|
||||
* 4. Otherwise create a new user and link.
|
||||
*
|
||||
* Step 2 also requires `email_verified !== false` on the OIDC side,
|
||||
* otherwise a malicious IdP could claim someone else's email.
|
||||
@@ -787,22 +673,6 @@ if (window.opener) {
|
||||
|
||||
if (stateDecoded.embedded_in_popup) {
|
||||
target = appendQueryParam(target, 'oidc_login', 'true');
|
||||
// `opener_origin` and `oidc_login` reach the popup as bare query
|
||||
// parameters, which say nothing about where they came from: the
|
||||
// URL a verified state produces is byte-identical to one anybody
|
||||
// can type. The popup treats the opener's origin as the app
|
||||
// identity to mint a token for, so it needs the integrity this
|
||||
// state already carries — re-signed here, at the one point where
|
||||
// the round trip is known to have actually happened.
|
||||
target = appendQueryParam(
|
||||
target,
|
||||
'opener_state',
|
||||
this.services.oidc.signPopupReturn({
|
||||
opener_origin: stateDecoded.opener_origin ?? null,
|
||||
msg_id: stateDecoded.msg_id ?? null,
|
||||
oidc_login: true,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
if (extraQueryParams) {
|
||||
|
||||
Loaded 100 of 788 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user