Compare commits

..
35 Commits
Author SHA1 Message Date
Sourabh Sharma 15134c99bf Fix mkdir dedupeName for existing directories (#3215) 2026-06-07 12:49:00 -07:00
Daniel Salazar 8262e3193e perf: don't batch get app stats (#3219) 2026-06-06 18:28:59 -07:00
Reynaldi Chernando 98a21c4ac0 More workers docs improvement (#3218)
* More workers docs improvement

* Add mention of workers alongside mention of user scope
2026-06-06 01:27:24 +07:00
ProgrammerIn-wonderland c52eb1b903 Ns/bug fixes 060426 (#3214)
* Harden backend auth

* Puter.js auth hardening
2026-06-05 13:55:23 -04:00
mohit_parmar_007 d6a71e50bb fix: remove unnecessary cache console.log statements (#3216) 2026-06-05 10:01:59 -07:00
Reynaldi Chernando a4b6ee0dd2 [PUT-1090] Improve worker docs (#3217)
* Improve worker docs

* refine
2026-06-05 21:54:56 +07:00
Daniel Salazar bb61103080 feat: add apps api into mcp (#3213) 2026-06-04 17:19:53 -07:00
Daniel Salazar 9bbc76b2dd drop metering alerts to low severity (#3212) 2026-06-04 14:01:44 -07:00
Daniel Salazar 918aba9c80 fix: reduce alarms further (#3211)
* feat; update bug-bounty

* fix: reduce alarms further
2026-06-04 10:43:50 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 3ceba9e3b3 Minor fix deployment docs (#3210)
* Minor fix deployment docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-04 23:48:25 +07:00
Daniel Salazar a480679a2d vis: make metering service alarm only go off on actual abuse (#3208) 2026-06-03 22:45:18 -07:00
Daniel Salazar e99ae40e93 chore: bump puter-js (#3207) 2026-06-03 18:59:07 -07:00
Daniel Salazar 3cd50fbaf3 move the token migration to root (#3206) 2026-06-03 18:54:22 -07:00
ProgrammerIn-wonderland 60dd1453d4 Fix issues (#3205) 2026-06-03 21:19:33 -04:00
ProgrammerIn-wonderland 07781cd681 allow overriding origin from workers.dev on mcp server (#3204) 2026-06-03 17:48:37 -04:00
ProgrammerIn-wonderland 2387d8dcc9 MCP whoami (#3203) 2026-06-03 17:13:13 -04:00
Reynaldi Chernando 723608f4db Add new puter cli (#3199)
* Add new cli

* update package lock

* fix readme and license
2026-06-04 01:51:31 +07:00
Reynaldi Chernando bd1dcde7b0 Add info for puterjs deployments (#3202)
* Add info for puterjs deployments

* update external links

* hide github actions method
2026-06-04 01:25:57 +07:00
ProgrammerIn-wonderland 5b8e41e66d MCP Server (#3197)
* beginnings of mcp

* oauth support

* add workers and better prompt engineering
2026-06-03 12:23:06 -04:00
Nariman Jelveh acc4796b99 Allow Cache-Control and Pragma headers (#3198) 2026-06-03 14:30:33 +02:00
jelveh 106e85d6ca Force revalidation on FileSystem reads 2026-06-03 14:15:22 +02:00
ProgrammerIn-wonderland 0ffc32f19e add whitelist for signed batch uploads (#3196) 2026-06-02 21:30:34 -04:00
Daniel Salazar d897cc5dd0 refactor: move subscriptions to webhook system (#3193)
* fix: dont sign write urls if they don't have write perm

* wip: new subscriptions

* refactor: move subscriptions to
2026-06-02 17:58:56 -07:00
Daniel Salazar 6d2f277ce2 fix: some error handling (#3190) 2026-05-31 18:58:27 -07:00
ProgrammerIn-wonderland 86f80b29c0 tighten RAO policy (#3188) 2026-05-29 20:52:21 -04:00
Daniel SalazarandProgrammerIn-wonderland 39243735e8 feat (put-1049): worker types (#3185)
* feat (put-1049): worket types

* Implement changes to worker types

---------

Co-authored-by: ProgrammerIn-wonderland <[email protected]>
2026-05-29 20:51:57 -04:00
ProgrammerIn-wonderland b3ff9d74b7 add browserjs to taskbar and RecommendedAppService (#3189) 2026-05-29 20:32:53 -04:00
ProgrammerIn-wonderland 7b7604440f Add serverless only filtering for togetherai chat models (#3187) 2026-05-29 14:33:55 -04:00
ProgrammerIn-wonderland 382d2b3c14 add minimax provider (#3171)
* add minimax provider
2026-05-29 13:36:10 -04:00
Anurag Pappula eda5379c17 fix(docs/search): handle search index loading and failure states (#3184)
* fix(docs/search): show loading state and replay query when index isn't ready

Previously performSearch rendered "No results found" while the
MiniSearch index was still loading, which is misleading on slow
networks. Track the last query as pendingQuery and replay it from
fetchSearchIndex once the index is ready; render a distinct
"Failed to load search index" message on fetch error so the
loading message doesn't persist indefinitely.

Fixes #3180.

* fix(docs/search): close stale-replay race and consolidate status messages

Drop pendingQuery synchronously in the input handler when the query
falls below the minimum length, so a fetchSearchIndex that resolves
within the 300ms debounce window after the user clears the field can
no longer replay the cleared query into an empty input.

Move the idle, loading, and failed copy into a single
renderSearchStatus(kind) helper so the four .search-no-results
renderings can only drift in one place.
2026-05-29 16:00:59 +07:00
Ron HernausandDaniel Salazar d2fee51844 Support PostgreSQL database backend (#3167)
* feat(database): add postgres database client

Adds the PostgreSQL database client, native bootstrap migration, SQL preparation helpers, and database config/factory wiring.\n\nRefs #3165.

* feat(database): make backend queries postgres-aware

Updates runtime SQL call sites for database-specific booleans, identifiers, insert-ignore, upserts, JSON extraction, intervals, and Postgres insert ids.\n\nRefs #3165.

* test(database): cover postgres client behavior

Adds unit coverage for SQL preparation, factory selection, write-result mapping, and transaction rollback/commit ordering, plus an env-gated PostgreSQL integration flow.\n\nRefs #3165.

* docs(self-hosting): document postgres database setup

Adds PostgreSQL configuration examples and migration path guidance for self-hosted deployments.\n\nRefs #3165.

* fix: harden postgres oidc tests

* fix(postgres): normalize query results and SQL prep

* fix(user): preserve normalized cache booleans

* test(postgres): run integration coverage with pgmock

* tests: add way to run all tests with postgres though slow

Also adding note that postgres is not in active use so might not work out the box

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-05-28 14:12:17 -07:00
Daniel Salazar 8ac4bad196 fix: readUrl perm check (#3183) 2026-05-28 11:44:16 -07:00
Reynaldi Chernando f140d7221c Add opus 4.8 (#3182)
* Add opus 4.8

* update alias

* handling opus 48 omit temperature
2026-05-28 13:27:46 -04:00
Daniel Salazar 631f0bda1b chore:deploy latest puter.js (#3181) 2026-05-28 09:58:33 -07:00
Reynaldi Chernando 1121314582 Docs search improvement (#3179)
* Init search improvement

* Add fuzzy implementation

* update package lock

* package lock matched with runner 24.16.0
2026-05-28 23:11:34 +07:00
134 changed files with 11946 additions and 3642 deletions

No files matched your search

+2
View File
@@ -70,3 +70,5 @@ coverage/
undefined
servers.json
config.*.json
volatile/
+34 -1
View File
@@ -17,18 +17,51 @@ Out-of-scope:
* Denial of Service (DoS), spam, or volumetric attacks.
* Physical security issues.
## Known Non-Issues (Please Check Before Submitting)
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
* **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we *do* evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
* **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
* **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
* **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
* **App metadata or app user-count "leaks".** This information is currently public by design.
* **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
* **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
* **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
* **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
* **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
* **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
* **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
## Rules of Engagement
To participate, you must:
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code or screenshots where applicable.
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see *Proof of Reproduction* below).
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
5. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
6. **Check the Known Non-Issues list**: Reports matching an item in the "Known Non-Issues" section above are not eligible and will be closed.
Reports that do not meet these guidelines may not be eligible for a reward.
## Proof of Reproduction
Reports must demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
* Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
* The **observed** result versus the **expected** result.
* For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
* For server-side bugs: a runnable proof-of-concept.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and are generally not eligible**. If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Please submit **one issue per report**. Bundled "audit packs" of many speculative findings will be declined; send each confirmed issue separately.
## Reporting Process
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
+22 -4
View File
@@ -159,17 +159,30 @@
// ── Database ────────────────────────────────────────────────────────
"database": {
// `sqlite` for single-node / dev; `mysql` for self-host with MariaDB.
// `sqlite` for single-node/dev, `mysql` for MariaDB/MySQL,
// or `postgres` for PostgreSQL.
//
// NOTE: `postgres` support is a community contribution and is not
// exercised by Puter.com production. It boots, passes its own
// integration tests against pgmock, and runs the common user/app/
// fsentry/session/permission/OIDC flows — but less-traveled code
// paths may surface MySQL/SQLite-isms that haven't been ported yet.
// Expect rough edges and please file issues if you hit one. For
// production self-hosting today, `mysql` (MariaDB) and `sqlite` are
// the supported defaults.
"engine": "sqlite",
// sqlite — file path on disk
"path": "volatile/runtime/puter-database.sqlite",
"targetVersion": 0,
// mysql — connection details
// mysql/postgres — connection details. PostgreSQL defaults to port
// 5432 when `engine` is `postgres`; MySQL/MariaDB normally use 3306.
"host": "",
"port": 3306,
"user": "",
"password": "",
"database": "",
// postgres may also use a URL instead of host/user/password fields:
// "connectionString": "postgres://puter:secret@localhost:5432/puter",
// Optional read-replica pool. Reads route here when populated.
"replica": {
"host": "",
@@ -178,9 +191,10 @@
"password": "",
"database": ""
}
// mysql self-host bootstrap: set `migrationPaths` to apply the bundled
// schema on first boot. Idempotent — safe to leave on.
// mysql/postgres self-host bootstrap: set `migrationPaths` to apply the
// bundled schema on first boot. Idempotent — safe to leave on.
// "migrationPaths": ["./src/backend/clients/database/migrations/mysql"]
// "migrationPaths": ["./src/backend/clients/database/migrations/postgres"]
},
// ── DynamoDB (KV store) ─────────────────────────────────────────────
@@ -252,6 +266,10 @@
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
},
"openrouter": {
"apiKey": "",
"apiBaseUrl": "https://openrouter.ai/api/v1"
+21 -1
View File
@@ -136,7 +136,7 @@ Why these knobs:
- `jwt_secret` + `jwt_secret_v2` — Puter signs new auth tokens with `jwt_secret_v2` (v2 token format, `kid: 'v2'` JWT header). `jwt_secret` is verify-only and lets tokens minted before this version (cookies still in users' browsers, stored API tokens) keep working. `allow_v1_tokens: true` keeps that fallback enabled. Fresh installs can leave it as-is; future versions will flip the flag to `false` and retire `jwt_secret` entirely once legacy tokens have drained.
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start=gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL schema on boot. `mysql_mig_1.sql` (tables) and `mysql_mig_2.sql` (default apps: editor, viewer, pdf, camera, player, recorder, git, dev-center, puter-linux). Idempotent — safe to re-run.
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
@@ -232,6 +232,26 @@ Change it in Settings after first login.
All optional. Drop any of the blocks below into `puter/config/config.json` and `docker compose restart puter`. See [config.template.jsonc](../config.template.jsonc) for the full list. Per-key documentation lives in [src/backend/types.ts](../src/backend/types.ts).
### PostgreSQL database
> **Community contribution — expect rough edges.** PostgreSQL support was contributed by the community and is not run by Puter.com production. It boots, applies the bundled schema, and exercises the common user/app/fsentry/session/permission/OIDC flows in the integration tests, but less-traveled SQL call sites may still need porting. If you hit a query that doesn't work on Postgres, please open an issue. For production self-hosting today, MariaDB/MySQL and SQLite are the supported defaults.
The bundled Docker Compose stack still defaults to MariaDB. To use PostgreSQL instead, run a PostgreSQL service yourself, point Puter at it, and use the PostgreSQL migration path:
```json
"database": {
"engine": "postgres",
"host": "postgres",
"port": 5432,
"user": "puter",
"password": "...",
"database": "puter",
"migrationPaths": ["/opt/puter/dist/src/backend/clients/database/migrations/postgres"]
}
```
You may use `"connectionString": "postgres://puter:...@postgres:5432/puter"` instead of the host/user/password fields. Puter only needs normal PostgreSQL connection details and the migration path; CloudNativePG is one possible Kubernetes operator, but no operator-specific manifests are required by Puter.
### Email (SMTP)
Used for password resets, email confirmation, and notifications. Without it those flows silently fail.
+1 -1
View File
@@ -84,7 +84,7 @@ export const handleAppTelemetryUsers = async (
const users = await clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
INNER JOIN user u ON p.user_id = u.id
INNER JOIN ${clients.db.quoteIdentifier('user')} u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
+1841 -2602
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -45,6 +45,7 @@
},
"scripts": {
"test:backend": "vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run build:ts",
@@ -0,0 +1,50 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Optional ClickHouse client.
*
* ClickHouse is NOT a hard dependency. Core Puter runs without it — the app
* stats read path (open / unique-user counts) falls back to the primary SQL
* database. `this.clients.clickhouse` is therefore typed as optional and is
* `undefined` in default and self-hosted setups; callers MUST branch on its
* presence and fall back to SQL.
*
* A production deployment can register a real ClickHouse client via an
* extension (`extension.registerClient('clickhouse', client)`) to offload the
* analytics queries off the primary database and keep the stats path fast at
* scale. When registered, the instance flows into `this.clients.clickhouse`
* everywhere it's typed.
*
* Only the surface the stats path actually consumes is declared here. Extend
* this interface (don't widen to `any`) when a new query shape is needed.
*/
export interface ClickhouseQueryResult {
json<T = Record<string, unknown>>(): Promise<T[]>;
}
export interface ClickhouseQueryParams {
query: string;
query_params?: Record<string, unknown>;
format?: string;
}
export interface ClickhouseClient {
query(params: ClickhouseQueryParams): Promise<ClickhouseQueryResult>;
}
+90 -3
View File
@@ -31,6 +31,8 @@ export interface BatchEntry {
values: unknown[];
}
type SqlJsonPath = readonly [string, ...string[]];
/**
* Base database client. Subclasses must override every method that throws here.
*
@@ -101,9 +103,10 @@ export class AbstractDatabaseClient extends PuterClient {
const cols = Object.keys(data);
const values = Object.values(data);
const sql =
`INSERT INTO \`${tableName}\` ` +
`(${cols.map((c) => `\`${c}\``).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`;
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
}
@@ -161,4 +164,88 @@ export class AbstractDatabaseClient extends PuterClient {
}
return choices.otherwise as T;
}
quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `\`${part.replaceAll('`', '``')}\``;
})
.join('.');
}
booleanLiteral(value: boolean): string {
return value ? '1' : '0';
}
booleanValue(value: boolean): boolean | 0 | 1 {
return value ? 1 : 0;
}
insertIgnoreInto(tableName: string): string {
const table = this.quoteIdentifier(tableName);
return this.case({
sqlite: `INSERT OR IGNORE INTO ${table}`,
postgres: `INSERT INTO ${table}`,
otherwise: `INSERT IGNORE INTO ${table}`,
});
}
insertIgnoreSuffix(): string {
return this.case({
postgres: ' ON CONFLICT DO NOTHING',
otherwise: '',
});
}
upsertClause(
conflictColumns: readonly string[],
updateColumns: readonly string[],
): string {
if (updateColumns.length === 0) {
throw new Error('upsertClause requires at least one update column');
}
const updateList = updateColumns
.map((column) => `${this.quoteIdentifier(column)} = ?`)
.join(', ');
return this.case({
mysql: `ON DUPLICATE KEY UPDATE ${updateList}`,
otherwise: `ON CONFLICT(${conflictColumns
.map((column) => this.quoteIdentifier(column))
.join(', ')}) DO UPDATE SET ${updateList}`,
});
}
jsonTextExtract(jsonExpression: string, path: SqlJsonPath): string {
const sqlitePath = `$${path.map((part) => `.${part}`).join('')}`;
return this.case({
sqlite: `json_extract(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)})`,
mysql: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
postgres: `${jsonExpression} #>> ARRAY[${path
.map((part) => this.sqlStringLiteral(part))
.join(', ')}]`,
otherwise: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
});
}
nullCoalesce(...expressions: readonly string[]): string {
if (expressions.length === 0) {
throw new Error('nullCoalesce requires at least one expression');
}
return `COALESCE(${expressions.join(', ')})`;
}
returningIdClause(): string {
return this.case({
postgres: ' RETURNING id',
otherwise: '',
});
}
protected sqlStringLiteral(value: string): string {
return `'${value.replaceAll("'", "''")}'`;
}
}
@@ -23,6 +23,7 @@ import { createPool, type Pool } from 'mysql2';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { SQLBatcher } from './SQLBatcher.js';
import { splitMysqlStatements } from './splitMysqlStatements.js';
import { compareMigrationFilenames } from './migrationFilenames.js';
import type { IConfig } from '../../types';
const RETRIABLE_ERROR_CODES = new Set([
@@ -44,36 +45,7 @@ const RETRIABLE_ERROR_MESSAGES = [
'ETIMEDOUT',
];
/**
* Comparator for MySQL migration filenames.
*
* Existing files are named `mysql_mig_<N>.sql` with unpadded N, so a
* plain lexical sort puts `mysql_mig_10.sql` BEFORE `mysql_mig_2.sql`.
* Pull the trailing integer out and sort numerically. Anything that
* doesn't match the `_<digits>.sql` shape (one-off, vendor dump) falls
* back to lexical comparison so the order stays deterministic, and
* unmatched names sort *after* numbered files so future numbered
* migrations don't get interleaved into a one-off's namespace.
*
* Exported only for the unit test — the production caller is the
* `runMigrations()` loop inside this file.
*/
export const compareMigrationFilenames = (a: string, b: string): number => {
const numericIndex = (name: string): number => {
const m = /_(\d+)\.sql$/.exec(name);
return m ? Number.parseInt(m[1], 10) : Number.NaN;
};
const na = numericIndex(a);
const nb = numericIndex(b);
if (Number.isFinite(na) && Number.isFinite(nb)) {
if (na !== nb) return na - nb;
} else if (Number.isFinite(na)) {
return -1;
} else if (Number.isFinite(nb)) {
return 1;
}
return a.localeCompare(b);
};
export { compareMigrationFilenames };
type PoolConfig = Parameters<typeof createPool>[0];
@@ -0,0 +1,328 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Pool } from 'pg';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { IConfig } from '../../types';
import type { PuterServer } from '../../server';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import {
createPgMockPostgresDatabaseClient,
POSTGRES_TEST_MIGRATIONS_PATH,
setupTestServer,
} from '../../testUtil.js';
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
const postgresUrl = process.env.PUTER_TEST_POSTGRES_URL;
const postgresMigrationsPath = POSTGRES_TEST_MIGRATIONS_PATH;
const postgresTestSchemaPattern = /^puter_test_[a-f0-9]{32}$/u;
const postgresIntegrationTimeoutMs = 180_000;
let postgresTestSchema: string | undefined;
let postgresTestUrl: string | undefined;
const postgresConfig = (overrides: Partial<IConfig> = {}): IConfig => {
if (postgresUrl && !postgresTestUrl) {
throw new Error('Postgres test schema was not initialized');
}
const { database: databaseOverrides, ...rootOverrides } = overrides;
const database = postgresUrl
? {
engine: 'postgres' as const,
inMemory: false,
connectionString: postgresTestUrl,
migrationPaths: [postgresMigrationsPath],
}
: {
engine: 'postgres' as const,
inMemory: true,
migrationPaths: [postgresMigrationsPath],
};
return {
port: 0,
extensions: [],
database: { ...database, ...(databaseOverrides ?? {}) },
...rootOverrides,
};
};
const quoteTestSchemaIdentifier = (schema: string): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
return `"${schema}"`;
};
const postgresConnectionStringForSchema = (
connectionString: string,
schema: string,
): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
const url = new URL(connectionString);
url.searchParams.set('options', `-c search_path=${schema}`);
return url.toString();
};
const createPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl) return;
const schema = `puter_test_${uuidv4().replaceAll('-', '')}`;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(`CREATE SCHEMA ${quoteTestSchemaIdentifier(schema)}`);
postgresTestSchema = schema;
postgresTestUrl = postgresConnectionStringForSchema(
postgresUrl,
schema,
);
} finally {
await pool.end();
}
};
const dropPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl || !postgresTestSchema) return;
const schema = postgresTestSchema;
postgresTestSchema = undefined;
postgresTestUrl = undefined;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(
`DROP SCHEMA IF EXISTS ${quoteTestSchemaIdentifier(schema)} CASCADE`,
);
} finally {
await pool.end();
}
};
describe('PostgresDatabaseClient integration', () => {
let server: PuterServer | undefined;
beforeEach(async () => {
await createPostgresTestSchema();
});
afterEach(async () => {
try {
await server?.shutdown();
} finally {
server = undefined;
await dropPostgresTestSchema();
}
});
it(
'applies the native migrations idempotently to an empty database',
async () => {
const config = postgresConfig();
const pgMockClient = postgresUrl
? undefined
: await createPgMockPostgresDatabaseClient(config);
try {
const firstClient =
pgMockClient?.client ?? new PostgresDatabaseClient(config);
try {
await firstClient.onServerStart();
} finally {
await firstClient.onServerShutdown();
}
const secondClient =
pgMockClient?.createClient() ??
new PostgresDatabaseClient(config);
try {
await secondClient.onServerStart();
const [systemUser] = await secondClient.read(
'SELECT `id`, `username` FROM `user` WHERE `username` = ?',
['system'],
);
const [devCenter] = await secondClient.read(
'SELECT `name`, `index_url` FROM `apps` WHERE `name` = ?',
['dev-center'],
);
expect(systemUser).toMatchObject({
id: 1,
username: 'system',
});
expect(devCenter).toMatchObject({
name: 'dev-center',
index_url:
'https://builtins.namespaces.puter.com/dev-center',
});
} finally {
await secondClient.onServerShutdown();
}
} finally {
pgMockClient?.destroy();
}
},
postgresIntegrationTimeoutMs,
);
it('starts the server and exercises user, app, fsentry, permission, and session flows', async () => {
const userStorageAllowance = 123_456_789;
server = await setupTestServer(
postgresConfig({
no_default_user: false,
is_storage_limited: true,
}),
);
const admin = await server.stores.user.getByUsername('admin');
expect(admin?.username).toBe('admin');
const username = `pg-${uuidv4().slice(0, 8)}`;
const createdUser = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: userStorageAllowance,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
createdUser,
);
const user = await server.stores.user.getById(createdUser.id);
if (!user) throw new Error('created user was not readable');
expect(user.username).toBe(username);
await expect(
server.stores.fsEntry.getUserStorageAllowance(user.id),
).resolves.toMatchObject({
max: userStorageAllowance,
});
const otherUsername = `pg-other-${uuidv4().slice(0, 8)}`;
const otherUser = await server.stores.user.create({
username: otherUsername,
uuid: uuidv4(),
password: null,
email: `${otherUsername}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const authResult = await server.services.auth.createSessionToken(
user,
{
ip: '127.0.0.1',
user_agent: 'postgres-integration-test',
},
);
const authenticated =
await server.services.auth.authenticateFromToken(authResult.token);
expect(authenticated?.user?.id).toBe(user?.id);
const devCenter = await server.stores.app.getByName('dev-center');
if (!devCenter) throw new Error('dev-center app was not seeded');
expect(devCenter.name).toBe('dev-center');
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
if (!documents) throw new Error('Documents directory was not created');
expect(documents.isDir).toBe(true);
const folder = await server.stores.fsEntry.createNonFileEntry({
userId: user.id,
parent: documents,
name: 'postgres-folder',
kind: 'directory',
});
const renamedFolder = await server.stores.fsEntry.updateEntry(
folder.uuid,
{
name: 'postgres-folder-renamed',
path: `/${username}/Documents/postgres-folder-renamed`,
},
);
expect(renamedFolder.name).toBe('postgres-folder-renamed');
await server.stores.permission.upsertUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
{ ok: true },
);
await expect(
server.stores.permission.hasUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
),
).resolves.toBe(true);
await server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
);
await expect(
server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
),
).resolves.toBeUndefined();
await expect(
server.stores.oidc.link(
otherUser.id,
'postgres-integration-test',
'subject-1',
null,
),
).rejects.toMatchObject({
statusCode: 409,
legacyCode: 'conflict',
});
const session = await server.stores.session.create(user.id, {
meta: { source: 'postgres-integration-test' },
});
const activeSession = await server.stores.session.getByUuid(
session.uuid,
);
expect(activeSession?.uuid).toBe(session.uuid);
const workerName = `worker-${uuidv4().slice(0, 8)}`;
const workerSession = await server.stores.session.getOrCreateWorker(
user.id,
{ workerName },
);
expect(workerSession?.kind).toBe('worker');
expect(workerSession?.meta?.worker_name).toBe(workerName);
await server.stores.session.removeByUuid(session.uuid);
await expect(
server.stores.session.getByUuid(session.uuid),
).resolves.toBeNull();
}, postgresIntegrationTimeoutMs);
});
@@ -0,0 +1,360 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { FieldDef, QueryResult } from 'pg';
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types';
import { DatabaseClientFactory } from './index.js';
import {
mapPostgresWriteResult,
PostgresDatabaseClient,
type PostgresPool,
type PostgresPoolClient,
} from './PostgresDatabaseClient.js';
type QueryCall = {
text: string;
values?: unknown[];
};
const postgresConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
},
});
const postgresReplicaConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
replica: {},
},
});
const field = (name: string, dataTypeID: number): FieldDef => ({
name,
tableID: 0,
columnID: 0,
dataTypeID,
dataTypeSize: -1,
dataTypeModifier: -1,
format: 'text',
});
const int8Field = (name: string): FieldDef => field(name, 20);
const textField = (name: string): FieldDef => field(name, 25);
const queryResult = (
rows: Record<string, unknown>[] = [],
rowCount = rows.length,
fields: FieldDef[] = [],
): QueryResult<Record<string, unknown>> => ({
command: '',
fields,
oid: 0,
rowCount,
rows,
});
class RecordingPoolClient implements PostgresPoolClient {
readonly calls: QueryCall[] = [];
released = false;
constructor(private readonly failOnText?: string) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (this.failOnText && text.includes(this.failOnText)) {
throw new Error(`forced query failure: ${text}`);
}
return queryResult([], text === 'ROLLBACK' ? 0 : 1);
}
release(): void {
this.released = true;
}
}
class RecordingPool implements PostgresPool {
readonly calls: QueryCall[] = [];
constructor(
private readonly client: RecordingPoolClient =
new RecordingPoolClient(),
private readonly nextResult: QueryResult<Record<string, unknown>> =
queryResult(),
) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (text === 'SELECT 1') return queryResult([{ ok: 1 }], 1);
return this.nextResult;
}
async connect(): Promise<PostgresPoolClient> {
return this.client;
}
async end(): Promise<void> {}
}
describe('PostgresDatabaseClient', () => {
it('is selected by the database factory', () => {
const client = new DatabaseClientFactory(postgresConfig());
expect(client).toBeInstanceOf(PostgresDatabaseClient);
});
it('maps pg write results to the shared WriteResult shape', () => {
expect(
mapPostgresWriteResult(queryResult([{ id: '42' }], 1)),
).toEqual({
insertId: 42,
affectedRows: 1,
anyRowsAffected: true,
});
expect(mapPostgresWriteResult(queryResult([], 0))).toEqual({
insertId: 0,
affectedRows: 0,
anyRowsAffected: false,
});
});
it('prepares write SQL at the pg boundary', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ id: '7' }], 1),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
const result = await client.write(
'INSERT INTO `apps` (`name`) VALUES (?) RETURNING id',
['editor'],
);
expect(result.insertId).toBe(7);
expect(pool.calls.at(-1)).toEqual({
text: 'INSERT INTO "apps" ("name") VALUES ($1) RETURNING id',
values: ['editor'],
});
});
it('normalizes int8 fields on read and primary read rows', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[
{
uuid: 'session-1',
created_at: '1710000000',
last_activity: '1710000001',
expires_at: '1710000002',
revoked_at: null,
},
],
1,
[
textField('uuid'),
int8Field('created_at'),
int8Field('last_activity'),
int8Field('expires_at'),
int8Field('revoked_at'),
],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
await expect(client.pread('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
});
it('rejects unsafe int8 values instead of losing precision', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[{ id: '9007199254740992' }],
1,
[int8Field('id')],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT `id` FROM `sessions`')).rejects.toThrow(
'safe integer',
);
});
it('normalizes tryHardRead rows returned by a replica', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '1' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '2' }], 1, [
int8Field('created_at'),
]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 2 }]);
});
it('normalizes tryHardRead rows returned by primary fallback', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '3' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([], 0, [int8Field('created_at')]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 3 }]);
});
it('runs batch writes in order and commits', async () => {
const conn = new RecordingPoolClient();
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
{
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
values: ['session-1'],
},
]);
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{
text: 'DELETE FROM "sessions" WHERE "uuid" = $1',
values: ['session-1'],
},
{ text: 'COMMIT', values: undefined },
]);
expect(conn.released).toBe(true);
});
it('rolls back and releases the pg connection when a batch write fails', async () => {
const conn = new RecordingPoolClient('UPDATE "user"');
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await expect(
client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]),
).rejects.toThrow('forced query failure');
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{ text: 'ROLLBACK', values: undefined },
]);
expect(conn.released).toBe(true);
});
});
@@ -0,0 +1,390 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { Pool, type PoolConfig, type QueryResult } from 'pg';
import {
AbstractDatabaseClient,
type BatchEntry,
type WriteResult,
} from './DatabaseClient';
import { compareMigrationFilenames } from './migrationFilenames.js';
import { preparePostgresSql } from './preparePostgresSql.js';
import { splitPostgresStatements } from './splitPostgresStatements.js';
import type { IConfig } from '../../types';
type PostgresEndpointConfig = {
host?: string;
port?: number;
user?: string;
password?: string;
database?: string;
connectionString?: string;
url?: string;
};
export interface PostgresQueryable {
query(query: string, values?: unknown[]): Promise<QueryResult>;
}
export interface PostgresPoolClient extends PostgresQueryable {
release(): void;
}
export interface PostgresPool extends PostgresQueryable {
connect(): Promise<PostgresPoolClient>;
end(): Promise<void>;
}
type PostgresPoolFactory = (poolConfig: PoolConfig) => PostgresPool;
enum Configuration {
SINGLE,
REPLICA,
}
const POSTGRES_INT8_OID = 20;
const INTEGER_TEXT_PATTERN = /^-?\d+$/u;
const normalizePostgresInt8 = (
value: unknown,
columnName: string,
): number | null | undefined => {
if (value === null || value === undefined) return value;
const parsed =
typeof value === 'bigint'
? Number(value)
: typeof value === 'number'
? value
: typeof value === 'string' && INTEGER_TEXT_PATTERN.test(value)
? Number(value)
: Number.NaN;
if (!Number.isSafeInteger(parsed)) {
throw new Error(
`[postgres] int8 column ${columnName} is outside JavaScript's safe integer range`,
);
}
return parsed;
};
const normalizePostgresRows = (
result: QueryResult,
): Record<string, unknown>[] => {
const int8Fields = result.fields.filter(
(field) => field.dataTypeID === POSTGRES_INT8_OID,
);
if (int8Fields.length === 0) {
return result.rows as Record<string, unknown>[];
}
return result.rows.map((row) => {
const normalized: Record<string, unknown> = { ...row };
for (const field of int8Fields) {
normalized[field.name] = normalizePostgresInt8(
normalized[field.name],
field.name,
);
}
return normalized;
});
};
export const mapPostgresWriteResult = (result: QueryResult): WriteResult => {
const affectedRows = result.rowCount ?? 0;
const rowId = result.rows[0]?.id;
const insertId =
typeof rowId === 'bigint'
? rowId
: typeof rowId === 'number'
? rowId
: typeof rowId === 'string' && rowId !== ''
? Number(rowId)
: 0;
const normalizedInsertId =
typeof insertId === 'number' && Number.isNaN(insertId) ? 0 : insertId;
return {
insertId: normalizedInsertId,
affectedRows,
anyRowsAffected: affectedRows > 0,
};
};
export class PostgresDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'postgres';
private primaryPool!: PostgresPool;
private replicaPool!: PostgresPool;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(
config: IConfig,
private readonly poolFactory: PostgresPoolFactory = (poolConfig) =>
new Pool(poolConfig) as unknown as PostgresPool,
) {
super(config);
}
override async onServerStart(): Promise<void> {
const dbConf = this.config.database!;
this.primaryPool = this.createPool(dbConf);
await this.primaryPool.query('SELECT 1');
console.log('[postgres] connected to primary');
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
await this.replicaPool.query('SELECT 1');
this.configuration = Configuration.REPLICA;
console.log('[postgres] connected to read-replica');
} else {
this.replicaPool = this.primaryPool;
this.configuration = Configuration.SINGLE;
}
await this.runMigrations();
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
const drainMs = 60_000;
console.log(
`[postgres] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools().catch((e) =>
console.error('[postgres] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools();
}
override quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `"${part.replaceAll('"', '""')}"`;
})
.join('.');
}
override booleanLiteral(value: boolean): string {
return value ? 'TRUE' : 'FALSE';
}
override booleanValue(value: boolean): boolean {
return value;
}
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.replicaPool, query, params);
return normalizePostgresRows(result);
}
override async pread(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.primaryPool, query, params);
return normalizePostgresRows(result);
}
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
const result = await this.query(this.primaryPool, query, params);
return mapPostgresWriteResult(result);
}
override async batchWrite(entries: BatchEntry[]): Promise<void> {
if (entries.length === 0) return;
const conn = await this.primaryPool.connect();
try {
await conn.query('BEGIN');
try {
for (const { statement, values } of entries) {
await this.query(conn, statement, values);
}
await conn.query('COMMIT');
} catch (err) {
await conn.query('ROLLBACK').catch(() => {});
throw err;
}
} finally {
conn.release();
}
}
override async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
if (this.configuration === Configuration.SINGLE) {
return this.read(query, params);
}
const primaryPromise = this.query(this.primaryPool, query, params);
try {
const replicaResult = await this.query(
this.replicaPool,
query,
params,
);
if (replicaResult.rows.length > 0) {
primaryPromise.catch(() => {});
return normalizePostgresRows(replicaResult);
}
} catch {
// fall through to primary
}
const primaryResult = await primaryPromise;
return normalizePostgresRows(primaryResult);
}
private async runMigrations(): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const conn = await this.primaryPool.connect();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) =>
f.endsWith('.sql') && f.startsWith('postgres'),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[postgres] migration path is unreadable: ${dir}`,
{ cause: e },
);
}
if (files.length === 0) {
console.log(`[postgres] no migrations in ${dir}`);
continue;
}
console.log(
`[postgres] running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const filePath = resolvePath(dir, file);
const contents = readFileSync(filePath, 'utf8');
const statements = splitPostgresStatements(contents);
await conn.query('BEGIN');
try {
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`[postgres] failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
await conn.query('COMMIT');
} catch (e) {
await conn.query('ROLLBACK').catch(() => {});
throw e;
}
console.log(
`[postgres] applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
}
private createPool(dbConf: PostgresEndpointConfig): PostgresPool {
const connectionString = dbConf.connectionString ?? dbConf.url;
if (connectionString) {
return this.poolFactory({
connectionString,
max: 30,
});
}
return this.poolFactory({
host: dbConf.host ?? '127.0.0.1',
port: dbConf.port ?? 5432,
user: dbConf.user ?? 'postgres',
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
max: 30,
});
}
private async query(
target: PostgresQueryable,
query: string,
params: unknown[] = [],
): Promise<QueryResult> {
const prepared = preparePostgresSql(query);
return target.query(prepared.text, params);
}
private async closeCurrentPools(): Promise<void> {
const tasks: Promise<void>[] = [];
if (this.primaryPool) tasks.push(this.primaryPool.end());
if (this.replicaPool && this.replicaPool !== this.primaryPool) {
tasks.push(this.replicaPool.end());
}
await Promise.all(tasks);
}
}
+4
View File
@@ -24,10 +24,12 @@ export {
} from './DatabaseClient';
export { SqliteDatabaseClient } from './SqliteDatabaseClient';
export { MySQLDatabaseClient } from './MySQLDatabaseClient';
export { PostgresDatabaseClient } from './PostgresDatabaseClient';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient } from './DatabaseClient';
import { MySQLDatabaseClient } from './MySQLDatabaseClient';
import { PostgresDatabaseClient } from './PostgresDatabaseClient';
import { SqliteDatabaseClient } from './SqliteDatabaseClient';
/**
@@ -41,6 +43,8 @@ export const DatabaseClientFactory = class DatabaseClientFactory {
switch (engine) {
case 'mysql':
return new MySQLDatabaseClient(config);
case 'postgres':
return new PostgresDatabaseClient(config);
case 'sqlite':
return new SqliteDatabaseClient(config);
default:
@@ -0,0 +1,43 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Comparator for migration filenames named `<engine>_mig_<N>.sql`.
*
* Existing MySQL files use unpadded integers, so lexical sorting places
* `*_mig_10.sql` before `*_mig_2.sql`. Pull the trailing integer out and sort
* numerically. Anything that does not match the `_<digits>.sql` shape falls
* back to lexical comparison and sorts after numbered files.
*/
export const compareMigrationFilenames = (a: string, b: string): number => {
const numericIndex = (name: string): number => {
const m = /_(\d+)\.sql$/.exec(name);
return m ? Number.parseInt(m[1], 10) : Number.NaN;
};
const na = numericIndex(a);
const nb = numericIndex(b);
if (Number.isFinite(na) && Number.isFinite(nb)) {
if (na !== nb) return na - nb;
} else if (Number.isFinite(na)) {
return -1;
} else if (Number.isFinite(nb)) {
return 1;
}
return a.localeCompare(b);
};
@@ -0,0 +1,724 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
CREATE TABLE IF NOT EXISTS "user" (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(36) NOT NULL UNIQUE,
username varchar(50) UNIQUE,
email varchar(256),
password varchar(225),
free_storage bigint,
max_subdomains integer,
taskbar_items text,
desktop_uuid varchar(36),
appdata_uuid varchar(36),
documents_uuid varchar(36),
pictures_uuid varchar(36),
videos_uuid varchar(36),
trash_uuid varchar(36),
trash_id integer,
appdata_id integer,
desktop_id integer,
documents_id integer,
pictures_id integer,
videos_id integer,
referrer varchar(64),
desktop_bg_url text,
desktop_bg_color varchar(20),
desktop_bg_fit varchar(16),
pass_recovery_token varchar(36),
requires_email_confirmation boolean NOT NULL DEFAULT FALSE,
email_confirm_code varchar(8),
email_confirm_token varchar(36),
email_confirmed boolean NOT NULL DEFAULT FALSE,
dev_first_name varchar(100),
dev_last_name varchar(100),
dev_paypal varchar(100),
dev_approved_for_incentive_program boolean DEFAULT FALSE,
dev_joined_incentive_program boolean DEFAULT FALSE,
suspended boolean,
unsubscribed boolean NOT NULL DEFAULT FALSE,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_activity_ts timestamp,
referral_code varchar(16) UNIQUE,
referred_by integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
unconfirmed_change_email varchar(256),
change_email_confirm_token varchar(256),
otp_secret text,
otp_enabled boolean DEFAULT FALSE,
otp_recovery_codes text,
stripe_customer_id varchar(40),
public_uuid varchar(36),
public_id integer,
clean_email varchar(256),
audit_metadata jsonb,
signup_ip varchar(45),
signup_ip_forwarded varchar(45),
signup_user_agent varchar(512),
signup_origin varchar(255),
signup_server varchar(255),
metadata jsonb DEFAULT '{}'::jsonb,
reputation smallint DEFAULT 100
);
CREATE INDEX IF NOT EXISTS idx_user_email ON "user" (email);
CREATE INDEX IF NOT EXISTS idx_user_pass_recovery_token ON "user" (pass_recovery_token);
CREATE INDEX IF NOT EXISTS idx_user_referrer ON "user" (referrer);
CREATE INDEX IF NOT EXISTS idx_user_email_confirm_token ON "user" (email_confirm_token);
CREATE INDEX IF NOT EXISTS idx_user_last_activity_ts ON "user" (last_activity_ts);
CREATE INDEX IF NOT EXISTS idx_user_referred_by ON "user" (referred_by);
CREATE INDEX IF NOT EXISTS idx_user_stripe_customer_id ON "user" (stripe_customer_id);
CREATE INDEX IF NOT EXISTS idx_user_clean_email ON "user" (clean_email);
CREATE INDEX IF NOT EXISTS idx_user_signup_ip ON "user" (signup_ip);
CREATE INDEX IF NOT EXISTS idx_user_signup_ip_forwarded ON "user" (signup_ip_forwarded);
CREATE INDEX IF NOT EXISTS idx_user_signup_user_agent ON "user" (signup_user_agent);
CREATE INDEX IF NOT EXISTS idx_user_signup_origin ON "user" (signup_origin);
CREATE INDEX IF NOT EXISTS idx_user_signup_server ON "user" (signup_server);
CREATE TABLE IF NOT EXISTS apps (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
owner_user_id integer REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
icon text,
name varchar(100) NOT NULL UNIQUE,
title varchar(100) NOT NULL,
description text,
godmode boolean DEFAULT FALSE,
maximize_on_start boolean DEFAULT FALSE,
index_url text NOT NULL,
approved_for_listing boolean DEFAULT FALSE,
approved_for_opening_items boolean DEFAULT FALSE,
approved_for_incentive_program boolean DEFAULT FALSE,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_review timestamp,
tags varchar(255),
app_owner integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
background boolean DEFAULT FALSE,
metadata jsonb,
protected boolean DEFAULT FALSE,
is_private boolean DEFAULT FALSE
);
CREATE INDEX IF NOT EXISTS idx_apps_owner_user_id ON apps (owner_user_id);
CREATE INDEX IF NOT EXISTS idx_apps_app_owner ON apps (app_owner);
CREATE INDEX IF NOT EXISTS idx_apps_owner_timestamp ON apps (owner_user_id, "timestamp" DESC);
CREATE INDEX IF NOT EXISTS idx_apps_listing_timestamp ON apps (approved_for_listing, "timestamp" DESC);
CREATE TABLE IF NOT EXISTS "group" (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) UNIQUE,
owner_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
extra jsonb,
metadata jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_group_owner_user_id ON "group" (owner_user_id);
CREATE TABLE IF NOT EXISTS app_filetype_association (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
type varchar(60) NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_app_filetype_association_app_id ON app_filetype_association (app_id);
CREATE INDEX IF NOT EXISTS idx_app_filetype_association_type ON app_filetype_association (type);
CREATE TABLE IF NOT EXISTS app_opens (
_id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_uid varchar(40) NOT NULL REFERENCES apps (uid) ON DELETE CASCADE ON UPDATE CASCADE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
ts integer NOT NULL,
human_ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_app_opens_user_id ON app_opens (user_id);
CREATE INDEX IF NOT EXISTS idx_app_opens_app_uid ON app_opens (app_uid);
CREATE INDEX IF NOT EXISTS idx_app_opens_uid_ts ON app_opens (app_uid, ts);
CREATE INDEX IF NOT EXISTS idx_app_opens_app_user ON app_opens (app_uid, user_id);
CREATE TABLE IF NOT EXISTS fsentries (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(36) NOT NULL UNIQUE,
bucket varchar(50),
bucket_region varchar(30),
public_token varchar(36) UNIQUE,
file_request_token varchar(36) UNIQUE,
is_shortcut boolean DEFAULT FALSE,
shortcut_to integer REFERENCES fsentries (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
parent_id integer REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
associated_app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
is_dir boolean DEFAULT FALSE,
layout varchar(30),
sort_by varchar(20),
sort_order varchar(10),
is_public boolean,
thumbnail text,
immutable boolean NOT NULL DEFAULT FALSE,
name varchar(767) NOT NULL,
metadata text,
modified integer NOT NULL,
created integer,
accessed integer,
size bigint,
symlink_path varchar(260),
is_symlink boolean DEFAULT FALSE,
parent_uid varchar(36),
path varchar(4096),
CONSTRAINT fsentries_parent_name_unique UNIQUE (parent_id, name)
);
CREATE INDEX IF NOT EXISTS idx_fsentries_name ON fsentries (name);
CREATE INDEX IF NOT EXISTS idx_fsentries_modified ON fsentries (modified);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_id ON fsentries (parent_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_is_dir ON fsentries (is_dir);
CREATE INDEX IF NOT EXISTS idx_fsentries_user_id ON fsentries (user_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_shortcut_to ON fsentries (shortcut_to);
CREATE INDEX IF NOT EXISTS idx_fsentries_associated_app_id ON fsentries (associated_app_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_bucket ON fsentries (bucket);
CREATE INDEX IF NOT EXISTS idx_fsentries_bucket_region ON fsentries (bucket_region);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_uid ON fsentries (parent_uid);
CREATE INDEX IF NOT EXISTS idx_fsentries_path ON fsentries (path);
CREATE INDEX IF NOT EXISTS idx_fsentries_accessed ON fsentries (accessed);
CREATE INDEX IF NOT EXISTS idx_fsentries_user_parent_name ON fsentries (user_id, parent_uid, name);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_uid_name ON fsentries (parent_uid, name);
CREATE TABLE IF NOT EXISTS fsentry_versions (
id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
fsentry_id integer NOT NULL REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_uuid varchar(36) NOT NULL,
version_id varchar(60) NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
message text,
ts_epoch integer
);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_fsentry_id ON fsentry_versions (fsentry_id);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_fsentry_uuid ON fsentry_versions (fsentry_uuid);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_user_id ON fsentry_versions (user_id);
CREATE TABLE IF NOT EXISTS subdomains (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(40) UNIQUE,
subdomain varchar(64) NOT NULL UNIQUE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
root_dir_id integer REFERENCES fsentries (id) ON DELETE SET NULL ON UPDATE CASCADE,
associated_app_id integer REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
ts timestamp DEFAULT CURRENT_TIMESTAMP,
app_owner integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
protected boolean DEFAULT FALSE,
domain varchar(265),
database_id varchar(40),
preamble_version varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_subdomains_user_id ON subdomains (user_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_root_dir_id ON subdomains (root_dir_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_associated_app_id ON subdomains (associated_app_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_app_owner ON subdomains (app_owner);
CREATE INDEX IF NOT EXISTS idx_subdomains_domain ON subdomains (domain);
CREATE INDEX IF NOT EXISTS idx_subdomains_root_user ON subdomains (root_dir_id, user_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_app_user ON subdomains (associated_app_id, user_id);
CREATE TABLE IF NOT EXISTS kv (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app varchar(40),
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
kkey_hash bigint NOT NULL,
kkey text NOT NULL,
value text,
migrated boolean DEFAULT FALSE,
"expireAt" timestamp,
CONSTRAINT kv_app_user_kkey_hash_unique UNIQUE (app, user_id, kkey_hash)
);
CREATE INDEX IF NOT EXISTS idx_kv_app ON kv (app);
CREATE INDEX IF NOT EXISTS idx_kv_user_id ON kv (user_id);
CREATE INDEX IF NOT EXISTS idx_kv_kkey_hash ON kv (kkey_hash);
CREATE TABLE IF NOT EXISTS sessions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
uuid varchar(40) NOT NULL,
meta jsonb,
created_at bigint DEFAULT 0,
last_activity bigint DEFAULT 0,
kind varchar(32) NOT NULL DEFAULT 'web'
CHECK (kind IN ('web', 'app', 'access_token', 'asset', 'worker')),
label varchar(255),
parent_session_id varchar(64),
last_ip varchar(64),
last_user_agent varchar(512),
revoked_at bigint,
expires_at bigint,
app_uid varchar(64),
legacy_token_uid varchar(64),
created_via varchar(32),
auth_id varchar(64),
access_token_uid varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions (user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_uuid ON sessions (uuid);
CREATE INDEX IF NOT EXISTS idx_sessions_user_revoked ON sessions (user_id, revoked_at);
CREATE INDEX IF NOT EXISTS idx_sessions_parent ON sessions (parent_session_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_user_app_active
ON sessions (user_id, app_uid)
WHERE kind = 'app' AND revoked_at IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_legacy_token_active
ON sessions (legacy_token_uid)
WHERE legacy_token_uid IS NOT NULL AND revoked_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_sessions_kind_user ON sessions (kind, user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_access_token_uid
ON sessions (access_token_uid)
WHERE access_token_uid IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_user_worker_active
ON sessions (user_id, COALESCE(app_uid, ''), (meta #>> ARRAY['worker_name']))
WHERE kind = 'worker' AND revoked_at IS NULL;
CREATE TABLE IF NOT EXISTS user_to_app_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
dt timestamp DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, app_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_utap_user_permission ON user_to_app_permissions (user_id, permission);
CREATE INDEX IF NOT EXISTS idx_utap_app_permission ON user_to_app_permissions (app_id, permission);
CREATE TABLE IF NOT EXISTS user_to_group_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (user_id, group_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_user_to_group_permissions_group_id ON user_to_group_permissions (group_id);
CREATE TABLE IF NOT EXISTS user_to_user_permissions (
issuer_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
holder_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (issuer_user_id, holder_user_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_user_to_user_permissions_holder_user_id ON user_to_user_permissions (holder_user_id);
CREATE TABLE IF NOT EXISTS dev_to_app_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (user_id, app_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_app_id ON dev_to_app_permissions (app_id);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_permission ON dev_to_app_permissions (permission);
CREATE TABLE IF NOT EXISTS access_token_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
token_uid varchar(40) NOT NULL,
authorizer_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
authorizer_app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_access_token_permissions_token_uid ON access_token_permissions (token_uid);
CREATE TABLE IF NOT EXISTS old_app_names (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_uid varchar(40) NOT NULL REFERENCES apps (uid) ON DELETE CASCADE,
name varchar(255) NOT NULL,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT unique_old_app_names_app_uid_name UNIQUE (app_uid, name)
);
CREATE INDEX IF NOT EXISTS idx_old_app_names_app_uid ON old_app_names (app_uid);
CREATE INDEX IF NOT EXISTS idx_old_app_names_name ON old_app_names (name);
CREATE TABLE IF NOT EXISTS jct_user_group (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE,
extra jsonb,
metadata jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_jct_user_group_user_id ON jct_user_group (user_id);
CREATE INDEX IF NOT EXISTS idx_jct_user_group_group_id ON jct_user_group (group_id);
CREATE TABLE IF NOT EXISTS notification (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
uid varchar(40) UNIQUE,
value jsonb NOT NULL,
acknowledged bigint,
shown bigint,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_notification_user_id ON notification (user_id);
CREATE TABLE IF NOT EXISTS share (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) UNIQUE,
issuer_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
recipient_email varchar(255) NOT NULL,
data jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_share_issuer_user_id ON share (issuer_user_id);
CREATE INDEX IF NOT EXISTS idx_share_recipient_email ON share (recipient_email);
CREATE TABLE IF NOT EXISTS feedback (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
message text,
ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_feedback_user_id ON feedback (user_id);
CREATE TABLE IF NOT EXISTS thread (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
parent_uid varchar(40) REFERENCES thread (uid) ON DELETE CASCADE ON UPDATE CASCADE,
owner_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
"schema" text,
text text NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_thread_parent_uid ON thread (parent_uid);
CREATE INDEX IF NOT EXISTS idx_thread_owner_user_id ON thread (owner_user_id);
CREATE TABLE IF NOT EXISTS user_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
metadata jsonb,
text text NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_user_comments_user_id ON user_comments (user_id);
CREATE INDEX IF NOT EXISTS idx_user_comments_uid ON user_comments (uid);
CREATE TABLE IF NOT EXISTS user_fsentry_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_id integer NOT NULL REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_comments_user_comment_id ON user_fsentry_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_comments_fsentry_id ON user_fsentry_comments (fsentry_id);
CREATE TABLE IF NOT EXISTS user_fsentry_version_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_version_id integer NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_version_comments_user_comment_id ON user_fsentry_version_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_version_comments_version_id ON user_fsentry_version_comments (fsentry_version_id);
CREATE TABLE IF NOT EXISTS user_group_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_group_comments_user_comment_id ON user_group_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_group_comments_group_id ON user_group_comments (group_id);
CREATE TABLE IF NOT EXISTS user_user_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
commented_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_user_comments_user_comment_id ON user_user_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_user_comments_commented_user_id ON user_user_comments (commented_user_id);
CREATE TABLE IF NOT EXISTS user_oidc_providers (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
provider varchar(64) NOT NULL,
provider_sub varchar(255) NOT NULL,
refresh_token text,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_user_oidc_providers_user_id ON user_oidc_providers (user_id);
CREATE INDEX IF NOT EXISTS idx_user_oidc_providers_provider ON user_oidc_providers (provider);
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_providers_provider_sub_unique ON user_oidc_providers (provider, provider_sub);
CREATE TABLE IF NOT EXISTS app_update_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
old_name varchar(50),
new_name varchar(50),
reason varchar(255)
);
CREATE INDEX IF NOT EXISTS idx_app_update_audit_app_id ON app_update_audit (app_id);
CREATE TABLE IF NOT EXISTS user_update_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
old_email varchar(256),
new_email varchar(256),
old_username varchar(50),
new_username varchar(50),
reason varchar(255)
);
CREATE INDEX IF NOT EXISTS idx_user_update_audit_user_id ON user_update_audit (user_id);
CREATE TABLE IF NOT EXISTS storage_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
is_subtract boolean NOT NULL DEFAULT FALSE,
amount bigint NOT NULL,
field_a varchar(16),
field_b varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_storage_audit_user_id ON storage_audit (user_id);
CREATE TABLE IF NOT EXISTS audit_user_to_app_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_app_permissions_user_id ON audit_user_to_app_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_app_permissions_app_id ON audit_user_to_app_permissions (app_id);
CREATE TABLE IF NOT EXISTS audit_dev_to_app_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_dev_to_app_permissions_user_id ON audit_dev_to_app_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_dev_to_app_permissions_app_id ON audit_dev_to_app_permissions (app_id);
CREATE TABLE IF NOT EXISTS audit_user_to_group_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
group_id integer REFERENCES "group" (id) ON DELETE SET NULL ON UPDATE CASCADE,
group_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(255),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_group_permissions_user_id ON audit_user_to_group_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_group_permissions_group_id ON audit_user_to_group_permissions (group_id);
CREATE TABLE IF NOT EXISTS audit_user_to_user_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
issuer_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
issuer_user_id_keep integer NOT NULL,
holder_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
holder_user_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_user_permissions_issuer_user_id ON audit_user_to_user_permissions (issuer_user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_user_permissions_holder_user_id ON audit_user_to_user_permissions (holder_user_id);
CREATE TABLE IF NOT EXISTS ai_usage (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
service_name varchar(64),
model_name varchar(128),
price_modifier varchar(40),
cost integer,
value_uint_1 integer,
value_uint_2 integer,
value_uint_3 integer,
value_uint_4 integer,
value_uint_5 integer,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_ai_usage_app_id ON ai_usage (app_id);
CREATE INDEX IF NOT EXISTS idx_ai_usage_service_name ON ai_usage (service_name);
CREATE INDEX IF NOT EXISTS idx_ai_usage_model_name ON ai_usage (model_name);
CREATE INDEX IF NOT EXISTS idx_ai_usage_price_modifier ON ai_usage (price_modifier);
CREATE INDEX IF NOT EXISTS idx_ai_usage_created_at ON ai_usage (created_at);
CREATE INDEX IF NOT EXISTS idx_ai_usage_user_timestamp ON ai_usage (user_id, created_at);
CREATE TABLE IF NOT EXISTS general_analytics (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
uid varchar(40) NOT NULL,
trace_id varchar(40),
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer,
server_id varchar(40),
actor_type varchar(40),
tags jsonb,
fields jsonb
);
CREATE INDEX IF NOT EXISTS idx_general_analytics_user_id ON general_analytics (user_id);
CREATE INDEX IF NOT EXISTS idx_general_analytics_app_id ON general_analytics (app_id);
CREATE TABLE IF NOT EXISTS monthly_usage_counts (
year integer NOT NULL,
month integer NOT NULL,
service_type varchar(40) NOT NULL,
service_name varchar(40) NOT NULL,
actor_key varchar(255) NOT NULL,
pricing_category jsonb NOT NULL,
pricing_category_hash bytea NOT NULL,
"count" integer DEFAULT 0,
value_uint_1 integer,
value_uint_2 integer,
value_uint_3 integer,
PRIMARY KEY (year, month, service_type, service_name, actor_key, pricing_category_hash)
);
CREATE TABLE IF NOT EXISTS service_usage_monthly (
"key" varchar(255) NOT NULL,
year integer NOT NULL,
month integer NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
"count" integer NOT NULL,
extra jsonb,
PRIMARY KEY ("key", year, month)
);
CREATE INDEX IF NOT EXISTS idx_service_usage_monthly_user_id ON service_usage_monthly (user_id);
CREATE INDEX IF NOT EXISTS idx_service_usage_monthly_app_id ON service_usage_monthly (app_id);
CREATE TABLE IF NOT EXISTS per_user_credit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL UNIQUE REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
amount bigint NOT NULL,
last_updated_at bigint NOT NULL
);
INSERT INTO "user" (id, uuid, username, metadata)
VALUES (1, '5d4adce0-a381-4982-9c02-6e2540026238', 'system', '{}'::jsonb)
ON CONFLICT (uuid) DO UPDATE SET username = EXCLUDED.username;
SELECT setval(pg_get_serial_sequence('"user"', 'id'), COALESCE((SELECT MAX(id) FROM "user"), 1), true);
INSERT INTO "group" (uid, owner_user_id, extra, metadata)
VALUES
('26bfb1fb-421f-45bc-9aa4-d81ea569e7a5', 1, '{"critical": true, "type": "default", "name": "system"}'::jsonb, '{"title": "System", "color": "#000000"}'::jsonb),
('ca342a5e-b13d-4dee-9048-58b11a57cc55', 1, '{"critical": true, "type": "default", "name": "admin"}'::jsonb, '{"title": "Admin", "color": "#a83232"}'::jsonb),
('78b1b1dd-c959-44d2-b02c-8735671f9997', 1, '{"critical": true, "type": "default", "name": "user"}'::jsonb, '{"title": "User", "color": "#3254a8"}'::jsonb),
('b7220104-7905-4985-b996-649fdcdb3c8f', 1, '{"critical": true, "type": "default", "name": "temp"}'::jsonb, '{"title": "Temp", "color": "#888888"}'::jsonb),
('3c2dfff7-d22a-41aa-a193-59a61dac4b64', 1, '{"type": "default", "name": "moderator"}'::jsonb, '{"title": "Moderator", "color": "#a432a8"}'::jsonb),
('5e8f251d-3382-4b0d-932c-7bb82f48652f', 1, '{"type": "default", "name": "developer"}'::jsonb, '{"title": "Developer", "color": "#32a852"}'::jsonb)
ON CONFLICT (uid) DO UPDATE SET
owner_user_id = EXCLUDED.owner_user_id,
extra = EXCLUDED.extra,
metadata = EXCLUDED.metadata;
INSERT INTO apps (
uid, owner_user_id, icon, name, title, description, index_url,
godmode, maximize_on_start, background,
approved_for_listing, approved_for_opening_items,
approved_for_incentive_program, tags, "timestamp"
)
VALUES
('app-3920851d-bda8-479b-9407-8517293c7d44', 1, NULL, 'pdf', 'PDF', '', 'https://pdf.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-5584fbf7-ed69-41fc-99cd-85da21b1ef51', 1, NULL, 'camera', 'Camera', 'Camera in the browser.', 'https://online-camera.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-11edfba2-1ed3-4e22-8573-47e88fb87d70', 1, NULL, 'player', 'Player', 'A free video player app in the browser.', 'https://simple-player.puter.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-7bdca1a4-6373-4c98-ad97-03ff2d608ca1', 1, NULL, 'recorder', 'Recorder', 'Online voice recorder in the browser with cloud storage.', 'https://voice-recorder.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-d7e9471f-e441-4d72-a5ab-75e96573b76b', 1, NULL, 'music-player', 'Music Player', 'A free music player app in the browser.', 'https://player.puter.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2026-05-10 00:00:00'),
('app-e3ac5486-da8c-42ad-8377-8728086e0980', 1, NULL, 'git', 'Git', 'Puter Git client', 'https://builtins.namespaces.puter.com/git', FALSE, FALSE, TRUE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-0b37f054-07d4-4627-8765-11bd23e889d4', 1, NULL, 'dev-center', 'Dev Center', 'This is the app that makes apps', 'https://builtins.namespaces.puter.com/dev-center', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-fbbdb72b-ad08-4cb4-86a1-de0f27cf2e1e', 1, NULL, 'puter-linux', 'Puter Linux', 'Linux emulator for Puter', 'https://builtins.namespaces.puter.com/emulator', TRUE, FALSE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-838dfbc4-bf8b-48c2-b47b-c4adc77fab58', 1, NULL, 'editor', 'Editor', 'Text editor', 'https://online-notepad.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-58282b08-990a-4906-95f7-fa37ff92452b', 1, NULL, 'draw', 'Draw', 'Image editor', 'https://draw.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'graphics', '2020-01-01 00:00:00'),
('app-0bef044f-918f-4cbf-a0c0-b4a17ee81085', 1, NULL, 'about', 'About', 'About Puter', 'https://about.puter.com', TRUE, FALSE, FALSE, FALSE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-a2ae72a4-1ba3-4a29-b5c0-6de1be5cf178', 1, NULL, 'app-center', 'App Center', 'Discover apps for Puter', 'https://app-center.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-93005ce0-80d1-50d9-9b1e-9c453c375d56', 1, NULL, 'markus', 'Markus', 'Markdown editor', 'https://markus.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-6f79ef7b-52b7-4b31-91c6-fc07b62e9396', 1, NULL, 'code', 'Code', 'Code editor', 'https://code.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-862fc09e-c7b8-4c30-b5b4-47b3cc5f5232', 1, NULL, 'memos', 'Memos', 'Notes and memos', 'https://memos.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-aeb8c03e-1144-4c57-bff5-3a1a7c17f9f0', 1, NULL, 'word-processor', 'Word Processor', 'Write documents', 'https://word-processor.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-c10a1999-22f5-4644-9960-9aaac0d4934e', 1, NULL, 'spreadsheet', 'Spreadsheet', 'Work with spreadsheets', 'https://spreadsheet.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-e9c1d58d-3d8d-4f0a-a85c-e383ef63bc29', 1, NULL, 'presentation', 'Presentation', 'Create presentations', 'https://presentation.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-fb7d9e42-8207-4fa0-b680-f0239327097f', 1, NULL, 'pdf-editor', 'PDF Editor', 'Edit PDF documents', 'https://pdf-editor.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00')
ON CONFLICT (uid) DO UPDATE SET
owner_user_id = EXCLUDED.owner_user_id,
name = EXCLUDED.name,
title = EXCLUDED.title,
description = EXCLUDED.description,
index_url = EXCLUDED.index_url,
godmode = EXCLUDED.godmode,
maximize_on_start = EXCLUDED.maximize_on_start,
background = EXCLUDED.background,
approved_for_listing = EXCLUDED.approved_for_listing,
approved_for_opening_items = EXCLUDED.approved_for_opening_items,
approved_for_incentive_program = EXCLUDED.approved_for_incentive_program,
tags = EXCLUDED.tags;
INSERT INTO user_to_group_permissions (user_id, group_id, permission, extra)
SELECT u.id, g.id, 'driver', '{}'::jsonb
FROM "user" u, "group" g
WHERE u.username = 'system'
AND g.uid = 'ca342a5e-b13d-4dee-9048-58b11a57cc55'
ON CONFLICT (user_id, group_id, permission) DO UPDATE SET extra = EXCLUDED.extra;
@@ -0,0 +1,85 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { preparePostgresSql } from './preparePostgresSql.js';
describe('preparePostgresSql', () => {
it('converts placeholders and backtick identifiers outside SQL literals', () => {
const prepared = preparePostgresSql(
'SELECT `user`.`id` FROM `user` WHERE `email` = ? AND `username` = ?',
);
expect(prepared).toEqual({
text: 'SELECT "user"."id" FROM "user" WHERE "email" = $1 AND "username" = $2',
parameterCount: 2,
});
});
it('leaves question marks inside strings and comments untouched', () => {
const prepared = preparePostgresSql(`
SELECT '?' AS literal, ?
-- ? in a comment
FROM \`apps\`
WHERE \`description\` = 'is this ok?'
/* and ? in a block comment */
AND \`name\` = ?
`);
expect(prepared.text).toContain("SELECT '?' AS literal, $1");
expect(prepared.text).toContain('-- ? in a comment');
expect(prepared.text).toContain('"description" = \'is this ok?\'');
expect(prepared.text).toContain('/* and ? in a block comment */');
expect(prepared.text).toContain('"name" = $2');
expect(prepared.parameterCount).toBe(2);
});
it('escapes double quotes inside converted identifiers', () => {
const prepared = preparePostgresSql(
'SELECT `odd"name` FROM `odd``table` WHERE `id` = ?',
);
expect(prepared).toEqual({
text: 'SELECT "odd""name" FROM "odd`table" WHERE "id" = $1',
parameterCount: 1,
});
});
it('does not scan placeholders inside dollar-quoted strings', () => {
const prepared = preparePostgresSql(
"SELECT $$?$$, $tag$`not_ident` ?$tag$, `id` FROM `user` WHERE `id` = ?",
);
expect(prepared).toEqual({
text: 'SELECT $$?$$, $tag$`not_ident` ?$tag$, "id" FROM "user" WHERE "id" = $1',
parameterCount: 1,
});
});
it('does not treat Postgres JSON operators as comments', () => {
const prepared = preparePostgresSql(
"SELECT * FROM `sessions` WHERE `user_id` = ? AND `meta` #>> ARRAY['worker_name'] = ? AND `expires_at` > ?",
);
expect(prepared).toEqual({
text: 'SELECT * FROM "sessions" WHERE "user_id" = $1 AND "meta" #>> ARRAY[\'worker_name\'] = $2 AND "expires_at" > $3',
parameterCount: 3,
});
});
});
@@ -0,0 +1,169 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
export interface PreparedPostgresSql {
text: string;
parameterCount: number;
}
type ScannerState =
| { kind: 'normal' }
| { kind: 'singleQuote' }
| { kind: 'doubleQuote' }
| { kind: 'backtickIdentifier' }
| { kind: 'lineComment' }
| { kind: 'blockComment' }
| { kind: 'dollarQuote'; tag: string };
const isLineCommentStart = (sql: string, index: number): boolean => {
if (sql[index] !== '-' || sql[index + 1] !== '-') return false;
const after = sql[index + 2];
return after === undefined || /\s/u.test(after);
};
const readDollarQuoteTag = (sql: string, index: number): string | null => {
const match = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/u.exec(sql.slice(index));
return match?.[0] ?? null;
};
export const preparePostgresSql = (sql: string): PreparedPostgresSql => {
let state: ScannerState = { kind: 'normal' };
let parameterCount = 0;
let out = '';
for (let i = 0; i < sql.length; i++) {
const char = sql[i]!;
const next = sql[i + 1];
switch (state.kind) {
case 'normal': {
const dollarTag =
char === '$' ? readDollarQuoteTag(sql, i) : null;
if (dollarTag) {
out += dollarTag;
i += dollarTag.length - 1;
state = { kind: 'dollarQuote', tag: dollarTag };
break;
}
if (char === "'") {
out += char;
state = { kind: 'singleQuote' };
break;
}
if (char === '"') {
out += char;
state = { kind: 'doubleQuote' };
break;
}
if (char === '`') {
out += '"';
state = { kind: 'backtickIdentifier' };
break;
}
if (isLineCommentStart(sql, i)) {
out += char;
if (char === '-') {
out += next ?? '';
i += 1;
}
state = { kind: 'lineComment' };
break;
}
if (char === '/' && next === '*') {
out += '/*';
i += 1;
state = { kind: 'blockComment' };
break;
}
if (char === '?') {
parameterCount += 1;
out += `$${parameterCount}`;
break;
}
out += char;
break;
}
case 'singleQuote':
out += char;
if (char === "'" && next === "'") {
out += next;
i += 1;
} else if (char === '\\' && next !== undefined) {
out += next;
i += 1;
} else if (char === "'") {
state = { kind: 'normal' };
}
break;
case 'doubleQuote':
out += char;
if (char === '"' && next === '"') {
out += next;
i += 1;
} else if (char === '"') {
state = { kind: 'normal' };
}
break;
case 'backtickIdentifier':
if (char === '`' && next === '`') {
out += '`';
i += 1;
} else if (char === '`') {
out += '"';
state = { kind: 'normal' };
} else if (char === '"') {
out += '""';
} else {
out += char;
}
break;
case 'lineComment':
out += char;
if (char === '\n') {
state = { kind: 'normal' };
}
break;
case 'blockComment':
out += char;
if (char === '*' && next === '/') {
out += '/';
i += 1;
state = { kind: 'normal' };
}
break;
case 'dollarQuote':
if (sql.startsWith(state.tag, i)) {
out += state.tag;
i += state.tag.length - 1;
state = { kind: 'normal' };
} else {
out += char;
}
break;
}
}
return { text: out, parameterCount };
};
@@ -0,0 +1,120 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
type ScannerState =
| { kind: 'normal' }
| { kind: 'singleQuote' }
| { kind: 'doubleQuote' }
| { kind: 'lineComment' }
| { kind: 'blockComment' }
| { kind: 'dollarQuote'; tag: string };
const readDollarQuoteTag = (sql: string, index: number): string | null => {
const match = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/u.exec(sql.slice(index));
return match?.[0] ?? null;
};
export const splitPostgresStatements = (sql: string): string[] => {
const statements: string[] = [];
let state: ScannerState = { kind: 'normal' };
let start = 0;
for (let i = 0; i < sql.length; i++) {
const char = sql[i]!;
const next = sql[i + 1];
switch (state.kind) {
case 'normal': {
const dollarTag =
char === '$' ? readDollarQuoteTag(sql, i) : null;
if (dollarTag) {
i += dollarTag.length - 1;
state = { kind: 'dollarQuote', tag: dollarTag };
break;
}
if (char === "'") {
state = { kind: 'singleQuote' };
break;
}
if (char === '"') {
state = { kind: 'doubleQuote' };
break;
}
if (char === '-' && next === '-') {
i += 1;
state = { kind: 'lineComment' };
break;
}
if (char === '/' && next === '*') {
i += 1;
state = { kind: 'blockComment' };
break;
}
if (char === ';') {
const statement = sql.slice(start, i).trim();
if (statement !== '') statements.push(statement);
start = i + 1;
}
break;
}
case 'singleQuote':
if (char === "'" && next === "'") {
i += 1;
} else if (char === '\\' && next !== undefined) {
i += 1;
} else if (char === "'") {
state = { kind: 'normal' };
}
break;
case 'doubleQuote':
if (char === '"' && next === '"') {
i += 1;
} else if (char === '"') {
state = { kind: 'normal' };
}
break;
case 'lineComment':
if (char === '\n') {
state = { kind: 'normal' };
}
break;
case 'blockComment':
if (char === '*' && next === '/') {
i += 1;
state = { kind: 'normal' };
}
break;
case 'dollarQuote':
if (sql.startsWith(state.tag, i)) {
i += state.tag.length - 1;
state = { kind: 'normal' };
}
break;
}
}
const finalStatement = sql.slice(start).trim();
if (finalStatement !== '') statements.push(finalStatement);
return statements;
};
+9
View File
@@ -18,6 +18,7 @@
*/
import type { IConfig, WithLifecycle } from '../types';
import type { ClickhouseClient } from './clickhouse/ClickhouseClient';
/**
* Extension-augmentable client registry. Extensions add their own client
@@ -40,6 +41,14 @@ export interface IExtensionClientInstances {
* override this for that name.
*/
[key: string]: unknown;
/**
* Optional ClickHouse analytics client. Absent by default — a production
* deployment registers it via an extension to speed up the app-stats path
* at scale (see {@link ClickhouseClient}). Always branch on its presence
* and fall back to SQL.
*/
clickhouse?: ClickhouseClient;
}
export interface IPuterClient<T extends WithLifecycle = WithLifecycle> {
+33 -1
View File
@@ -17,6 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { isAccessTokenActor, isAppActor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { driversContainers } from '../../exports.js';
import {
@@ -89,6 +90,16 @@ export class AppController extends PuterController {
// actor calls this, the app id is already on the token — clients
// don't re-send it in the body. Fall back to `actor.app.uid`
// before 400-ing for a missing body field.
//
// Authorization: only two callers are trusted to report opens —
// 1. a root user actor (plain session, no `.app` and no access
// token), e.g. the GUI launching apps on behalf of the user;
// 2. the app-under-user actor for the app being reported, i.e.
// `actor.app.uid === app_uid`.
// Everything else — access tokens (regardless of issuer), asset
// tokens, app actors reporting for a *different* app — is denied,
// otherwise any authenticated party could inflate another app's
// open count.
router.post(
'/rao',
{
@@ -96,8 +107,9 @@ export class AppController extends PuterController {
requireAuth: true,
},
async (req, res) => {
const actor = req.actor;
const bodyAppUid = req.body?.app_uid;
const actorAppUid = req.actor?.app?.uid;
const actorAppUid = actor?.app?.uid;
const app_uid =
typeof bodyAppUid === 'string' && bodyAppUid.length > 0
? bodyAppUid
@@ -108,6 +120,26 @@ export class AppController extends PuterController {
});
}
// Access tokens (and any other non-user/non-app identity,
// e.g. asset tokens) are not allowed to report opens —
// they're shared / scoped credentials and shouldn't drive
// analytics counters.
if (isAccessTokenActor(actor)) {
throw new HttpError(
403,
'Access tokens cannot report app opens',
{ legacyCode: 'forbidden' },
);
}
if (isAppActor(actor) && app_uid !== actorAppUid) {
throw new HttpError(
403,
'App actors can only report opens for their own app',
{ legacyCode: 'forbidden' },
);
}
const app = await this.appStore.getByUid(app_uid);
if (!app)
throw new HttpError(404, 'App not found', {
@@ -37,6 +37,7 @@ import type { EventClient } from '../../clients/event/EventClient.js';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { requireUserActorGate } from '../../core/http/middleware/gates.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -794,6 +795,65 @@ describe('AuthController.handleLogout', () => {
});
});
describe('AuthController account-lifecycle route gating', () => {
const routeOptions = (method: string, path: string) => {
const proto = Object.getPrototypeOf(controller) as {
__puterRoutes?: Array<{
method: string;
path: string;
options?: Record<string, unknown>;
}>;
};
const route = (proto.__puterRoutes ?? []).find(
(r) =>
r.method.toLowerCase() === method.toLowerCase() &&
r.path === path,
);
expect(route, `route ${method} ${path} not found`).toBeDefined();
return route!.options ?? {};
};
it('POST /logout requires a human user actor', () => {
const opts = routeOptions('post', '/logout');
expect(opts.requireUserActor).toBe(true);
expect(opts.antiCsrf).toBe(true);
});
it('GET /get-anticsrf-token requires a human user actor', () => {
const opts = routeOptions('get', '/get-anticsrf-token');
expect(opts.requireUserActor).toBe(true);
});
it('requireUserActorGate rejects app-under-user and access-token actors', () => {
const gate = requireUserActorGate();
const run = (actor: Partial<Actor>) =>
new Promise<unknown>((resolve) => {
gate(
{ actor } as never,
{} as never,
(err?: unknown) => resolve(err),
);
});
return (async () => {
const appActor = await run({
user: { uuid: 'u1' },
app: { uid: 'app-1' },
} as Partial<Actor>);
expect(appActor).toMatchObject({ statusCode: 403 });
const tokenActor = await run({
user: { uuid: 'u1' },
accessToken: { uid: 'tok-1' },
} as Partial<Actor>);
expect(tokenActor).toMatchObject({ statusCode: 403 });
const human = await run({ user: { uuid: 'u1' } } as Partial<Actor>);
expect(human).toBeUndefined();
})();
});
});
// ── Token grants: user → user / app / group ─────────────────────────
describe('AuthController grant flows', () => {
@@ -764,7 +764,7 @@ export class AuthController extends PuterController {
// -- Logout ------------------------------------------------------
@Post('/logout', {
requireAuth: true,
requireUserActor: true,
allowUnconfirmed: true,
antiCsrf: true,
})
@@ -1634,7 +1634,9 @@ export class AuthController extends PuterController {
// -- Anti-CSRF token generation ----------------------------------
@Get('/get-anticsrf-token', {
requireAuth: true,
// Anti-CSRF tokens are only consumed by `requireUserActor` routes,
// so issuance is scoped to the same actor kind for consistency.
requireUserActor: true,
allowUnconfirmed: true,
})
async handleGetAntiCsrfToken(req: Request, res: Response): Promise<void> {
@@ -2335,8 +2337,8 @@ export class AuthController extends PuterController {
}
await this.clients.db.write(
'UPDATE `user` SET `otp_enabled` = 1 WHERE `uuid` = ?',
[user.uuid],
'UPDATE `user` SET `otp_enabled` = ? WHERE `uuid` = ?',
[this.clients.db.booleanValue(true), user.uuid],
);
await this.stores.user.invalidateById(user.id);
@@ -2371,8 +2373,8 @@ export class AuthController extends PuterController {
});
await this.clients.db.write(
'UPDATE `user` SET `otp_enabled` = 0, `otp_recovery_codes` = NULL, `otp_secret` = NULL WHERE `uuid` = ?',
[user.uuid],
'UPDATE `user` SET `otp_enabled` = ?, `otp_recovery_codes` = NULL, `otp_secret` = NULL WHERE `uuid` = ?',
[this.clients.db.booleanValue(false), user.uuid],
);
await this.stores.user.invalidateById(user.id);
@@ -2789,7 +2791,6 @@ export class AuthController extends PuterController {
router.post(
'/auth/migrate-token',
{
subdomain: 'api',
rateLimit: {
scope: 'migrate-token',
limit: 20,
@@ -33,7 +33,7 @@ import type { DriverController } from './DriverController.js';
// The DriverController under test is the same instance the live request
// pipeline uses, so its iface→driver registry is populated from real
// drivers (puter-kvstore, puter-apps, puter-subdomains, …). The HTTP
// handlers (`#handleCall`, `#handleListInterfaces`, `#handleXd`) are
// handlers (`#handleCall`, `#handleListInterfaces`) are
// private — we exercise the public lookup API (`resolve` / list / get
// default) which the handlers themselves delegate to.
@@ -116,7 +116,7 @@ describe('DriverController.resolve', () => {
});
});
// ── Route handlers (#handleCall, #handleListInterfaces, #handleXd) ─
// ── Route handlers (#handleCall, #handleListInterfaces) ─────────────
// The handlers are private class fields. We capture references to them by
// invoking `registerRoutes` with a fake router whose `post`/`get` save the
@@ -189,10 +189,7 @@ const makeRes = (): MockRes => {
return res;
};
const makeReq = (
body: Record<string, unknown> = {},
actor?: Actor,
): Request =>
const makeReq = (body: Record<string, unknown> = {}, actor?: Actor): Request =>
({
body,
actor,
@@ -338,11 +335,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
res as unknown as Response,
() => {},
),
routes['POST /call'](req, res as unknown as Response, () => {}),
);
const body = res.body as {
success: boolean;
@@ -374,11 +367,9 @@ describe('DriverController.#handleCall (via captured router)', () => {
// Register it through the controller's private map by re-running
// its #buildIfaceMap path: easier to just stash it into the
// existing iface map directly via TypeScript-defeating cast.
const internalDrivers = (
controller as unknown as {
['#drivers']: Map<string, Map<string, unknown>>;
}
);
const internalDrivers = controller as unknown as {
['#drivers']: Map<string, Map<string, unknown>>;
};
// Access the actual private slot via the well-known getter
// pattern doesn't work for `#`-private fields; instead, re-run
// registerRoutes after stashing on the bag — but that's already
@@ -413,13 +404,12 @@ describe('DriverController.#handleCall (via captured router)', () => {
>;
const original = kv.set;
kv.set = async function (...args: unknown[]) {
const { Context } = await import(
'../../core/context.js'
);
const { Context } = await import('../../core/context.js');
Context.set('driverMetadata', { providerUsed: 'kv-direct' });
return (
original as (...x: unknown[]) => Promise<unknown>
).apply(this, args);
return (original as (...x: unknown[]) => Promise<unknown>).apply(
this,
args,
);
};
try {
const res = makeRes();
@@ -432,11 +422,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
res as unknown as Response,
() => {},
),
routes['POST /call'](req, res as unknown as Response, () => {}),
);
const body = res.body as {
metadata?: Record<string, unknown>;
@@ -497,31 +483,11 @@ describe('DriverController.#handleListInterfaces', () => {
});
});
describe('DriverController.#handleXd', () => {
let routes: Captured;
beforeAll(() => {
routes = captureRoutes(controller);
});
it('serves the iframe-bridge HTML with text/html content-type', () => {
const res = makeRes();
routes['GET /xd'](
makeReq(),
res as unknown as Response,
() => {},
);
expect(res.contentType).toBe('text/html');
expect(res.sentBody).toMatch(/<!DOCTYPE html>/);
expect(res.sentBody).toMatch(/\/drivers\/call/);
});
});
describe('DriverController.registerRoutes', () => {
it('registers POST /call, GET /list-interfaces, and GET /xd', () => {
it('registers POST /call and GET /list-interfaces', () => {
const routes = captureRoutes(controller);
expect(routes['POST /call']).toBeInstanceOf(Function);
expect(routes['GET /list-interfaces']).toBeInstanceOf(Function);
expect(routes['GET /xd']).toBeInstanceOf(Function);
});
});
@@ -26,8 +26,6 @@ import {
checkDriverRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import type { WithLifecycle } from '../../types';
import type { DriverMeta } from '../../drivers/meta.js';
import {
isDriverStreamResult,
@@ -35,120 +33,12 @@ import {
resolveDriverMethodConcurrent,
resolveDriverMethodRateLimit,
} from '../../drivers/meta.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import type { WithLifecycle } from '../../types';
import { PuterController } from '../types.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
// -- /drivers/xd payload ---------------------------------------------
//
// Self-contained HTML/JS shipped to the iframe consumer. Listens for
// postMessage events shaped as `{ id, interface, method, params }`,
// forwards to `/drivers/call`, and posts `{ id, result }` back to the
// originating window.
//
// Wire-shape note: the postMessage uses `params` (puter-js's historical
// name) but `/drivers/call` expects `args`; this bridge translates.
const XD_SCRIPT = /* js */ `
(function () {
const call = async ({ interface_name, method_name, params }) => {
const response = await fetch('/drivers/call', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
interface: interface_name,
method: method_name,
args: params,
}),
});
return await response.json();
};
const fcall = async ({ interface_name, method_name, params }) => {
const form = new FormData();
form.append('interface', interface_name);
form.append('method', method_name);
for (const k in params) {
form.append(k, params[k]);
}
const response = await fetch('/drivers/call', {
method: 'POST',
body: form,
});
return await response.json();
};
window.addEventListener('message', async (event) => {
const { id, interface: iface, method, params } = event.data || {};
let has_file = false;
for (const k in params) {
if (params[k] instanceof File) {
has_file = true;
break;
}
}
const result = has_file
? await fcall({ interface_name: iface, method_name: method, params })
: await call({ interface_name: iface, method_name: method, params });
if (event.source) {
event.source.postMessage({ id, result }, event.origin);
}
});
})();
`;
const XD_HTML = `<!DOCTYPE html>
<html>
<head>
<title>Puter Driver API</title>
<script>
document.addEventListener('DOMContentLoaded', function () {
${XD_SCRIPT}
});
</script>
</head>
<body></body>
</html>`;
// -- Controller ------------------------------------------------------
/**
* Routes driver RPC calls through a unified HTTP surface.
*
* - `POST /drivers/call` — invoke `<iface>.<method>(args)` on the
* registered driver after a per-actor permission + rate-limit check.
* Stream-shaped results are piped directly; everything else is
* returned as JSON.
* - `GET /drivers/list-interfaces` — enumerate registered driver
* interfaces with their default + alternate implementations.
* - `GET /drivers/xd` — legacy iframe bridge; serves an HTML page that
* proxies `postMessage` RPCs to `/drivers/call` on the same origin.
*
* Holds an internal iface → driverName → instance map built at
* construction time from `this.drivers`. Extensions that register
* additional drivers end up in that bag before this controller is
* instantiated, so they show up here automatically.
*/
/**
* Catch-all upstream-error translator for the driver boundary.
*
* Drivers that wrap a third-party SDK (OpenAI, Anthropic, etc.) often
* let the SDK's own error class bubble — those carry an HTTP `.status`
* but are plain `Error` subclasses, not `HttpError`s, so they would
* otherwise hit the global error handler as unexpected 500s and page
* PagerDuty. Repackage them with `upstream_*` legacy codes so the
* alarm gate (server.ts) treats them as upstream failures and only
* pages on the two we actually care about (rate-limit / auth).
*
* Status extraction covers the shapes we've seen in the wild:
* - `.status` / `.statusCode` (OpenAI / Anthropic / Together / Google GenAI)
* - `.response.status` (Replicate's `ApiError`)
* - `.$metadata.httpStatusCode` (AWS SDK v3, e.g. Polly)
* - status sniffed from the message string (last resort, for providers
* that re-wrap their SDK error in `new Error(msg)` and lose the field)
*
* `HttpError`s thrown by drivers pass through untouched.
*/
const extractUpstreamStatus = (e: {
status?: number;
statusCode?: number;
@@ -273,11 +163,6 @@ export class DriverController extends PuterController {
{ subdomain: 'api', requireAuth: true },
this.#handleListInterfaces,
);
router.get(
'/xd',
{ subdomain: 'api', requireAuth: true },
this.#handleXd,
);
}
// -- Handlers ----------------------------------------------------
@@ -489,11 +374,6 @@ export class DriverController extends PuterController {
res.json(out);
};
#handleXd = (_req: Request, res: Response): void => {
res.type('text/html');
res.send(XD_HTML);
};
// -- Internals ---------------------------------------------------
#buildIfaceMap(): void {
@@ -953,6 +953,82 @@ describe('FSController.mkdirEntry', () => {
expect(body.isDir).toBe(true);
});
it('dedupes an existing directory when dedupe_name is true', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/hello`;
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target, dedupe_name: true },
actor,
}),
res,
),
);
const body = captured.body as {
path: string;
name: string;
isDir: boolean;
};
expect(body.path).toBe(`/${username}/Documents/hello (1)`);
expect(body.name).toBe('hello (1)');
expect(body.isDir).toBe(true);
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/hello (1)`,
),
).toMatchObject({ isDir: true });
});
it('requires parent write when deduping an existing directory', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const target = `/${username}/AppData/${appUid}`;
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target },
actor: userActor,
}),
makeRes().res,
),
);
await expect(
withActor(appActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target, dedupe_name: true },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/AppData/${appUid} (1)`,
),
).toBeNull();
});
it('expands ~/ in the path to the user home', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
+15 -2
View File
@@ -1039,13 +1039,15 @@ export class FSController extends PuterController {
legacyCode: 'bad_request',
});
const dedupeName =
this.#toBoolean(body.dedupe_name ?? body.dedupeName) ?? false;
await this.#assertCanCreate(actor, path);
if (dedupeName) await this.#assertCanDedupeCreate(actor, path);
const entry = await this.services.fs.mkdir(userId, {
path,
overwrite: this.#toBoolean(body.overwrite) ?? false,
dedupeName:
this.#toBoolean(body.dedupe_name ?? body.dedupeName) ?? false,
dedupeName,
createMissingParents:
this.#toBoolean(
body.create_missing_parents ??
@@ -1310,6 +1312,17 @@ export class FSController extends PuterController {
await this.#assertAccess(actor, parentForCheck, 'write');
}
async #assertCanDedupeCreate(actor: Actor, targetPath: string) {
const existing = await this.stores.fsEntry.getEntryByPath(targetPath);
if (!existing) return;
const parent = pathPosix.dirname(targetPath);
await this.#assertAccess(
actor,
parent === '/' ? targetPath : parent,
'write',
);
}
async #assertAccess(
actor: Actor,
path: string,
@@ -206,6 +206,78 @@ describe('LegacyFSController.mkdir', () => {
expect(fetched?.isDir).toBe(true);
});
it('dedupes an existing directory when dedupe_name is true', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const parent = `/${username}/Documents`;
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { parent, path: 'hello' },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { parent, path: 'hello', dedupe_name: true },
actor,
}),
res,
),
);
const body = captured.body as Record<string, unknown>;
expect(body).toMatchObject({
path: `${parent}/hello (1)`,
name: 'hello (1)',
is_dir: true,
});
expect(
await server.stores.fsEntry.getEntryByPath(`${parent}/hello (1)`),
).toMatchObject({ isDir: true });
});
it('requires parent write when deduping an existing directory', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const parent = `/${username}/AppData`;
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: { parent, path: appUid },
actor: userActor,
}),
makeRes().res,
),
);
await expect(
withActor(appActor, () =>
controller.mkdir(
makeReq({
body: { parent, path: appUid, dedupe_name: true },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
expect(
await server.stores.fsEntry.getEntryByPath(
`${parent}/${appUid} (1)`,
),
).toBeNull();
});
it("rejects writing into another user's home with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
@@ -1010,6 +1082,76 @@ describe('LegacyFSController.sign', () => {
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('refuses an app actor signing for a different app (403)', async () => {
// An app-under-user actor may only mint a token for its own app;
// requesting a different app's UID is rejected.
const { actor: userActor } = await makeUser();
const targetApp = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `victim-${uuidv4()}`,
title: 'Victim app',
index_url: 'https://example.test/victim.html',
},
{ ownerUserId: userActor.user!.id! },
);
const attackerActor: Actor = {
...userActor,
app: { uid: `attacker-${uuidv4()}` },
};
const { res } = makeRes();
await expect(
withActor(attackerActor, () =>
controller.sign(
makeReq({
body: {
items: [{}],
app_uid: targetApp.uid,
},
actor: attackerActor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 403, legacyCode: 'forbidden' });
});
it('lets an app actor sign for its own app', async () => {
const { actor: userActor } = await makeUser();
const ownApp = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `self-${uuidv4()}`,
title: 'Self app',
index_url: 'https://example.test/self.html',
},
{ ownerUserId: userActor.user!.id! },
);
const appActor: Actor = { ...userActor, app: { uid: ownApp.uid } };
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.sign(
makeReq({
body: { items: [{}], app_uid: ownApp.uid },
actor: appActor,
}),
res,
),
);
const body = captured.body as { token?: string };
expect(typeof body.token).toBe('string');
});
});
// ── writeFile (validation paths) ────────────────────────────────────
@@ -1089,6 +1231,85 @@ describe('LegacyFSController.openItem', () => {
});
});
// ── openItem: write_url stripping for read-only callers ─────────────
//
// Mirrors `/sign` and `/readdir`, which already strip `write_url` when the
// caller only proved read. Without these gates, /open_item handed out a
// valid write signature to read-only sharees — `/writeFile`'s own ACL
// re-check would still reject the write, but the leak shape (a signed
// write URL escaping the access boundary) is the same one those other
// endpoints already defend against.
describe('LegacyFSController.openItem (write_url stripping)', () => {
beforeAll(() => {
(
controller as unknown as { config: { api_base_url?: string } }
).config.api_base_url = 'http://api.test.local';
});
it('returns write_url for the owner (who has write)', async () => {
const { actor } = await makeUser();
const target = `/${actor.user!.username}/Documents/owned.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor }),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.openItem(
makeReq({ body: { path: target }, actor }),
res,
),
);
const body = captured.body as {
signature: { read_url?: string; write_url?: string };
};
expect(body.signature.read_url).toBeDefined();
expect(body.signature.write_url).toBeDefined();
expect(body.signature.write_url).toContain('writeFile');
});
it('strips write_url for a read-only sharee (the fix)', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const target = `/${victim.actor.user!.username}/Documents/shared-ro.txt`;
await withActor(victim.actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor: victim.actor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
await server.services.permission.grantUserUserPermission(
victim.actor,
attacker.actor.user!.username!,
`fs:${entry!.uuid}:read`,
{},
);
const { res, captured } = makeRes();
await withActor(attacker.actor, () =>
controller.openItem(
makeReq({
body: { uid: entry!.uuid },
actor: attacker.actor,
}),
res,
),
);
const body = captured.body as {
signature: { read_url?: string; write_url?: string };
};
// Sharee still gets read_url (they have read).
expect(body.signature.read_url).toBeDefined();
// …but write_url is stripped (they don't have write).
expect(body.signature.write_url).toBeUndefined();
});
});
// ── requestAppRootDir ───────────────────────────────────────────────
describe('LegacyFSController.requestAppRootDir', () => {
@@ -427,17 +427,33 @@ export class LegacyFSController extends PuterController {
const normalizedTarget = targetPath.startsWith('/')
? targetPath
: `/${targetPath}`;
const dedupeName =
getBoolean(body, 'dedupe_name', 'change_name') ?? false;
await assertCanCreate(
this.services.acl,
this.services.fs,
actor,
normalizedTarget,
);
if (dedupeName) {
const existing =
await this.stores.fsEntry.getEntryByPath(normalizedTarget);
if (existing) {
const parent = pathPosix.dirname(normalizedTarget);
await assertAccess(
this.services.acl,
this.services.fs,
actor,
parent === '/' ? normalizedTarget : parent,
'write',
);
}
}
const entry = await this.services.fs.mkdir(userId, {
path: targetPath,
overwrite: getBoolean(body, 'overwrite') ?? false,
dedupeName: getBoolean(body, 'dedupe_name', 'change_name') ?? false,
dedupeName,
createMissingParents:
getBoolean(
body,
@@ -1391,6 +1407,20 @@ export class LegacyFSController extends PuterController {
'read',
);
// Downgrade the envelope when the caller only proved read.
// `/writeFile`'s ACL re-check would still block the write, but
// returning `write_url` to a read-only caller is the same
// privilege-leak shape that `/sign` and `/readdir` strip.
const writeOk = await this.services.acl.check(
actor,
{
path: entry.path,
resolveAncestors: () =>
this.services.fs.getAncestorChain(entry.path),
},
'write',
);
const suggested =
(await this.services.suggestedApps?.getSuggestedApps({
name: entry.name,
@@ -1417,7 +1447,13 @@ export class LegacyFSController extends PuterController {
}
const signingCfg = signingConfigFromAppConfig(this.config);
const signature = { ...signEntry(entry, signingCfg), path: entry.path };
const signed = signEntry(entry, signingCfg);
const signature = writeOk
? { ...signed, path: entry.path }
: (() => {
const { write_url: _, ...rest } = signed;
return { ...rest, path: entry.path };
})();
res.json({
signature,
token,
@@ -165,7 +165,7 @@ export class StaticPagesController extends PuterController {
const domain = this.config.domain ?? req.hostname;
const origin = `${req.protocol}://${domain}`;
const apps = (await this.clients.db.read(
'SELECT `name` FROM `apps` WHERE `approved_for_listing` = 1',
`SELECT \`name\` FROM \`apps\` WHERE \`approved_for_listing\` = ${this.clients.db.booleanLiteral(true)}`,
)) as Array<{ name: string }>;
const urls = [
`<url><loc>${req.protocol}://docs.${domain}/</loc></url>`,
@@ -252,7 +252,7 @@ export class StaticPagesController extends PuterController {
const [dupe] = (await this.clients.db.read(
`SELECT EXISTS(
SELECT 1 FROM \`user\` WHERE (\`email\` = ? OR \`clean_email\` = ?)
AND \`email_confirmed\` = 1
AND \`email_confirmed\` = ${this.clients.db.booleanLiteral(true)}
AND \`password\` IS NOT NULL
) AS email_exists`,
[user.email, cleanEmail],
@@ -243,7 +243,9 @@ export async function resolveOwnedAppForHostedSite(opts: {
const uniqueCandidates = [...new Set(urlCandidates)];
if (uniqueCandidates.length === 0) return null;
const privateFilter = opts.requirePrivate ? 'AND `is_private` = 1 ' : '';
const privateFilter = opts.requirePrivate
? `AND \`is_private\` = ${opts.db.booleanLiteral(true)} `
: '';
const placeholders = uniqueCandidates.map(() => '?').join(', ');
const rows = await opts.db.read(
`SELECT * FROM apps WHERE owner_user_id = ? ${privateFilter}AND index_url IN (${placeholders}) LIMIT 2`,
@@ -43,6 +43,7 @@ import { XAIProvider } from './providers/xai/XAIProvider.js';
import { ZAIProvider } from './providers/zai/ZAIProvider.js';
import { AlibabaProvider } from './providers/alibaba/AlibabaProvider.js';
import { MoonshotProvider } from './providers/moonshot/MoonshotProvider.js';
import { MiniMaxProvider } from './providers/minimax/MiniMaxProvider.js';
import type {
IChatCompleteResult,
IChatModel,
@@ -869,6 +870,18 @@ export class ChatCompletionDriver extends PuterDriver {
);
}
const minimax = providers['minimax'];
const minimaxKey = readKey(minimax);
if (minimaxKey) {
this.#providers['minimax'] = new MiniMaxProvider(
{
apiKey: minimaxKey,
apiBaseUrl: minimax?.apiBaseUrl as string | undefined,
},
metering,
);
}
const zai = providers['zai'];
const zaiKey = readKey(zai);
if (zaiKey) {
@@ -229,10 +229,13 @@ export class ClaudeProvider implements IChatProvider {
reasoningEffort: requestedReasoningEffort,
maxTokens: max_tokens,
});
// Opus 4.7 errors on non-default sampling params; omit temperature entirely.
// Opus 4.7 and 4.8 error on non-default sampling params; omit temperature entirely.
// Other models require temperature=1 when thinking is enabled.
const isOpus47 = modelUsed.id === 'claude-opus-4-7';
const resolvedTemperature = isOpus47
const omitsTemperature = [
'claude-opus-4-7',
'claude-opus-4-8',
].includes(modelUsed.id);
const resolvedTemperature = omitsTemperature
? undefined
: thinkingConfig
? 1
@@ -21,6 +21,37 @@ import type { IChatModel } from '../../types.js';
// Hardcoded from https://models.dev/api.json
export const CLAUDE_MODELS: IChatModel[] = [
{
puterId: 'anthropic:anthropic/claude-opus-4-8',
id: 'claude-opus-4-8',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2026-01',
release_date: '2026-05-28',
aliases: [
'claude-opus',
'claude-opus-latest',
'claude-opus-4-8-latest',
'claude-opus-4.8',
'claude-opus-4-8',
'anthropic/claude-opus-4-8',
],
name: 'Claude Opus 4.8',
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 500,
ephemeral_5m_input_tokens: 500 * 1.25,
ephemeral_1h_input_tokens: 500 * 2,
cache_read_input_tokens: 500 * 0.1,
output_tokens: 2500,
},
context: 1000000,
max_tokens: 128000,
},
{
puterId: 'anthropic:anthropic/claude-opus-4-7',
id: 'claude-opus-4-7',
@@ -30,8 +61,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
knowledge: '2026-01',
release_date: '2026-04-16',
aliases: [
'claude-opus',
'claude-opus-latest',
'claude-opus-4-7-latest',
'claude-opus-4.7',
'claude-opus-4-7',
@@ -0,0 +1,355 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
type MockInstance,
} from 'vitest';
import { SYSTEM_ACTOR } from '../../../../core/actor.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { PuterServer } from '../../../../server.js';
import { setupTestServer } from '../../../../testUtil.js';
import { withTestActor } from '../../../integrationTestUtil.js';
import { MINIMAX_MODELS } from './models.js';
import { MiniMaxProvider } from './MiniMaxProvider.js';
const { createMock, openAICtor } = vi.hoisted(() => {
const createMock = vi.fn();
const openAICtor = vi.fn();
return { createMock, openAICtor };
});
vi.mock('openai', () => {
const OpenAICtor = vi.fn().mockImplementation(function (
this: Record<string, unknown>,
opts: unknown,
) {
openAICtor(opts);
this.chat = { completions: { create: createMock } };
});
return { OpenAI: OpenAICtor, default: { OpenAI: OpenAICtor } };
});
let server: PuterServer;
let recordSpy: MockInstance<MeteringService['utilRecordUsageObject']>;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const makeProvider = () => {
const provider = new MiniMaxProvider(
{ apiKey: 'test-key' },
server.services.metering,
);
return { provider };
};
const asAsyncIterable = <T>(items: T[]): AsyncIterable<T> => ({
async *[Symbol.asyncIterator]() {
for (const item of items) {
yield item;
}
},
});
beforeEach(() => {
createMock.mockReset();
openAICtor.mockReset();
recordSpy = vi.spyOn(server.services.metering, 'utilRecordUsageObject');
});
afterEach(() => {
vi.restoreAllMocks();
});
describe('MiniMaxProvider construction', () => {
it('points the OpenAI SDK at the MiniMax base URL with the configured key', () => {
makeProvider();
expect(openAICtor).toHaveBeenCalledTimes(1);
expect(openAICtor).toHaveBeenCalledWith({
apiKey: 'test-key',
baseURL: 'https://api.minimax.io/v1',
});
});
it('allows overriding the MiniMax API base URL', () => {
new MiniMaxProvider(
{
apiKey: 'test-key',
apiBaseUrl: 'https://example.test/v1',
},
server.services.metering,
);
expect(openAICtor).toHaveBeenCalledWith({
apiKey: 'test-key',
baseURL: 'https://example.test/v1',
});
});
});
describe('MiniMaxProvider model catalog', () => {
it('returns minimax-m2.7 as the default', () => {
const { provider } = makeProvider();
expect(provider.getDefaultModel()).toBe('minimax-m2.7');
});
it('exposes the static MINIMAX_MODELS list verbatim from models()', () => {
const { provider } = makeProvider();
expect(provider.models()).toBe(MINIMAX_MODELS);
});
it('list() flattens canonical ids and aliases', () => {
const { provider } = makeProvider();
const ids = provider.list();
for (const model of MINIMAX_MODELS) {
expect(ids).toContain(model.id);
for (const alias of model.aliases ?? []) {
expect(ids).toContain(alias);
}
}
expect(ids).toContain('minimax-m2.7');
expect(ids).toContain('MiniMax-M2.7');
expect(ids).toContain('minimax/minimax-m2.7');
});
it('uses MiniMax completion-token limits instead of the context window', () => {
for (const model of MINIMAX_MODELS) {
expect(model.context).toBe(204_800);
expect(model.max_tokens).toBe(196_608);
}
});
});
describe('MiniMaxProvider.complete request shape', () => {
const baseCompletion = {
choices: [
{
message: { content: 'hi', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 1, completion_tokens: 1 },
};
it('sends the case-sensitive upstream MiniMax model id', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hello' }],
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.model).toBe('MiniMax-M2.7');
expect(args.messages).toEqual([{ role: 'user', content: 'hello' }]);
expect(args.max_tokens).toBe(1000);
});
it('resolves the upstream-cased alias to the canonical API model', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'MiniMax-M2.7-highspeed',
messages: [{ role: 'user', content: 'hello' }],
}),
);
expect(createMock.mock.calls[0]![0].model).toBe(
'MiniMax-M2.7-highspeed',
);
expect(recordSpy).toHaveBeenCalledWith(
expect.any(Object),
expect.anything(),
'minimax:minimax-m2.7-highspeed',
expect.any(Object),
);
});
it('passes standard chat completion options through', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
const tools = [
{
type: 'function',
function: {
name: 'lookup',
parameters: { type: 'object', properties: {} },
},
},
];
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
tools,
tool_choice: 'auto',
max_tokens: 256,
temperature: 0.4,
top_p: 0.9,
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.tools).toBe(tools);
expect(args.tool_choice).toBe('auto');
expect(args.max_tokens).toBe(256);
expect(args.temperature).toBe(0.4);
expect(args.top_p).toBe(0.9);
});
it('clamps oversized max_tokens to the MiniMax completion limit', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.1',
messages: [{ role: 'user', content: 'hi' }],
max_tokens: 200_000,
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.model).toBe('MiniMax-M2.1');
expect(args.max_tokens).toBe(196_608);
});
it('only sets stream_options.include_usage when streaming', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
stream: false,
}),
);
const [nonStreamArgs] = createMock.mock.calls[0]!;
expect(nonStreamArgs.stream).toBe(false);
expect('stream_options' in nonStreamArgs).toBe(false);
createMock.mockReturnValueOnce(asAsyncIterable([]));
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
stream: true,
}),
);
const [streamArgs] = createMock.mock.calls[1]!;
expect(streamArgs.stream).toBe(true);
expect(streamArgs.stream_options).toEqual({ include_usage: true });
});
});
describe('MiniMaxProvider.complete output and metering', () => {
it('returns the first choice and records MiniMax usage with model costs', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: { content: 'hi there', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: {
prompt_tokens: 100,
completion_tokens: 50,
prompt_tokens_details: { cached_tokens: 10 },
},
});
const result = await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
}),
);
expect(result).toMatchObject({
message: { content: 'hi there', role: 'assistant' },
finish_reason: 'stop',
});
expect((result as { usage: unknown }).usage).toEqual({
prompt_tokens: 100,
completion_tokens: 50,
cached_tokens: 10,
});
expect(recordSpy).toHaveBeenCalledTimes(1);
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
expect(usage).toEqual({
prompt_tokens: 100,
completion_tokens: 50,
cached_tokens: 10,
});
expect(actor).toBe(SYSTEM_ACTOR);
expect(prefix).toBe('minimax:minimax-m2.7');
expect(overrides).toEqual({
prompt_tokens: 100 * 30,
completion_tokens: 50 * 120,
cached_tokens: 10 * 6,
});
});
it('rethrows errors raised by the OpenAI client unchanged', async () => {
const { provider } = makeProvider();
const apiError = new Error('MiniMax exploded');
createMock.mockRejectedValueOnce(apiError);
await expect(
withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'boom' }],
}),
),
).rejects.toBe(apiError);
expect(recordSpy).not.toHaveBeenCalled();
});
});
describe('MiniMaxProvider.checkModeration', () => {
it('throws because MiniMax provider does not implement moderation', () => {
const { provider } = makeProvider();
expect(() => provider.checkModeration('anything')).toThrow(
/not implemented/i,
);
});
});
@@ -0,0 +1,135 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { OpenAI } from 'openai';
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
import { Context } from '../../../../core/context.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import type { IChatProvider, ICompleteArguments } from '../../types.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
import { MINIMAX_MODELS } from './models.js';
type MiniMaxConfig = {
apiKey: string;
apiBaseUrl?: string;
};
export class MiniMaxProvider implements IChatProvider {
#openai: OpenAI;
#meteringService: MeteringService;
#defaultModel = 'minimax-m2.7';
constructor(config: MiniMaxConfig, meteringService: MeteringService) {
this.#openai = new OpenAI({
apiKey: config.apiKey,
baseURL: config.apiBaseUrl ?? 'https://api.minimax.io/v1',
});
this.#meteringService = meteringService;
}
getDefaultModel() {
return this.#defaultModel;
}
models() {
return MINIMAX_MODELS;
}
list() {
const modelIds: string[] = [];
for (const model of this.models()) {
modelIds.push(model.id);
if (model.aliases) {
modelIds.push(...model.aliases);
}
}
return modelIds;
}
async complete({
messages,
stream,
model,
tools,
tool_choice,
max_tokens,
temperature,
top_p,
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
const actor = Context.get('actor');
const availableModels = this.models();
const modelUsed =
availableModels.find((m) =>
[m.id, ...(m.aliases || [])].includes(model),
) || availableModels.find((m) => m.id === this.getDefaultModel())!;
messages = await OpenAIUtil.process_input_messages(messages);
const requestedMaxTokens = max_tokens ?? 1000;
const completion = await this.#openai.chat.completions.create({
messages,
model: modelUsed.apiModel,
...(tools ? { tools } : {}),
...(tool_choice !== undefined ? { tool_choice } : {}),
max_tokens: Math.min(requestedMaxTokens, modelUsed.max_tokens),
...(temperature !== undefined ? { temperature } : {}),
...(top_p !== undefined ? { top_p } : {}),
stream,
...(stream
? {
stream_options: { include_usage: true },
}
: {}),
} as ChatCompletionCreateParams);
return OpenAIUtil.handle_completion_output({
usage_calculator: ({ usage }) => {
const trackedUsage = usage
? OpenAIUtil.extractMeteredUsage(usage)
: {
prompt_tokens: 0,
completion_tokens: 0,
cached_tokens: 0,
};
const costsOverride = Object.fromEntries(
Object.entries(trackedUsage).map(([key, value]) => {
return [key, value * Number(modelUsed.costs[key] ?? 0)];
}),
);
this.#meteringService.utilRecordUsageObject(
trackedUsage,
actor!,
`minimax:${modelUsed.id}`,
costsOverride,
);
return trackedUsage;
},
stream,
completion,
});
}
checkModeration(
_text: string,
): ReturnType<IChatProvider['checkModeration']> {
throw new Error('Method not implemented.');
}
}
@@ -0,0 +1,77 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { IChatModel } from '../../types.js';
import { usdPerMToken } from '../../utils/pricing.js';
const CONTEXT_WINDOW = 204_800;
const MAX_OUTPUT_TOKENS = 196_608;
type MiniMaxChatModel = IChatModel & {
apiModel: string;
};
const minimaxModel = (
apiModel: string,
name: string,
costs: IChatModel['costs'],
): MiniMaxChatModel => {
const id = apiModel.toLowerCase();
return {
puterId: `minimax:minimax/${id}`,
id,
apiModel,
name,
aliases: [`minimax/${id}`, apiModel, `minimax/${apiModel}`],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: CONTEXT_WINDOW,
max_tokens: MAX_OUTPUT_TOKENS,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs,
};
};
// Hardcoded from MiniMax OpenAI-compatible API docs and pay-as-you-go pricing:
// https://platform.minimax.io/docs/api-reference/text-openai-api
// https://platform.minimax.io/docs/guides/pricing-paygo
export const MINIMAX_MODELS: MiniMaxChatModel[] = [
minimaxModel('MiniMax-M2.7', 'MiniMax M2.7', usdPerMToken(0.3, 1.2, 0.06)),
minimaxModel(
'MiniMax-M2.7-highspeed',
'MiniMax M2.7 Highspeed',
usdPerMToken(0.6, 2.4, 0.06),
),
minimaxModel('MiniMax-M2.5', 'MiniMax M2.5', usdPerMToken(0.3, 1.2, 0.03)),
minimaxModel(
'MiniMax-M2.5-highspeed',
'MiniMax M2.5 Highspeed',
usdPerMToken(0.6, 2.4, 0.03),
),
minimaxModel('MiniMax-M2.1', 'MiniMax M2.1', usdPerMToken(0.3, 1.2, 0.03)),
minimaxModel(
'MiniMax-M2.1-highspeed',
'MiniMax M2.1 Highspeed',
usdPerMToken(0.6, 2.4, 0.03),
),
minimaxModel('MiniMax-M2', 'MiniMax M2', usdPerMToken(0.3, 1.2, 0.03)),
];
@@ -21,8 +21,8 @@ import { Together } from 'together-ai';
import { Context } from '../../../../core/context.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { kv } from '../../../../util/kvSingleton.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
import { IChatModel, IChatProvider, ICompleteArguments } from '../../types.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
const TOGETHER_AI_CHAT_COST_MAP = {
prompt_tokens: 'input',
@@ -51,7 +51,9 @@ export class TogetherAIProvider implements IChatProvider {
let models: IChatModel[] | undefined = kv.get(this.#kvKey);
if (models) return models;
const apiModels = await this.#together.models.list();
const apiModels = await this.#together.models.list({
query: { serverless: 'true' },
});
models = [];
for (const model of apiModels) {
if (
@@ -18,22 +18,10 @@
*/
import type { IChatModel } from '../../types.js';
import { usdPerMToken } from '../../utils/pricing.js';
const CENTS_PER_USD = 100;
const MTOK = 1_000_000;
const K = 1_000;
const usdPerMToken = (
inputUsd: number,
outputUsd: number,
cachedInputUsd = 0,
) => ({
tokens: MTOK,
prompt_tokens: inputUsd * CENTS_PER_USD,
completion_tokens: outputUsd * CENTS_PER_USD,
cached_tokens: cachedInputUsd * CENTS_PER_USD,
});
const textModel = (
id: string,
name: string,
@@ -0,0 +1,40 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { ModelCost } from '../types.js';
const CENTS_PER_USD = 100;
const MTOK = 1_000_000;
/**
* Builds a `costs` block (currency `usd-cents`, per million tokens) from
* per-million-token USD prices. Providers list pricing in USD/MTok, so this
* keeps the source numbers readable while emitting the cents-based shape the
* driver expects.
*/
export const usdPerMToken = (
inputUsd: number,
outputUsd: number,
cachedReadUsd = 0,
): ModelCost => ({
tokens: MTOK,
prompt_tokens: inputUsd * CENTS_PER_USD,
completion_tokens: outputUsd * CENTS_PER_USD,
cached_tokens: cachedReadUsd * CENTS_PER_USD,
});
@@ -146,7 +146,10 @@ export class ImageGenerationDriver extends PuterDriver {
await this.#assertWriteAccess(actor, resolvedOutputPath);
}
let modelId = args.model?.trim().toLowerCase();
let modelId =
typeof args.model === 'string'
? args.model.trim().toLowerCase()
: undefined;
let intendedProvider =
args.provider ?? (Context.get('driverName') as string | undefined);
+83
View File
@@ -0,0 +1,83 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterEach, describe, expect, it } from 'vitest';
import { extension } from './extensions.ts';
import { clientsContainers } from './exports.ts';
/**
* The `extension.import('client')` proxy does NOT return `undefined` for
* clients that were never registered — it hands back a placeholder proxy that
* throws on property access. Extensions reaching for an OPTIONAL client (e.g.
* the ClickHouse analytics client, absent on plain self-hosts) must therefore
* probe for a real method behind a try/catch rather than trusting truthiness.
*
* These tests lock it so a future proxy change can't silently
* reintroduce the "truthy-but-throws" footgun.
*/
describe('extension.import("client") optional client access', () => {
const clients = extension.import('client') as Record<
string,
{ query?: unknown } | undefined
>;
afterEach(() => {
delete clientsContainers.notRegistered;
delete clientsContainers.optionalThing;
});
it('returns a truthy-but-throwing placeholder for an unregistered client', () => {
delete clientsContainers.notRegistered;
const placeholder = clients.notRegistered;
// Truthy — so a bare `if (clients.x)` check is NOT safe.
expect(placeholder).toBeTruthy();
// ...and accessing a method on it throws.
expect(() => (placeholder as { query: unknown }).query).toThrow();
});
it('the try/catch probe pattern reports absence safely', () => {
delete clientsContainers.optionalThing;
const probe = () => {
try {
return typeof clients.optionalThing?.query === 'function'
? clients.optionalThing
: null;
} catch {
return null;
}
};
expect(probe()).toBeNull();
});
it('the same probe returns a registered client exposing the method', () => {
const fake = { query: async () => undefined };
clientsContainers.optionalThing =
fake as unknown as (typeof clientsContainers)[string];
const probe = () => {
try {
return typeof clients.optionalThing?.query === 'function'
? clients.optionalThing
: null;
} catch {
return null;
}
};
expect(probe()).toBe(fake);
});
});
+4 -1
View File
@@ -56,11 +56,12 @@
"lorem-ipsum": "^2.0.8",
"mime-types": "^2.1.35",
"murmurhash": "^2.0.1",
"mysql2": "^3.21.1",
"mysql2": "^3.22.4",
"nodemailer": "^8.0.7",
"openai": "^6.34.0",
"otpauth": "^9.2.4",
"parse-domain": "^8.2.2",
"pg": "^8.21.0",
"prompt-sync": "^4.2.0",
"replicate": "^1.0.0",
"sharp": "^0.34.5",
@@ -77,9 +78,11 @@
"@types/busboy": "^1.5.4",
"@types/jsonwebtoken": "^9.0.10",
"@types/node": "^24.0.0",
"@types/pg": "^8.6.1",
"@types/validator": "^13.15.10",
"chai": "^4.3.7",
"nodemon": "^3.1.0",
"pgmock": "^1.0.3",
"typescript": "^5.9.3",
"vite": "^8.0.0",
"vitest": "^4.0.14"
+2
View File
@@ -601,6 +601,8 @@ export class PuterServer {
'Content-Type',
'Accept',
'Authorization',
'Cache-Control',
'Pragma',
'sentry-trace',
'baggage',
'Depth',
@@ -32,6 +32,7 @@ const RECOMMENDED_APP_NAMES = [
'camera',
'music-player',
'recorder',
'browser',
'memos',
'word-processor',
'spreadsheet',
@@ -23,6 +23,7 @@ import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import { AuthService } from './AuthService.js';
function createAuthService(): AuthService {
@@ -1181,6 +1182,24 @@ describe('AuthService (integration)', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('createWorkerAppToken refuses an app actor targeting a different app (403)', async () => {
const user = await makeUser();
const actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
app: { uid: `app-${uuidv4()}` },
} as Actor;
await expect(
authService.createWorkerAppToken(
actor,
`app-${uuidv4()}`,
'wk-x',
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
// ── Revocation flow ────────────────────────────────────────
it('revokeSession on a worker session — authenticate returns reauth.session_revoked', async () => {
@@ -1298,6 +1317,20 @@ describe('AuthService (integration)', () => {
expect(a).toBe(b);
expect(a).toMatch(/^app-/);
});
it.each([
'javascript:alert(document.domain)',
'data:text/html,<script>alert(1)</script>',
'file:///etc/passwd',
'vbscript:msgbox(1)',
])('throws 400 for non-http(s) scheme %s', async (origin) => {
// These parse fine via `new URL()` but must never become a
// bootstrap app `index_url` — that would be a stored XSS /
// code-execution vector when launched as `iframe.src`.
await expect(
authService.appUidFromOrigin(origin),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('getUserAppToken', () => {
@@ -1347,6 +1380,67 @@ describe('AuthService (integration)', () => {
// same app session row.
expect(decodedFirst.session_uid).toBe(decodedSecond.session_uid);
});
// Delegation scope: a scoped actor (app-under-user or access-token)
// may only mint a token for its own app; only a root user session
// may request a token for an arbitrary app.
it('lets an app actor mint a token for its own app', async () => {
const user = await makeUser();
const ownApp = `app-${uuidv4()}`;
const actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
app: { uid: ownApp },
} as Actor;
const token = await authService.getUserAppToken(actor, ownApp);
const decoded = server.services.token.verify('auth', token) as {
app_uid: string;
};
expect(decoded.app_uid).toBe(ownApp);
});
it('refuses an app actor minting a token for a different app (403)', async () => {
const user = await makeUser();
const actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
app: { uid: `app-${uuidv4()}` },
} as Actor;
await expect(
authService.getUserAppToken(actor, `app-${uuidv4()}`),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
it('refuses an access-token actor minting an app token (403)', async () => {
const user = await makeUser();
const issuer = {
user: { id: user.id, uuid: user.uuid, username: user.username },
} as Actor;
const actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
accessToken: { uid: `tok-${uuidv4()}`, issuer, authorized: null },
} as Actor;
await expect(
authService.getUserAppToken(actor, `app-${uuidv4()}`),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
it('lets a root user session mint a token for any app', async () => {
const user = await makeUser();
const actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
} as Actor;
const anyApp = `app-${uuidv4()}`;
const token = await authService.getUserAppToken(actor, anyApp);
const decoded = server.services.token.verify('auth', token) as {
app_uid: string;
};
expect(decoded.app_uid).toBe(anyApp);
});
});
describe('createAccessToken / revokeAccessToken', () => {
@@ -1449,6 +1543,114 @@ describe('AuthService (integration)', () => {
),
).rejects.toMatchObject({ statusCode: 403 });
});
// Regression for the post-#1001 token-inval check: an
// app-under-user actor must be able to mint a token for a file
// inside its own AppData. Without the `app-owns-appdata`
// implicator the issuer-subset gate 403s these — even though
// ACLService already allows the equivalent fs.read via its own
// short-circuit. puter-js getReadURL is the canonical caller.
it('app-under-user actor can mint fs:<uuid>:read for a file inside its own AppData', async () => {
const user = await makeUser();
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
user,
);
const appUid = `app-${uuidv4()}`;
await server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[
appUid,
`n-${appUid}`,
`t-${appUid}`,
`https://${appUid}.example/`,
user.id,
],
);
const appDataPath = `/${user.username}/AppData/${appUid}`;
await server.services.fs.mkdir(user.id, {
path: appDataPath,
createMissingParents: true,
} as never);
const body = Buffer.from('hello');
await server.services.fs.write(user.id, {
fileMetadata: {
path: `${appDataPath}/note.txt`,
size: body.byteLength,
contentType: 'text/plain',
},
fileContent: body,
} as never);
const fileEntry = await server.stores.fsEntry.getEntryByPath(
`${appDataPath}/note.txt`,
);
expect(fileEntry).not.toBeNull();
const appActor: Actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
app: { id: 0, uid: appUid },
} as Actor;
const jwt = await authService.createAccessToken(appActor, [
[`fs:${fileEntry!.uuid}:read`],
]);
expect(typeof jwt).toBe('string');
});
// Negative side of the implicator: a file the user owns but
// that lives *outside* the app's AppData must still be
// rejected — the issuer-subset gate is the only thing
// preventing an authorized app from minting a token over
// arbitrary user-owned uuids.
it('app-under-user actor cannot mint fs:<uuid>:read for a user-owned file outside its AppData', async () => {
const user = await makeUser();
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
user,
);
const appUid = `app-${uuidv4()}`;
await server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[
appUid,
`n-${appUid}`,
`t-${appUid}`,
`https://${appUid}.example/`,
user.id,
],
);
const body = Buffer.from('secret');
await server.services.fs.write(user.id, {
fileMetadata: {
path: `/${user.username}/Documents/secret.txt`,
size: body.byteLength,
contentType: 'text/plain',
},
fileContent: body,
} as never);
const fileEntry = await server.stores.fsEntry.getEntryByPath(
`/${user.username}/Documents/secret.txt`,
);
expect(fileEntry).not.toBeNull();
const appActor: Actor = {
user: { id: user.id, uuid: user.uuid, username: user.username },
app: { id: 0, uid: appUid },
} as Actor;
await expect(
authService.createAccessToken(appActor, [
[`fs:${fileEntry!.uuid}:read`],
]),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
});
describe('private-asset / public hosted-actor tokens', () => {
+28
View File
@@ -357,6 +357,7 @@ export class AuthService extends PuterService {
legacyCode: 'bad_request',
});
}
this.#assertAppDelegationAllowed(actor, appUid);
const auth_id = this.#authIdFor(actor.user as UserRow);
const session = await this.stores.session.getOrCreateWorker(
actor.user.id,
@@ -384,6 +385,22 @@ export class AuthService extends PuterService {
return user.uuid;
}
/**
* Scope app token delegation by actor kind. An app-under-user or
* access-token actor is bound to a single app and may only mint a token
* for that same app; only a root user session may request a token for an
* arbitrary app (the GUI's app-launch delegation).
*/
#assertAppDelegationAllowed(actor: Actor, appUid: string): void {
if ((actor.app || actor.accessToken) && actor.app?.uid !== appUid) {
throw new HttpError(
403,
'Actor cannot mint a token for another app',
{ legacyCode: 'forbidden' },
);
}
}
/**
* Convert a jsonwebtoken-style `expiresIn` (seconds, or `'1h'`/`'30d'`)
* into an absolute unix-seconds timestamp for the session row. Returns
@@ -1057,6 +1074,7 @@ export class AuthService extends PuterService {
throw new HttpError(403, 'Actor must be a user', {
legacyCode: 'forbidden',
});
this.#assertAppDelegationAllowed(actor, appUid);
// Request-context (IP / UA) isn't available on the Actor shape —
// the app row's `last_ip` / `last_user_agent` start NULL and get
@@ -1633,6 +1651,16 @@ export class AuthService extends PuterService {
#originFromUrl(url: string): string | null {
try {
const parsed = new URL(url);
// A real web origin is always http(s). `new URL()` happily parses
// `javascript:`, `data:`, `file:`, `vbscript:`, etc.; if one of
// those slips through it ends up persisted as an app `index_url`
// (see AppStore.createFromOrigin) and later loaded as `iframe.src`
// — an XSS/code-execution primitive. Reject anything that isn't
// http(s) so the bootstrap path matches AppDriver's validateUrl
// allow-list.
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
return null;
}
return `${parsed.protocol}//${parsed.hostname}${parsed.port ? `:${parsed.port}` : ''}`;
} catch {
return null;
+54 -5
View File
@@ -206,6 +206,51 @@ export class FSService extends PuterService {
},
});
// -- app-owns-appdata -----------------------------------------
// Mirror of the ACLService short-circuit at ACLService.check:
// an app-under-user actor implicitly holds fs:<uuid>:* on any
// entry inside its own /<username>/AppData/<appUid> subtree.
// ACL paths (fs.read etc.) already accept these via that
// short-circuit, but permissionService.checkMany — used by
// createAccessToken's issuer-subset gate — bypasses ACL, so
// without this implicator an app can fs.read its appdata but
// can't mint a token for the same fs:<uuid>:read it just read.
permissions.registerImplicator({
id: 'app-owns-appdata',
shortcut: true,
matches: (permission: string): boolean => {
return (
permission.startsWith('fs:') ||
permission.startsWith(`${MANAGE_PERM_PREFIX}:fs:`) ||
permission.startsWith(
`${MANAGE_PERM_PREFIX}:${MANAGE_PERM_PREFIX}:fs:`,
)
);
},
check: async ({ actor, permission }): Promise<unknown> => {
if (!actor.app || actor.accessToken) return undefined;
const username = actor.user?.username;
const appUid = actor.app.uid;
if (!username || !appUid) return undefined;
const stripped = permission.replaceAll(
`${MANAGE_PERM_PREFIX}:`,
'',
);
const uid = PermissionUtil.split(stripped)[1];
if (!uid) return undefined;
const entry = await fsEntryStore.getEntryByUuid(uid);
if (!entry) return undefined;
const root = `/${username}/AppData/${appUid}`;
if (entry.path === root || entry.path.startsWith(`${root}/`)) {
return {};
}
return undefined;
},
});
// -- fs-access-levels exploder ----------------------------------
// `fs:UUID:see` implies `[list, read, write, manage:fs:UUID]`.
// ACLService.check already expands the same-family chain
@@ -2795,10 +2840,13 @@ export class FSService extends PuterService {
const existing = await this.stores.fsEntry.getEntryByPath(targetPath);
if (existing) {
if (existing.isDir) {
// A directory already exists at path: idempotent success.
return existing;
}
if (input.overwrite) {
if (input.dedupeName) {
name = await this.#findDedupedName(parent, name);
} else {
// A directory already exists at path: idempotent success.
return existing;
}
} else if (input.overwrite) {
// Remove the non-directory occupant then create the dir.
await this.remove(userId, {
entry: existing,
@@ -3080,11 +3128,12 @@ export class FSService extends PuterService {
*/
async removeAllForUser(userId: number): Promise<void> {
const pageSize = 5000;
const falseLiteral = this.clients.db.booleanLiteral(false);
// Files-first loop: delete backing S3 objects in batches, then DB rows.
for (;;) {
const files = (await this.clients.db.read(
`SELECT uuid, bucket, bucket_region FROM fsentries
WHERE user_id = ? AND is_dir = 0 AND (is_shortcut = 0 OR is_shortcut IS NULL) AND (is_symlink = 0 OR is_symlink IS NULL)
WHERE user_id = ? AND is_dir = ${falseLiteral} AND (is_shortcut = ${falseLiteral} OR is_shortcut IS NULL) AND (is_symlink = ${falseLiteral} OR is_symlink IS NULL)
LIMIT ${pageSize}`,
[userId],
)) as Array<{
@@ -321,6 +321,115 @@ describe('MeteringService', () => {
});
});
// ── overuse alarm ────────────────────────────────────────────────
describe('overuse alarm', () => {
const wasOveruseAlarmed = (alarmSpy: ReturnType<typeof vi.spyOn>) =>
alarmSpy.mock.calls.some(
(call) =>
typeof call[0] === 'string' &&
call[0].includes('usage exceeded'),
);
it('does not alarm when a single large request crosses the limit in one shot', async () => {
const bigActor: Actor = { user: makeUser() };
const sub = await target.getActorSubscription(bigActor);
const alarmSpy = vi.spyOn(server.clients.alarm, 'create');
// Previous usage was 0 (under the allowance) — one big request that
// blows straight past several multiples is legitimate, not abuse.
await target.incrementUsage(
bigActor,
'ai:chat',
1,
sub.monthUsageAllowance * 5,
);
expect(wasOveruseAlarmed(alarmSpy)).toBe(false);
alarmSpy.mockRestore();
});
it('does not alarm on further usage past the limit until the next multiple is crossed', async () => {
const overActor: Actor = { user: makeUser() };
const sub = await target.getActorSubscription(overActor);
// Take them just over the allowance (into the 1x–2x band).
await target.incrementUsage(
overActor,
'ai:chat',
1,
sub.monthUsageAllowance,
);
// A small further expense stays within the same band — no new
// multiple crossed, so it shouldn't page.
const alarmSpy = vi.spyOn(server.clients.alarm, 'create');
await target.incrementUsage(overActor, 'ai:chat', 1, 1_000);
expect(wasOveruseAlarmed(alarmSpy)).toBe(false);
alarmSpy.mockRestore();
});
it('alarms when a whole multiple of the allowance is crossed while already over', async () => {
const overActor: Actor = { user: makeUser() };
const sub = await target.getActorSubscription(overActor);
// First expense takes them to the limit (1x) — no alarm yet.
await target.incrementUsage(
overActor,
'ai:chat',
1,
sub.monthUsageAllowance,
);
// Spy only on the expense that crosses into 2x while already over.
const alarmSpy = vi.spyOn(server.clients.alarm, 'create');
await target.incrementUsage(
overActor,
'ai:chat',
1,
sub.monthUsageAllowance,
);
expect(alarmSpy).toHaveBeenCalledWith(
expect.stringContaining('usage exceeded'),
expect.stringContaining('exceeded their usage allowance'),
expect.objectContaining({ totalUsage: expect.any(Number) }),
// Non-paging severity — records and de-dupes but doesn't page on-call.
'warning',
);
alarmSpy.mockRestore();
});
it('does not alarm while purchased credits still cover the overage', async () => {
const creditActor: Actor = { user: makeUser() };
const sub = await target.getActorSubscription(creditActor);
await target.updateAddonCredit(
creditActor.user.uuid,
5_000_000_000,
);
// Cross to 2x — would page if not for the credits covering it.
await target.incrementUsage(
creditActor,
'ai:chat',
1,
sub.monthUsageAllowance,
);
const alarmSpy = vi.spyOn(server.clients.alarm, 'create');
await target.incrementUsage(
creditActor,
'ai:chat',
1,
sub.monthUsageAllowance,
);
expect(wasOveruseAlarmed(alarmSpy)).toBe(false);
alarmSpy.mockRestore();
});
});
// ── batchIncrementUsages ─────────────────────────────────────────
describe('batchIncrementUsages', () => {
@@ -184,7 +184,7 @@ export class MeteringService extends PuterService {
PERIOD_ESCAPE,
);
const appId = actor.app?.uid || GLOBAL_APP_KEY;
const userId = actor.user.uuid;
const userId = actor.user.uuid!;
const pathAndAmountMap = {
total: totalCost,
[`${escapedUsageType}.units`]: usageAmount,
@@ -360,7 +360,7 @@ export class MeteringService extends PuterService {
}
const appId = actor.app?.uid || GLOBAL_APP_KEY;
const userId = actor.user.uuid;
const userId = actor.user.uuid!;
const actorUsageKey = `${METRICS_PREFIX}:actor:${userId}:${currentMonth}`;
const actorUsagesPromise = this.stores.kv
@@ -893,20 +893,30 @@ export class MeteringService extends PuterService {
incrementCost,
} = ctx;
const allowedMultiple = Math.floor(
actorUsages.total / actorSubscription.monthUsageAllowance,
);
const previousMultiple = Math.floor(
(actorUsages.total - incrementCost) /
actorSubscription.monthUsageAllowance,
);
const isOver2x = allowedMultiple >= 2;
const crossedThreshold = previousMultiple < allowedMultiple;
const allowance = actorSubscription.monthUsageAllowance;
// No metered allowance to exceed (e.g. unlimited policies) — nothing to flag.
if (!(allowance > 0)) return;
const previousUsage = actorUsages.total - incrementCost;
const allowedMultiple = Math.floor(actorUsages.total / allowance);
const previousMultiple = Math.floor(previousUsage / allowance);
// Only alarm if the actor was ALREADY at or past their allowance before
// this expense arrived. A single large request that jumps past the limit
// in one shot (previous usage still under the allowance) is legitimate
// and shouldn't page.
const wasAlreadyOverLimit = previousUsage >= allowance;
// And only when this expense crosses into a new whole multiple of the
// allowance (2x, 3x, …) rather than on every expense once over — that
// first-over multiple is 2x, since being already over means the previous
// multiple was at least 1.
const crossedMultiple = previousMultiple < allowedMultiple;
const hasNoAddonCredit =
(actorAddons.purchasedCredits || 0) <=
(actorAddons.consumedPurchaseCredits || 0);
if (!(isOver2x && crossedThreshold && hasNoAddonCredit)) return;
if (!(wasAlreadyOverLimit && crossedMultiple && hasNoAddonCredit))
return;
this.clients.alarm.create(
`metering usage exceeded by user: ${actor.user?.username}`,
@@ -923,6 +933,8 @@ export class MeteringService extends PuterService {
totalUsage: actorUsages.total,
monthUsageAllowance: actorSubscription.monthUsageAllowance,
},
// Expected-but-worth-tracking signal — record/de-dupe it but don't page on-call.
'warning',
);
}
+49 -109
View File
@@ -20,6 +20,7 @@
import { v4 as uuidv4 } from 'uuid';
import { PuterStore } from '../types';
import { HttpError } from '../../core/http/HttpError.js';
import { validateUrl } from '../../util/validation.js';
/**
* Persistence + cache for the `apps` table.
@@ -47,11 +48,11 @@ const OLD_APP_NAME_TTL_MONTHS = 3;
// backends without splitting the cap by driver.
const BULK_QUERY_CHUNK_SIZE = 200;
// Top-level all-time open/user counts: hot path, slow to compute, refreshed
// periodically by one instance and read via MGET on every app list/read.
// Top-level all-time open/user counts: hot path, slow to compute. Cached
// lazily on read (pipelined MGET on every app list/read; misses query the
// analytics source once and backfill) and left to autoexpire — these counts
// tolerate being slightly stale, so there's no background refresh.
const STATS_CACHE_TTL_SECONDS = 30 * 60;
const STATS_REFRESH_INTERVAL_MS = 15 * 60 * 1000;
const STATS_REFRESH_LOCK_KEY = 'appStatsLastRefresh';
// Period helpers for detailed/grouped stats. Ported from v1
// AppInformationService — queries go straight to ClickHouse/MySQL on demand
@@ -106,9 +107,18 @@ const READ_ONLY_COLUMNS = new Set([
'protected',
'is_private',
]);
const APP_BOOLEAN_COLUMNS = new Set([
'godmode',
'maximize_on_start',
'approved_for_listing',
'approved_for_opening_items',
'approved_for_incentive_program',
'background',
'protected',
'is_private',
]);
export class AppStore extends PuterStore {
#appStatsInterval;
// -- Reads --------------------------------------------------------
async getByUid(uid) {
@@ -341,7 +351,7 @@ export class AppStore extends PuterStore {
const colList = columns.map((c) => `\`${c}\``).join(', ');
const result = await this.clients.db.write(
`INSERT INTO \`apps\` (${colList}) VALUES (${placeholders})`,
`INSERT INTO \`apps\` (${colList}) VALUES (${placeholders})${this.clients.db.returningIdClause()}`,
values,
);
const insertId = result?.insertId;
@@ -365,6 +375,13 @@ export class AppStore extends PuterStore {
* `#isOriginBootstrapApp` checks.
*/
async createFromOrigin(uid, origin) {
// Bootstrap apps persist `origin` straight into `index_url`, which is
// later loaded as `iframe.src`. Enforce the same http(s) scheme
// allow-list as the AppDriver create/update path so a caller-supplied
// `javascript:`/`data:`/`file:` origin can never become a stored,
// launchable code-execution vector.
validateUrl(origin, { key: 'origin' });
const fields = {
name: uid,
title: uid,
@@ -380,7 +397,7 @@ export class AppStore extends PuterStore {
const colList = columns.map((c) => `\`${c}\``).join(', ');
const result = await this.clients.db.write(
`INSERT INTO \`apps\` (${colList}) VALUES (${placeholders})`,
`INSERT INTO \`apps\` (${colList}) VALUES (${placeholders})${this.clients.db.returningIdClause()}`,
values,
);
const insertId = result?.insertId;
@@ -698,6 +715,7 @@ export class AppStore extends PuterStore {
async #resolveByOldName(name) {
const cutoffClause = this.clients.db.case({
sqlite: `datetime('now', '-${OLD_APP_NAME_TTL_MONTHS} months')`,
postgres: `(NOW() - INTERVAL '${OLD_APP_NAME_TTL_MONTHS} months')`,
otherwise: `(NOW() - INTERVAL ${OLD_APP_NAME_TTL_MONTHS} MONTH)`,
});
@@ -842,7 +860,10 @@ export class AppStore extends PuterStore {
const out = {};
for (const [k, v] of Object.entries(fields)) {
if (READ_ONLY_COLUMNS.has(k)) continue;
out[k] = v;
out[k] =
APP_BOOLEAN_COLUMNS.has(k) && v !== null && v !== undefined
? this.clients.db.booleanValue(Boolean(v))
: v;
}
return out;
}
@@ -976,7 +997,7 @@ export class AppStore extends PuterStore {
const period = options.period ?? 'all';
const grouping = options.grouping;
const timeRange = this.#computeTimeRange(period, options.createdAt);
const clickhouse = globalThis.clickhouseClient;
const clickhouse = this.clients.clickhouse;
if (grouping) {
if (!MYSQL_DATE_FORMATS[grouping]) {
@@ -1003,7 +1024,7 @@ export class AppStore extends PuterStore {
const out = new Map();
if (uids.length === 0) return out;
const clickhouse = globalThis.clickhouseClient;
const clickhouse = this.clients.clickhouse;
if (clickhouse) {
const res = await clickhouse.query({
query: `
@@ -1136,6 +1157,7 @@ export class AppStore extends PuterStore {
const periodExpr = this.clients.db.case({
mysql: `DATE_FORMAT(FROM_UNIXTIME(ts), '${timeFormat}')`,
sqlite: `STRFTIME('${timeFormat}', datetime(ts, 'unixepoch'))`,
postgres: `TO_CHAR(TO_TIMESTAMP(ts), '${this.#postgresDateFormat(grouping)}')`,
otherwise: `DATE_FORMAT(FROM_UNIXTIME(ts), '${timeFormat}')`,
});
const rows = await this.clients.db.read(
@@ -1156,6 +1178,23 @@ export class AppStore extends PuterStore {
return this.#assembleGroupedResult(processed, allPeriods, grouping);
}
#postgresDateFormat(grouping) {
switch (grouping) {
case 'hour':
return 'YYYY-MM-DD HH24:00:00';
case 'day':
return 'YYYY-MM-DD';
case 'week':
return 'YYYY-WW';
case 'month':
return 'YYYY-MM';
case 'year':
return 'YYYY';
default:
return 'YYYY-MM-DD';
}
}
#assembleGroupedResult(rows, allPeriods, grouping) {
// Totals come from raw rows so they survive even if a row's
// period key fails to match an `allPeriods` entry (e.g. week
@@ -1334,103 +1373,4 @@ export class AppStore extends PuterStore {
}
return 1 + Math.ceil((firstThursday - target) / 604800000);
}
// -- Refresh loop -------------------------------------------------
async #refreshAppStats() {
// Cross-instance lock: if another node refreshed within the window,
// skip. TTL slightly longer than the interval so the guard survives
// jitter in the setInterval drift.
try {
const last = parseInt(
(await this.clients.redis.get(STATS_REFRESH_LOCK_KEY)) || '0',
10,
);
const now = Date.now();
if (now - last < STATS_REFRESH_INTERVAL_MS) return;
await this.clients.redis.set(
STATS_REFRESH_LOCK_KEY,
String(now),
'EX',
Math.floor(STATS_REFRESH_INTERVAL_MS / 1000) + 60,
);
} catch {
// Keep going — better to over-refresh than miss updates entirely.
}
const clickhouse = globalThis.clickhouseClient;
let rows;
try {
if (clickhouse) {
const res = await clickhouse.query({
query: `
SELECT app_uid,
count(_id) AS open_count,
count(DISTINCT user_id) AS user_count
FROM app_opens
GROUP BY app_uid
`,
format: 'JSONEachRow',
});
rows = await res.json();
} else {
rows = await this.clients.db.read(
`SELECT app_uid,
COUNT(_id) AS open_count,
COUNT(DISTINCT user_id) AS user_count
FROM app_opens
GROUP BY app_uid`,
);
}
} catch (e) {
console.warn('[AppStore] refresh app stats failed:', e);
return;
}
if (!rows?.length) return;
try {
const pipeline = this.clients.redis.pipeline();
for (const row of rows) {
if (!row.app_uid) continue;
pipeline.set(
this.#openCountCacheKey(row.app_uid),
String(parseInt(row.open_count, 10) || 0),
'EX',
STATS_CACHE_TTL_SECONDS,
);
pipeline.set(
this.#userCountCacheKey(row.app_uid),
String(parseInt(row.user_count, 10) || 0),
'EX',
STATS_CACHE_TTL_SECONDS,
);
}
await pipeline.exec();
} catch (e) {
console.warn('[AppStore] refresh app stats cache write failed:', e);
}
}
onServerStart() {
// Kick off one refresh immediately so the cache is warm before the
// first client request (failing silently is fine — getAppsStats
// falls back to an on-demand query).
this.#refreshAppStats().catch(() => {});
// Jitter prevents every node refreshing on the same tick when a
// cluster boots together.
this.#appStatsInterval = setInterval(
() => {
this.#refreshAppStats().catch(() => {});
},
STATS_REFRESH_INTERVAL_MS + Math.floor(Math.random() * 500),
);
}
onServerShutdown() {
if (this.#appStatsInterval) {
clearInterval(this.#appStatsInterval);
this.#appStatsInterval = undefined;
}
}
}
+164
View File
@@ -0,0 +1,164 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupTestServer } from '../../testUtil.ts';
// Stats cache TTL the store backfills with (mirrors STATS_CACHE_TTL_SECONDS).
const STATS_CACHE_TTL_SECONDS = 30 * 60;
let counter = 0;
const freshUid = () => `app-test-${Date.now()}-${counter++}`;
describe('AppStore app stats (cache-on-read)', () => {
let server;
let appStore;
let db;
let redis;
const openKey = (uid) => `${appStore.appStatsCachePrefix}open:${uid}`;
const userKey = (uid) => `${appStore.appStatsCachePrefix}user:${uid}`;
const insertOpen = (uid, userId, ts) =>
db.write(
'INSERT INTO app_opens (app_uid, user_id, ts) VALUES (?, ?, ?)',
[uid, userId, ts],
);
// The store backfills the cache fire-and-forget (not awaited), so reads
// must poll for the key to land instead of assuming it's there.
const waitForKey = async (key, timeoutMs = 2000) => {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const v = await redis.get(key);
if (v != null) return v;
await new Promise((r) => setTimeout(r, 5));
}
throw new Error(`cache key ${key} was never populated`);
};
beforeAll(async () => {
server = await setupTestServer();
appStore = server.stores.app;
db = server.clients.db;
redis = server.clients.redis;
});
afterAll(async () => {
await server?.shutdown();
});
it('computes all-time open/unique-user counts from SQL when no ClickHouse client is registered', async () => {
const uid = freshUid();
// 3 opens across 2 distinct users.
await insertOpen(uid, 1, 1700000000);
await insertOpen(uid, 1, 1700000100);
await insertOpen(uid, 2, 1700000200);
const stats = await appStore.getAppsStats([uid]);
expect(stats.get(uid)).toEqual({
open_count: 3,
user_count: 2,
referral_count: null,
});
});
it('caches counts on read with an autoexpire TTL', async () => {
const uid = freshUid();
await insertOpen(uid, 7, 1700000000);
// Nothing should have warmed the cache before the first read — there
// is no background bulk refresh anymore.
expect(await redis.get(openKey(uid))).toBeNull();
expect(await redis.get(userKey(uid))).toBeNull();
await appStore.getAppsStats([uid]);
expect(await waitForKey(openKey(uid))).toBe('1');
expect(await waitForKey(userKey(uid))).toBe('1');
// Set to autoexpire — TTL is positive and bounded by the cache window.
const ttl = await redis.ttl(openKey(uid));
expect(ttl).toBeGreaterThan(0);
expect(ttl).toBeLessThanOrEqual(STATS_CACHE_TTL_SECONDS);
});
it('serves cached counts on subsequent reads without re-querying', async () => {
const uid = freshUid();
await insertOpen(uid, 1, 1700000000);
await insertOpen(uid, 2, 1700000100);
// Warm the cache, then wait for the backfill to land.
await appStore.getAppsStats([uid]);
await waitForKey(openKey(uid));
// Drop the underlying rows: a fresh query would now return zero.
await db.write('DELETE FROM app_opens WHERE app_uid = ?', [uid]);
const stats = await appStore.getAppsStats([uid]);
expect(stats.get(uid)).toEqual({
open_count: 2,
user_count: 2,
referral_count: null,
});
});
it('resolves apps with no opens to zero counts', async () => {
const uid = freshUid();
const stats = await appStore.getAppsStats([uid]);
expect(stats.get(uid)).toEqual({
open_count: 0,
user_count: 0,
referral_count: null,
});
});
it('uses the ClickHouse client when one is registered', async () => {
const uid = freshUid();
let receivedUids;
// Inject a fake ClickHouse client onto the shared clients object the
// store reads from. No SQL rows exist for this uid, so a non-zero
// result can only come from ClickHouse.
server.clients.clickhouse = {
query: async ({ query_params }) => {
receivedUids = query_params?.uids;
return {
json: async () => [
{ app_uid: uid, open_count: '42', user_count: '7' },
],
};
},
};
try {
const stats = await appStore.getAppsStats([uid]);
expect(stats.get(uid)).toEqual({
open_count: 42,
user_count: 7,
referral_count: null,
});
expect(receivedUids).toEqual([uid]);
} finally {
delete server.clients.clickhouse;
}
});
});
@@ -0,0 +1,63 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types';
import { FSEntryStore } from './FSEntryStore.js';
describe('FSEntryStore', () => {
it('quotes camelCase storage allowance aliases for Postgres', async () => {
const queries: string[] = [];
const config: IConfig = {
port: 0,
extensions: [],
is_storage_limited: true,
storage_capacity: 1000,
};
const clients = {
db: {
quoteIdentifier: (identifier: string) => `"${identifier}"`,
read: async (query: string) => {
queries.push(query);
if (query.includes('SUM(size)')) {
return [{ totalUsage: 321 }];
}
return [{ freeStorage: 654 }];
},
},
event: {
emitAndWait: async () => undefined,
},
};
const store = new FSEntryStore(
config,
clients as ConstructorParameters<typeof FSEntryStore>[1],
{} as ConstructorParameters<typeof FSEntryStore>[2],
);
await expect(store.getUserStorageAllowance(42)).resolves.toEqual({
curr: 321,
max: 654,
});
expect(queries).toEqual([
'SELECT COALESCE(SUM(size), 0) AS "totalUsage" FROM fsentries WHERE user_id = ?',
'SELECT free_storage AS "freeStorage" FROM "user" WHERE id = ? LIMIT 1',
]);
});
});
+76 -41
View File
@@ -60,10 +60,7 @@ export class FSEntryStore extends PuterStore {
declare protected stores: LayerInstances<typeof puterStores>;
#insertIgnoreIntoFsentriesSql(): string {
return this.clients.db.case({
sqlite: 'INSERT OR IGNORE INTO fsentries',
otherwise: 'INSERT IGNORE INTO fsentries',
});
return this.clients.db.insertIgnoreInto('fsentries');
}
// JSON aggregation of associated subdomain rows, keyed on fsentries.id.
@@ -86,6 +83,16 @@ export class FSEntryStore extends PuterStore {
FROM subdomains sd
WHERE sd.root_dir_id = fsentries.id
)`,
postgres: `(
SELECT COALESCE(
json_agg(
json_build_object('uuid', sd.uuid, 'subdomain', sd.subdomain)
),
'[]'::json
)
FROM subdomains sd
WHERE sd.root_dir_id = fsentries.id
)`,
});
}
@@ -659,6 +666,8 @@ export class FSEntryStore extends PuterStore {
const insertRows: unknown[] = [];
const valuePlaceholders: string[] = [];
const expectedUuidByPath = new Map<string, string>();
const trueLiteral = this.clients.db.booleanLiteral(true);
const falseLiteral = this.clients.db.booleanLiteral(false);
for (const dirPath of pathsAtDepth) {
const parentPath = pathPosix.dirname(dirPath);
const parentEntry =
@@ -679,7 +688,7 @@ export class FSEntryStore extends PuterStore {
const expectedUuid = uuidv4();
expectedUuidByPath.set(dirPath, expectedUuid);
valuePlaceholders.push(
'(?, ?, ?, ?, ?, ?, 1, ?, ?, ?, 0, 0)',
`(?, ?, ?, ?, ?, ?, ${trueLiteral}, ?, ?, ?, ${falseLiteral}, 0)`,
);
insertRows.push(
expectedUuid,
@@ -709,7 +718,7 @@ export class FSEntryStore extends PuterStore {
accessed,
immutable,
size
) VALUES ${valuePlaceholders.join(', ')}`,
) VALUES ${valuePlaceholders.join(', ')}${this.clients.db.insertIgnoreSuffix()}`,
insertRows,
);
} catch {
@@ -812,6 +821,8 @@ export class FSEntryStore extends PuterStore {
: await this.#ensureDirectoryPath(parentPath, userId, true);
const dirName = pathPosix.basename(normalizedPath);
const now = Math.floor(Date.now() / 1000);
const trueLiteral = this.clients.db.booleanLiteral(true);
const falseLiteral = this.clients.db.booleanLiteral(false);
let insertError: unknown = null;
try {
@@ -829,7 +840,7 @@ export class FSEntryStore extends PuterStore {
accessed,
immutable,
size
) VALUES (?, ?, ?, ?, ?, ?, 1, ?, ?, ?, 0, 0)`,
) VALUES (?, ?, ?, ?, ?, ?, ${trueLiteral}, ?, ?, ?, ${falseLiteral}, 0)${this.clients.db.insertIgnoreSuffix()}`,
[
uuidv4(),
userId,
@@ -1693,8 +1704,10 @@ export class FSEntryStore extends PuterStore {
for (const entry of userEntries) {
const parentEntry = parentByPath.get(entry.parentPath);
if (!parentEntry) {
throw new Error(
`Failed to resolve parent directory for ${entry.targetPath}`,
throw new HttpError(
404,
`Parent directory not found for ${entry.targetPath}`,
{ legacyCode: 'dest_does_not_exist' },
);
}
@@ -1765,11 +1778,13 @@ export class FSEntryStore extends PuterStore {
entry.input.associatedAppId ?? null,
entry.input.isPublic === undefined
? null
: entry.input.isPublic
? 1
: 0,
: this.clients.db.booleanValue(
entry.input.isPublic,
),
entry.input.thumbnail ?? null,
entry.input.immutable ? 1 : 0,
this.clients.db.booleanValue(
Boolean(entry.input.immutable),
),
entry.fileName,
entry.targetPath,
entry.metadataJson,
@@ -1844,13 +1859,15 @@ export class FSEntryStore extends PuterStore {
for (const entry of insertChunk) {
const parentEntry = parentByPath.get(entry.parentPath);
if (!parentEntry) {
throw new Error(
`Failed to resolve parent directory for ${entry.targetPath}`,
throw new HttpError(
404,
`Parent directory not found for ${entry.targetPath}`,
{ legacyCode: 'dest_does_not_exist' },
);
}
valuePlaceholders.push(
'(?, ?, ?, ?, ?, ?, ?, 0, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)',
`(?, ?, ?, ?, ?, ?, ?, ${this.clients.db.booleanLiteral(false)}, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
);
values.push(
entry.input.uuid,
@@ -1862,11 +1879,13 @@ export class FSEntryStore extends PuterStore {
entry.input.associatedAppId ?? null,
entry.input.isPublic === undefined
? null
: entry.input.isPublic
? 1
: 0,
: this.clients.db.booleanValue(
entry.input.isPublic,
),
entry.input.thumbnail ?? null,
entry.input.immutable ? 1 : 0,
this.clients.db.booleanValue(
Boolean(entry.input.immutable),
),
entry.fileName,
entry.targetPath,
entry.metadataJson,
@@ -1903,7 +1922,7 @@ export class FSEntryStore extends PuterStore {
created,
accessed,
size
) VALUES ${valuePlaceholders.join(', ')}`,
) VALUES ${valuePlaceholders.join(', ')}${this.clients.db.insertIgnoreSuffix()}`,
values,
);
},
@@ -1974,8 +1993,10 @@ export class FSEntryStore extends PuterStore {
for (const entry of racedCandidates) {
const parentEntry = parentByPath.get(entry.parentPath);
if (!parentEntry) {
throw new Error(
`Failed to resolve parent directory for ${entry.targetPath}`,
throw new HttpError(
404,
`Parent directory not found for ${entry.targetPath}`,
{ legacyCode: 'dest_does_not_exist' },
);
}
const existing = racedExistingByPath.get(entry.targetPath);
@@ -2055,11 +2076,13 @@ export class FSEntryStore extends PuterStore {
entry.input.associatedAppId ?? null,
entry.input.isPublic === undefined
? null
: entry.input.isPublic
? 1
: 0,
: this.clients.db.booleanValue(
entry.input.isPublic,
),
entry.input.thumbnail ?? null,
entry.input.immutable ? 1 : 0,
this.clients.db.booleanValue(
Boolean(entry.input.immutable),
),
entry.fileName,
entry.targetPath,
entry.metadataJson,
@@ -2273,9 +2296,13 @@ export class FSEntryStore extends PuterStore {
? `/${input.name}`
: `${parentPath}/${input.name}`;
const isDir = input.kind === 'directory' ? 1 : 0;
const isShortcut = input.kind === 'shortcut' ? 1 : 0;
const isSymlink = input.kind === 'symlink' ? 1 : 0;
const isDir = this.clients.db.booleanValue(input.kind === 'directory');
const isShortcut = this.clients.db.booleanValue(
input.kind === 'shortcut',
);
const isSymlink = this.clients.db.booleanValue(
input.kind === 'symlink',
);
await this.clients.db.write(
`INSERT INTO fsentries (
@@ -2315,12 +2342,10 @@ export class FSEntryStore extends PuterStore {
input.associatedAppId ?? null,
input.metadata ?? null,
input.thumbnail ?? null,
input.immutable ? 1 : 0,
this.clients.db.booleanValue(Boolean(input.immutable)),
input.isPublic === undefined || input.isPublic === null
? null
: input.isPublic
? 1
: 0,
: this.clients.db.booleanValue(input.isPublic),
now,
now,
now,
@@ -2578,10 +2603,12 @@ export class FSEntryStore extends PuterStore {
if (patch.isPublic !== undefined)
push(
'is_public',
patch.isPublic === null ? null : patch.isPublic ? 1 : 0,
patch.isPublic === null
? null
: this.clients.db.booleanValue(patch.isPublic),
);
if (patch.immutable !== undefined)
push('immutable', patch.immutable ? 1 : 0);
push('immutable', this.clients.db.booleanValue(patch.immutable));
if (patch.associatedAppId !== undefined)
push('associated_app_id', patch.associatedAppId);
if (patch.layout !== undefined) push('layout', patch.layout);
@@ -2629,7 +2656,7 @@ export class FSEntryStore extends PuterStore {
async getRootEntryForUser(userId: number): Promise<FSEntry | null> {
const rows = (await this.clients.db.read(
`SELECT ${this.#selectFsentriesColumns()} FROM fsentries
WHERE user_id = ? AND parent_uid IS NULL AND is_dir = 1
WHERE user_id = ? AND parent_uid IS NULL AND is_dir = ${this.clients.db.booleanLiteral(true)}
ORDER BY id ASC LIMIT 1`,
[userId],
)) as unknown as FSEntryRow[];
@@ -2669,9 +2696,13 @@ export class FSEntryStore extends PuterStore {
if (oldPath && oldPath !== '/' && oldPath !== newPath) {
const likePattern = `${this.#escapeLikePattern(oldPath)}/%`;
const oldLen = oldPath.length;
const rewrittenPath = this.clients.db.case({
postgres: '? || SUBSTR(path, ?)',
otherwise: 'CONCAT(?, SUBSTR(path, ?))',
});
await this.clients.db.write(
`UPDATE fsentries
SET path = CONCAT(?, SUBSTR(path, ?)),
SET path = ${rewrittenPath},
modified = ?
WHERE user_id = ? AND path LIKE ? ESCAPE '!'`,
[newPath, oldLen + 1, now, userId, likePattern],
@@ -2720,9 +2751,13 @@ export class FSEntryStore extends PuterStore {
// CONCAT(?, SUBSTR(path, ? + 1)) to rewrite just the prefix portion.
const oldPrefixLen = normalizedOld.length;
const rewrittenPath = this.clients.db.case({
postgres: '? || SUBSTR(path, ?)',
otherwise: 'CONCAT(?, SUBSTR(path, ?))',
});
const result = await this.clients.db.write(
`UPDATE fsentries
SET path = CONCAT(?, SUBSTR(path, ?)),
SET path = ${rewrittenPath},
modified = ?
WHERE user_id = ? AND path LIKE ? ESCAPE '!'`,
[normalizedNew, oldPrefixLen + 1, now, userId, likePattern],
@@ -2771,11 +2806,11 @@ export class FSEntryStore extends PuterStore {
): Promise<{ curr: number; max: number }> {
const [usageRows, userRows] = await Promise.all([
this.clients.db.read(
'SELECT COALESCE(SUM(size), 0) AS totalUsage FROM fsentries WHERE user_id = ?',
`SELECT COALESCE(SUM(size), 0) AS ${this.clients.db.quoteIdentifier('totalUsage')} FROM fsentries WHERE user_id = ?`,
[userId],
) as Promise<{ totalUsage: number }[]>,
this.clients.db.read(
'SELECT free_storage AS freeStorage FROM user WHERE id = ? LIMIT 1',
`SELECT free_storage AS ${this.clients.db.quoteIdentifier('freeStorage')} FROM ${this.clients.db.quoteIdentifier('user')} WHERE id = ? LIMIT 1`,
[userId],
) as Promise<{ freeStorage: number | null }[]>,
]);
+1
View File
@@ -159,6 +159,7 @@ export class GroupStore extends PuterStore {
}): Promise<string> {
const windowClause = this.clients.db.case<string>({
sqlite: "datetime('now', '-1 hour')",
postgres: "NOW() - INTERVAL '1 hour'",
otherwise: 'NOW() - INTERVAL 1 HOUR',
});
const [countRow] = await this.clients.db.read(
+10 -5
View File
@@ -20,6 +20,15 @@
import { PuterStore } from '../types';
import { HttpError } from '../../core/http/HttpError.js';
const isUniqueConstraintError = (e) => {
return (
e?.message?.includes('UNIQUE') ||
e?.code === 'SQLITE_CONSTRAINT' ||
e?.code === 'ER_DUP_ENTRY' ||
e?.code === '23505'
);
};
/**
* CRUD over the `user_oidc_providers` table.
*
@@ -54,11 +63,7 @@ export class OIDCStore extends PuterStore {
);
return;
} catch (e) {
const isUnique =
e.message?.includes('UNIQUE') ||
e.code === 'SQLITE_CONSTRAINT' ||
e.code === 'ER_DUP_ENTRY';
if (!isUnique) throw e;
if (!isUniqueConstraintError(e)) throw e;
}
// UNIQUE(provider, provider_sub) collision — either we're re-linking
+74
View File
@@ -0,0 +1,74 @@
import { describe, expect, it, vi } from 'vitest';
import { OIDCStore } from './OIDCStore.js';
type OidcRow = {
user_id: number;
provider: string;
provider_sub: string;
};
const createPostgresUniqueError = (): Error & { code: string } => {
return Object.assign(
new Error(
'duplicate key value violates unique constraint "idx_user_oidc_providers_provider_sub_unique"',
),
{ code: '23505' },
);
};
const createStore = (rows: readonly OidcRow[]) => {
const db = {
write: vi.fn(
async (_sql: string, _params: readonly unknown[]): Promise<void> => {
throw createPostgresUniqueError();
},
),
read: vi.fn(
async (
_sql: string,
_params: readonly unknown[],
): Promise<readonly OidcRow[]> => rows,
),
};
const store = new OIDCStore({}, { db });
return { db, store };
};
describe('OIDCStore', () => {
it('treats a Postgres unique violation as idempotent for an existing same-user link', async () => {
const { db, store } = createStore([
{
user_id: 123,
provider: 'test-provider',
provider_sub: 'subject-1',
},
]);
await expect(
store.link(123, 'test-provider', 'subject-1'),
).resolves.toBeUndefined();
expect(db.read).toHaveBeenCalledWith(
expect.stringContaining('user_oidc_providers'),
['test-provider', 'subject-1'],
);
});
it('rejects a Postgres unique violation for an existing different-user link', async () => {
const { store } = createStore([
{
user_id: 456,
provider: 'test-provider',
provider_sub: 'subject-1',
},
]);
await expect(
store.link(123, 'test-provider', 'subject-1'),
).rejects.toMatchObject({
statusCode: 409,
legacyCode: 'conflict',
});
});
});
@@ -169,11 +169,10 @@ export class PermissionStore extends PuterStore {
permission: string,
extra: Record<string, unknown>,
): Promise<void> {
const upsertClause = this.clients.db.case<string>({
mysql: 'ON DUPLICATE KEY UPDATE `extra` = ?',
otherwise:
'ON CONFLICT(`holder_user_id`, `issuer_user_id`, `permission`) DO UPDATE SET `extra` = ?',
});
const upsertClause = this.clients.db.upsertClause(
['holder_user_id', 'issuer_user_id', 'permission'],
['extra'],
);
await this.clients.db.write(
'INSERT INTO `user_to_user_permissions` (`holder_user_id`, `issuer_user_id`, `permission`, `extra`) ' +
`VALUES (?, ?, ?, ?) ${upsertClause}`,
@@ -262,11 +261,10 @@ export class PermissionStore extends PuterStore {
permission: string,
extra: Record<string, unknown>,
): Promise<void> {
const upsertClause = this.clients.db.case<string>({
mysql: 'ON DUPLICATE KEY UPDATE `extra` = ?',
otherwise:
'ON CONFLICT(`user_id`, `app_id`, `permission`) DO UPDATE SET `extra` = ?',
});
const upsertClause = this.clients.db.upsertClause(
['user_id', 'app_id', 'permission'],
['extra'],
);
await this.clients.db.write(
'INSERT INTO `user_to_app_permissions` (`user_id`, `app_id`, `permission`, `extra`) ' +
`VALUES (?, ?, ?, ?) ${upsertClause}`,
@@ -353,11 +351,10 @@ export class PermissionStore extends PuterStore {
permission: string,
extra: Record<string, unknown>,
): Promise<void> {
const upsertClause = this.clients.db.case<string>({
mysql: 'ON DUPLICATE KEY UPDATE `extra` = ?',
otherwise:
'ON CONFLICT(`user_id`, `app_id`, `permission`) DO UPDATE SET `extra` = ?',
});
const upsertClause = this.clients.db.upsertClause(
['user_id', 'app_id', 'permission'],
['extra'],
);
await this.clients.db.write(
'INSERT INTO `dev_to_app_permissions` (`user_id`, `app_id`, `permission`, `extra`) ' +
`VALUES (?, ?, ?, ?) ${upsertClause}`,
@@ -443,11 +440,10 @@ export class PermissionStore extends PuterStore {
permission: string,
extra: Record<string, unknown>,
): Promise<void> {
const upsertClause = this.clients.db.case<string>({
mysql: 'ON DUPLICATE KEY UPDATE `extra` = ?',
otherwise:
'ON CONFLICT(`user_id`, `group_id`, `permission`) DO UPDATE SET `extra` = ?',
});
const upsertClause = this.clients.db.upsertClause(
['user_id', 'group_id', 'permission'],
['extra'],
);
await this.clients.db.write(
'INSERT INTO `user_to_group_permissions` (`user_id`, `group_id`, `permission`, `extra`) ' +
`VALUES (?, ?, ?, ?) ${upsertClause}`,
+20 -17
View File
@@ -55,14 +55,15 @@ export const WORKER_WINDOW_SECONDS = 99 * 365 * 24 * 60 * 60; // 99y (virtually
// FK / type violations must bubble up; otherwise the caller caches a
// row that was never inserted.
// better-sqlite3 surfaces SqliteError.code; mysql2 surfaces .code and
// .errno (1062 = ER_DUP_ENTRY).
// .errno (1062 = ER_DUP_ENTRY); pg surfaces SQLSTATE 23505.
function isUniqueViolation(err) {
if (!err) return false;
const code = err.code;
if (
code === 'SQLITE_CONSTRAINT_UNIQUE' ||
code === 'SQLITE_CONSTRAINT_PRIMARYKEY' ||
code === 'ER_DUP_ENTRY'
code === 'ER_DUP_ENTRY' ||
code === '23505'
) {
return true;
}
@@ -462,7 +463,7 @@ export class SessionStore extends PuterStore {
*
* `appUid` is allowed null for user-scoped workers — the partial
* unique index treats those distinctly (SQLite via NULL-distinct
* semantics, MySQL via IFNULL in the generated key).
* semantics, MySQL/Postgres via COALESCE in the generated key).
*
* @param userId - User row id (numeric).
* @param opts.appUid - App UID or null for user-scoped workers.
@@ -598,9 +599,16 @@ export class SessionStore extends PuterStore {
return cached;
}
const coalesceLastIp = this.clients.db.nullCoalesce('`last_ip`', "''");
const coalesceBoundIp = this.clients.db.nullCoalesce('?', "''");
const coalesceLastUserAgent = this.clients.db.nullCoalesce(
'`last_user_agent`',
"''",
);
const coalesceBoundUserAgent = this.clients.db.nullCoalesce('?', "''");
const selectOldest = () =>
this.clients.db.read(
"SELECT * FROM `sessions` WHERE `kind` = 'web' AND `user_id` = ? AND `created_via` = 'legacy_backfill' AND IFNULL(`last_ip`, '') = IFNULL(?, '') AND IFNULL(`last_user_agent`, '') = IFNULL(?, '') AND `revoked_at` IS NULL AND (`expires_at` IS NULL OR `expires_at` > ?) ORDER BY `id` ASC LIMIT 1",
`SELECT * FROM \`sessions\` WHERE \`kind\` = 'web' AND \`user_id\` = ? AND \`created_via\` = 'legacy_backfill' AND ${coalesceLastIp} = ${coalesceBoundIp} AND ${coalesceLastUserAgent} = ${coalesceBoundUserAgent} AND \`revoked_at\` IS NULL AND (\`expires_at\` IS NULL OR \`expires_at\` > ?) ORDER BY \`id\` ASC LIMIT 1`,
[opts.userId, ip, ua, now],
);
@@ -713,7 +721,7 @@ export class SessionStore extends PuterStore {
/** Update user-level last activity timestamp. */
async updateUserActivity(userId, lastActivityTs) {
await this.clients.db.write(
'UPDATE `user` SET `last_activity_ts` = ? WHERE `id` = ? AND (`last_activity_ts` IS NULL OR `last_activity_ts` < ?) LIMIT 1',
'UPDATE `user` SET `last_activity_ts` = ? WHERE `id` = ? AND (`last_activity_ts` IS NULL OR `last_activity_ts` < ?)',
[lastActivityTs, userId, lastActivityTs],
);
}
@@ -915,23 +923,18 @@ export class SessionStore extends PuterStore {
/**
* Active worker session for (userId, appUid, workerName). Matches
* the partial unique index `idx_sessions_user_worker_active`.
* `appUid` is allowed null for user-scoped workers; IFNULL keeps
* `appUid` is allowed null for user-scoped workers; COALESCE keeps
* the comparison correct since SQL `= NULL` doesn't match.
*
* MySQL's `JSON_EXTRACT` returns a JSON-typed value with embedded
* quoting (`"name"` rather than `name`), which never equals a
* plain string bind. Use `JSON_UNQUOTE(JSON_EXTRACT(...))` on
* MySQL to strip that. SQLite's `json_extract` already returns the
* unwrapped scalar so the literal form works there.
*/
async #selectWorkerRow(userId, appUid, workerName) {
const now = nowSeconds();
const workerNameExpr = this.clients.db.case({
sqlite: "json_extract(`meta`, '$.worker_name')",
otherwise: "JSON_UNQUOTE(JSON_EXTRACT(`meta`, '$.worker_name'))",
});
const workerNameExpr = this.clients.db.jsonTextExtract('`meta`', [
'worker_name',
]);
const appUidExpr = this.clients.db.nullCoalesce('`app_uid`', "''");
const appUidBound = this.clients.db.nullCoalesce('?', "''");
const rows = await this.clients.db.read(
`SELECT * FROM \`sessions\` WHERE \`kind\` = 'worker' AND \`user_id\` = ? AND IFNULL(\`app_uid\`, '') = IFNULL(?, '') AND ${workerNameExpr} = ? AND \`revoked_at\` IS NULL AND (\`expires_at\` IS NULL OR \`expires_at\` > ?) LIMIT 1`,
`SELECT * FROM \`sessions\` WHERE \`kind\` = 'worker' AND \`user_id\` = ? AND ${appUidExpr} = ${appUidBound} AND ${workerNameExpr} = ? AND \`revoked_at\` IS NULL AND (\`expires_at\` IS NULL OR \`expires_at\` > ?) LIMIT 1`,
[userId, appUid ?? null, workerName, now],
);
return this.#normalizeRow(rows[0]);
+57
View File
@@ -0,0 +1,57 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { setupTestServer } from '../../testUtil.ts';
import { PuterServer } from '../../server.ts';
describe('UserStore', () => {
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
it('keeps cached booleans normalized after update', async () => {
const username = `us-${Math.random().toString(36).slice(2, 10)}`;
const user = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
});
await server.stores.user.update(user.id, {
email_confirmed: false,
requires_email_confirmation: true,
});
const cachedUser = await server.stores.user.getById(user.id);
expect(cachedUser?.email_confirmed).toBe(false);
expect(cachedUser?.requires_email_confirmation).toBe(true);
expect(typeof cachedUser?.email_confirmed).toBe('boolean');
expect(typeof cachedUser?.requires_email_confirmation).toBe('boolean');
});
});
+36 -9
View File
@@ -85,6 +85,15 @@ const LATIN1_USER_COLUMNS: ReadonlySet<string> = new Set([
'username',
'clean_email',
]);
const USER_BOOLEAN_COLUMNS: ReadonlySet<string> = new Set([
'requires_email_confirmation',
'email_confirmed',
'dev_approved_for_incentive_program',
'dev_joined_incentive_program',
'suspended',
'unsubscribed',
'otp_enabled',
]);
const assertLatin1Writable = (fields: Record<string, unknown>): void => {
for (const [key, value] of Object.entries(fields)) {
@@ -336,7 +345,7 @@ export class UserStore extends PuterStore {
signup_server,
referrer,
last_activity_ts)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)${this.clients.db.returningIdClause()}`,
[
fields.username,
fields.email,
@@ -344,7 +353,9 @@ export class UserStore extends PuterStore {
fields.password,
fields.uuid,
fields.free_storage ?? null,
fields.requires_email_confirmation ? 1 : 0,
this.clients.db.booleanValue(
Boolean(fields.requires_email_confirmation),
),
fields.email_confirm_code ?? null,
fields.email_confirm_token ?? null,
fields.audit_metadata
@@ -380,13 +391,23 @@ export class UserStore extends PuterStore {
userId: number,
patch: Record<string, unknown>,
): Promise<void> {
const keys = Object.keys(patch);
const dbPatch: Record<string, unknown> = {};
for (const [key, value] of Object.entries(patch)) {
dbPatch[key] =
USER_BOOLEAN_COLUMNS.has(key) &&
value !== null &&
value !== undefined
? this.clients.db.booleanValue(Boolean(value))
: value;
}
const keys = Object.keys(dbPatch);
if (keys.length === 0) return;
assertLatin1Writable(patch);
assertLatin1Writable(dbPatch);
const setClause = keys.map((k) => `\`${k}\` = ?`).join(', ');
const values = keys.map((k) => patch[k]);
const values = keys.map((k) => dbPatch[k]);
await this.clients.db.write(
`UPDATE \`user\` SET ${setClause} WHERE \`id\` = ?`,
@@ -395,7 +416,7 @@ export class UserStore extends PuterStore {
const fresh = await this.getByProperty('id', userId, { force: true });
if (fresh) {
await this.#refreshCache({ ...fresh, ...patch });
await this.#refreshCache(fresh);
} else {
await this.invalidateById(userId);
}
@@ -428,13 +449,19 @@ export class UserStore extends PuterStore {
`UPDATE \`user\`
SET \`email\` = NULL,
\`clean_email\` = NULL,
\`email_confirmed\` = 0,
\`requires_email_confirmation\` = 0,
\`email_confirmed\` = ?,
\`requires_email_confirmation\` = ?,
\`email_confirm_code\` = NULL,
\`email_confirm_token\` = NULL
WHERE \`id\` != ?
AND (\`email\` = ? OR \`clean_email\` = ?)`,
[userId, email, cleanEmailValue],
[
this.clients.db.booleanValue(false),
this.clients.db.booleanValue(false),
userId,
email,
cleanEmailValue,
],
);
}
+125 -4
View File
@@ -1,8 +1,96 @@
import { deepMerge } from '../../tools/lib/configMigration.mjs';
import { PuterServer } from './server';
import { IConfig } from './types';
import { puterClients } from './clients';
import {
PostgresDatabaseClient,
type PostgresPool,
} from './clients/database/PostgresDatabaseClient';
import type { PoolConfig } from 'pg';
export const setupTestServer = async (configOverrides?: IConfig) => {
export const POSTGRES_TEST_MIGRATIONS_PATH =
'src/backend/clients/database/migrations/postgres';
type PgMockPostgresHarness = {
client: PostgresDatabaseClient;
createClient: () => PostgresDatabaseClient;
destroy: () => void;
};
const usesPgMockPostgres = (config: IConfig): boolean => {
const database = config.database;
if (database?.engine !== 'postgres' || database.inMemory !== true) {
return false;
}
return !(
database.connectionString ||
database.url ||
database.host ||
database.port ||
database.user ||
database.password ||
database.database ||
database.replica
);
};
export const createPgMockPostgresDatabaseClient = async (
config: IConfig,
): Promise<PgMockPostgresHarness> => {
const [{ PostgresMock }, { Pool }] = await Promise.all([
import('pgmock'),
import('pg'),
]);
const mock = await PostgresMock.create();
const pgMockConfig = mock.getNodePostgresConfig();
const createPgMockStream = pgMockConfig.stream;
const poolFactory = (poolConfig: PoolConfig): PostgresPool =>
new Pool({
...poolConfig,
database: 'postgres',
...pgMockConfig,
stream: () => {
const stream = createPgMockStream();
stream.ref = () => stream;
stream.unref = () => stream;
return stream;
},
}) as unknown as PostgresPool;
const createClient = () => new PostgresDatabaseClient(config, poolFactory);
return {
client: createClient(),
createClient,
destroy: () => mock.destroy(),
};
};
/**
* When `PUTER_TEST_DB_ENGINE=postgres` is set, `setupTestServer` swaps its
* default sqlite test database for an in-memory Postgres backed by pgmock
* (with the bundled Postgres migrations applied on boot). Tests that
* explicitly override `database` still win — the env var only affects the
* implicit default used by callers that don't pass any DB overrides.
*
* Recognized values: `postgres` → pgmock. Anything else (including unset) →
* the original sqlite-in-memory default.
*/
const testDatabaseDefault = (): IConfig['database'] => {
const engine = (process.env.PUTER_TEST_DB_ENGINE ?? '').toLowerCase();
if (engine === 'postgres') {
return {
engine: 'postgres',
inMemory: true,
migrationPaths: [POSTGRES_TEST_MIGRATIONS_PATH],
};
}
return { engine: 'sqlite', inMemory: true };
};
export const setupTestServer = async (
configOverrides?: IConfig,
): Promise<PuterServer> => {
// read default config json
const defaultConfig = await import('../../config.default.json', {
with: {
@@ -14,7 +102,7 @@ export const setupTestServer = async (configOverrides?: IConfig) => {
deepMerge(defaultConfig, {
extensions: [],
port: 0,
database: { engine: 'sqlite', inMemory: true },
database: testDatabaseDefault(),
dynamo: { inMemory: true, bootstrapTables: true },
redis: { useMock: true },
s3: { localConfig: { inMemory: true } },
@@ -22,8 +110,41 @@ export const setupTestServer = async (configOverrides?: IConfig) => {
no_devwatch: true,
}),
configOverrides ?? {},
) as IConfig;
let pgMockClient: PgMockPostgresHarness | undefined;
if (usesPgMockPostgres(config)) {
const database = config.database;
if (!database) {
throw new Error('Postgres test database config is missing');
}
database.migrationPaths ??= [POSTGRES_TEST_MIGRATIONS_PATH];
pgMockClient = await createPgMockPostgresDatabaseClient(config);
}
const server = new PuterServer(
config,
pgMockClient ? { ...puterClients, db: pgMockClient.client } : undefined,
);
const server = new PuterServer(config);
await server.start(true);
if (pgMockClient) {
const originalShutdown = server.shutdown.bind(server);
let destroyPgMock: (() => void) | undefined = pgMockClient.destroy;
server.shutdown = async () => {
try {
await originalShutdown();
} finally {
destroyPgMock?.();
destroyPgMock = undefined;
}
};
}
try {
await server.start(true);
} catch (e) {
pgMockClient?.destroy();
throw e;
}
return server;
};
+11 -5
View File
@@ -306,7 +306,7 @@ export interface IS3Config {
}
export interface IDatabaseConfig {
engine: 'sqlite' | 'mysql';
engine: 'sqlite' | 'mysql' | 'postgres';
// sqlite
/**
* SQLite database file path. Defaults to `':memory:'` (the
@@ -317,7 +317,8 @@ export interface IDatabaseConfig {
/**
* Force in-memory SQLite (ignores `path`). Equivalent to
* `path: ':memory:'`. Intended for tests so each suite gets a
* pristine in-process database.
* pristine in-process database. Test utilities also use
* `engine: 'postgres'` with `inMemory: true` to run against pgmock.
*/
inMemory?: boolean;
targetVersion?: number;
@@ -327,17 +328,21 @@ export interface IDatabaseConfig {
user?: string;
password?: string;
database?: string;
connectionString?: string;
url?: string;
replica?: {
host?: string;
port?: number;
user?: string;
password?: string;
database?: string;
connectionString?: string;
url?: string;
};
/**
* Ordered list of directories whose `.sql` files are run sequentially at
* server start (mysql engine only). Files within a directory are sorted
* lexically; directories are processed in array order. Files MUST be
* server start (mysql/postgres engines). Numbered migration filenames sort
* numerically; directories are processed in array order. Files MUST be
* idempotent — there is no per-file applied-state tracking.
* Relative paths resolve from `process.cwd()`.
*/
@@ -560,7 +565,8 @@ interface IConfigOptional {
redis: IRedisConfig;
pager: IPagerConfig;
email: IEmailConfig;
clickhouse: IClickhouseConfig;
/** Optional — only set when a ClickHouse analytics client is wired in. */
clickhouse?: IClickhouseConfig;
cf_file_cache: ICfFileCacheConfig;
// -- Rate limiting -----------------------------------------------
+1
View File
@@ -59,6 +59,7 @@ const DEFAULT_TASKBAR_ITEMS: TaskbarEntry[] = [
{ name: 'code', type: 'app' },
{ name: 'camera', type: 'app' },
{ name: 'recorder', type: 'app' },
{ name: 'browser', type: 'app' },
];
export async function getTaskbarItems(
+4 -3
View File
@@ -81,10 +81,11 @@ export async function generateDefaultFsentries(
]),
];
// Each row: uuid, parent_uid, user_id, name, path, created, modified
// is_dir and immutable are hardcoded to 1.
// Each row: uuid, parent_uid, user_id, name, path, created, modified.
// is_dir and immutable are hardcoded to the database's true literal.
const trueLiteral = db.booleanLiteral(true);
const placeholders = rows
.map(() => '(?, ?, ?, ?, ?, 1, ?, ?, 1)')
.map(() => `(?, ?, ?, ?, ?, ${trueLiteral}, ?, ?, ${trueLiteral})`)
.join(', ');
const params: unknown[] = [];
for (const [uuid, parent, name, path] of rows) {
+17
View File
@@ -27,6 +27,16 @@ const isCi = process.env.CI === 'true';
const backendDir = __dirname;
const repoRoot = path.resolve(backendDir, '../..');
// pgmock boots a WASM-emulated Postgres on every `setupTestServer()` call —
// migrations alone take ~60s, and the WASM VM serializes badly across
// concurrent emulator instances. So in pgmock mode we (a) bump hook/test
// timeouts well beyond the 10s default and (b) disable file parallelism so
// only one pgmock VM runs at a time. Tests still race their own logic
// internally; we only serialize the *files*.
const isPgmockMode =
(process.env.PUTER_TEST_DB_ENGINE ?? '').toLowerCase() === 'postgres';
const pgmockTimeoutMs = 600_000;
// Vite 8's oxc transform leaves TC39 stage-3 decorators in place
// (used by `@Controller`/`@Post`), so they reach Node verbatim and
// crash with "SyntaxError: Invalid or unexpected token". Pre-transform
@@ -72,6 +82,13 @@ export default defineConfig(({ mode }) => ({
},
test: {
globals: true,
...(isPgmockMode
? {
testTimeout: pgmockTimeoutMs,
hookTimeout: pgmockTimeoutMs,
fileParallelism: false,
}
: {}),
coverage: {
provider: 'v8',
reporter: isCi
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Puter Technologies Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+58
View File
@@ -0,0 +1,58 @@
# `@heyputer/cli`
> **Beta (0.x).** Deploy static sites and serverless workers to Puter from the
> terminal.
## Install
```sh
npm install @heyputer/cli
```
Requires Node 18+.
## Authentication
Log in once via the browser and your token is stored for later commands. For
automation, set `PUTER_AUTH_TOKEN` and the CLI skips login entirely.
```sh
puter login # web browser flow (interactive)
echo "$TOKEN" | puter login --with-token # token via stdin
puter logout
puter whoami
```
## Sites
Deploy a static directory to a `*.puter.site` subdomain, then list, inspect, or
remove your sites. Run with no arguments and the CLI prompts for the directory
and subdomain.
```sh
puter site deploy [dir] [subdomain] # both positional, both optional
puter site list
puter site get <subdomain>
puter site delete <subdomain> [-y]
```
## Workers
Deploy a JavaScript file as a serverless worker served at `<name>.puter.work`,
then list, inspect, or remove it.
```sh
puter worker deploy [file] [name]
puter worker list
puter worker get <name>
puter worker delete <name> [-y]
```
## Apps (read-only)
Browse the apps registered on your account. These commands are read-only.
```sh
puter app list
puter app get <name>
```
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env node
import { createRequire } from 'node:module';
import { Command } from 'commander';
import { action } from '../src/lib/errors.js';
import { loginCommand } from '../src/commands/login.js';
import { logoutCommand } from '../src/commands/logout.js';
import { whoamiCommand } from '../src/commands/whoami.js';
import {
siteDeploy,
siteList,
siteGet,
siteDelete,
} from '../src/commands/site.js';
import {
workerDeploy,
workerList,
workerGet,
workerDelete,
} from '../src/commands/worker.js';
import { appList, appGet } from '../src/commands/app.js';
// The Puter.js SDK emits duplicate "stray" rejections for failed API calls in
// addition to rejecting the promise we await. We already route the awaited
// error through action()/fail(), so swallow the strays to avoid a hard crash.
// Set PUTER_DEBUG=1 to surface them.
process.on('unhandledRejection', (reason) => {
if (process.env.PUTER_DEBUG) console.error('unhandledRejection:', reason);
});
const require = createRequire(import.meta.url);
const { version } = require('../package.json');
const program = new Command();
program
.name('puter')
.description('CLI for the Puter platform — deploy sites and workers. (beta)')
.version(version, '-v, --version');
// --- auth ------------------------------------------------------------------
program
.command('login')
.description('Log in to Puter (web browser, or --with-token)')
.option('--with-token', 'read an auth token from stdin')
.action(action(loginCommand));
program
.command('logout')
.description('Clear the stored auth token')
.action(action(logoutCommand));
program
.command('whoami')
.description('Show the current account')
.action(action(whoamiCommand));
// --- site ------------------------------------------------------------------
const site = program.command('site').description('Manage static sites');
site
.command('deploy')
.description('Deploy a static directory to <subdomain>.puter.site')
.argument('[dir]', 'directory to deploy')
.argument('[subdomain]', 'target subdomain')
.action(action(siteDeploy));
site
.command('list')
.description('List owned subdomains')
.action(action(siteList));
site
.command('get')
.description('Show details for one subdomain')
.argument('<subdomain>')
.action(action(siteGet));
site
.command('delete')
.description('Remove a subdomain')
.argument('<subdomain>')
.option('-y, --yes', 'skip confirmation')
.action(action(siteDelete));
// --- worker ----------------------------------------------------------------
const worker = program.command('worker').description('Manage serverless workers');
worker
.command('deploy')
.description('Deploy or replace a serverless worker')
.argument('[file]', "worker's JS file")
.argument('[name]', 'worker name')
.action(action(workerDeploy));
worker
.command('list')
.description('List workers')
.action(action(workerList));
worker
.command('get')
.description('Show details for one worker')
.argument('<name>')
.action(action(workerGet));
worker
.command('delete')
.description('Delete a worker')
.argument('<name>')
.option('-y, --yes', 'skip confirmation')
.action(action(workerDelete));
// --- app (read-only, beta) -------------------------------------------------
const app = program.command('app').description('Inspect apps (read-only)');
app
.command('list')
.description('List apps')
.action(action(appList));
app
.command('get')
.description('Show details for one app')
.argument('<name>')
.action(action(appGet));
program.parseAsync(process.argv);
+31
View File
@@ -0,0 +1,31 @@
{
"name": "@heyputer/cli",
"version": "0.1.0",
"description": "CLI for Puter Platform - manage your sites and workers from the terminal.",
"license": "MIT",
"author": "Puter Technologies Inc.",
"type": "module",
"bin": {
"puter": "./bin/puter.js"
},
"main": "bin/puter.js",
"files": [
"bin",
"src",
"README.md"
],
"engines": {
"node": ">=18"
},
"scripts": {
"start": "node ./bin/puter.js",
"test": "echo \"Error: no test specified\" && exit 1"
},
"dependencies": {
"@clack/prompts": "^0.7.0",
"@heyputer/puter.js": "latest",
"chalk": "^5.3.0",
"commander": "^12.1.0",
"conf": "^13.0.0"
}
}
+43
View File
@@ -0,0 +1,43 @@
// Read-only for beta (spec §7). An "app" is a registered desktop-OS entry on
// top of hosting; defining "app deploy" is deferred.
import { ensureClient } from '../lib/auth.js';
import { CLIError } from '../lib/errors.js';
import * as ui from '../lib/ui.js';
function appsApi(puter) {
const api = puter.apps ?? puter.app;
if (!api || typeof api.list !== 'function') {
throw new CLIError('App commands are not available in this SDK build.');
}
return api;
}
export async function appList() {
const puter = await ensureClient();
const apps = (await appsApi(puter).list()) ?? [];
if (apps.length === 0) {
ui.info('No apps.');
return;
}
for (const a of apps) {
const name = a?.name ?? String(a);
ui.out(a?.title ? `${name}\t${ui.dim(a.title)}` : name);
}
}
export async function appGet(nameArg) {
const puter = await ensureClient();
let app;
try {
app = await appsApi(puter).get(nameArg);
} catch (err) {
throw new CLIError(`Could not fetch '${nameArg}': ${err.message}`);
}
if (!app) throw new CLIError(`App '${nameArg}' not found.`);
ui.out(`name: ${app.name ?? nameArg}`);
if (app.title) ui.out(`title: ${app.title}`);
if (app.index_url || app.url) ui.out(`url: ${app.index_url ?? app.url}`);
}
+48
View File
@@ -0,0 +1,48 @@
import { isInteractive, canPrompt } from '../lib/env.js';
import { saveToken } from '../lib/config.js';
import { makeClient } from '../lib/client.js';
import { interactiveLogin, readStdin } from '../lib/auth.js';
import { CLIError } from '../lib/errors.js';
import * as ui from '../lib/ui.js';
const BETA_NOTICE =
ui.dim('Note: the Puter CLI is in beta (0.x) — behavior may change.');
export async function loginCommand(opts) {
let token;
if (opts.withToken) {
// Token comes from stdin, never argv. If stdin is a TTY there's nothing
// piped in and a read would hang — fail fast with guidance instead.
if (canPrompt()) {
throw new CLIError(
'No token piped in.',
{ hint: 'Usage: echo $TOKEN | puter login --with-token' },
);
}
token = await readStdin();
if (!token) {
throw new CLIError('Empty token received on stdin.');
}
} else if (isInteractive() && canPrompt()) {
token = await interactiveLogin();
} else {
throw new CLIError(
'Cannot log in non-interactively without a token.',
{ hint: 'Pipe one in: echo $TOKEN | puter login --with-token' },
);
}
// Verify the token before persisting it.
const puter = makeClient(token);
let user;
try {
user = await puter.auth.getUser();
} catch (err) {
throw new CLIError(`Token rejected: ${err.message}`);
}
saveToken(token);
ui.success(`Logged in as ${ui.bold(user?.username ?? 'unknown')}`);
ui.status(BETA_NOTICE);
}
+11
View File
@@ -0,0 +1,11 @@
import { getStoredToken, clearToken } from '../lib/config.js';
import * as ui from '../lib/ui.js';
export async function logoutCommand() {
if (!getStoredToken()) {
ui.info('No stored login to clear.');
return;
}
clearToken();
ui.success('Logged out.');
}
+258
View File
@@ -0,0 +1,258 @@
import fs from 'node:fs';
import path from 'node:path';
import * as clack from '@clack/prompts';
import { isInteractive, SITE_DOMAIN } from '../lib/env.js';
import { ensureClient } from '../lib/auth.js';
import { randName } from '../lib/client.js';
import { walk } from '../lib/fswalk.js';
import { CLIError } from '../lib/errors.js';
import * as ui from '../lib/ui.js';
// --- subdomain helpers (spec §5.3): liberal in, strict out ----------------
function normalizeSubdomain(input) {
let s = String(input).trim().toLowerCase();
// accept a pasted full host like "my-app.puter.site"
const suffix = `.${SITE_DOMAIN}`;
if (s.endsWith(suffix)) s = s.slice(0, -suffix.length);
return s;
}
function subdomainError(s) {
if (!s) return 'Subdomain is required.';
if (!/^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$/.test(s)) {
return 'Use only lowercase letters, numbers and hyphens (not at the ends).';
}
return undefined;
}
function siteUrl(subdomain) {
return `https://${subdomain}.${SITE_DOMAIN}`;
}
function bail(value) {
if (clack.isCancel(value)) throw new CLIError('Cancelled.');
return value;
}
// OS/junk files the platform ignores (uploading only these yields an empty
// batch → EMPTY_UPLOAD), so we strip them before uploading.
const IGNORED_NAMES = new Set(['.DS_Store', 'Thumbs.db', '.localized']);
function isIgnored(relPath) {
return IGNORED_NAMES.has(relPath.split('/').pop());
}
// Build a File the SDK's upload() accepts, preserving the file's relative path
// so nested directories are recreated. The SDK overwrites .filepath/.fullPath
// with the basename but reads .finalPath first, so that's where the rel path
// has to go.
function toUploadFile(buf, relPath) {
const file = new File([buf], relPath.split('/').pop());
file.finalPath = relPath;
return file;
}
// --- deploy (spec §5.5) ---------------------------------------------------
export async function siteDeploy(dirArg, subArg, opts) {
const interactive = isInteractive();
// Non-interactive: both positionals are required (spec §5.1).
if (!interactive && (!dirArg || !subArg)) {
throw new CLIError(
'Both a directory and subdomain are required in non-interactive mode.',
{ hint: 'Usage: puter site deploy <dir> <subdomain>' },
);
}
// 1a. Resolve + validate directory (before auth, so the footgun hint shows
// even when not logged in; the dir prompt doesn't need a client).
let dirInput = dirArg;
if (!dirInput && interactive) {
dirInput = bail(
await clack.text({ message: 'Directory to deploy', initialValue: '.' }),
);
}
const dir = path.resolve(dirInput);
if (!fs.existsSync(dir) || !fs.statSync(dir).isDirectory()) {
// Positional footgun mitigation (spec §5.4): a lone arg is the directory.
throw new CLIError(`No such directory '${dirInput}'.`, {
hint: `(To deploy the current folder to that subdomain, run: puter site deploy . ${dirInput})`,
});
}
const puter = await ensureClient();
// 1b. Resolve + validate subdomain (re-prompt interactively).
let subdomain;
if (subArg) {
subdomain = normalizeSubdomain(subArg);
const err = subdomainError(subdomain);
if (err) throw new CLIError(`Invalid subdomain: ${err}`);
} else if (interactive) {
const suggested = await randName(puter);
subdomain = normalizeSubdomain(
bail(
await clack.text({
message: `Subdomain (.${SITE_DOMAIN})`,
initialValue: suggested,
validate: (v) => subdomainError(normalizeSubdomain(v)),
}),
),
);
}
// Collect files, dropping OS junk the platform ignores.
const files = walk(dir).filter((f) => !isIgnored(f.rel));
// Empty-directory guard.
if (files.length === 0) {
if (interactive) {
const go = bail(
await clack.confirm({
message: `'${dirInput}' has no uploadable files. Deploy anyway?`,
initialValue: false,
}),
);
if (!go) throw new CLIError('Cancelled.');
} else {
ui.warn(`'${dirInput}' has no uploadable files — deploying anyway.`);
}
}
// 2. Echo resolved values (spec §5.2) then act.
ui.status(`Deploying ${ui.bold(dirInput)} → ${ui.bold(siteUrl(subdomain).replace('https://', ''))}`);
ui.debug('resolved dir:', dir);
ui.debug('subdomain:', subdomain);
ui.debug(`uploading ${files.length} file(s):`);
for (const f of files) ui.debug(' -', f.rel);
// 3. Atomic, versioned folder (spec §5.6). dedupeName and createMissingParents
// conflict: createMissingParents gives mkdir `-p` semantics, so when the
// `deployment` folder already exists the server returns it as-is instead of
// deduping — clobbering the previous version. So we ensure the parent exists
// first (tolerating "already exists"), then create the deployment folder with
// dedupe only. Each deploy then gets its own auto-numbered folder
// (deployment, deployment (1), ...) and older versions are preserved.
try {
await puter.fs.mkdir(`~/Sites/${subdomain}`, { createMissingParents: true });
} catch (err) {
// Parent already exists (the common case after the first deploy) — fine.
ui.debug('parent mkdir note:', err?.message ?? JSON.stringify(err));
}
const folder = await puter.fs.mkdir(`~/Sites/${subdomain}/deployment`, {
dedupeName: true,
});
const targetPath = folder?.path ?? folder;
ui.debug('mkdir returned:', JSON.stringify(folder));
ui.debug('target path:', targetPath);
// 4. Upload the whole tree in one batch. Each File carries its relative path
// (via finalPath) so nested folders are recreated under targetPath;
// createMissingParents builds those intermediate folders server-side.
const items = files.map((f) => toUploadFile(fs.readFileSync(f.full), f.rel));
const sp = ui.spinner(`Uploading ${items.length} file(s)...`);
try {
await puter.fs.upload(items, targetPath, {
overwrite: true,
createMissingParents: true,
});
sp.stop(`Uploaded ${items.length} file(s).`);
} catch (err) {
sp.stop('Upload failed.');
ui.debug('upload error object:', JSON.stringify(err));
throw new CLIError(`Upload failed: ${err.message ?? JSON.stringify(err)}`);
}
// 5. Point the subdomain at the new folder.
let existing = null;
try {
existing = await puter.hosting.get(subdomain);
} catch {
existing = null; // treat "not found" as creatable
}
if (existing) {
if (interactive) {
const go = bail(
await clack.confirm({
message: `Update existing site '${subdomain}' to this deploy?`,
initialValue: true,
}),
);
if (!go) throw new CLIError('Cancelled.');
}
await puter.hosting.update(subdomain, targetPath);
} else {
try {
await puter.hosting.create(subdomain, targetPath);
} catch (err) {
throw new CLIError(
`Could not create subdomain '${subdomain}': ${err.message}`,
{ hint: 'It may be taken by another account — try a different name.' },
);
}
}
ui.success('Deployed.');
ui.out(siteUrl(subdomain));
}
// --- list / get / delete --------------------------------------------------
export async function siteList() {
const puter = await ensureClient();
const sites = (await puter.hosting.list()) ?? [];
if (sites.length === 0) {
ui.info('No sites yet. Deploy one with: puter site deploy');
return;
}
for (const s of sites) {
const sub = s?.subdomain ?? s?.name ?? String(s);
ui.out(`${sub}\t${ui.url(siteUrl(sub))}`);
}
}
export async function siteGet(subdomainArg) {
const puter = await ensureClient();
const subdomain = normalizeSubdomain(subdomainArg);
let site;
try {
site = await puter.hosting.get(subdomain);
} catch (err) {
throw new CLIError(`Could not fetch '${subdomain}': ${err.message}`);
}
if (!site) throw new CLIError(`Site '${subdomain}' not found.`);
ui.out(`subdomain: ${site.subdomain ?? subdomain}`);
ui.out(`url: ${siteUrl(site.subdomain ?? subdomain)}`);
const root = site.root_dir ?? site.path ?? site.dir_path;
if (root) ui.out(`root: ${typeof root === 'object' ? root.path : root}`);
}
export async function siteDelete(subdomainArg, opts) {
const puter = await ensureClient();
const subdomain = normalizeSubdomain(subdomainArg);
if (isInteractive() && !opts.yes) {
const go = bail(
await clack.confirm({
message: `Delete site '${subdomain}'? This removes the subdomain.`,
initialValue: false,
}),
);
if (!go) throw new CLIError('Cancelled.');
}
try {
await puter.hosting.delete(subdomain);
} catch (err) {
throw new CLIError(`Could not delete '${subdomain}': ${err.message}`);
}
ui.success(`Deleted '${subdomain}'.`);
}
+16
View File
@@ -0,0 +1,16 @@
import { ensureClient } from '../lib/auth.js';
import { CLIError } from '../lib/errors.js';
import * as ui from '../lib/ui.js';
export async function whoamiCommand() {
const puter = await ensureClient();
let user;
try {
user = await puter.auth.getUser();
} catch (err) {
throw new CLIError(`Could not fetch account: ${err.message}`);
}
ui.out(user?.username ?? '(unknown)');
if (user?.email) ui.status(ui.dim(user.email));
}
+175
View File
@@ -0,0 +1,175 @@
import fs from 'node:fs';
import path from 'node:path';
import * as clack from '@clack/prompts';
import { isInteractive, WORKER_DOMAIN } from '../lib/env.js';
import { ensureClient } from '../lib/auth.js';
import { randName } from '../lib/client.js';
import { CLIError } from '../lib/errors.js';
import * as ui from '../lib/ui.js';
function bail(value) {
if (clack.isCancel(value)) throw new CLIError('Cancelled.');
return value;
}
// Workers are served at <name>.puter.work (the SDK lowercases the name).
function workerUrl(name) {
return `https://${String(name).toLowerCase()}.${WORKER_DOMAIN}`;
}
function nameError(s) {
if (!s) return 'Name is required.';
if (!/^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$/i.test(s)) {
return 'Use only letters, numbers and hyphens (not at the ends).';
}
return undefined;
}
// --- deploy (spec §6): get-first, then branch -----------------------------
export async function workerDeploy(fileArg, nameArg, opts) {
const interactive = isInteractive();
if (!interactive && (!fileArg || !nameArg)) {
throw new CLIError(
'Both a file and name are required in non-interactive mode.',
{ hint: 'Usage: puter worker deploy <file> <name>' },
);
}
// Resolve + validate entry file before auth (no cwd-equivalent default).
let fileInput = fileArg;
if (!fileInput && interactive) {
fileInput = bail(
await clack.text({
message: "Worker's JavaScript file",
validate: (v) => (v && v.trim() ? undefined : 'A file is required'),
}),
);
}
const file = path.resolve(fileInput);
if (!fs.existsSync(file) || !fs.statSync(file).isFile()) {
throw new CLIError(`No such file '${fileInput}'.`);
}
const puter = await ensureClient();
// Resolve name.
let name = nameArg;
if (!name && interactive) {
const suggested = await randName(puter);
name = bail(
await clack.text({
message: 'Worker name',
initialValue: suggested,
validate: nameError,
}),
);
}
const nameErr = nameError(name);
if (nameErr) throw new CLIError(`Invalid name: ${nameErr}`);
ui.status(`Deploying ${ui.bold(fileInput)} → worker ${ui.bold(name)}`);
// 1. Does it already exist?
let existing = null;
try {
existing = await puter.workers.get(name);
} catch {
existing = null;
}
// 2/3. Overwrite the backing file in place; register only if new.
const remotePath = `~/Workers/${name}.js`;
const code = fs.readFileSync(file);
const sp = ui.spinner('Uploading worker...');
let created = null;
try {
await puter.fs.write(remotePath, code, {
overwrite: true,
createMissingParents: true,
});
if (!existing) {
created = await puter.workers.create(name, remotePath);
sp.stop('Worker created.');
} else {
sp.stop('Worker file updated.');
}
} catch (err) {
sp.stop('Deploy failed.');
throw new CLIError(`Deploy failed: ${err.message}`);
}
// Beta caveat (spec §6 / §9): no liveness signal to confirm the deploy.
ui.warn(
'Beta: there is no readiness signal yet, so an effective deploy cannot be confirmed.',
);
// Prefer the URL the API returns; fall back to the canonical form.
ui.out(created?.url ?? existing?.url ?? workerUrl(name));
}
// --- list / get / delete --------------------------------------------------
export async function workerList() {
const puter = await ensureClient();
const workers = (await puter.workers.list()) ?? [];
if (workers.length === 0) {
ui.info('No workers yet. Deploy one with: puter worker deploy');
return;
}
for (const w of workers) {
const name = w?.name ?? String(w);
ui.out(`${name}\t${ui.url(w?.url ?? workerUrl(name))}`);
}
}
export async function workerGet(nameArg) {
const puter = await ensureClient();
let worker;
try {
worker = await puter.workers.get(nameArg);
} catch (err) {
throw new CLIError(`Could not fetch '${nameArg}': ${err.message}`);
}
if (!worker) throw new CLIError(`Worker '${nameArg}' not found.`);
ui.out(`name: ${worker.name ?? nameArg}`);
ui.out(`url: ${worker.url ?? workerUrl(worker.name ?? nameArg)}`);
if (worker.file_path || worker.path) {
ui.out(`file: ${worker.file_path ?? worker.path}`);
}
}
export async function workerDelete(nameArg, opts) {
const puter = await ensureClient();
if (isInteractive() && !opts.yes) {
const go = bail(
await clack.confirm({
message: `Delete worker '${nameArg}'?`,
initialValue: false,
}),
);
if (!go) throw new CLIError('Cancelled.');
}
// Delete the worker first, then its backing file (the order the platform
// wants: unregister before removing the file it points at).
try {
await puter.workers.delete(nameArg);
} catch (err) {
throw new CLIError(`Could not delete '${nameArg}': ${err.message}`);
}
const remotePath = `~/Workers/${nameArg}.js`;
try {
await puter.fs.delete(remotePath);
} catch (err) {
// Worker is already gone; the file may not exist or have a different name.
ui.warn(`Worker deleted, but could not remove ${remotePath}: ${err?.message ?? err}`);
}
ui.success(`Deleted worker '${nameArg}'.`);
}
+57
View File
@@ -0,0 +1,57 @@
// Authentication flows + the gate every authenticated command goes through.
import * as clack from '@clack/prompts';
import { isInteractive, canPrompt } from './env.js';
import { getToken, saveToken } from './config.js';
import { makeClient, getAuthTokenViaBrowser } from './client.js';
import { CLIError } from './errors.js';
function bailIfCancelled(value) {
if (clack.isCancel(value)) {
throw new CLIError('Cancelled.');
}
return value;
}
// Read a token from stdin (the `--with-token` model — keeps it out of argv,
// shell history and `ps`).
export async function readStdin() {
const chunks = [];
for await (const chunk of process.stdin) chunks.push(chunk);
return Buffer.concat(chunks).toString('utf8').trim();
}
// Interactive login. Web-browser flow only — username/password login is no
// longer supported by the platform. Returns a token.
export async function interactiveLogin() {
try {
return await getAuthTokenViaBrowser();
} catch (err) {
throw new CLIError(`Web login failed: ${err.message}`, {
hint: 'You can instead run: echo $TOKEN | puter login --with-token',
});
}
}
// The gate: resolve a token (env → stored → inline login) and return a client.
export async function ensureClient() {
let token = getToken();
if (!token) {
if (isInteractive() && canPrompt()) {
const proceed = bailIfCancelled(
await clack.confirm({ message: 'Not logged in — log in now?' }),
);
if (!proceed) {
throw new CLIError('Not authenticated.');
}
token = await interactiveLogin();
saveToken(token);
} else {
throw new CLIError(
'Not authenticated.',
{ hint: "Set PUTER_AUTH_TOKEN or run 'puter login'." },
);
}
}
return makeClient(token);
}
+79
View File
@@ -0,0 +1,79 @@
// Puter.js SDK bootstrap for Node.
//
// The SDK ships a CommonJS entry (`init.cjs`); we're an ESM package, so it's
// loaded via createRequire. `init(token)` returns a configured `puter` object
// exposing .auth / .fs / .hosting / .workers, etc.
import { createRequire } from 'node:module';
const require = createRequire(import.meta.url);
let sdk;
function loadSdk() {
if (sdk) return sdk;
try {
sdk = require('@heyputer/puter.js/src/init.cjs');
} catch (err) {
throw new Error(
`Could not load the Puter.js SDK (@heyputer/puter.js). ` +
`Run \`npm install\` in the CLI directory.\n${err.message}`,
);
}
return sdk;
}
// The SDK's Node XMLHttpRequest shim (xhrshim.js) calls
// `resp.headers.get('content-type').includes(...)` with no null-guard, which
// throws when a response omits Content-Type — exactly what the signed-URL PUT
// uploads (the fast, parallel signed-batch-write path) return. Patch the global
// Headers.get so a missing content-type reads as '' instead of null; this keeps
// the fast upload path working. Idempotent. Remove once the SDK null-guards it.
function patchHeadersContentType() {
if (typeof Headers === 'undefined') return;
if (Headers.prototype.__puterCliContentTypePatched) return;
const original = Headers.prototype.get;
Headers.prototype.get = function (name) {
const value = original.call(this, name);
if (value === null && String(name).toLowerCase() === 'content-type') {
return '';
}
return value;
};
Headers.prototype.__puterCliContentTypePatched = true;
}
// Build an authenticated client from a token.
export function makeClient(token) {
const { init } = loadSdk();
patchHeadersContentType();
return init(token);
}
// Browser-based auth flow (used by interactive `login`). Returns a token.
export async function getAuthTokenViaBrowser() {
const { getAuthToken } = loadSdk();
if (typeof getAuthToken !== 'function') {
throw new Error('Web login is not available in this SDK build.');
}
return getAuthToken();
}
// Best-effort random domain-safe name from the SDK, with a local fallback
// so prompts still get a sensible pre-fill if randName is unavailable.
const ADJ = ['swift', 'brave', 'lucky', 'calm', 'bright', 'bold', 'sunny', 'cosmic'];
const NOUN = ['otter', 'falcon', 'maple', 'comet', 'harbor', 'cedar', 'pixel', 'delta'];
export async function randName(puter) {
try {
if (puter && typeof puter.randName === 'function') {
const n = puter.randName();
return n && typeof n.then === 'function' ? await n : n;
}
} catch {
// fall through to local generator
}
const a = ADJ[Math.floor(Math.random() * ADJ.length)];
const n = NOUN[Math.floor(Math.random() * NOUN.length)];
const suffix = Math.floor(Math.random() * 9000) + 1000;
return `${a}-${n}-${suffix}`;
}
+52
View File
@@ -0,0 +1,52 @@
// Token storage (spec §4.3).
//
// Isolated accessors so logout/multi-account only ever touch this file.
// Stored in a future-proof multi-account shape from day one even though
// multi-account itself is deferred:
//
// { "accounts": { "default": { "token": "..." } }, "active": "default" }
//
// Plaintext-with-permissions (chmod 0600) is the accepted bar for a deploy
// CLI; conf's encryptionKey is obfuscation, not security, so we don't use it.
import fs from 'node:fs';
import Conf from 'conf';
const config = new Conf({ projectName: 'puter-cli' });
export const configPath = config.path;
function lockDown() {
// conf doesn't restrict permissions by default — owner read/write only.
try {
fs.chmodSync(config.path, 0o600);
} catch {
// best effort (e.g. Windows / file not yet flushed)
}
}
function activeProfile() {
return config.get('active') || 'default';
}
export function saveToken(token) {
config.set('accounts.default.token', token);
config.set('active', 'default');
lockDown();
}
// The stored token only (no env). Used by `whoami`/`logout`-adjacent display.
export function getStoredToken() {
return config.get(`accounts.${activeProfile()}.token`);
}
// Full resolution order for authenticated commands (spec §4.2):
// 1. PUTER_AUTH_TOKEN 2. stored token
export function getToken() {
return process.env.PUTER_AUTH_TOKEN ?? getStoredToken();
}
export function clearToken() {
config.delete(`accounts.${activeProfile()}.token`);
lockDown();
}
+30
View File
@@ -0,0 +1,30 @@
// Environment / interactivity detection (spec §3).
//
// "Interactive" = we may prompt because a human is attached to both ends.
// stream.isTTY is `true` on a terminal and `undefined` otherwise, so we coerce
// with Boolean(...) rather than comparing === true.
export function isInteractive() {
if (process.env.CI) return false; // respect the CI convention
return Boolean(process.stdin.isTTY && process.stdout.isTTY);
}
// Whether stdin specifically can be prompted on (the load-bearing check).
export function canPrompt() {
if (process.env.CI) return false;
return Boolean(process.stdin.isTTY);
}
// Spinners / animations / color are gated on stdout only — skip them when
// output is piped to a file even if stdin is still a terminal.
export function canAnimate() {
return Boolean(process.stdout.isTTY) && !process.env.CI;
}
// Default API + hosting endpoints, overridable for self-hosted instances.
export const API_ORIGIN =
process.env.PUTER_API_ORIGIN || 'https://api.puter.com';
export const SITE_DOMAIN =
process.env.PUTER_SITE_DOMAIN || 'puter.site';
export const WORKER_DOMAIN =
process.env.PUTER_WORKER_DOMAIN || 'puter.work';
+64
View File
@@ -0,0 +1,64 @@
// Uniform error handling. CLIError carries a user-facing message plus an
// optional hint and exit code; anything else is treated as an unexpected
// failure (still non-zero exit, so automation notices).
import chalk from 'chalk';
export class CLIError extends Error {
constructor(message, { hint, code = 1 } = {}) {
super(message);
this.name = 'CLIError';
this.hint = hint;
this.exitCode = code;
}
}
// Puter SDK rejections are often plain objects like { status, message } or
// { error: { message } } rather than Error instances — dig out something
// human-readable before falling back to a stringified object.
export function messageOf(err) {
if (!err) return String(err);
if (typeof err === 'string') return err;
return (
err.message ||
err.error?.message ||
(typeof err.error === 'string' ? err.error : null) ||
(() => {
try {
return JSON.stringify(err);
} catch {
return String(err);
}
})()
);
}
// The Puter SDK holds an open connection that keeps the event loop alive, so
// commands won't exit on their own. Force an exit after flushing both streams
// (writing '' invokes the callback once pending output has drained), so piped
// output is never truncated.
export function flushAndExit(code) {
let pending = 2;
const done = () => {
if (--pending === 0) process.exit(code);
};
process.stdout.write('', done);
process.stderr.write('', done);
}
export function fail(err) {
const message = messageOf(err);
console.error(chalk.red('Error:') + ' ' + message);
if (err instanceof CLIError && err.hint) {
console.error(chalk.dim(err.hint));
}
flushAndExit(err instanceof CLIError ? err.exitCode : 1);
}
// Wrap an async commander action so it always exits cleanly: 0 on success,
// non-zero (via fail) on error.
export function action(fn) {
return (...args) => {
Promise.resolve(fn(...args)).then(() => flushAndExit(0), fail);
};
}
+23
View File
@@ -0,0 +1,23 @@
// Recursively list regular files in a local directory.
// Returns { full, rel } where `rel` is a POSIX path relative to the root,
// suitable for building remote Puter paths.
import fs from 'node:fs';
import path from 'node:path';
export function walk(root) {
const files = [];
(function recurse(dir, base) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
const rel = base ? `${base}/${entry.name}` : entry.name;
if (entry.isDirectory()) {
recurse(full, rel);
} else if (entry.isFile()) {
files.push({ full, rel });
}
// symlinks / sockets / fifos are skipped
}
})(root, '');
return files;
}
+66
View File
@@ -0,0 +1,66 @@
// Output helpers.
//
// Convention: status, progress, prompts and spinners go to STDERR; actual
// data (URLs, JSON, list rows) goes to STDOUT. That way `puter site list`
// can be piped without status noise contaminating the data stream.
import chalk from 'chalk';
import * as clack from '@clack/prompts';
import { canAnimate } from './env.js';
// Data → stdout
export function out(line = '') {
console.log(line);
}
// Status → stderr
export function status(line = '') {
console.error(line);
}
export function success(msg) {
console.error(chalk.green('✔') + ' ' + msg);
}
export function warn(msg) {
console.error(chalk.yellow('!') + ' ' + msg);
}
export function info(msg) {
console.error(chalk.dim(msg));
}
// Verbose diagnostics → stderr, only when PUTER_DEBUG is set.
export function debug(...args) {
if (process.env.PUTER_DEBUG) {
console.error(chalk.magenta('[debug]'), ...args);
}
}
export function url(u) {
return chalk.cyan(u);
}
export function bold(s) {
return chalk.bold(s);
}
export function dim(s) {
return chalk.dim(s);
}
// Spinner that degrades to a one-line stderr message when output isn't a TTY.
export function spinner(startText) {
if (canAnimate()) {
const s = clack.spinner();
s.start(startText);
return s;
}
console.error(startText);
return {
message() {},
stop(text) {
if (text) console.error(text);
},
};
}
+73 -21
View File
@@ -9,8 +9,36 @@ const { encode } = require('html-entities');
const { JSDOM } = require('jsdom');
const yaml = require('js-yaml');
const esbuild = require('esbuild');
const MiniSearch = require('minisearch');
const { generatePlayground } = require('./src/playground');
// Index-time tokenizer. Splits on whitespace AND punctuation, and ALSO emits
// contiguous pairwise (2-gram) joins of the parts:
// "Node.js" → ["node", "js", "nodejs"]
// "puter.ai.chat()" → ["puter", "ai", "chat", "puterai", "aichat"]
// This lets queries like "nodejs", "puterai", or "aichat" match directly.
function indexTokenize (text) {
if ( !text ) return [];
const tokens = [];
for ( const chunk of text.split(/[\n\r\p{Z}]+/u) ) {
if ( !chunk ) continue;
const parts = chunk.split(/\p{P}+/u).filter(Boolean);
// Individual parts
for ( const part of parts ) tokens.push(part);
// Pairwise adjacent joins (linear in parts.length)
for ( let i = 0; i + 1 < parts.length; i++ ) {
tokens.push(parts[i] + parts[i + 1]);
}
}
return tokens;
}
const MINISEARCH_INDEX_CONFIG = {
fields: ['title', 'text', 'pageTitle'],
storeFields: ['title', 'pageTitle', 'path', 'anchor', 'text'],
tokenize: indexTokenize,
};
const site = 'https://docs.puter.com';
let usedPlaygroundExamples = new Set();
@@ -866,14 +894,33 @@ IMPORTANT: when creating an app, include a link to 'https://developer.puter.com'
fs.writeFileSync(outputFile, outputContent, 'utf8');
};
function markdownToPlainText (markdown) {
function splitHtmlIntoSections (markdown) {
const html = marked.parse(markdown);
const dom = new JSDOM(html);
const body = dom.window.document.body;
const dom = new JSDOM();
const div = dom.window.document.createElement('div');
div.innerHTML = html;
const sections = [];
let current = { heading: null, slug: '', textParts: [] };
return div.textContent.replace(/\s+/g, ' ').trim();
for ( const child of body.children ) {
if ( child.tagName === 'H2' ) {
sections.push(current);
current = {
heading: child.textContent.trim(),
slug: child.id || '',
textParts: [],
};
} else {
current.textParts.push(child.textContent);
}
}
sections.push(current);
return sections.map(s => ({
heading: s.heading,
slug: s.slug,
text: s.textParts.join(' ').replace(/\s+/g, ' ').trim(),
}));
}
const generateSearchIndex = () => {
@@ -881,23 +928,30 @@ const generateSearchIndex = () => {
const outputFile = path.join(currentDir, 'dist', 'index.json');
const json = [];
const pushPage = (pageTitle, pagePath, markdown) => {
const { content } = parseFrontMatter(markdown);
const sections = splitHtmlIntoSections(content);
sections.forEach((section) => {
const isIntro = section.heading === null;
json.push({
title: isIntro ? pageTitle : section.heading,
pageTitle,
path: pagePath,
anchor: isIntro ? '' : section.slug,
text: section.text,
});
});
};
const indexFile = path.join(currentDir, 'src', 'index.md');
const indexMarkdown = fs.readFileSync(indexFile, 'utf8');
json.push({
title: 'Puter.js',
path: '',
text: markdownToPlainText(indexMarkdown),
});
pushPage('Puter.js', '', indexMarkdown);
sidebar.forEach((item) => {
if ( item.source ) {
const file = path.join(currentDir, 'src', item.source);
const markdown = fs.readFileSync(file, 'utf8');
json.push({
title: item.title_tag ?? item.title,
path: item.path,
text: markdownToPlainText(markdown),
});
pushPage(item.title_tag ?? item.title, item.path, markdown);
}
if ( item.children && Array.isArray(item.children) ) {
@@ -905,17 +959,15 @@ const generateSearchIndex = () => {
if ( child.source ) {
const file = path.join(currentDir, 'src', child.source);
const markdown = fs.readFileSync(file, 'utf8');
json.push({
title: child.title_tag ?? child.title,
path: child.path,
text: markdownToPlainText(markdown),
});
pushPage(child.title_tag ?? child.title, child.path, markdown);
}
});
}
});
fs.writeFileSync(outputFile, JSON.stringify(json), 'utf8');
const miniSearch = new MiniSearch(MINISEARCH_INDEX_CONFIG);
miniSearch.addAll(json.map((doc, id) => ({ id, ...doc })));
fs.writeFileSync(outputFile, JSON.stringify(miniSearch.toJSON()), 'utf8');
};
// Main execution
+1
View File
@@ -27,6 +27,7 @@
"js-yaml": "^4.1.0",
"jsdom": "^26.1.0",
"marked": "^11.1.1",
"minisearch": "^7.2.0",
"nwsapi": "^2.2.23"
},
"devDependencies": {
+2
View File
@@ -9,6 +9,8 @@ It comes with a comprehensive but familiar file system operations including writ
With Puter.js, you don't need to worry about setting up storage infrastructure such as configuring buckets, managing CDNs, or ensuring availability, since everything is handled for you. Additionally, with the [User-Pays Model](/user-pays-model/), you don't have to worry about storage or bandwidth costs, as users of your application cover their own usage.
<div class="info"><strong>Need to share data across users?</strong> Each user's files live in their own account, so one user can't read another's data. To keep centralized files that every user reads from and writes to, use a <a href="/Workers/">Serverless Worker</a> — its code can act on the worker owner's resources, giving all users one shared backend.</div>
## Features
<div style="overflow:hidden; margin-bottom: 30px;">
+2
View File
@@ -9,6 +9,8 @@ It supports various operations such as set, get, delete, list keys, increment an
Puter.js handles all the infrastructure for you, so you don't need to set up servers, handle scaling, or manage backups. And thanks to the [User-Pays Model](/user-pays-model/), you don't have to worry about storage, read, or write costs, as users of your application cover their own usage.
<div class="info"><strong>Need to share data across users?</strong> Each user's key-value store lives in their own account, so one user can't read another's data. To keep a single, centralized store that every user reads from and writes to, use a <a href="/Workers/">Serverless Worker</a> — its code can act on the worker owner's resources, giving all users one shared backend.</div>
## Features
<div style="overflow:hidden; margin-bottom: 30px;">
+65 -1
View File
@@ -5,9 +5,11 @@ description: Run and manage serverless JavaScript funcitons in the cloud.
Serverless Workers are serverless functions that run JavaScript code in the cloud.
Workers run server-side, which makes them a good fit for centralized application data and backend logic. See [Integration with Puter.js](/Workers/router/#integration-with-puterjs) for how worker code accesses Puter resources.
## Router
Workers use a router-based system to handle HTTP requests and can integrate with Puter's cloud services like file storage, key-value databases, and AI APIs. Workers are perfect for building backend services, REST APIs, webhooks, and data processing pipelines.
Workers use a router-based system to handle HTTP requests and can integrate with Puter's cloud services like file storage, key-value databases, and AI APIs. Workers are perfect for building backend services, REST APIs, webhooks, shared data stores, and data processing pipelines.
### Examples
@@ -139,3 +141,65 @@ You can see various Puter.js workers management features in action from the foll
- [Get a worker](/playground/workers-get/)
- [Workers Management](/playground/workers-management/)
- [Authenticated Worker Requests](/playground/workers-exec/)
## Deployment
Once your worker is ready, you can put it online on a free `*.puter.work` subdomain.
<div class="info">A worker is created once and keeps its name and URL. To ship changes, overwrite its source file rather than creating a new worker — see <a href="/Workers/create/#updating-a-worker">Updating a worker</a>.</div>
### Publish from puter.com
The quickest way to publish a worker is to create it on [puter.com](https://puter.com) and publish it.
<ol>
<li>
Create a <code>.js</code> file containing your worker code.
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/workers/code.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
<li>
Right-click the file and choose <strong>Publish as Worker</strong>.
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/workers/publish-workers.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
<li>
Pick a name and click <strong>Publish</strong>. Your worker is live at <code>https://your-worker.puter.work</code>.
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/workers/published.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
</ol>
<!-- ### Automate with GitHub Actions
If your worker's code lives on GitHub, you can redeploy it automatically on every push using the [Puter Worker Deploy Action](https://github.com/HeyPuter/puter-worker-deploy-action).
Add a workflow file at `.github/workflows/deploy-worker.yml`:
```yaml
name: Deploy Worker to Puter
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy worker
uses: HeyPuter/puter-worker-deploy-action@v1
with:
worker_name: my-api # publishes to my-api.puter.work
source_path: worker # the folder containing your worker
entry_file: index.js # the worker's entry file
puter_path: ~/workers/my-api # where to store the files on Puter
puter_token: ${{ secrets.PUTER_TOKEN }}
```
<div class="info">Store your Puter auth token as a GitHub Actions secret named <code>PUTER_TOKEN</code>. See the <a href="https://github.com/HeyPuter/puter-worker-deploy-action">action's README</a> for how to obtain a token and for the full list of options.</div> -->
+20 -1
View File
@@ -10,6 +10,8 @@ Creates and deploys a new worker from a JavaScript file containing [router](../r
<div class="info">After a worker is created or updated, full propagation may take between 5 to 30 seconds to fully take effect across all edge servers. </div>
<div class="info">A worker is tied to its <strong>name</strong> — you create it <strong>once</strong> and keep that name. To deploy changes, don't call <code>create()</code> again with a new name; instead overwrite the worker's source file (see <a href="#updating-a-worker">Updating a worker</a> below). Recreating under a different name leaves the old worker live at its old URL while your callers end up pointing at an orphaned one.</div>
## Syntax
@@ -85,4 +87,21 @@ puter.workers.create('my-api', 'api-server.js')
</script>
</body>
</html>
```
```
## Updating a worker
A worker keeps the same name and URL for its whole lifetime. You create it once with `create()`; after that, you **update it by overwriting its source file**, not by creating a new worker.
[`puter.workers.get()`](/Workers/get/) returns the worker's [`file_path`](/Objects/workerinfo), so you can write your new code back to it:
```js
// Look up the deployed worker's source file
const info = await puter.workers.get('my-api');
// Overwrite it with your new code — this redeploys the worker
// at the same name and URL
await puter.fs.write(info.file_path, updatedWorkerCode);
```
The worker redeploys from that file, so `https://my-api.puter.work` keeps serving — now running your updated code. Anything already calling the worker keeps working without changes.
+96 -22
View File
@@ -28,35 +28,88 @@ The router object supports standard HTTP methods and provides a clean way to org
### Handler Parameters
Route handlers receive structured parameters:
Route handlers receive a single object as their parameter, which can be destructured into the following properties:
- `request` - The incoming [HTTP request](https://developer.mozilla.org/en-US/docs/Web/API/Request).
- `user` - The user object, contains `user.puter` (available when called via [`puter.workers.exec()`](/Workers/exec/))
- `user.puter` - The user's Puter resources (KV, FS, AI, etc.)
- `params` - URL parameters (for dynamic routes)
- `me` - The deployer's Puter object (your own Puter resources for KV, FS, AI, etc.)
- `user` - An object representing the user who made the request to this worker. It has a `puter` property (`user.puter`) that gives you access to that user's own Puter resources — KV, FS, AI, etc. Only available when the worker is called via [`puter.workers.exec()`](/Workers/exec/).
- `params` - Route parameters captured from the path (see [Route Parameters](#route-parameters))
## Global Objects
When writing worker code, you have access to several global objects:
When writing worker code, you have access to these global objects:
- `router` - The router object for defining API endpoints
- `me.puter` - The deployer's Puter object (your own Puter resources for KV, FS, AI, etc.)
**Note**: `me.puter` refers to the deployer's (your) Puter resources, while `user.puter` refers to the user's resources when they execute your worker with their own token.
- `me` - An object representing you, the worker's owner. It has a `puter` property (`me.puter`) that gives you access to your own Puter resources — KV, FS, AI, etc.
## Integration with Puter.js
Just like in apps or websites, you can use Puter.js in workers to access AI, cloud storage, key-value stores, and databases.
The difference is where the resources are utilized. Normally with Puter.js, all resources belong to your users; each user has their own storage and databases. Workers give you the flexibility in using resources:
The difference is *whose* resources you use. A worker gives you two `.puter` objects to work with, and operations are billed to whichever one you call:
- **Worker context** (`me.puter`) - Store data in your own storage and databases. Use this for shared application data, server-side logic, and centralized resources that you control.
- **User context** (`user.puter`) - Keep data in each user's own storage and databases. This maintains the default [User-Pays model](/user-pays-model/) while still executing logic server-side.
- **`me.puter`** is the **worker context** — your own resources, as the owner. Use this for shared application data, server-side logic, and centralized resources you control. Operations run against your account and are billed to you.
- **`user.puter`** is the **user context** — the resources of the user who called the worker (available when it's executed via [`puter.workers.exec()`](/Workers/exec/), which runs it with their token). This keeps the default [User-Pays model](/user-pays-model/): each user's data stays in their own storage, billed to them, while your logic still runs server-side.
> The `user` object is available when the worker is executed via `puter.workers.exec()` in the frontend and contains the user's own Puter resources.
So you can mix and match within the same codebase — some endpoints reading and writing your own data (`me.puter`), others acting on the calling user's data (`user.puter`).
This means you can choose which parts of your app use centralized resources (your storage/database) versus user-specific resources, all from the same codebase.
## Route Parameters
Sometimes part of a path isn't fixed — like a post ID or a username. You can capture these segments by prefixing them with a colon (`:`) in the route path. Each captured segment becomes a property on the `params` object, keyed by the name you gave it.
```js
router.get("/api/posts/:category/:id", async ({ params }) => {
const { category, id } = params;
return { category, id };
});
```
A request to `/api/posts/tech/42` matches this route and gives you:
- `params.category` → `"tech"`
- `params.id` → `"42"`
You can use as many route parameters as you need. Captured values are always strings, so convert them yourself if you expect a number.
## Wildcard Routes
While a route parameter (`:name`) matches a single segment, a **wildcard** (`*name`) matches the rest of the path — any number of segments. Like a route parameter, the matched value is available on `params`, keyed by the name after the `*`.
```js
router.get("/files/*path", async ({ params }) => {
// A request to /files/images/avatars/me.png gives:
// params.path === "images/avatars/me.png"
return { path: params.path };
});
```
A common use is a catch-all route for unmatched paths — define it last so it only runs when nothing else matched (see the [404 Handler](#examples) example below).
## CORS
Every response from your worker automatically includes `Access-Control-Allow-Origin: *`, so **simple cross-origin requests work out of the box** — a basic `GET` or `POST` from another origin just works, no extra code.
Some requests need a **CORS preflight** first: the browser sends an `OPTIONS` request and waits for the allowed methods and headers before sending the real one. This happens when the request uses a method like `PUT` or `DELETE`, or carries custom headers (e.g. `Authorization`).
To handle this, you can add an `OPTIONS` handler that returns the methods and headers you want to allow:
```js
router.options("/*path", async () => {
return new Response(null, {
status: 204,
headers: {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",
"Access-Control-Allow-Headers": "Content-Type, Authorization, puter-auth",
},
});
});
```
This answers the preflight for any path with the CORS headers the browser expects, so your other routes work cross-origin.
<div class="info">The <code>puter-auth</code> header is important: when you call your worker with <a href="/Workers/exec/"><code>puter.workers.exec()</code></a>, it attaches the user's Puter token in a <code>puter-auth</code> header so the worker can act on the calling user's behalf (this is what populates <code>user.puter</code>). Because that's a custom header, the browser runs a preflight first — so <code>puter-auth</code> must be listed in <code>Access-Control-Allow-Headers</code>, otherwise the preflight fails and the request never reaches your worker.</div>
If you need different CORS rules per endpoint — for example, restricting the allowed methods or headers on a specific route — define an `OPTIONS` handler on that individual path instead of using the wildcard.
## Examples
@@ -91,14 +144,14 @@ router.post("/api/user", async ({ request }) => {
});
```
<strong class="example-title">URL Parameters</strong>
<strong class="example-title">Query Parameters</strong>
```js
router.post("/api/user*tag", async ({ request }) => {
// Get URL parameters
router.get("/api/search", async ({ request }) => {
// Read query string parameters from the URL
const url = new URL(request.url);
const queryParam = url.searchParams.get("param");
return { processed: true };
const query = url.searchParams.get("q");
return { query };
});
```
@@ -112,13 +165,12 @@ router.post("/api/user", async ({ request }) => {
});
```
<strong class="example-title">URL Parameters</strong>
<strong class="example-title">Route Parameters</strong>
Use `:paramName` in your route path to capture dynamic segments:
Use `:name` in your route path to capture route parameters:
```js
router.get("/api/posts/:category/:id", async ({ request, params }) => {
// Dynamic route with parameters
const { category, id } = params;
return { category, id };
});
@@ -210,6 +262,28 @@ router.post("/api/risky-operation", async ({ request }) => {
});
```
<strong class="example-title">Worker Context vs User Context</strong>
The same operation can run against either Puter account. Here, one endpoint reads from the calling user's KV store (`user.puter`), the other from your own (`me.puter`).
```js
// Read from the calling user's KV store (user context)
router.get("/api/kv/user/get", async ({ request, user }) => {
const url = new URL(request.url);
const key = url.searchParams.get("key");
const value = await user.puter.kv.get(key);
return { value };
});
// Read from the worker owner's KV store (worker context)
router.get("/api/kv/worker/get", async ({ request }) => {
const url = new URL(request.url);
const key = url.searchParams.get("key");
const value = await me.puter.kv.get(key);
return { value };
});
```
<strong class="example-title">File System Integration</strong>
```js
+105
View File
@@ -0,0 +1,105 @@
---
title: TypeScript Types
description: Type-checking and autocomplete for Puter Workers via @heyputer/worker-types.
platforms: [workers]
---
The [`@heyputer/worker-types`](https://www.npmjs.com/package/@heyputer/worker-types) package adds TypeScript type definitions for the worker runtime — the `router`, `me`, `my`, `myself`, `puter_auth`, and `puter_endpoint` globals — plus typed route handlers with automatic `params` inference from path literals.
It's purely a development-time aid: it adds nothing to your deployed worker bundle.
## Install
```sh
npm install --save-dev @heyputer/worker-types
```
## Convention: `*.worker.js`
We recommend naming worker files `*.worker.js` (or `*.worker.ts`). This makes the worker-y parts of your project obvious in a file listing and lets you scope the worker globals to just those files — so `router`, `me`, etc. don't leak into the rest of your code.
The Puter GUI's **New > Worker** action creates files as `New Worker.worker.js` and includes the types reference at the top automatically.
## Setup
Pick whichever style fits your project.
### File-scoped (works for any project)
Add a triple-slash reference at the top of each `*.worker.js` / `*.worker.ts` file. This is the line the GUI now adds for you:
```js
/// <reference types="@heyputer/worker-types" />
router.get('/api/hello', ({ request }) => {
return { msg: 'hello' };
});
```
### Project-wide for worker files only
For projects with many workers, add a worker-only `tsconfig.workers.json` that includes only `*.worker.ts` and pulls in the globals:
```json
{
"extends": "./tsconfig.json",
"compilerOptions": {
"types": ["@heyputer/worker-types"]
},
"include": ["**/*.worker.ts"]
}
```
Then exclude the same files from your main `tsconfig.json`:
```json
{
"exclude": ["**/*.worker.ts"]
}
```
Build both with `tsc -p tsconfig.json && tsc -p tsconfig.workers.json`, or wire them up with TypeScript [project references](https://www.typescriptlang.org/docs/handbook/project-references.html).
### Named imports
For users who prefer being explicit:
```ts
import type { Handler, Router, WorkerEvent } from '@heyputer/worker-types';
const getPost: Handler<{ id: string }> = ({ params }) => ({ id: params.id });
router.get('/posts/:id', getPost);
```
Importing anything from the package also pulls the globals into that file, so you don't also need the triple-slash reference.
## Param inference
Path literals are parsed at the type level, so destructured `params` get exact keys without any annotation:
```ts
router.get('/posts/:postId/comments/:commentId', ({ params }) => {
params.postId; // string
params.commentId; // string
});
router.get('/files/*path', ({ params }) => {
params.path; // string — wildcard captures the remainder
});
```
## What's typed
| Global | Type | Description |
|---|---|---|
| `router` | `Router` | Register handlers via `get`/`post`/`put`/`delete`/`options`/`custom`. |
| `me` | `{ puter: Puter }` | Deployer's Puter context (FS, KV, AI, auth, etc). |
| `my`, `myself` | `{ puter: Puter }` | Aliases for `me`. |
| `puter_auth` | `string` | Deployer's auth token (Cloudflare secret binding). |
| `puter_endpoint` | `string` | Puter API endpoint. |
Handler events expose:
- `request` — standard [`Request`](https://developer.mozilla.org/en-US/docs/Web/API/Request)
- `params` — route params, inferred from the path literal
- `user` / `requestor` — caller's Puter context, present only when invoked with a `puter-auth` header (e.g. via [`puter.workers.exec()`](/Workers/exec/))
+12
View File
@@ -421,6 +421,18 @@ ul code {
margin-bottom: 4px;
}
/* Lighter page-title prefix so the matched section heading stays prominent
in the "Page Title › Section" format. */
.search-result-title-page {
color: #888888;
font-weight: 400;
}
.search-result-sep {
color: #cccccc;
margin: 0 6px;
}
.search-result-text {
font-size: 14px;
color: #666666;
+231 -109
View File
@@ -1,10 +1,52 @@
import $ from 'jquery';
import MiniSearch from 'minisearch';
// Global search index
let searchIndex = [];
let miniSearch = null;
let searchTimeout = null;
let selectedSearchResult = -1;
// Last query the user typed while the index was still loading; replayed by
// fetchSearchIndex() once the index becomes available so the user doesn't
// have to retype. Cleared on success, failure, or when the query falls
// below the minimum length.
let pendingQuery = null;
// True after fetchSearchIndex() has thrown at least once; lets performSearch
// distinguish "still loading" from "load failed" without waiting forever.
let indexLoadFailed = false;
// Query-time tokenizer. Splits on whitespace AND punctuation only — does
// NOT emit the joined-without-punctuation form. The index (built in
// build.js's indexTokenize) already pre-baked the joined variants, so
// queries just need to produce the user's natural tokens and let
// exact/prefix/fuzzy match against the indexed terms.
function queryTokenize (text) {
if ( !text ) return [];
const tokens = [];
for ( const chunk of text.split(/[\n\r\p{Z}]+/u) ) {
if ( !chunk ) continue;
const parts = chunk.split(/\p{P}+/u).filter(Boolean);
for ( const part of parts ) tokens.push(part);
}
return tokens;
}
const MINISEARCH_CONFIG = {
fields: ['title', 'text', 'pageTitle'],
storeFields: ['title', 'pageTitle', 'path', 'anchor', 'text'],
tokenize: queryTokenize,
searchOptions: {
boost: { title: 3, pageTitle: 2, text: 1 },
fuzzy: term => {
if ( term.length <= 3 ) return 0;
if ( term.length <= 7 ) return 1;
return 2;
},
prefix: term => term.length >= 3,
combineWith: 'AND',
},
};
const commandIcon = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-command-icon lucide-command"><path d="M15 6v12a3 3 0 1 0 3-3H6a3 3 0 1 0 3 3V6a3 3 0 1 0-3 3h12a3 3 0 1 0-3-3"/></svg>';
$(document).ready(function () {
@@ -86,11 +128,16 @@ $(document).ready(function () {
clearTimeout(searchTimeout);
}
// Set new timeout for debouncing
searchTimeout = setTimeout(async () => {
if ( searchIndex.length == 0 ) {
await fetchSearchIndex();
}
// Synchronously drop any pending replay when the input is too
// short, so a late index load can't resurrect stale results
// after the user clears the field within the debounce window.
if ( !query || query.length < 2 ) {
pendingQuery = null;
}
// Set new timeout for debouncing. performSearch handles the
// not-yet-loaded case itself, so the input handler stays sync.
searchTimeout = setTimeout(() => {
performSearch(query);
}, 300);
});
@@ -100,14 +147,26 @@ $(document).ready(function () {
});
async function fetchSearchIndex () {
indexLoadFailed = false;
try {
const response = await fetch('/index.json');
const data = await response.json();
searchIndex = data;
console.log('Search index loaded:', `${searchIndex.length } items`);
const text = await response.text();
miniSearch = MiniSearch.loadJSON(text, MINISEARCH_CONFIG);
console.log('Search index loaded:', `${miniSearch.documentCount } items`);
// Replay the query the user typed while we were still loading.
if ( pendingQuery !== null ) {
const q = pendingQuery;
pendingQuery = null;
performSearch(q);
}
} catch ( error ) {
console.error('Failed to load search index:', error);
searchIndex = [];
miniSearch = null;
indexLoadFailed = true;
if ( pendingQuery !== null ) {
pendingQuery = null;
renderSearchStatus('failed');
}
}
}
function escapeHtml (text) {
@@ -119,123 +178,177 @@ function escapeHtml (text) {
.replace(/'/g, '&#39;');
}
function escapeRegex (text) {
return text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
function generateTextFragment (matchedText, prefix = '', suffix = '') {
const encodedText = encodeURIComponent(matchedText);
const encodedPrefix = prefix ? `${encodeURIComponent(prefix) }-,` : '';
const encodedSuffix = suffix ? `,-${ encodeURIComponent(suffix)}` : '';
return `#:~:text=${encodedPrefix}${encodedText}${encodedSuffix}`;
return `text=${encodedPrefix}${encodedText}${encodedSuffix}`;
}
// Build the final href for a search result, combining the page path with
// optional section anchor and optional text fragment per the WICG spec.
function buildResultUrl (result) {
let url = `${result.path}/`;
if ( result.anchor ) url += `#${result.anchor}`;
if ( result.textFragment ) {
url += result.anchor
? `:~:${result.textFragment}`
: `#:~:${result.textFragment}`;
}
return url;
}
// Single source of truth for the empty-state UI so the idle / loading /
// failed copy can only drift in one place.
function renderSearchStatus (kind) {
const messages = {
idle: 'Start typing to search...',
loading: 'Loading search index...',
failed: 'Failed to load search index. Please refresh.',
};
$('.search-results').html(
`<div class="search-no-results">${messages[kind]}</div>`
);
}
function performSearch (query) {
if ( !query || query.length < 2 ) {
$('.search-results').html(
'<div class="search-no-results">Start typing to search...</div>');
// Drop any pending replay so the user isn't surprised by a stale
// search firing after they cleared the input.
pendingQuery = null;
renderSearchStatus('idle');
return;
}
const titleResults = [];
const textResults = [];
if ( !miniSearch ) {
if ( indexLoadFailed ) {
renderSearchStatus('failed');
return;
}
// Index still loading; remember the query so fetchSearchIndex
// can replay it on success.
pendingQuery = query;
renderSearchStatus('loading');
return;
}
const hits = miniSearch.search(query).slice(0, 15);
const results = hits.map(hit => decorateHit(hit, query));
updateSearchResults(results);
}
function textPreview (text, maxLen) {
if ( !text ) return '';
if ( text.length <= maxLen ) return text;
return `${text.substring(0, maxLen) }...`;
}
// Apply exact-substring highlighting on top of an engine hit. Priority:
// 1. Query found in section title → mark the title, no text fragment
// (anchor jump alone lands on the heading).
// 2. Query found in section body → mark a snippet around the match,
// attach a text-fragment URL so the browser scrolls to the phrase.
// 3. Fuzzy-only hit (no exact match either place) → no marks, no fragment;
// we just link to the section anchor.
function decorateHit (hit, query) {
const queryLower = query.toLowerCase();
const queryRegex = new RegExp(`(${escapeRegex(query)})`, 'i');
searchIndex.forEach((item) => {
const titleMatch = item.title.toLowerCase().indexOf(queryLower);
if ( titleMatch !== -1 ) {
const highlightedTitle = escapeHtml(item.title).replace(
new RegExp(`(${escapeHtml(query)})`, 'i'),
'<mark>$1</mark>');
const base = {
pageTitle: hit.pageTitle,
path: hit.path,
anchor: hit.anchor,
};
titleResults.push({
title: highlightedTitle,
path: item.path,
text: escapeHtml(item.text.substring(0, 60) + (item.text.length > 60 ? '...' : '')),
textFragment: '',
});
if ( hit.title.toLowerCase().includes(queryLower) ) {
return {
...base,
title: escapeHtml(hit.title).replace(queryRegex, '<mark>$1</mark>'),
text: escapeHtml(textPreview(hit.text, 80)),
textFragment: '',
};
}
const textIdx = hit.text.toLowerCase().indexOf(queryLower);
if ( textIdx !== -1 ) {
const { snippet, textFragment } = buildBodySnippet(hit.text, query, textIdx);
return {
...base,
title: escapeHtml(hit.title),
text: snippet,
textFragment,
};
}
// Fuzzy-only hit
return {
...base,
title: escapeHtml(hit.title),
text: escapeHtml(textPreview(hit.text, 120)),
textFragment: '',
};
}
// Build a word-aligned snippet around an exact match in body text, plus the
// `text=prefix-,match,-suffix` fragment that lets the browser highlight the
// phrase in-page after navigation.
function buildBodySnippet (text, query, matchIdx) {
const queryLen = query.length;
// ±50 chars of context around the match
const contextStart = Math.max(0, matchIdx - 50);
const contextEnd = Math.min(text.length, matchIdx + queryLen + 50);
const contextText = text.substring(contextStart, contextEnd);
const words = contextText.split(/\s+/);
// Find which words the match spans
const matchStart = matchIdx;
const matchEnd = matchIdx + queryLen;
let matchStartWordIndex = -1;
let matchEndWordIndex = -1;
let currentPos = contextStart;
for ( let i = 0; i < words.length; i++ ) {
const wordStart = currentPos;
const wordEnd = wordStart + words[i].length;
if ( wordStart < matchEnd && wordEnd > matchStart ) {
if ( matchStartWordIndex === -1 ) matchStartWordIndex = i;
matchEndWordIndex = i;
}
currentPos = wordEnd + 1; // +1 for separating space
}
const textLower = item.text.toLowerCase();
let searchOffset = 0;
const matchedWords = matchStartWordIndex !== -1
? words.slice(matchStartWordIndex, matchEndWordIndex + 1).join(' ')
: words[0] || '';
// Find all matches in the text
while ( true ) {
const textMatch = textLower.indexOf(queryLower, searchOffset);
if ( textMatch === -1 ) break;
// Adjacent words tighten the text-fragment so it doesn't match elsewhere
const fragmentPrefix = matchStartWordIndex > 0
? words[matchStartWordIndex - 1] : '';
const fragmentSuffix = matchEndWordIndex < words.length - 1
? words[matchEndWordIndex + 1] : '';
const textFragment = generateTextFragment(matchedWords,
fragmentPrefix,
fragmentSuffix);
// Extract 50 chars before and after the match
const contextStart = Math.max(0, textMatch - 50);
const contextEnd = Math.min(item.text.length,
textMatch + query.length + 50);
const contextText = item.text.substring(contextStart, contextEnd);
// Display: up to 4 words on either side, with ellipses if trimmed
const startWord = Math.max(0, matchStartWordIndex - 4);
const endWord = Math.min(words.length, matchEndWordIndex + 5);
let displayText = words.slice(startWord, endWord).join(' ');
if ( startWord > 0 ) displayText = `...${ displayText}`;
if ( endWord < words.length ) displayText = `${displayText }...`;
// Split into words
const words = contextText.split(/\s+/);
const snippet = escapeHtml(displayText).replace(
new RegExp(`(${escapeRegex(query)})`, 'i'),
'<mark>$1</mark>');
// Find all words that intersect with the match range
const matchStart = textMatch;
const matchEnd = textMatch + query.length;
let matchStartWordIndex = -1;
let matchEndWordIndex = -1;
let currentPos = contextStart;
for ( let i = 0; i < words.length; i++ ) {
const wordStart = currentPos;
const wordEnd = wordStart + words[i].length;
// Check if this word intersects with the match
if ( wordStart < matchEnd && wordEnd > matchStart ) {
if ( matchStartWordIndex === -1 ) {
matchStartWordIndex = i;
}
matchEndWordIndex = i;
}
currentPos = wordEnd + 1; // +1 for space
}
// Get the complete matched text (all words that contain the match)
const matchedWords =
matchStartWordIndex !== -1
? words.slice(matchStartWordIndex, matchEndWordIndex + 1).join(' ')
: words[0] || '';
// Get prefix and suffix for text fragment (closest words)
const fragmentPrefix =
matchStartWordIndex > 0 ? words[matchStartWordIndex - 1] : '';
const fragmentSuffix =
matchEndWordIndex < words.length - 1
? words[matchEndWordIndex + 1]
: '';
// Generate text fragment
const textFragment = generateTextFragment(matchedWords,
fragmentPrefix,
fragmentSuffix);
// Create display text (max 4 words before/after)
const startWord = Math.max(0, matchStartWordIndex - 4);
const endWord = Math.min(words.length, matchEndWordIndex + 5);
const displayWords = words.slice(startWord, endWord);
let displayText = displayWords.join(' ');
if ( startWord > 0 ) displayText = `...${ displayText}`;
if ( endWord < words.length ) displayText = `${displayText }...`;
// Highlight the matched text in display
const highlightedChunk = escapeHtml(displayText).replace(
new RegExp(`(${escapeHtml(query)})`, 'i'),
'<mark>$1</mark>');
textResults.push({
title: item.title,
path: item.path,
text: highlightedChunk,
textFragment: textFragment,
});
searchOffset = textMatch + 1;
}
});
updateSearchResults([...titleResults, ...textResults]);
return { snippet, textFragment };
}
function updateSearchResults (results) {
@@ -248,11 +361,20 @@ function updateSearchResults (results) {
let html = '';
results.slice(0, 15).forEach((result, index) => {
const url = `${result.path }/${ result.textFragment || ''}`;
const url = buildResultUrl(result);
// For section entries, show "Page Title › Section Heading" inline. For
// intro entries (no anchor), the result IS the page, so just the title.
// `result.title` may already contain <mark> from title-match highlighting,
// so it's not escaped here; pageTitle is plain text and must be escaped.
const titleHtml = result.anchor && result.pageTitle
? `<span class="search-result-title-page">${escapeHtml(result.pageTitle)}</span>`
+ '<span class="search-result-sep">›</span>'
+ result.title
: result.title;
html += `
<div class="search-result" data-index="${index}">
<a href="${url}" class="search-result-link skip-insta-load">
<div class="search-result-title">${result.title}</div>
<div class="search-result-title">${titleHtml}</div>
<div class="search-result-text">${result.text}</div>
</a>
</div>
+83
View File
@@ -0,0 +1,83 @@
---
title: Deployments
description: Deploy your Puter.js app anywhere, or host your website directly on Puter.
---
Once you've integrated Puter.js into your app, the next step is getting it online.
Puter.js is a regular JavaScript library, so your app deploys like any other website. You can ship it to any hosting platform you already use, or host it directly on [Puter](https://puter.com).
## Deploy anywhere
Because Puter.js runs entirely in the browser, there's no special backend to provision. Build your app as you normally would and serve the output from any static or web hosting provider, such as <a href="https://vercel.com" rel="nofollow">Vercel</a>, <a href="https://pages.cloudflare.com" rel="nofollow">Cloudflare Pages</a>, <a href="https://www.netlify.com" rel="nofollow">Netlify</a>, or <a href="https://pages.github.com" rel="nofollow">GitHub Pages</a>.
<div class="info">Your app should be served over HTTPS by a web server, like any of the hosting providers above.</div>
No extra configuration is required. Your app keeps talking to Puter's services from the browser, wherever it's hosted.
## Deploy to Puter
Puter can also host your website for you, on a free `*.puter.site` subdomain.
### Publish from puter.com
The quickest way to publish a website is to upload it on [puter.com](https://puter.com) and publish it.
<ol>
<li>
Right-click on the desktop and create a new folder for your website's files.
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/free-hosting/create-directory.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
<li>
Open the folder, right-click inside it, and choose <strong>Upload Here</strong> to upload your website's files (your <code>index.html</code> and any other assets).
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/free-hosting/upload-here.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
<li>
Right-click the folder and choose <strong>Publish as Website</strong>.
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/free-hosting/publish-website.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
<li>
Pick a subdomain and click <strong>Publish</strong>. Your site goes live instantly at <code>https://your-subdomain.puter.site</code>.
<figure style="margin: 30px 0;">
<img src="https://developer.puter.com/assets/img/free-hosting/published.webp" style="width: 100%; max-width: 600px; margin: 0 auto; display: block; border-radius: 6px;">
</figure>
</li>
</ol>
<!-- ### Automate with GitHub Actions
If your code lives on GitHub, you can redeploy your site automatically on every push using the [Puter Subdomain Deploy Action](https://github.com/HeyPuter/puter-subdomain-deploy-action).
Add a workflow file at `.github/workflows/deploy.yml`:
```yaml
name: Deploy to Puter
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy website
uses: HeyPuter/puter-subdomain-deploy-action@v1
with:
subdomain: my-site # publishes to my-site.puter.site
source_path: dist # the folder to deploy (e.g. your build output)
puter_path: ~/sites/my-site # where to store the files on Puter
puter_token: ${{ secrets.PUTER_TOKEN }}
```
<div class="info">Store your Puter auth token as a GitHub Actions secret named <code>PUTER_TOKEN</code>. See the <a href="https://github.com/HeyPuter/puter-subdomain-deploy-action">action's README</a> for how to obtain a token and for the full list of options.</div>
If your project has a build step, run it before the deploy step (for example `npm ci && npm run build`) and point `source_path` at the build output. -->
+2
View File
@@ -28,6 +28,8 @@ Once the user has been authenticated, your app will get a few things by default:
<div class="info"><strong>Apps are sandboxed by default!</strong> Apps are not able to access any files, directories, or data outside of their own directory and key-value store within a user's account. This is to ensure that apps can't access any data or resources that they shouldn't have access to.</div>
<div class="info"><strong>Need to share data across users?</strong> Because each user's storage lives in their own account, one user can't see another's data. To keep a single, centralized store that every user reads from and writes to, use a <a href="/Workers/">Serverless Worker</a> — its code can act on the worker owner's resources, giving all users one shared backend.</div>
Your app will also be able to use the following services by default:
- **AI**: Your app will be able to use the AI services provided by Puter.com. This includes chat, txt2img, img2txt, and more.
+13
View File
@@ -28,6 +28,11 @@ let sidebar = [
source: '/frameworks.md',
path: '/frameworks',
},
{
title: 'Deployments',
source: '/deployments.md',
path: '/deployments',
},
{
title: 'Examples',
source: '/examples.md',
@@ -350,6 +355,14 @@ let sidebar = [
source: '/Workers/router.md',
path: '/Workers/router',
},
{
title: 'TypeScript Types',
page_title: 'TypeScript Types for Workers',
title_tag: 'TypeScript Types',
icon: '/assets/img/object.svg',
source: '/Workers/types.md',
path: '/Workers/types',
},
{
title: '<code>create()</code>',
page_title: '<code>puter.workers.create()</code>',
+194 -64
View File
@@ -19,17 +19,21 @@
import UIWindowSaveAccount from '../UIWindowSaveAccount.js';
function buildRecentAppsHTML () {
function buildRecentAppsHTML() {
let h = '';
if ( window.launch_apps?.recent?.length > 0 ) {
if (window.launch_apps?.recent?.length > 0) {
h += '<div class="bento-recent-apps-grid">';
// Show up to 6 recent apps (2 columns x 3 rows)
const recentApps = window.launch_apps.recent.slice(0, 6);
for ( const app_info of recentApps ) {
for (const app_info of recentApps) {
// if title, name and uuid are the same and index_url is set, then show the hostname of index_url
if ( app_info.name === app_info.title && app_info.name === app_info.uuid && app_info.index_url ) {
if (
app_info.name === app_info.title &&
app_info.name === app_info.uuid &&
app_info.index_url
) {
app_info.title = new URL(app_info.index_url).hostname;
app_info.target_link = app_info.index_url;
}
@@ -44,9 +48,12 @@ function buildRecentAppsHTML () {
h += '</div>';
} else {
h += '<div class="bento-recent-apps-empty">';
h += '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5">';
h += '<rect x="3" y="3" width="7" height="7"/><rect x="14" y="3" width="7" height="7"/>';
h += '<rect x="14" y="14" width="7" height="7"/><rect x="3" y="14" width="7" height="7"/>';
h +=
'<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5">';
h +=
'<rect x="3" y="3" width="7" height="7"/><rect x="14" y="3" width="7" height="7"/>';
h +=
'<rect x="14" y="14" width="7" height="7"/><rect x="3" y="14" width="7" height="7"/>';
h += '</svg>';
h += '<p>No recent apps yet</p>';
h += '<span>Apps you use will appear here</span>';
@@ -56,21 +63,25 @@ function buildRecentAppsHTML () {
return h;
}
function buildUsageHTML () {
function buildUsageHTML() {
let h = '';
h += '<div class="bento-usage-grid">';
// Your Plan section
h += '<div class="bento-usage-section bento-usage-card bento-plan-section">';
h +=
'<div class="bento-usage-section bento-usage-card bento-plan-section">';
h += '<a href="#" class="bento-usage-card-header bento-plan-header">';
h += `<h3>${i18n('your_plan')}</h3>`;
h += '<span class="bento-usage-card-arrow">›</span>';
h += '</a>';
h += '<div class="bento-usage-card-info bento-plan-info">';
h += '<span class="bento-usage-card-used bento-plan-name">--</span>';
h += '<span class="bento-usage-card-details bento-plan-details"><span class="bento-plan-badge"></span></span>';
h +=
'<span class="bento-usage-card-details bento-plan-details"><span class="bento-plan-badge"></span></span>';
h += '</div>';
h += '<a href="#" class="bento-plan-upgrade" style="display: none;">Upgrade →</a>';
h += '<div class="bento-plan-warning" style="display: none;"></div>';
h +=
'<a href="#" class="bento-plan-upgrade" style="display: none;">Upgrade →</a>';
h += '</div>';
// Storage section
@@ -83,8 +94,10 @@ function buildUsageHTML () {
h += '<div class="bento-usage-card-bar bento-storage-bar"></div>';
h += '</div>';
h += '<div class="bento-usage-card-info">';
h += '<span class="bento-usage-card-used bento-storage-used">-- Used</span>';
h += '<span class="bento-usage-card-details"><span class="bento-storage-percent">--%</span> of <span class="bento-storage-capacity">--</span></span>';
h +=
'<span class="bento-usage-card-used bento-storage-used">-- Used</span>';
h +=
'<span class="bento-usage-card-details"><span class="bento-storage-percent">--%</span> of <span class="bento-storage-capacity">--</span></span>';
h += '</div>';
h += '</div>';
@@ -98,8 +111,10 @@ function buildUsageHTML () {
h += '<div class="bento-usage-card-bar bento-resources-bar"></div>';
h += '</div>';
h += '<div class="bento-usage-card-info">';
h += '<span class="bento-usage-card-used bento-resources-used">-- Used</span>';
h += '<span class="bento-usage-card-details"><span class="bento-resources-percent">--%</span> of <span class="bento-resources-capacity">--</span></span>';
h +=
'<span class="bento-usage-card-used bento-resources-used">-- Used</span>';
h +=
'<span class="bento-usage-card-details"><span class="bento-resources-percent">--%</span> of <span class="bento-resources-capacity">--</span></span>';
h += '</div>';
h += '</div>';
@@ -112,9 +127,10 @@ const TabHome = {
label: 'Home',
icon: '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5"><path d="M3 9l9-7 9 7v11a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2z"/><polyline points="9 22 9 12 15 12 15 22"/></svg>',
html () {
html() {
const username = window.user?.username || 'User';
const profilePicture = window.user?.profile?.picture || window.icons['profile.svg'];
const profilePicture =
window.user?.profile?.picture || window.icons['profile.svg'];
let h = '';
h += '<div class="bento-container">';
@@ -129,9 +145,10 @@ const TabHome = {
h += `<h1 class="bento-username username">${html_encode(username)}</h1>`;
h += `<p class="bento-tagline">${i18n('your_personal_internet_computer')}</p>`;
// Show warning if account is temporary/unsaved
if ( window.user?.is_temp ) {
if (window.user?.is_temp) {
h += '<button class="bento-save-account-warning">';
h += '<svg style="width: 16px; height: 16px;" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" width="48px" height="48px" viewBox="0 0 48 48"><g transform="translate(0, 0)"><path d="M45.521,39.04L27.527,5.134c-1.021-1.948-3.427-2.699-5.375-1.679-.717,.376-1.303,.961-1.679,1.679L2.479,39.04c-.676,1.264-.635,2.791,.108,4.017,.716,1.207,2.017,1.946,3.42,1.943H41.993c1.403,.003,2.704-.736,3.42-1.943,.743-1.226,.784-2.753,.108-4.017ZM23.032,15h1.937c.565,0,1.017,.467,1,1.031l-.438,14c-.017,.54-.459,.969-1,.969h-1.062c-.54,0-.983-.429-1-.969l-.438-14c-.018-.564,.435-1.031,1-1.031Zm.968,25c-1.657,0-3-1.343-3-3s1.343-3,3-3,3,1.343,3,3-1.343,3-3,3Z" fill="var(--dashboard-warning-icon)"></path></g></svg>';
h +=
'<svg style="width: 16px; height: 16px;" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" width="48px" height="48px" viewBox="0 0 48 48"><g transform="translate(0, 0)"><path d="M45.521,39.04L27.527,5.134c-1.021-1.948-3.427-2.699-5.375-1.679-.717,.376-1.303,.961-1.679,1.679L2.479,39.04c-.676,1.264-.635,2.791,.108,4.017,.716,1.207,2.017,1.946,3.42,1.943H41.993c1.403,.003,2.704-.736,3.42-1.943,.743-1.226,.784-2.753,.108-4.017ZM23.032,15h1.937c.565,0,1.017,.467,1,1.031l-.438,14c-.017,.54-.459,.969-1,.969h-1.062c-.54,0-.983-.429-1-.969l-.438-14c-.018-.564,.435-1.031,1-1.031Zm.968,25c-1.657,0-3-1.343-3-3s1.343-3,3-3,3,1.343,3,3-1.343,3-3,3Z" fill="var(--dashboard-warning-icon)"></path></g></svg>';
h += `<span>${i18n('save_session')}</span>`;
h += '</button>';
}
@@ -141,14 +158,17 @@ const TabHome = {
// Recent apps card (rectangle)
h += '<div class="bento-card bento-recent">';
h += '<a href="#" class="bento-card-fancy-header" data-target-tab="apps">';
h +=
'<a href="#" class="bento-card-fancy-header" data-target-tab="apps">';
h += '<div class="bento-card-fancy-icon bento-card-fancy-icon-apps">';
h += '<svg viewBox="0 0 24 24" fill="currentColor"><rect x="3" y="3" width="7" height="7" rx="1.5"/><rect x="14" y="3" width="7" height="7" rx="1.5"/><rect x="3" y="14" width="7" height="7" rx="1.5"/><rect x="14" y="14" width="7" height="7" rx="1.5"/></svg>';
h +=
'<svg viewBox="0 0 24 24" fill="currentColor"><rect x="3" y="3" width="7" height="7" rx="1.5"/><rect x="14" y="3" width="7" height="7" rx="1.5"/><rect x="3" y="14" width="7" height="7" rx="1.5"/><rect x="14" y="14" width="7" height="7" rx="1.5"/></svg>';
h += '</div>';
h += '<div class="bento-card-fancy-text">';
h += '<h2>Apps</h2>';
h += '<span class="bento-card-fancy-subtitle">';
h += '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><circle cx="12" cy="12" r="10"/><polyline points="12 6 12 12 16 14"/></svg>';
h +=
'<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><circle cx="12" cy="12" r="10"/><polyline points="12 6 12 12 16 14"/></svg>';
h += 'Recently used';
h += '</span>';
h += '</div>';
@@ -168,14 +188,17 @@ const TabHome = {
// Usage card (spans full width on second row)
h += '<div class="bento-card bento-usage">';
h += '<a href="#" class="bento-card-fancy-header" data-target-tab="usage">';
h +=
'<a href="#" class="bento-card-fancy-header" data-target-tab="usage">';
h += '<div class="bento-card-fancy-icon bento-card-fancy-icon-usage">';
h += '<svg viewBox="0 0 16 16" fill="currentColor"><path d="M8 4a.5.5 0 0 1 .5.5V6a.5.5 0 0 1-1 0V4.5A.5.5 0 0 1 8 4M3.732 5.732a.5.5 0 0 1 .707 0l.915.914a.5.5 0 1 1-.708.708l-.914-.915a.5.5 0 0 1 0-.707M2 10a.5.5 0 0 1 .5-.5h1.586a.5.5 0 0 1 0 1H2.5A.5.5 0 0 1 2 10m9.5 0a.5.5 0 0 1 .5-.5h1.5a.5.5 0 0 1 0 1H12a.5.5 0 0 1-.5-.5m.754-4.246a.39.39 0 0 0-.527-.02L7.547 9.31a.91.91 0 1 0 1.302 1.258l3.434-4.297a.39.39 0 0 0-.029-.518z"/><path fill-rule="evenodd" d="M0 10a8 8 0 1 1 15.547 2.661c-.442 1.253-1.845 1.602-2.932 1.25C11.309 13.488 9.475 13 8 13c-1.474 0-3.31.488-4.615.911-1.087.352-2.49.003-2.932-1.25A8 8 0 0 1 0 10m8-7a7 7 0 0 0-6.603 9.329c.203.575.923.876 1.68.63C4.397 12.533 6.358 12 8 12s3.604.532 4.923.96c.757.245 1.477-.056 1.68-.631A7 7 0 0 0 8 3"/></svg>';
h +=
'<svg viewBox="0 0 16 16" fill="currentColor"><path d="M8 4a.5.5 0 0 1 .5.5V6a.5.5 0 0 1-1 0V4.5A.5.5 0 0 1 8 4M3.732 5.732a.5.5 0 0 1 .707 0l.915.914a.5.5 0 1 1-.708.708l-.914-.915a.5.5 0 0 1 0-.707M2 10a.5.5 0 0 1 .5-.5h1.586a.5.5 0 0 1 0 1H2.5A.5.5 0 0 1 2 10m9.5 0a.5.5 0 0 1 .5-.5h1.5a.5.5 0 0 1 0 1H12a.5.5 0 0 1-.5-.5m.754-4.246a.39.39 0 0 0-.527-.02L7.547 9.31a.91.91 0 1 0 1.302 1.258l3.434-4.297a.39.39 0 0 0-.029-.518z"/><path fill-rule="evenodd" d="M0 10a8 8 0 1 1 15.547 2.661c-.442 1.253-1.845 1.602-2.932 1.25C11.309 13.488 9.475 13 8 13c-1.474 0-3.31.488-4.615.911-1.087.352-2.49.003-2.932-1.25A8 8 0 0 1 0 10m8-7a7 7 0 0 0-6.603 9.329c.203.575.923.876 1.68.63C4.397 12.533 6.358 12 8 12s3.604.532 4.923.96c.757.245 1.477-.056 1.68-.631A7 7 0 0 0 8 3"/></svg>';
h += '</div>';
h += '<div class="bento-card-fancy-text">';
h += `<h2>${i18n('usage')}</h2>`;
h += '<span class="bento-card-fancy-subtitle">';
h += '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M22 12h-4l-3 9L9 3l-3 9H2"/></svg>';
h +=
'<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M22 12h-4l-3 9L9 3l-3 9H2"/></svg>';
h += 'Monthly overview';
h += '</span>';
h += '</div>';
@@ -189,9 +212,33 @@ const TabHome = {
return h;
},
init ($el_window) {
init($el_window) {
this.loadRecentApps($el_window);
this.loadUsageData($el_window);
// Auto-refresh sources:
// 1. `puter:subscription:changed` — fired by upgrade dialog
// after subscribe / change / cancel / uncancel.
// 2. Visibility / focus — user returning to puter (e.g. from
// Stripe Customer Portal in another tab). Portal mutations
// bypass our dispatch, so we re-pull state + broadcast.
const refresh = () => this.loadUsageData($el_window);
const refreshAndBroadcast = async () => {
refresh();
try {
await window.refresh_user_data?.(puter.authToken);
} catch {}
try {
window.dispatchEvent(
new CustomEvent('puter:subscription:changed'),
);
} catch {}
};
window.addEventListener('puter:subscription:changed', refresh);
const onVisibility = () => {
if (document.visibilityState === 'visible') refreshAndBroadcast();
};
document.addEventListener('visibilitychange', onVisibility);
window.addEventListener('focus', refreshAndBroadcast);
// Handle app clicks
$el_window.on('click', '.bento-recent-app', function (e) {
@@ -199,23 +246,30 @@ const TabHome = {
e.stopPropagation();
const appName = $(this).attr('data-app-name');
const targetLink = $(this).attr('data-target-link');
if ( targetLink && targetLink !== '' ) {
if (targetLink && targetLink !== '') {
window.open(targetLink, '_blank');
}
else if ( appName ) {
} else if (appName) {
window.open(`/app/${appName}`, '_blank');
}
});
// Handle "View details" link clicks
$el_window.on('click', '.bento-view-more, .bento-usage-card-header, .bento-card-fancy-header[data-target-tab]', function (e) {
e.preventDefault();
const targetTab = $(this).attr('data-target-tab');
if ( targetTab ) {
// Trigger click on the corresponding sidebar item
$el_window.find(`.dashboard-sidebar-item[data-section="${targetTab}"]`).click();
}
});
$el_window.on(
'click',
'.bento-view-more, .bento-usage-card-header, .bento-card-fancy-header[data-target-tab]',
function (e) {
e.preventDefault();
const targetTab = $(this).attr('data-target-tab');
if (targetTab) {
// Trigger click on the corresponding sidebar item
$el_window
.find(
`.dashboard-sidebar-item[data-section="${targetTab}"]`,
)
.click();
}
},
);
// Handle desktop switch button
$el_window.on('click', '.bento-desktop-switch-btn', function () {
@@ -237,15 +291,15 @@ const TabHome = {
draggable_body: false,
},
}).then(function (is_saved) {
if ( is_saved ) {
if (is_saved) {
$el_window.find('.bento-save-account-warning').hide();
}
});
});
},
async loadRecentApps ($el_window) {
if ( ! window.launch_apps?.recent?.length ) {
async loadRecentApps($el_window) {
if (!window.launch_apps?.recent?.length) {
try {
window.launch_apps = await $.ajax({
url: `${window.api_origin}/get-launch-apps?icon_size=64`,
@@ -253,36 +307,89 @@ const TabHome = {
async: true,
contentType: 'application/json',
headers: {
'Authorization': `Bearer ${window.auth_token}`,
Authorization: `Bearer ${window.auth_token}`,
},
});
} catch (e) {
console.error('Failed to load launch apps:', e);
}
}
$el_window.find('.bento-recent-apps-container').html(buildRecentAppsHTML());
$el_window
.find('.bento-recent-apps-container')
.html(buildRecentAppsHTML());
},
async loadUsageData ($el_window) {
// Load plan data
async loadUsageData($el_window) {
// Load plan data — fetch live from /marketplace/subscriptions/current
// rather than reading `window.user.subscription` (which is set once
// from whoami at page-load and goes stale after subscribe / portal
// cancel until a hard refresh).
try {
const hasSubscription = window.user?.subscription?.active;
const planName = window.user?.subscription?.offering?.name || 'free';
let subscription = null;
try {
const resp = await fetch(
`${window.api_origin}/marketplace/subscriptions/current`,
{
headers: { Authorization: `Bearer ${puter.authToken}` },
},
);
if (resp.ok) {
const data = await resp.json();
subscription = data?.subscription ?? null;
}
} catch {
// fall through to free state
}
const pastDue =
!!subscription && subscription.status === 'past_due';
// `past_due` keeps benefits during Stripe's dunning window, so
// it still counts as having a plan — we just flag it.
const hasSubscription =
!!subscription &&
(subscription.status === 'active' ||
subscription.status === 'trialing' ||
subscription.status === 'cancel_pending' ||
pastDue);
const planName = subscription?.tier || 'free';
$el_window.find('.bento-plan-name').text(i18n(planName));
if ( hasSubscription ) {
$el_window.find('.bento-plan-badge').text('Current').addClass('active');
// Reset state-dependent classes / warning each (re)render.
const $badge = $el_window
.find('.bento-plan-badge')
.removeClass('active free past-due');
const $warning = $el_window
.find('.bento-plan-warning')
.hide()
.text('');
if (hasSubscription) {
if (pastDue) {
$badge.text('Payment past due').addClass('past-due');
$warning
.text(
'Your last payment failed. Update your payment method to keep your subscription — access will be revoked shortly.',
)
.show();
} else {
$badge.text('Current').addClass('active');
}
$el_window.find('.bento-plan-upgrade').text('Manage →').show();
} else {
$el_window.find('.bento-plan-badge').text('Upgrade for more features').addClass('free');
$badge.text('Upgrade for more features').addClass('free');
$el_window.find('.bento-plan-upgrade').show();
}
$el_window.find('.bento-plan-upgrade').off('click.billing').on('click.billing', (e) => {
e.preventDefault();
window.puterLegacyBilling?.openSubscriptionsDialog?.();
});
$el_window
.find('.bento-plan-upgrade')
.off('click.billing')
.on('click.billing', (e) => {
e.preventDefault();
if (typeof window.UIUpgradeAccount === 'function') {
new window.UIUpgradeAccount().open_as_window();
}
});
} catch (e) {
console.error('Failed to load plan data:', e);
}
@@ -290,17 +397,23 @@ const TabHome = {
// Load storage data
try {
const res = await puter.fs.space();
let usage_percentage = (res.used / res.capacity * 100).toFixed(0);
let usage_percentage = ((res.used / res.capacity) * 100).toFixed(0);
usage_percentage = usage_percentage > 100 ? 100 : usage_percentage;
let general_used = res.used;
if ( res.host_used ) {
if (res.host_used) {
general_used = res.host_used;
}
$el_window.find('.bento-storage-used').text(`${window.byte_format(general_used)} Used`);
$el_window.find('.bento-storage-capacity').text(window.byte_format(res.capacity));
$el_window.find('.bento-storage-percent').text(`${usage_percentage}%`);
$el_window
.find('.bento-storage-used')
.text(`${window.byte_format(general_used)} Used`);
$el_window
.find('.bento-storage-capacity')
.text(window.byte_format(res.capacity));
$el_window
.find('.bento-storage-percent')
.text(`${usage_percentage}%`);
$el_window.find('.bento-storage-bar').css({
width: `${usage_percentage}%`,
'background-color': window.usage_bar_color(usage_percentage),
@@ -315,21 +428,38 @@ const TabHome = {
let monthlyAllowance = res.allowanceInfo?.monthUsageAllowance;
let remaining = res.allowanceInfo?.remaining;
let totalUsage = monthlyAllowance - remaining;
let totalUsagePercentage = (totalUsage / monthlyAllowance * 100).toFixed(0);
let totalUsagePercentage = (
(totalUsage / monthlyAllowance) *
100
).toFixed(0);
$el_window.find('.bento-resources-used').text(`${window.number_format(totalUsage / 100_000_000, { decimals: 2, prefix: '$' })} Used`);
$el_window.find('.bento-resources-capacity').text(window.number_format(monthlyAllowance / 100_000_000, { decimals: 2, prefix: '$' }));
$el_window.find('.bento-resources-percent').text(`${totalUsagePercentage}%`);
$el_window
.find('.bento-resources-used')
.text(
`${window.number_format(totalUsage / 100_000_000, { decimals: 2, prefix: '$' })} Used`,
);
$el_window
.find('.bento-resources-capacity')
.text(
window.number_format(monthlyAllowance / 100_000_000, {
decimals: 2,
prefix: '$',
}),
);
$el_window
.find('.bento-resources-percent')
.text(`${totalUsagePercentage}%`);
$el_window.find('.bento-resources-bar').css({
width: `${totalUsagePercentage}%`,
'background-color': window.usage_bar_color(totalUsagePercentage),
'background-color':
window.usage_bar_color(totalUsagePercentage),
});
} catch (e) {
console.error('Failed to load monthly usage data:', e);
}
},
onActivate ($el_window) {
onActivate($el_window) {
this.loadRecentApps($el_window);
this.loadUsageData($el_window);
},
+1 -1
View File
@@ -356,7 +356,7 @@ async function UIWindowLogin (options) {
},
onAppend: function (this_window) {
if ( options.authError ) {
$(this_window).find('.login-error-msg').html(options.authError).fadeIn();
$(this_window).find('.login-error-msg').html(html_encode(options.authError)).fadeIn();
}
if ( ! window.disable_login_autofocus )
{
+1 -1
View File
@@ -138,7 +138,7 @@ function UIWindowSignup (options) {
center: true,
onAppend: function (el_window) {
if ( options.authError ) {
$(el_window).find('.signup-error-msg').html(options.authError).fadeIn();
$(el_window).find('.signup-error-msg').html(html_encode(options.authError)).fadeIn();
}
if ( ! window.disable_signup_autofocus )
{
+22
View File
@@ -1473,6 +1473,28 @@ input.myapps-search::placeholder {
border: 1px solid var(--dashboard-success-border);
}
.dashboard .bento-plan-badge.past-due {
display: inline-block;
padding: 2px 8px;
border-radius: 999px;
font-size: 12px;
font-weight: 500;
color: var(--dashboard-danger-text);
background: var(--dashboard-danger-background);
border: 1px solid var(--dashboard-danger-border);
}
.dashboard .bento-plan-warning {
margin-top: 10px;
padding: 8px 10px;
border-radius: 8px;
color: var(--dashboard-danger-text);
background: var(--dashboard-danger-background);
border: 1px solid var(--dashboard-danger-border);
font-size: 12px;
line-height: 1.4;
}
.dashboard .bento-plan-upgrade {
font-size: 14px;
font-weight: 500;
Loaded 100 of 134 files, more files were not shown because too many files have changed in this diff. Show more