Compare commits

..
51 Commits
Author SHA1 Message Date
Daniel Salazar 63f128457e remove driver usage endpoint (#3017) 2026-05-08 10:18:26 -07:00
Daniel Salazar 5e8e9a852d test: add unit tests for ai-chat utils (#3016)
Covers `Messages` (normalize_single_message / normalize_messages /
extract_and_remove_system_messages / extract_text), `FunctionCalling`
(normalize_json_schema / normalize_tools_object /
make_openai_tools / make_claude_tools), and `Streaming`
(AIChatStream / AIChatMessageStream / AIChatTextStream /
AIChatToolUseStream). All three are pure data transforms — Streaming
is exercised against a real `AIChatStream` wired to a buffering
Writable, so no method-level spies are needed.

Closes #2956
2026-05-08 10:17:46 -07:00
Daniel Salazar 3166ad933c test: add unit tests for OpenAIUtil shape normalization (#3015)
Covers the Puter↔OpenAI shape conversion (`process_input_messages`,
`process_input_messages_responses_api`), usage extraction
(`extractMeteredUsage`, `create_usage_calculator`), the streaming
handlers (`create_chat_stream_handler` and the responses-API variant),
and the non-stream `handle_completion_output[_responses_api]` paths
including moderation, missing-content, and per-provider deviations.
Streaming is exercised via a real `AIChatStream` wired to a buffering
Writable so no method-level spies are needed.

Closes #2955
2026-05-08 10:17:42 -07:00
Reynaldi Chernando 7e5f64e04b Fix minor typo for tls cert in self hosting docs (#3014) 2026-05-08 09:23:06 -07:00
jelveh 908c02fd99 fix ASCII art for PUTER.JS 2026-05-08 08:04:21 -07:00
jelveh 38fcc6d6bd Detect fullpage apps early and skip desktop load 2026-05-07 22:17:27 -07:00
Daniel Salazar 16652347c6 test: add tests for HomepageController (#3010)
Covers the early-return guard when the homepage service is unwired,
shell rendering at /, /settings, /dashboard, /action, /@:username,
canonical_url + meta inheritance from gui_params, /app/:name (404 +
shell, embedded app metadata, JSON / object metadata, malformed JSON),
and /show/* launching the explorer with the trimmed path.

Closes #2973
2026-05-07 21:41:16 -07:00
Daniel Salazar 5f28b988ec test: add tests for SystemController (#3009)
Covers /healthcheck (live status, 503 on failure, fallback when service
unavailable), /version (semver parsing, env defaults), /contactUs
(validation, persistence, email best-effort, default support address),
/whoarewe, /lsmod (interface grouping + driverName fallback), and the
prepareShutdown drain-flag flip.

Closes #2971
2026-05-07 21:40:27 -07:00
Daniel Salazar 160631b158 test: add tests for OIDCController (#3008)
Covers the provider listing route, the auth-start redirect (404 / 400 /
500 / 302), all three callback flows (login / signup / revalidate)
including state validation, suspended accounts, unconfirmed-email
linking refusal, redirect_uri origin clamping, and the
revalidate-done landing page. Routes are collected via PuterRouter and
invoked with stubbed services that return prefab data.

Closes #2966
2026-05-07 21:38:47 -07:00
Daniel Salazar 3ab9923f0c fix: ai provider integration tests (#3011)
* fix: ai provider integration tests

* change name for ai test
2026-05-07 21:38:40 -07:00
Daniel Salazar cc799e069f test: add unit tests for fileInput driver utility (#3007)
Covers `loadFileInput` (validation, data-URL decoding, FS-path resolution,
maxBytes enforcement, ACL gating) and `inferFilenameFromUrlOrPath`. Stubs
stores/fsService with prefab data so tests don't require a real server.

Closes #2957
2026-05-07 21:38:27 -07:00
Daniel Salazar 9a7d3bac5f test: add tests for DesktopController (#3006)
Covers /set-desktop-bg (partial updates, null clearing, type validation),
/update-taskbar-items (JSON serialization), /set_layout (layout
allowlist, ownership gate, path/uuid resolution), and /set_sort_by
(sort_by/sort_order allowlists, default ordering). Routes are
collected via PuterRouter and driven with stub stores returning
prefab fsentry rows.

Closes #2970
2026-05-07 21:37:34 -07:00
Daniel Salazar 0a9a4f629d test: add tests for StaticPagesController (#3005)
Covers /robots.txt (bot disallow + sitemap), /sitemap.xml (docs +
approved-listing apps), /unsubscribe (missing/invalid uuid, idempotent
re-runs), and /confirm-email-by-token (token mismatch, already-confirmed
short-circuit, duplicate-email rejection, success path with verified-
group promotion + best-effort side channels). Stub stores hold prefab
user rows; the DB stub returns canned EXISTS / SELECT results.

Closes #2975
2026-05-07 21:36:50 -07:00
Daniel Salazar 32c2c0c1a5 test: add tests for AppController (#3004)
Covers /apps, /apps/nameAvailable, /rao, /apps/:name (single + batch),
/query/app (max-entries cap, marketplace shape, hidden-app gating), and
/app-icon (default fallback, data-URL decoding, MIME allowlist, CDN
redirect). Routes are collected via PuterRouter; AppDriver is injected
through the shared `driversContainers` registry, and stub stores return
prefab app rows.

Closes #2969
2026-05-07 21:35:49 -07:00
Daniel Salazar 508d652194 test: add tests for StaticAssetsController (#3003)
Covers each config-gated branch (client_libs_root, puterjs_root,
gui_assets_root, builtin_apps) using real temp directories so the
existsSync checks see real files instead of mocked module behavior.
Asserts both registration outcomes (which routes appear, on which
subdomains) and the file resolution rules for the puter.js / puter.dev.js
fallback.

Closes #2974
2026-05-07 21:35:17 -07:00
jelveh 471ba94ba2 Update index.js 2026-05-07 20:04:43 -07:00
jelveh 66e24a07d4 Update console banner and messaging 2026-05-07 19:49:01 -07:00
jelveh 5ffac54c86 experimental cute CTA 2026-05-07 19:39:06 -07:00
ProgrammerIn-wonderland c5b12b61d9 Unconditionally send Access-Control-Allow-Private-Network, fix LNA fix in GUI (#2954) 2026-05-07 16:38:04 -07:00
Daniel Salazar 8d5e9ef9cf fix: types for open ai driver (#2950)
* fix: types for open ai driver

* fix: add extra time to tests

* fix: test

* fix: together ai tests

* fix: tests
2026-05-07 14:48:47 -07:00
Daniel Salazar 25fe9a3cdf fix: add coverage reports for changes (#2949)
* tests: ai integraton tests

* fix: add coverage reports for this too
2026-05-07 13:55:09 -07:00
Daniel Salazar 05425d20a7 chore: cleanup types for tests (#2948) 2026-05-07 13:40:58 -07:00
Nariman Jelveh ff10e47d67 Show fullscreen overlay when signup is blocked (#2947) 2026-05-07 11:37:23 -07:00
Daniel Salazar 6c19dfbafd fix: linux build (#2946) 2026-05-07 11:23:10 -07:00
Nariman Jelveh 360cbf48af feat: support code in puter.signup.validate error responses (#2945)
Extensions listening on the `puter.signup.validate` event can now set a
`code` field (in addition to `message`) when blocking signups. The code
is forwarded through HttpError and appears in the JSON error response,
giving clients a stable, machine-readable signal to act on.

Covers both the AuthController (password signup) and OIDCService paths.
Adds tests for the new field.
2026-05-07 10:11:08 -07:00
Daniel Salazar bbe93defe9 tests: add some more driver tests (#2944) 2026-05-07 10:04:39 -07:00
Reynaldi Chernando 5e01c13b31 Fix duplicate textract result (#2943) 2026-05-07 09:15:28 -07:00
Daniel Salazar 4b37f35803 test: tests for fs endpoints (#2942) 2026-05-06 22:55:53 -07:00
Daniel Salazar 775c9c49e2 test: add harness and some examples (#2941)
* test: add harness and some examples

* feat: tests ran on pr
2026-05-06 22:43:22 -07:00
Nariman Jelveh e35260924e Scope .myapps-search to input and tweak UI styles 2026-05-06 17:59:54 -07:00
Nariman Jelveh 90aad8918c Support light/dark sidebar logo 2026-05-06 17:54:37 -07:00
Nariman Jelveh 46974beba1 Replace gradients with flat colors in system buttons 2026-05-06 17:47:45 -07:00
Shruc 5e9dc33be1 expose costs in ai chat response (#2921)
* expose costs

* microcents
2026-05-06 15:59:16 -07:00
Daniel Salazar 1e271f188e fix: bring back old app names functionality and fix build (#2937) 2026-05-06 15:48:25 -07:00
jelveh f333597754 Update IPC.js 2026-05-06 15:40:38 -07:00
jelveh 3b33d16356 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-05-06 15:33:21 -07:00
jelveh 2289d61d3c Update UIPrompt.js 2026-05-06 15:33:13 -07:00
Daniel Salazar 8e02f38bad fix: templates (#2936) 2026-05-06 15:31:41 -07:00
angelinazzz1809 5b65a6bc78 Update UIPrompt.js (#2930)
In the app environment, the prompt input was initialized using `options.value`, so `defaultValue` was not reflected.

This change ensures that `defaultValue` is used when provided, while keeping backward compatibility with `options.value`.
2026-05-06 15:27:28 -07:00
ProgrammerIn-wonderland 872d2c01d4 Add support for log in with Apple (#2935)
* Add experimental login with Apple support

* Fix revalidate URL

* Fix OIDC provider revalidate in whoami as well
2026-05-06 16:47:24 -04:00
jelveh b7f199ddbc Normalize string options to object 2026-05-06 11:59:28 -07:00
jelveh 906c00af97 Support options.defaultColor in color picker 2026-05-06 11:48:47 -07:00
Shruc f00d52f879 per model allowed params on replicate image, normalize ratio (#2898)
* replicate allowed params, normalize ratio

* update docs
2026-05-06 14:16:30 -04:00
Devendra Reddy Pennabadianddevareddy05 7a1a885e89 fix: inline http(s) image URLs for Moonshot vision models (#2929) (#2931)
Co-authored-by: devareddy05 <[email protected]>
2026-05-06 13:53:52 -04:00
Daniel Salazar ff8c275317 chore: package-lock.json (#2934) 2026-05-06 10:52:22 -07:00
Devendra Reddy Pennabadianddevareddy05 b3bf979cac fix: support defaultValue in showColorPicker app env (#2895) (#2932)
Co-authored-by: devareddy05 <[email protected]>
2026-05-06 10:46:32 -07:00
Reynaldi Chernando eb9e316ed5 Update Z.AI vision model id to z-ai/z-ai for consistency (#2928) 2026-05-06 02:41:04 -04:00
Daniel Salazar 224ea5f7f8 fix: extensions (#2926) 2026-05-05 20:55:53 -07:00
Daniel Salazar c1dacab4c2 extension events after boot fixes (#2925) 2026-05-05 17:42:55 -07:00
Daniel Salazar 8d0b8d4905 fix: app errors (#2924) 2026-05-05 17:20:31 -07:00
jelveh e7190f442e Update README.md 2026-05-05 16:49:01 -07:00
115 changed files with 16031 additions and 1957 deletions

No files matched your search

@@ -0,0 +1,49 @@
name: AI Provider Integration Tests
# Hits real provider APIs with cheap models, so this workflow is
# manually triggered only. Each provider's test reads its credential
# from PUTER_TEST_AI_<PROVIDER>_API_KEY and skips itself silently if
# the secret is missing.
on:
workflow_dispatch:
permissions:
contents: read
jobs:
ai-test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run AI provider integration tests
# Vitest treats positional args as filename substring filters.
# The pattern matches every `*.integration.test.ts` file and
# nothing else — the regular backend test suite stays out.
run: npm run test:backend -- integration.test
env:
CI: 'true'
PUTER_TEST_AI_CLAUDE_API_KEY: ${{ secrets.PUTER_TEST_AI_CLAUDE_API_KEY }}
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
PUTER_TEST_AI_GEMINI_API_KEY: ${{ secrets.PUTER_TEST_AI_GEMINI_API_KEY }}
PUTER_TEST_AI_GROQ_API_KEY: ${{ secrets.PUTER_TEST_AI_GROQ_API_KEY }}
PUTER_TEST_AI_MISTRAL_API_KEY: ${{ secrets.PUTER_TEST_AI_MISTRAL_API_KEY }}
PUTER_TEST_AI_DEEPSEEK_API_KEY: ${{ secrets.PUTER_TEST_AI_DEEPSEEK_API_KEY }}
PUTER_TEST_AI_XAI_API_KEY: ${{ secrets.PUTER_TEST_AI_XAI_API_KEY }}
PUTER_TEST_AI_OPENROUTER_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENROUTER_API_KEY }}
PUTER_TEST_AI_TOGETHER_API_KEY: ${{ secrets.PUTER_TEST_AI_TOGETHER_API_KEY }}
PUTER_TEST_AI_MOONSHOT_API_KEY: ${{ secrets.PUTER_TEST_AI_MOONSHOT_API_KEY }}
PUTER_TEST_AI_ZAI_API_KEY: ${{ secrets.PUTER_TEST_AI_ZAI_API_KEY }}
PUTER_TEST_AI_ELEVENLABS_API_KEY: ${{ secrets.PUTER_TEST_AI_ELEVENLABS_API_KEY }}
+77
View File
@@ -0,0 +1,77 @@
name: Backend Tests
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- ref: ${{ github.base_ref }}
artifact: base
- ref: ${{ github.head_ref }}
artifact: pr
steps:
- name: Checkout ${{ matrix.ref }}
uses: actions/checkout@v4
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run backend tests with coverage
run: npm run test:backend -- --coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: backend-coverage-${{ matrix.artifact }}
path: src/backend/coverage/
retention-days: 14
report-coverage:
needs: test
if: always()
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download PR coverage
uses: actions/download-artifact@v4
with:
name: backend-coverage-pr
path: src/backend/coverage
- name: Download base coverage
uses: actions/download-artifact@v4
with:
name: backend-coverage-base
path: coverage-base
- name: Report coverage on PR
uses: davelosert/vitest-coverage-report-action@v2
with:
json-summary-path: src/backend/coverage/coverage-summary.json
json-final-path: src/backend/coverage/coverage-final.json
json-summary-compare-path: coverage-base/coverage-summary.json
+1 -3
View File
@@ -40,7 +40,7 @@ cd puter
npm install
npm start
```
**→** This should launch Puter at http://puter.localhost:4100 (or the next available port).
**→** This should launch Puter at http://puter.localhost:4100
<br/>
@@ -59,8 +59,6 @@ curl -fsSL https://puter.com/selfhost | sh
irm https://puter.com/selfhost?os=windows | iex
```
<br/>
**→** For more details, see [Self-Hosting Puter](./doc/self-hosting.md).
<br/>
+18 -1
View File
@@ -14,7 +14,7 @@ Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker
- **Docker** with the `compose` plugin.
- A **domain** with DNS access — you need a wildcard record (`*.your-domain.com` → server IP). Puter routes by subdomain (`api.<domain>`, `site.<domain>`, `app.<domain>`).
- Optional: **TLS certs** (or `certbot` to grab them — see Step 4).
- Optional: **TLS certs** (or `certbot` to grab them — see Step 3).
## What's running
@@ -260,6 +260,23 @@ To require email confirmation before login, also set `"strict_email_verification
Add `https://puter.<your-domain>/auth/oidc/callback/login` to the OAuth client's authorized redirect URIs in the Google Cloud Console. For non-Google providers, replace `google` with a custom id and supply `authorization_endpoint` / `token_endpoint` / `userinfo_endpoint` explicitly.
### Sign in with Apple
```json
"oidc": {
"providers": {
"apple": {
"client_id": "com.example.your-service-id",
"team_id": "YOUR_TEAM_ID",
"key_id": "YOUR_KEY_ID",
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----"
}
}
}
```
The `client_id` is your Apple Services ID. `team_id`, `key_id`, and `private_key` come from the Apple Developer Portal (Keys section — create a key with "Sign in with Apple" enabled). The `private_key` is the contents of the `.p8` file Apple provides. Add `https://puter.<your-domain>/auth/oidc/callback/login` and `https://puter.<your-domain>/auth/oidc/callback/signup` as return URLs in the Apple Services ID configuration.
### AI providers
Any provider with a key set is auto-enabled. Same shape as `ollama` above:
+5 -7
View File
@@ -1,11 +1,11 @@
import { extension } from '@heyputer/backend/src/extensions';
import { PuterService } from '@heyputer/backend/src/services/types.js';
import { nativeImport } from '@heyputer/backend/src/util/nativeImport.js';
import { spawn, type ChildProcess } from 'node:child_process';
import { existsSync, readFileSync } from 'node:fs';
import { createRequire } from 'node:module';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { extension } from '@heyputer/backend/src/extensions';
import { PuterService } from '@heyputer/backend/src/services/types.js';
import { nativeImport } from '@heyputer/backend/src/util/nativeImport.js';
const requireFromHere = createRequire(__filename);
const webpack = requireFromHere('webpack') as typeof import('webpack');
@@ -121,9 +121,7 @@ const defaultWebpackEntries: WebpackEntry[] = [
class DevWatcherService extends PuterService {
#processes: Array<{ name: string; proc: ChildProcess }> = [];
#watchers: Array<{
close: (handler: (err?: Error | null) => void) => void;
}> = [];
#watchers: ReturnType<ReturnType<typeof webpack>['watch']>[] = [];
#started = false;
#packageRoot = findPackageRoot();
@@ -159,7 +157,7 @@ class DevWatcherService extends PuterService {
this.#watchers.map(
(watcher) =>
new Promise<void>((resolve) => {
watcher.close((err) => {
watcher?.close((err) => {
if (err) {
console.warn(
'[devwatch] failed to close webpack watcher:',
+3
View File
@@ -0,0 +1,3 @@
{
"type": "commonjs"
}
+89
View File
@@ -0,0 +1,89 @@
import { GetObjectCommand } from '@aws-sdk/client-s3';
import {
afterAll,
beforeAll,
describe,
expect,
it,
} from 'vitest';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleThumbnailCreated } from './thumbnails.ts';
// 1x1 transparent PNG — smallest valid image sharp will accept.
const TINY_PNG_BASE64 =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
const BUCKET = 'puter-local';
const streamToBuffer = async (
body: { transformToByteArray: () => Promise<Uint8Array> } | undefined,
): Promise<Buffer> => {
if (!body) throw new Error('s3 GetObject returned no body');
return Buffer.from(await body.transformToByteArray());
};
describe('thumbnails extension — handleThumbnailCreated', () => {
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server.shutdown();
});
it('uploads a valid data: URL thumbnail to S3 and rewrites event.url to an s3:// pointer', async () => {
const s3 = server.clients.s3.get();
const event: Record<string, unknown> = {
url: `data:image/png;base64,${TINY_PNG_BASE64}`,
};
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
expect(typeof event.url).toBe('string');
const newUrl = event.url as string;
expect(newUrl.startsWith(`s3://${BUCKET}/`)).toBe(true);
const key = newUrl.slice(`s3://${BUCKET}/`.length);
const obj = await s3.send(
new GetObjectCommand({ Bucket: BUCKET, Key: key }),
);
expect(obj.ContentType).toBe('image/png');
const expected = Buffer.from(TINY_PNG_BASE64, 'base64');
const actual = await streamToBuffer(obj.Body as never);
expect(actual.equals(expected)).toBe(true);
});
it('sets event.url to null when the data: URL does not decode to a valid image', async () => {
const s3 = server.clients.s3.get();
const event: Record<string, unknown> = {
url: `data:image/png;base64,${Buffer.from('not an image').toString('base64')}`,
};
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
expect(event.url).toBeNull();
});
it('leaves event.url untouched when the URL is not a data: URL', async () => {
const s3 = server.clients.s3.get();
const original = 'https://example.com/thumb.png';
const event: Record<string, unknown> = { url: original };
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
expect(event.url).toBe(original);
});
it('returns without writing to S3 when event.url is missing', async () => {
const s3 = server.clients.s3.get();
const event: Record<string, unknown> = {};
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
expect(event.url).toBeUndefined();
});
});
+21 -8
View File
@@ -99,7 +99,10 @@ async function decodeAndValidateThumbnail(
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer.
extension.on('thumbnail.created', async (event: Record<string, unknown>) => {
export async function handleThumbnailCreated(
event: Record<string, unknown>,
deps: { s3: S3Client; bucketName: string },
): Promise<void> {
const url = event.url;
if (typeof url !== 'string' || !url.startsWith('data:')) return;
@@ -110,24 +113,34 @@ extension.on('thumbnail.created', async (event: Record<string, unknown>) => {
}
const key = crypto.randomUUID();
event.url = `s3://${thumbnailBucketName}/${key}`;
event.url = `s3://${deps.bucketName}/${key}`;
await getClient().send(
await deps.s3.send(
new PutObjectCommand({
Bucket: thumbnailBucketName,
Bucket: deps.bucketName,
Key: key,
Body: decoded.data,
ContentType: decoded.mimeType,
}),
);
});
}
extension.on(
'thumbnail.created',
async (_key, event: Record<string, unknown>) => {
await handleThumbnailCreated(event, {
s3: getClient(),
bucketName: thumbnailBucketName,
});
},
);
// ── thumbnail.upload.prepare ────────────────────────────────────────
// Generate pre-signed upload URLs so the client can PUT directly to S3.
extension.on(
'thumbnail.upload.prepare',
async (event: Record<string, unknown>) => {
async (_key, event: Record<string, unknown>) => {
if (!event || !Array.isArray(event.items)) return;
const presignClient = getPresignClient();
@@ -169,7 +182,7 @@ extension.on(
// ── thumbnail.read ──────────────────────────────────────────────────
// Convert s3:// or legacy https:// thumbnails to signed URLs.
extension.on('thumbnail.read', async (entry: Record<string, unknown>) => {
extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
const thumb = entry.thumbnail;
if (typeof thumb !== 'string' || !thumb) return;
const presignClient = getPresignClient();
@@ -233,7 +246,7 @@ extension.on('thumbnail.read', async (entry: Record<string, unknown>) => {
extension.on(
'fs.remove.node',
async ({ target }: { target: Record<string, unknown> }) => {
async (_key, { target }: { target: Record<string, unknown> }) => {
const thumbnailUrl = target.thumbnail as string | undefined;
if (!thumbnailUrl || !thumbnailUrl.startsWith('s3://')) return;
+5 -6
View File
@@ -100,13 +100,12 @@ extension.get(
// OIDC revalidate URL for password-less accounts
if (oidcOnly) {
try {
const providers = await services.oidc.getEnabledProviderIds();
const provider = providers?.[0];
const provider = await services.oidc.getLinkedProviderForUser(
user.id as number,
);
if (provider) {
const callbackUrl =
services.oidc.getCallbackUrl?.('login') ?? '';
const origin = callbackUrl.replace(
/\/auth\/oidc\/callback\/login$/,
const origin = (extension.config.origin ?? '').replace(
/\/$/,
'',
);
details.oidc_revalidate_url = `${origin}/auth/oidc/${provider}/start?flow=revalidate&user_uuid=${encodeURIComponent(user.uuid)}`;
+1862 -1255
View File
File diff suppressed because it is too large. Load diff
+5 -8
View File
@@ -13,6 +13,7 @@
"@eslint/js": "^9.35.0",
"@playwright/test": "^1.56.1",
"@stylistic/eslint-plugin": "^5.3.1",
"@types/better-sqlite3": "^7.6.13",
"@types/express": "^5.0.0",
"@types/mime-types": "^3.0.1",
"@typescript-eslint/eslint-plugin": "^8.46.1",
@@ -22,6 +23,7 @@
"chalk": "^4.1.0",
"clean-css": "^5.3.2",
"dotenv": "^16.4.5",
"esbuild": "^0.28.0",
"eslint": "^9.35.0",
"eslint-config-prettier": "^10.1.8",
"eslint-plugin-prettier": "^5.5.5",
@@ -36,19 +38,17 @@
"typescript": "^5.4.5",
"uglify-js": "^3.17.4",
"vite-plugin-static-copy": "^3.3.0",
"vitest": "^4.1.5",
"webpack": "^5.88.2",
"webpack-cli": "^5.1.1",
"yaml": "^2.8.1"
},
"scripts": {
"test": "npx vitest run --config=src/backend/vitest.config.ts && node src/backend/tools/test.mjs",
"test:puterjs-api": "vitest run tests/puterJsApiTests",
"test:backend": "npm run build:ts; vitest run --config=src/backend/vitest.config.ts",
"test:backend-coverage": "npm run build:ts; vitest run --config=src/backend/vitest.config.ts",
"test:backend": "vitest run --config src/backend/vitest.config.ts ",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run build:ts",
"dev": "npm run build:ts && node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"dev": "npm start",
"build": "npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
"check-translations": "node tools/check-translations.js",
@@ -78,9 +78,6 @@
"javascript-time-ago": "^2.5.11",
"open": "^10.1.0"
},
"optionalDependencies": {
"sharp": "^0.34.4"
},
"engines": {
"node": ">=24.0.0"
}
@@ -20,8 +20,8 @@
import { existsSync, readFileSync } from 'fs';
import { basename, extname, join, resolve } from 'path';
import { createContext, runInContext } from 'vm';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
const MIGRATIONS_DIR = resolve(__dirname, './migrations/sqlite');
@@ -76,6 +76,7 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[41, ['0045_user_oidc_providers.sql']],
[42, ['0046_is-private-apps.sql']],
[43, ['0047_app-url-updates.sql']],
[44, ['0048_old-app-names-unique-tuple.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -95,7 +96,9 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
override async onServerStart(): Promise<void> {
const Database = (await import('better-sqlite3')).default;
const dbPath = this.config.database?.path ?? ':memory:';
const dbPath = this.config.database?.inMemory
? ':memory:'
: (this.config.database?.path ?? ':memory:');
const isNew = dbPath === ':memory:' || !existsSync(dbPath);
this.db = new Database(dbPath);
@@ -453,6 +456,13 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
version: 43,
check: () => this.hasColumn('apps', 'is_private'),
},
{
version: 44,
check: () =>
this.hasRow(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'old_app_names' AND sql LIKE '%UNIQUE%app_uid%name%'",
),
},
];
let inferredUserVersion = 0;
@@ -0,0 +1,48 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- Add UNIQUE(app_uid, name) on `old_app_names` to mirror the SQLite
-- schema (after migration 0048) and let AppStore use ON DUPLICATE KEY
-- UPDATE to refresh the timestamp when the same app re-records the
-- same old name. The MySQL table previously had no uniqueness on these
-- columns, so we deduplicate first (keeping the most recent row per
-- (app_uid, name) pair) before adding the constraint.
--
-- Idempotent: a stored procedure inspects INFORMATION_SCHEMA before
-- adding the index, so re-running the migration directory is safe.
DROP PROCEDURE IF EXISTS _puter_dedup_old_app_names;
DELIMITER //
CREATE PROCEDURE _puter_dedup_old_app_names()
BEGIN
DELETE oa FROM `old_app_names` oa
JOIN `old_app_names` ob
ON oa.`app_uid` = ob.`app_uid`
AND oa.`name` = ob.`name`
AND (oa.`timestamp` < ob.`timestamp`
OR (oa.`timestamp` = ob.`timestamp` AND oa.`id` < ob.`id`));
END//
DELIMITER ;
CALL _puter_dedup_old_app_names();
DROP PROCEDURE IF EXISTS _puter_dedup_old_app_names;
DROP PROCEDURE IF EXISTS _puter_add_unique_old_app_names;
DELIMITER //
CREATE PROCEDURE _puter_add_unique_old_app_names()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'old_app_names'
AND INDEX_NAME = 'unique_old_app_names_app_uid_name'
) THEN
ALTER TABLE `old_app_names`
ADD UNIQUE KEY `unique_old_app_names_app_uid_name` (`app_uid`, `name`);
END IF;
END//
DELIMITER ;
CALL _puter_add_unique_old_app_names();
DROP PROCEDURE IF EXISTS _puter_add_unique_old_app_names;
@@ -0,0 +1,41 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Replace UNIQUE(name) on old_app_names with UNIQUE(app_uid, name).
-- The original constraint blocked the same name from cycling through
-- multiple apps over time and made it impossible for AppStore to use
-- ON CONFLICT(app_uid, name) DO UPDATE to refresh the timestamp when
-- an app re-records its previous name. SQLite has no ALTER TABLE
-- DROP CONSTRAINT, so we recreate the table.
CREATE TABLE `old_app_names_new` (
`id` INTEGER PRIMARY KEY AUTOINCREMENT,
`app_uid` char(40) NOT NULL,
`name` varchar(100) NOT NULL,
`timestamp` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE (`app_uid`, `name`),
FOREIGN KEY (`app_uid`) REFERENCES `apps`(`uid`) ON DELETE CASCADE
);
INSERT INTO `old_app_names_new` (`id`, `app_uid`, `name`, `timestamp`)
SELECT `id`, `app_uid`, `name`, `timestamp` FROM `old_app_names`;
DROP TABLE `old_app_names`;
ALTER TABLE `old_app_names_new` RENAME TO `old_app_names`;
CREATE INDEX `idx_old_app_names_name` ON `old_app_names` (`name`);
+32 -18
View File
@@ -36,33 +36,43 @@ import {
UpdateCommand,
} from '@aws-sdk/lib-dynamodb';
import { NodeHttpHandler } from '@smithy/node-http-handler';
//@ts-expect-error - no types available for dynalite
import dynalite from 'dynalite';
import { randomUUID } from 'node:crypto';
import { once } from 'node:events';
import { Agent as httpsAgent } from 'node:https';
import { PuterClient } from '../types';
import type { IConfig, IDynamoConfig } from '../../types';
import { HttpError } from '../../core/http';
import type { IConfig, IDynamoConfig } from '../../types';
import { PuterClient } from '../types';
const LOCAL_DYNAMO_PATH_KEY = ':memory:';
const LOCAL_DYNAMO_MEMORY_PREFIX = ':memory:';
const localDynaliteEndpointPromises = new Map<string, Promise<string>>();
const MAX_BATCH_WRITE_ITEMS = 25;
const MAX_BATCH_WRITE_RETRIES = 8;
const BATCH_WRITE_RETRY_BASE_MS = 25;
const getDynalitePathKey = (path?: string) => {
if (path === ':memory:') return LOCAL_DYNAMO_PATH_KEY;
// In-memory mode gives each DDBClient its own unique key, which keys
// into a fresh dynalite server. This is what test parallelism needs:
// two clients in the same Node process don't share state. Persistent
// paths still share by `path`, so prod-like reuse keeps working.
const getDynalitePathKey = (path?: string, inMemory?: boolean) => {
if (inMemory) return `${LOCAL_DYNAMO_MEMORY_PREFIX}#${randomUUID()}`;
if (path === ':memory:')
return `${LOCAL_DYNAMO_MEMORY_PREFIX}#${randomUUID()}`;
return path || './volatile/runtime/puter-ddb';
};
const isMemoryPathKey = (pathKey: string) =>
pathKey.startsWith(LOCAL_DYNAMO_MEMORY_PREFIX);
const getOrCreateLocalDynaliteEndpoint = async (pathKey: string) => {
let endpointPromise = localDynaliteEndpointPromises.get(pathKey);
if (endpointPromise) return endpointPromise;
endpointPromise = (async () => {
const dynaliteOptions =
pathKey === LOCAL_DYNAMO_PATH_KEY
? { createTableMs: 0 }
: { createTableMs: 0, path: pathKey };
const dynaliteOptions = isMemoryPathKey(pathKey)
? { createTableMs: 0 }
: { createTableMs: 0, path: pathKey };
const dynaliteInstance = dynalite(dynaliteOptions);
const dynaliteServer = dynaliteInstance.listen(0, '127.0.0.1');
@@ -107,10 +117,18 @@ export class DDBClient extends PuterClient {
#documentClient: DynamoDBDocumentClient | null = null;
#localInitPromise: Promise<void> | null = null;
#ddbConfig: IDynamoConfig;
// Resolved once at construction. In-memory mode generates a unique
// key per instance so parallel clients don't share state, but
// `recreateClient()` reuses this same key and so reuses the server.
#localPathKey: string;
constructor(config: IConfig) {
super(config);
this.#ddbConfig = config.dynamo ?? {};
this.#localPathKey = getDynalitePathKey(
this.#ddbConfig.path,
this.#ddbConfig.inMemory,
);
if (this.#ddbConfig.aws) {
this.#bindAwsClient();
@@ -388,13 +406,8 @@ export class DDBClient extends PuterClient {
ReturnConsumedCapacity: 'TOTAL',
});
try {
const client = await this.#getDocumentClient();
return await client.send(command);
} catch (error) {
console.error('DDB Update Error', error);
throw error;
}
const client = await this.#getDocumentClient();
return client.send(command);
}
async createTableIfNotExists(
@@ -480,8 +493,9 @@ export class DDBClient extends PuterClient {
}
async #bindLocalClient() {
const pathKey = getDynalitePathKey(this.#ddbConfig.path);
const endpoint = await getOrCreateLocalDynaliteEndpoint(pathKey);
const endpoint = await getOrCreateLocalDynaliteEndpoint(
this.#localPathKey,
);
const ddbClient = new DynamoDBClient({
credentials: {
+1 -1
View File
@@ -69,7 +69,7 @@ The Puter Team
html: `
<p>Hi{{#if owner_username}} {{owner_username}}{{/if}},</p>
<p>
Please update <a href="https://apps.puter.com/app/{{app_name}}">{{app_title}}</a>.
Please update <a href="https://puter.com/app/{{app_name}}">{{app_title}}</a>.
</p>
<p><strong>Requested updates:</strong></p>
<blockquote>{{nl2br message}}</blockquote>
@@ -0,0 +1,213 @@
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import { PuterServer } from '../../server.ts';
import { setupTestServer } from '../../testUtil.ts';
import type { EventClient } from './EventClient.js';
describe('EventClient', () => {
let server: PuterServer;
let target: EventClient;
beforeAll(async () => {
server = await setupTestServer();
target = server.clients.event as unknown as EventClient;
});
afterAll(async () => {
await server.shutdown();
});
// Each test gets a fresh key so listeners registered by earlier tests
// never collide with later ones (the client has no public way to
// unsubscribe).
let key: string;
beforeEach(() => {
key = `test.${Math.random().toString(36).slice(2)}`;
});
describe('on / emit', () => {
it('invokes a listener registered for the exact key', () => {
const listener = vi.fn();
target.on(key, listener);
target.emit(key, { hello: 'world' }, { source: 'test' });
expect(listener).toHaveBeenCalledTimes(1);
expect(listener).toHaveBeenCalledWith(
key,
{ hello: 'world' },
{ source: 'test' },
);
});
it('does not invoke listeners registered under a different key', () => {
const other = vi.fn();
target.on(`${key}.other`, other);
target.emit(key, {}, {});
expect(other).not.toHaveBeenCalled();
});
it('invokes every listener registered for the same key', () => {
const a = vi.fn();
const b = vi.fn();
target.on(key, a);
target.on(key, b);
target.emit(key, {}, {});
expect(a).toHaveBeenCalledTimes(1);
expect(b).toHaveBeenCalledTimes(1);
});
it('does nothing when emitting a key with no listeners', () => {
// Just asserts that no exception is thrown.
expect(() =>
target.emit(`${key}.nobody-home`, {}, {}),
).not.toThrow();
});
it('continues firing later listeners after one throws', () => {
const errSpy = vi
.spyOn(console, 'error')
.mockImplementation(() => {});
const bad = vi.fn(() => {
throw new Error('boom');
});
const good = vi.fn();
target.on(key, bad);
target.on(key, good);
target.emit(key, {}, {});
expect(bad).toHaveBeenCalledTimes(1);
expect(good).toHaveBeenCalledTimes(1);
errSpy.mockRestore();
});
});
describe('wildcard subscriptions', () => {
it('matches every dot-extended descendant of a wildcard prefix', () => {
const listener = vi.fn();
target.on(`${key}.*`, listener);
target.emit(`${key}.foo`, { v: 1 }, {});
target.emit(`${key}.foo.bar`, { v: 2 }, {});
expect(listener).toHaveBeenCalledTimes(2);
expect(listener).toHaveBeenNthCalledWith(
1,
`${key}.foo`,
{ v: 1 },
{},
);
expect(listener).toHaveBeenNthCalledWith(
2,
`${key}.foo.bar`,
{ v: 2 },
{},
);
});
it('fires both wildcard and exact-key subscribers for one emit', () => {
const wild = vi.fn();
const exact = vi.fn();
target.on(`${key}.*`, wild);
target.on(`${key}.thing`, exact);
target.emit(`${key}.thing`, {}, {});
expect(wild).toHaveBeenCalledTimes(1);
expect(exact).toHaveBeenCalledTimes(1);
});
it('does not fire a wildcard listener for the prefix itself', () => {
const wild = vi.fn();
target.on(`${key}.*`, wild);
target.emit(key, {}, {});
expect(wild).not.toHaveBeenCalled();
});
it('fires wildcards at every nesting level for a deep emit', () => {
const top = vi.fn();
const mid = vi.fn();
target.on(`${key}.*`, top);
target.on(`${key}.a.*`, mid);
target.emit(`${key}.a.b.c`, {}, {});
expect(top).toHaveBeenCalledTimes(1);
expect(mid).toHaveBeenCalledTimes(1);
});
});
describe('emitAndWait', () => {
it('awaits async listeners before resolving', async () => {
let resolved = false;
target.on(key, async () => {
await new Promise((r) => setTimeout(r, 10));
resolved = true;
});
await target.emitAndWait(key, {}, {});
expect(resolved).toBe(true);
});
it('runs listeners sequentially so later ones see earlier mutations', async () => {
target.on(key, (_k, data) => {
(data as { steps: string[] }).steps.push('first');
});
target.on(key, async (_k, data) => {
await new Promise((r) => setTimeout(r, 5));
(data as { steps: string[] }).steps.push('second');
});
const data = { steps: [] as string[] };
await target.emitAndWait(key, data, {});
expect(data.steps).toEqual(['first', 'second']);
});
it('continues the chain when a listener throws', async () => {
const errSpy = vi
.spyOn(console, 'error')
.mockImplementation(() => {});
const after = vi.fn();
target.on(key, () => {
throw new Error('boom');
});
target.on(key, after);
await target.emitAndWait(key, {}, {});
expect(after).toHaveBeenCalledTimes(1);
errSpy.mockRestore();
});
it('awaits wildcard listeners too', async () => {
let resolved = false;
target.on(`${key}.*`, async () => {
await new Promise((r) => setTimeout(r, 10));
resolved = true;
});
await target.emitAndWait(`${key}.child`, {}, {});
expect(resolved).toBe(true);
});
});
describe('lifecycle hooks', () => {
it('onServerStart emits a serverStart event', () => {
const listener = vi.fn();
target.on('serverStart', listener);
target.onServerStart();
expect(listener).toHaveBeenCalledWith('serverStart', {}, {});
});
it('onServerPrepareShutdown emits a serverPrepareShutdown event', () => {
const listener = vi.fn();
target.on('serverPrepareShutdown', listener);
target.onServerPrepareShutdown();
expect(listener).toHaveBeenCalledWith(
'serverPrepareShutdown',
{},
{},
);
});
it('onServerShutdown emits a serverShutdown event', () => {
const listener = vi.fn();
target.on('serverShutdown', listener);
target.onServerShutdown();
expect(listener).toHaveBeenCalledWith('serverShutdown', {}, {});
});
});
});
@@ -17,10 +17,12 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { PuterClient } from './types';
import { extensionStore } from '../../extensions';
import { PuterClient } from '../types';
import { EventListener, EventMap, ListenKey, MatchingEvents } from './types';
export class EventClient extends PuterClient {
#eventListeners = {};
#eventListeners: Partial<Record<ListenKey, EventListener[]>> = {};
onServerStart() {
this.emit('serverStart', {}, {});
@@ -49,18 +51,19 @@ export class EventClient extends PuterClient {
* under their literal `<prefix>.*` string. No regex, no per-emit
* scan of every listener.
*
* @param {string} key
* @param {unknown} data
* @param {object} meta
*/
emit(key, data, meta) {
emit<T extends keyof EventMap>(key: T, data: EventMap[T], meta: unknown) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey =
const matchKey = (
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`;
const listeners = this.#eventListeners[matchKey];
: `${parts.slice(0, i + 1).join('.')}.*`
) as ListenKey;
const extensionListeners = extensionStore.events[matchKey];
const listeners = (this.#eventListeners[matchKey] || []).concat(
extensionListeners || [],
);
if (!listeners) continue;
for (const listener of listeners) {
this.#emitEvent(listener, key, data, meta);
@@ -80,20 +83,23 @@ export class EventClient extends PuterClient {
* Listeners run sequentially in the order they're registered so an
* earlier handler's mutation is visible to later ones. A listener that
* throws is logged (same as `emit`) and the chain continues.
*
* @param {string} key
* @param {unknown} data
* @param {object} meta
* @returns {Promise<void>}
*/
async emitAndWait(key, data, meta) {
async emitAndWait<T extends keyof EventMap>(
key: T,
data: EventMap[T],
meta: unknown,
) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey =
const matchKey = (
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`;
const listeners = this.#eventListeners[matchKey];
: `${parts.slice(0, i + 1).join('.')}.*`
) as ListenKey;
const extensionListeners = extensionStore.events[matchKey];
const listeners = (this.#eventListeners[matchKey] || []).concat(
extensionListeners || [],
);
if (!listeners) continue;
for (const listener of listeners) {
try {
@@ -117,17 +123,25 @@ export class EventClient extends PuterClient {
* to `emit()` — wildcard subscribers can branch on the triggering
* event name.
*
* @param {string} key
* @param {(key: string, data: unknown, meta: object) => void} callback
*/
on(key, callback) {
if (!this.#eventListeners[key]) {
this.#eventListeners[key] = [];
}
this.#eventListeners[key].push(callback);
on<P extends ListenKey>(
key: P,
callback: (
key: MatchingEvents<P>,
data: EventMap[MatchingEvents<P>],
meta: unknown,
) => Promise<void> | void,
) {
const listeners: EventListener[] =
this.#eventListeners[key] ?? (this.#eventListeners[key] = []);
listeners.push(callback as EventListener);
}
async #emitEvent(listener, key, data, meta) {
async #emitEvent<T extends keyof EventMap>(
listener: EventListener,
key: T,
data: EventMap[T],
meta: unknown,
) {
try {
await listener(key, data, meta);
} catch (e) {
+271
View File
@@ -0,0 +1,271 @@
/**
* Shape of every payload emitted on the event bus, keyed by event name.
*
* Domain objects (User, Actor, FSEntry, Socket, ...) are typed as `unknown`
* here on purpose — pulling their real types in would couple this module to
* most of the backend and risk import cycles. Refine at the listener if
* you need narrowed access.
*
* Conventions:
* - `*.validate` events carry an `allow` flag listeners flip to reject;
* they tend to grow listener-specific fields, so they accept extras via
* an index signature.
* - `outer.gui.*` events ride the `{ user_id_list, response }` envelope
* the SocketService fans out to user-scoped channels.
*/
// GUI write events spread an entry plus per-event metadata into `response`.
// The exact field set varies by emit site (FSController / LegacyFSController /
// WebDAVController each project a slightly different shape) so the envelope
// stays loose; listeners narrow on whichever fields they actually consume.
type GuiEvent<R = Record<string, unknown>> = {
user_id_list: number[];
response: R;
};
export type EventMap = {
// ---- Server lifecycle ----
serverStart: Record<string, never>;
serverPrepareShutdown: Record<string, never>;
serverShutdown: Record<string, never>;
// ---- AI ----
'ai.prompt.validate': {
username: string;
intended_service: string;
parameters: unknown;
allow?: boolean;
abuse?: unknown;
custom?: unknown;
[key: string]: unknown;
};
'ai.prompt.complete': {
username: string;
completionId: string;
intended_service: string;
parameters: unknown;
result: unknown;
model_used: string;
service_used: string;
};
'ai.prompt.cost-calculated': {
completionId: string;
username: string;
usage: unknown;
input_tokens: number;
output_tokens: number;
input_ucents: number;
output_ucents: number;
total_ucents: number;
costs_currency: string;
model_used: string;
service_used: string;
intended_service: string;
model_details: {
id: string;
provider: string;
input_cost_key: string;
output_cost_key: string;
costs: unknown;
costs_currency: string;
};
};
'ai.log.image': {
actor: unknown;
completionId: string;
parameters: unknown;
intended_service: string;
model_used: string;
service_used: string;
};
// ---- Apps ----
'app.changed': {
app_uid: string;
action: string;
app?: unknown;
old_app?: unknown;
};
'app.new-icon': { app_uid: string; data_url: string };
'app.opened': { app_uid: string; user_id: number; ts: number };
'app.rename': {
app_uid: string;
old_name: string;
new_name: string;
app: unknown;
};
'app.from-origin': { origin: string };
'app.privateAccess.check': {
app: unknown;
actor: unknown;
result: {
allowed: boolean;
reason?: string;
redirectUrl?: string;
checkedBy?: string;
};
};
'app.privateAccess.resolveLaunch': {
app: unknown;
actor: unknown;
result: { allowed: boolean; reason?: string; error?: string };
};
// ---- Auth / signup ----
'puter.signup.validate': {
allow: boolean;
email?: string;
ip?: string | null;
source?: 'oidc';
req?: unknown;
data?: unknown;
abuse?: unknown;
custom?: unknown;
[key: string]: unknown;
};
'puter.signup.success': {
user_id: number;
user_uuid: string;
email: string;
username: string;
ip?: string | null;
[key: string]: unknown;
};
'email.validate': {
email: string;
allow: boolean;
message: string | null;
[key: string]: unknown;
};
'user.save_account': {
user_id: number;
old_username?: string;
new_username?: string;
email?: string;
};
'user.email-confirmed': {
user_id: number;
user_uid: string;
email: string;
};
'user.username-changed': {
user_id: number;
old_username: string;
new_username: string;
};
'user.email-changed': { user_id: number; new_email: string };
// ---- Filesystem ----
'fs.copy.node': {
source: unknown;
copy: unknown;
sourceObjectKey: string;
copyObjectKey: string;
};
'fs.move.node': { node: unknown; fromPath: string; toPath: string };
'fs.remove.node': { node: unknown; entry: unknown; target: unknown };
'fs.storage.upload-progress': {
upload_tracker: unknown;
context: unknown;
meta: {
user_id: number;
userId: number;
item_uid: string;
item_path: string;
[key: string]: unknown;
};
};
'storage.quota.bonus': { userId: number; extra: number };
// ---- Outer / GUI broadcast ----
'outer.cacheUpdate': {
cacheKey: string[];
data?: unknown;
ttlSeconds?: number;
};
'outer.fs.write-hash': { hash: string; uuid: string };
'outer.gui.item.added': GuiEvent;
'outer.gui.item.updated': GuiEvent;
'outer.gui.item.moved': GuiEvent;
'outer.gui.item.pending': GuiEvent;
'outer.gui.item.removed': GuiEvent;
'outer.gui.notif.ack': GuiEvent<{ uid: string }>;
'outer.gui.notif.persisted': GuiEvent<{ uid: string }>;
'outer.gui.notif.message': GuiEvent<{ uid: string; notification: unknown }>;
'outer.gui.notif.unreads': GuiEvent<{
unreads: { uid: string; notification: unknown }[];
}>;
// ---- Subdomains ----
'subdomain.delete': { subdomain: string };
'subdomain.update': { subdomain: string };
// ---- Thumbnails ----
'thumbnail.read': {
uri: string;
size?: string;
thumbnail?: string | null;
};
'thumbnail.created': { url: string };
'thumbnail.upload.prepare': {
items: { index: number; item_uid: string }[];
uploadUrl?: string;
thumbnailUrl?: string;
};
// ---- Web sockets ----
'web.socket.connected': { socket: unknown; user: unknown };
'web.socket.user-connected': { socket: unknown; user: unknown };
// ---- Extension hooks / misc ----
'puter.gui.addons': {
prependHeadContent?: string[];
prependBodyContent?: string[];
};
'whoami.details': {
user: unknown;
details: {
uuid?: string;
username?: string;
email?: string;
app_name?: string;
};
isUser: boolean;
};
'wisp.get-policy': {
app: unknown;
actor: unknown;
allow: boolean;
policy?: unknown;
};
'ip.validate': { allow: boolean; ip: string };
} & {
// SocketService re-emits each fanout-eligible event under
// `sent-to-user.<wireName>` so per-user channels can subscribe by wire name.
[K in `sent-to-user.${string}`]: { user_id: number; response: unknown };
};
export type EventKey = keyof EventMap & string;
// "a.b.c" -> "a.*" | "a.b.*"
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
? Extract<EventKey, `${Prefix}.${string}`>
: P & EventKey;
export type EventListener<K extends EventKey = EventKey> = (
key: K,
data: EventMap[K],
meta: unknown,
) => Promise<void> | void;
+1 -1
View File
@@ -20,7 +20,7 @@
import { AlarmClient } from './alarm/AlarmClient';
import { DatabaseClientFactory } from './database';
import { EmailClient } from './email/EmailClient';
import { EventClient } from './EventClient';
import { EventClient } from './event/EventClient';
import { DDBClient } from './dynamodb/DDBClient';
import { RedisClient } from './redis/RedisClient';
import { S3Client } from './s3/S3Client';
+23
View File
@@ -19,6 +19,29 @@
import type { IConfig, WithLifecycle } from '../types';
/**
* Extension-augmentable client registry. Extensions add their own client
* instance types via TypeScript declaration merging:
*
* declare module '@heyputer/backend/clients/types' {
* interface IExtensionClientInstances {
* myClient: MyClient;
* }
* }
*
* Augmentations flow into `this.clients` everywhere it's typed (PuterStore,
* PuterService, PuterController, PuterDriver) and into the
* `extension.import('client')` proxy.
*/
export interface IExtensionClientInstances {
/**
* Open index signature so reads of extension-only client keys return
* `unknown` instead of a type error. Concrete declaration-merged keys
* override this for that name.
*/
[key: string]: unknown;
}
export interface IPuterClient<T extends WithLifecycle = WithLifecycle> {
new (config: IConfig): T;
}
@@ -123,7 +123,7 @@ export class AppController extends PuterController {
}
try {
this.clients.event?.emit(
this.clients.event?.emitAndWait(
'app.opened',
{
app_uid,
@@ -0,0 +1,709 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock
// redis) and re-registers AppController's inline lambda routes onto a
// fresh PuterRouter so each handler is reachable. Tests run against
// the live wired AppDriver, AppStore, DB client, and EventClient —
// no method spies. Apps are created via the real AppDriver so
// ownership / approval / metadata behave exactly like prod.
let server: PuterServer;
let router: PuterRouter;
interface CrudQDriver {
create: (args: Record<string, unknown>) => Promise<Record<string, unknown>>;
read: (args: Record<string, unknown>) => Promise<Record<string, unknown>>;
select: (args: Record<string, unknown>) => Promise<unknown[]>;
isNameAvailable: (name: string) => Promise<boolean>;
}
let driver: CrudQDriver;
beforeAll(async () => {
server = await setupTestServer();
router = new PuterRouter();
server.controllers.apps.registerRoutes(router);
driver = server.drivers.apps as unknown as CrudQDriver;
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `apc-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const withActor = <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
const uniqueName = (prefix: string) =>
`${prefix}-${Math.random().toString(36).slice(2, 10)}`;
const uniqueIndexUrl = () =>
`https://example-${Math.random().toString(36).slice(2, 10)}.test/`;
const createApp = async (
actor: Actor,
overrides: Record<string, unknown> = {},
): Promise<Record<string, unknown>> =>
withActor(actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'Test App',
description: 'desc',
index_url: uniqueIndexUrl(),
...overrides,
},
}),
);
interface CapturedResponse {
statusCode: number;
body: unknown;
headers: Record<string, string>;
redirectStatus?: number;
redirectUrl?: string;
}
const makeReq = (init: {
body?: unknown;
query?: Record<string, unknown>;
params?: Record<string, unknown>;
actor?: Actor | unknown;
}): Request => {
return {
body: init.body ?? {},
query: init.query ?? {},
params: init.params ?? {},
headers: {},
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = {
statusCode: 200,
body: undefined,
headers: {},
};
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
send: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
set: vi.fn((key: string, value: string) => {
if (typeof key === 'string') {
captured.headers[key.toLowerCase()] = value;
}
return res;
}),
setHeader: vi.fn(() => res),
redirect: vi.fn((status: number | string, url?: string) => {
if (typeof status === 'number' && typeof url === 'string') {
captured.redirectStatus = status;
captured.redirectUrl = url;
} else if (typeof status === 'string') {
captured.redirectStatus = 302;
captured.redirectUrl = status;
}
return res;
}),
};
return { res: res as unknown as Response, captured };
};
const findHandler = (method: string, path: string): RequestHandler => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route.handler;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler = findHandler(method, path);
await handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
};
// ── GET /apps ───────────────────────────────────────────────────────
describe('AppController GET /apps', () => {
it('returns the apps the caller can edit (filtered by user-can-edit)', async () => {
const owner = await makeUser();
const stranger = await makeUser();
const app = await createApp(owner.actor, { name: uniqueName('mine') });
// Owner sees their own app.
const { res: ownerRes, captured: ownerCaptured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'get',
'/apps',
makeReq({ actor: owner.actor }),
ownerRes,
),
);
const ownerApps = ownerCaptured.body as Array<{ uid: string }>;
expect(ownerApps.some((a) => a.uid === app.uid)).toBe(true);
// A different user does not.
const { res: strangerRes, captured: strangerCaptured } = makeRes();
await withActor(stranger.actor, () =>
callRoute(
'get',
'/apps',
makeReq({ actor: stranger.actor }),
strangerRes,
),
);
const strangerApps = strangerCaptured.body as Array<{ uid: string }>;
expect(strangerApps.every((a) => a.uid !== app.uid)).toBe(true);
});
});
// ── GET /apps/nameAvailable ─────────────────────────────────────────
describe('AppController GET /apps/nameAvailable', () => {
it('returns available=true for an unused name', async () => {
const { actor } = await makeUser();
const name = uniqueName('avail');
const { res, captured } = makeRes();
await withActor(actor, () =>
callRoute(
'get',
'/apps/nameAvailable',
makeReq({ query: { name }, actor }),
res,
),
);
expect(captured.body).toEqual({ name, available: true });
});
it('returns available=false once an app with that name exists', async () => {
const { actor } = await makeUser();
const name = uniqueName('taken');
await createApp(actor, { name });
const { res, captured } = makeRes();
await withActor(actor, () =>
callRoute(
'get',
'/apps/nameAvailable',
makeReq({ query: { name }, actor }),
res,
),
);
expect(captured.body).toEqual({ name, available: false });
});
it('throws 400 when `name` query param is missing', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
callRoute(
'get',
'/apps/nameAvailable',
makeReq({ query: {}, actor }),
res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── POST /rao ───────────────────────────────────────────────────────
describe('AppController POST /rao', () => {
it('throws 400 when neither body nor actor carries app_uid', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
callRoute(
'post',
'/rao',
makeReq({ body: {}, actor }),
res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 404 when the supplied app_uid does not exist', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
callRoute(
'post',
'/rao',
makeReq({
body: { app_uid: 'app-no-such-thing' },
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('records the app_open in the DB on success', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
const { res, captured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'post',
'/rao',
makeReq({ body: { app_uid: app.uid }, actor: owner.actor }),
res,
),
);
expect(captured.body).toEqual({});
const rows = (await server.clients.db.read(
'SELECT `app_uid`, `user_id` FROM `app_opens` WHERE `app_uid` = ? AND `user_id` = ?',
[app.uid, owner.userId],
)) as Array<{ app_uid: string; user_id: number }>;
expect(rows).toHaveLength(1);
});
it('falls back to actor.app.uid when the body omits app_uid', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
const { res, captured } = makeRes();
const actorWithApp: unknown = {
...owner.actor,
app: { uid: app.uid },
};
await withActor(owner.actor, () =>
callRoute(
'post',
'/rao',
makeReq({ body: {}, actor: actorWithApp }),
res,
),
);
expect(captured.body).toEqual({});
const rows = (await server.clients.db.read(
'SELECT `app_uid` FROM `app_opens` WHERE `app_uid` = ? AND `user_id` = ?',
[app.uid, owner.userId],
)) as Array<{ app_uid: string }>;
expect(rows.length).toBeGreaterThan(0);
});
});
// ── GET /apps/:name (single + pipe-batched) ─────────────────────────
describe('AppController GET /apps/:name', () => {
it('throws 404 for a single missing app', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
callRoute(
'get',
'/apps/:name',
makeReq({
params: { name: 'no-such-app' },
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('returns a single app object for a single name', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
const { res, captured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'get',
'/apps/:name',
makeReq({
params: { name: app.name },
actor: owner.actor,
}),
res,
),
);
expect(Array.isArray(captured.body)).toBe(false);
const body = captured.body as Record<string, unknown>;
expect(body.uid).toBe(app.uid);
expect(body.privateAccess).toBeDefined();
});
it('returns an array (with nulls for unknowns) for pipe-separated batch', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
const { res, captured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'get',
'/apps/:name',
makeReq({
params: { name: `${app.name}|missing-app` },
actor: owner.actor,
}),
res,
),
);
expect(Array.isArray(captured.body)).toBe(true);
const arr = captured.body as unknown[];
expect(arr).toHaveLength(2);
expect((arr[0] as Record<string, unknown>).uid).toBe(app.uid);
expect(arr[1]).toBeNull();
});
});
// ── POST /query/app ─────────────────────────────────────────────────
describe('AppController POST /query/app', () => {
it('returns [] for a non-array body', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
await withActor(actor, () =>
callRoute(
'post',
'/query/app',
makeReq({ body: { not: 'an-array' }, actor }),
res,
),
);
expect(captured.body).toEqual([]);
});
it('throws 400 when the array exceeds the 200-entry cap', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
callRoute(
'post',
'/query/app',
makeReq({
body: new Array(201).fill('x'),
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('returns approved-for-listing apps with the v1 marketplace shape', async () => {
const owner = await makeUser();
// Name must not start with `app-` — the controller treats names
// with that prefix as UID lookups.
const app = await createApp(owner.actor, {
name: uniqueName('marketplace'),
description: 'cool desc',
});
// approved_for_listing is admin-controlled and goes through the
// store's READ_ONLY filter — flip it via a direct DB write
// and invalidate the store's cache so the stale row doesn't
// get served.
await server.clients.db.write(
'UPDATE `apps` SET `approved_for_listing` = 1 WHERE `uid` = ?',
[app.uid],
);
await server.stores.app.invalidateByUid(app.uid as string);
// Stranger queries by name and gets the v1 marketplace shape.
const stranger = await makeUser();
const { res, captured } = makeRes();
await withActor(stranger.actor, () =>
callRoute(
'post',
'/query/app',
makeReq({ body: [app.name], actor: stranger.actor }),
res,
),
);
const body = captured.body as Array<Record<string, unknown>>;
expect(body).toHaveLength(1);
expect(body[0]).toMatchObject({
uuid: app.uid,
name: app.name,
description: 'cool desc',
});
// Internal fields must be omitted.
expect(body[0]?.id).toBeUndefined();
expect(body[0]?.owner_user_id).toBeUndefined();
expect(body[0]?.index_url).toBeUndefined();
});
it('skips selectors that are empty/oversize/non-string', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
await withActor(actor, () =>
callRoute(
'post',
'/query/app',
makeReq({
body: [
'', // empty
'a'.repeat(201), // oversize
123, // wrong type
],
actor,
}),
res,
),
);
expect(captured.body).toEqual([]);
});
it('hides unapproved, protected, non-owned apps from strangers', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor, {
name: uniqueName('hidden'),
});
// The /query/app gate only fires when AppDriver.read denies
// access — and that only happens for `protected` apps the
// caller doesn't own / hasn't been granted access to.
// `protected` is admin-controlled (READ_ONLY in AppStore), so
// we flip it via a direct DB write.
await server.clients.db.write(
'UPDATE `apps` SET `protected` = 1 WHERE `uid` = ?',
[app.uid],
);
await server.stores.app.invalidateByUid(app.uid as string);
const stranger = await makeUser();
const { res, captured } = makeRes();
await withActor(stranger.actor, () =>
callRoute(
'post',
'/query/app',
makeReq({ body: [app.name], actor: stranger.actor }),
res,
),
);
// Existence isn't surfaced — caller can't enumerate private apps.
expect(captured.body).toEqual([]);
});
it('shows owners their own unapproved apps', async () => {
const owner = await makeUser();
// Avoid `app-` prefix so the controller looks up by name.
const app = await createApp(owner.actor, {
name: uniqueName('mine'),
});
const { res, captured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'post',
'/query/app',
makeReq({ body: [app.name], actor: owner.actor }),
res,
),
);
const body = captured.body as Array<Record<string, unknown>>;
expect(body).toHaveLength(1);
expect(body[0]?.uuid).toBe(app.uid);
});
it('looks up by UID when the selector starts with `app-`', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
await server.clients.db.write(
'UPDATE `apps` SET `approved_for_listing` = 1 WHERE `uid` = ?',
[app.uid],
);
await server.stores.app.invalidateByUid(app.uid as string);
const { res, captured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'post',
'/query/app',
makeReq({ body: [app.uid], actor: owner.actor }),
res,
),
);
const body = captured.body as Array<Record<string, unknown>>;
expect(body).toHaveLength(1);
expect(body[0]?.name).toBe(app.name);
});
});
// ── GET /app-icon/:app_uid(/:size) ──────────────────────────────────
describe('AppController GET /app-icon/:app_uid', () => {
it('returns 400 for a missing app_uid param', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/app-icon/:app_uid',
makeReq({ params: { app_uid: '' } }),
res,
);
expect(captured.statusCode).toBe(400);
});
it('returns 400 for an unsupported size param', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/app-icon/:app_uid/:size',
makeReq({ params: { app_uid: 'app-1', size: '999' } }),
res,
);
expect(captured.statusCode).toBe(400);
});
it('serves the default icon when the app row has no icon', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
const { res, captured } = makeRes();
await callRoute(
'get',
'/app-icon/:app_uid',
makeReq({ params: { app_uid: app.uid } }),
res,
);
// Default icon is SVG; CSP sandbox locks down script execution.
expect(captured.headers['content-type']).toContain('image/svg+xml');
expect(captured.headers['content-security-policy']).toContain(
'sandbox',
);
expect(Buffer.isBuffer(captured.body)).toBe(true);
});
it('decodes a data URL icon and serves the declared MIME', async () => {
const owner = await makeUser();
const png = Buffer.from('mock-png-bytes');
const dataUrl = `data:image/png;base64,${png.toString('base64')}`;
const app = await createApp(owner.actor, { icon: dataUrl });
const { res, captured } = makeRes();
await callRoute(
'get',
'/app-icon/:app_uid',
makeReq({ params: { app_uid: app.uid } }),
res,
);
expect(captured.headers['content-type']).toBe('image/png');
expect(Buffer.isBuffer(captured.body)).toBe(true);
expect((captured.body as Buffer).equals(png)).toBe(true);
});
it('falls back to the default icon when the data-URL MIME is not allowlisted', async () => {
const owner = await makeUser();
// text/html is NOT in the icon allowlist — must NOT be echoed back.
// AppDriver.create rejects non-image MIMEs at write time, so we
// bypass and store the dangerous icon directly in the DB.
const app = await createApp(owner.actor);
const dataUrl = `data:text/html;base64,${Buffer.from('<script>alert(1)</script>').toString('base64')}`;
await server.clients.db.write(
'UPDATE `apps` SET `icon` = ? WHERE `uid` = ?',
[dataUrl, app.uid],
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/app-icon/:app_uid',
makeReq({ params: { app_uid: app.uid } }),
res,
);
// Falls back to default icon (SVG).
expect(captured.headers['content-type']).toContain('image/svg+xml');
});
it('prepends the `app-` prefix when omitted from the param', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
// Strip the prefix; controller should re-add it before lookup.
const stripped = String(app.uid).replace(/^app-/, '');
const { res, captured } = makeRes();
await callRoute(
'get',
'/app-icon/:app_uid',
makeReq({ params: { app_uid: stripped } }),
res,
);
// Either the default icon (no `icon` column on the row) or a
// configured one — both come back as 200, not 404.
expect(captured.statusCode).toBe(200);
});
});
+12 -1
View File
@@ -404,6 +404,7 @@ export class AuthController extends PuterController {
no_temp_user: false,
requires_email_confirmation: false,
message: null,
code: null,
};
try {
await this.clients.event?.emitAndWait(
@@ -418,6 +419,11 @@ export class AuthController extends PuterController {
throw new HttpError(
403,
validateEvent.message ?? 'Signup blocked',
{
...(validateEvent.code
? { code: validateEvent.code }
: {}),
},
);
}
if (is_temp && validateEvent.no_temp_user) {
@@ -425,7 +431,12 @@ export class AuthController extends PuterController {
403,
validateEvent.message ??
'Temporary accounts are disabled',
{ legacyCode: 'must_login_or_signup' },
{
legacyCode: 'must_login_or_signup',
...(validateEvent.code
? { code: validateEvent.code }
: {}),
},
);
}
const force_email_confirmation = Boolean(
@@ -0,0 +1,114 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { EventClient } from '../../clients/EventClient.js';
import { HttpError } from '../../core/http/HttpError.js';
let server: PuterServer;
let eventClient: EventClient;
beforeAll(async () => {
server = await setupTestServer();
eventClient = server.clients.event as unknown as EventClient;
});
afterAll(async () => {
await server?.shutdown();
});
describe('puter.signup.validate event', () => {
it('supports code in the validate event when allow is false', async () => {
eventClient.on('puter.signup.validate', (_key, data) => {
const event = data as {
allow: boolean;
message: string | null;
code: string | null;
};
event.allow = false;
event.message = 'Region not supported';
event.code = 'region_blocked';
});
const validateEvent = {
req: {},
data: {},
ip: '127.0.0.1',
email: '[email protected]',
allow: true,
no_temp_user: false,
requires_email_confirmation: false,
message: null as string | null,
code: null as string | null,
};
await eventClient.emitAndWait(
'puter.signup.validate',
validateEvent,
{},
);
expect(validateEvent.allow).toBe(false);
expect(validateEvent.message).toBe('Region not supported');
expect(validateEvent.code).toBe('region_blocked');
// Verify the HttpError constructed from this event carries the code
const err = new HttpError(
403,
validateEvent.message ?? 'Signup blocked',
{
...(validateEvent.code
? { code: validateEvent.code }
: {}),
},
);
expect(err.statusCode).toBe(403);
expect(err.message).toBe('Region not supported');
expect(err.code).toBe('region_blocked');
});
it('omits code from HttpError when extension does not set it', async () => {
const validateEvent = {
req: {},
data: {},
ip: '127.0.0.1',
email: '[email protected]',
allow: false,
no_temp_user: false,
requires_email_confirmation: false,
message: 'Blocked',
code: null as string | null,
};
const err = new HttpError(
403,
validateEvent.message ?? 'Signup blocked',
{
...(validateEvent.code
? { code: validateEvent.code }
: {}),
},
);
expect(err.statusCode).toBe(403);
expect(err.message).toBe('Blocked');
expect(err.code).toBeUndefined();
});
});
@@ -0,0 +1,199 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { BroadcastController } from './BroadcastController.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one PuterServer and exercises the live BroadcastController
// against the wired BroadcastService. The default test config has no
// configured peers, so every signed-incoming path falls through to the
// service's "unknown peer" gate (403). That's the right blast radius
// for these tests — we cover header validation + error→HTTP wiring,
// and trust BroadcastService's own tests for the crypto path.
let server: PuterServer;
let controller: BroadcastController;
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.broadcast as unknown as BroadcastController;
});
afterAll(async () => {
await server?.shutdown();
});
interface CapturedResponse {
statusCode: number;
body: unknown;
}
const makeReq = (init: {
body?: unknown;
rawBody?: Buffer;
headers?: Record<string, string>;
}): Request => {
return {
body: init.body ?? {},
rawBody: init.rawBody,
query: {},
headers: init.headers ?? {},
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
setHeader: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
// Standard signed-payload headers used across cases. Real verification
// fails because no peer is configured — the test config doesn't set up
// `broadcast_peers` — so we land on the service's "Unknown peer" gate
// rather than an HMAC mismatch.
const signedHeaders = (): Record<string, string> => ({
'x-broadcast-peer-id': 'unknown-peer',
'x-broadcast-timestamp': String(Math.floor(Date.now() / 1000)),
'x-broadcast-nonce': '1',
'x-broadcast-signature': 'a'.repeat(64),
});
// ── /broadcast/webhook ──────────────────────────────────────────────
describe('BroadcastController.webhook', () => {
it('returns 400 when rawBody is missing', async () => {
const { res, captured } = makeRes();
await controller.webhook(
makeReq({
body: { events: [] },
headers: signedHeaders(),
}),
res,
);
expect(captured.statusCode).toBe(400);
expect(captured.body).toMatchObject({
error: { message: expect.stringContaining('body') },
});
});
it('returns 400 when the JSON body is not an object', async () => {
const { res, captured } = makeRes();
await controller.webhook(
makeReq({
body: 'not an object',
rawBody: Buffer.from('"not an object"'),
headers: signedHeaders(),
}),
res,
);
expect(captured.statusCode).toBe(400);
});
it('returns 400 when the body has neither `events` nor a single-event shape', async () => {
const raw = Buffer.from('{}');
const { res, captured } = makeRes();
await controller.webhook(
makeReq({
body: {},
rawBody: raw,
headers: signedHeaders(),
}),
res,
);
expect(captured.statusCode).toBe(400);
expect(captured.body).toMatchObject({
error: { message: expect.stringContaining('payload') },
});
});
it('returns 403 when the peer-id header is missing', async () => {
const raw = Buffer.from('{"events":[]}');
const headers = signedHeaders();
delete headers['x-broadcast-peer-id'];
const { res, captured } = makeRes();
await controller.webhook(
makeReq({
body: { events: [] },
rawBody: raw,
headers,
}),
res,
);
expect(captured.statusCode).toBe(403);
expect(captured.body).toMatchObject({
error: { message: expect.stringContaining('Peer-Id') },
});
});
it('returns 403 for an unknown peer-id (no configured webhook secret)', async () => {
const raw = Buffer.from('{"events":[{"key":"x","data":{},"meta":{}}]}');
const { res, captured } = makeRes();
await controller.webhook(
makeReq({
body: { events: [{ key: 'x', data: {}, meta: {} }] },
rawBody: raw,
headers: signedHeaders(),
}),
res,
);
expect(captured.statusCode).toBe(403);
expect(captured.body).toMatchObject({
error: { message: expect.stringContaining('Unknown peer') },
});
});
it('reads only the first value when a header is repeated as an array', async () => {
// Express normally collapses duplicates to a string, but tests
// can supply arrays — the controller's `headerOnce` helper picks
// index 0. Verify by sending an unknown peer-id as `[id, junk]`
// and confirming we still hit the 403 path (not a 400 from the
// body parsing path that runs before peer lookup).
const raw = Buffer.from('{"events":[]}');
const headers = signedHeaders() as unknown as Record<
string,
string | string[]
>;
headers['x-broadcast-peer-id'] = ['unknown-peer', 'second-value'];
const { res, captured } = makeRes();
await controller.webhook(
makeReq({
body: { events: [] },
rawBody: raw,
headers: headers as Record<string, string>,
}),
res,
);
expect(captured.statusCode).toBe(403);
});
});
@@ -0,0 +1,511 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock
// redis) and re-registers DesktopController's inline lambda routes
// onto a fresh PuterRouter so each handler is reachable. Each test
// makes its own user via `makeUser` and exercises the live controller
// against the real wired stores (user, fsEntry) and DB client.
let server: PuterServer;
let router: PuterRouter;
beforeAll(async () => {
server = await setupTestServer();
router = new PuterRouter();
server.controllers.desktop.registerRoutes(router);
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `dc-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
interface CapturedResponse {
statusCode: number;
body: unknown;
}
const makeReq = (init: {
body?: unknown;
actor?: Actor;
}): Request => {
return {
body: init.body ?? {},
query: {},
headers: {},
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
setHeader: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
const findHandler = (method: string, path: string): RequestHandler => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route.handler;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler = findHandler(method, path);
await handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
};
// ── /set-desktop-bg ─────────────────────────────────────────────────
describe('DesktopController POST /set-desktop-bg', () => {
it('persists url/color/fit on the user row', async () => {
const { actor, userId } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({
body: {
url: 'https://cdn.test/wallpaper.png',
color: '#000000',
fit: 'cover',
},
actor,
}),
res,
);
expect(captured.body).toEqual({});
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.desktop_bg_url).toBe(
'https://cdn.test/wallpaper.png',
);
expect(refreshed?.desktop_bg_color).toBe('#000000');
expect(refreshed?.desktop_bg_fit).toBe('cover');
});
it('persists only the supplied fields (partial update)', async () => {
const { actor, userId } = await makeUser();
const { res } = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { color: '#ffffff' }, actor }),
res,
);
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.desktop_bg_color).toBe('#ffffff');
// Untouched fields remain at their column default.
expect(refreshed?.desktop_bg_url).toBeFalsy();
expect(refreshed?.desktop_bg_fit).toBeFalsy();
});
it('passes through `null` to clear a field', async () => {
const { actor, userId } = await makeUser();
// First populate, then clear.
const populate = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({
body: { url: 'https://cdn.test/x.png' },
actor,
}),
populate.res,
);
const before = await server.stores.user.getById(userId);
expect(before?.desktop_bg_url).toBe('https://cdn.test/x.png');
const clear = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { url: null }, actor }),
clear.res,
);
const after = await server.stores.user.getById(userId);
expect(after?.desktop_bg_url).toBeNull();
});
it('throws 400 when url is not a string or null', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { url: 123 }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when no fields are provided', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── /update-taskbar-items ───────────────────────────────────────────
describe('DesktopController POST /update-taskbar-items', () => {
it('persists items as a JSON-encoded string', async () => {
const { actor, userId } = await makeUser();
const items = [{ name: 'editor' }, { name: 'browser' }];
const { res } = makeRes();
await callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: { items }, actor }),
res,
);
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.taskbar_items).toBe(JSON.stringify(items));
});
it('throws 400 when items is missing', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when items is not an array', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: { items: 'oops' }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── /set_layout ─────────────────────────────────────────────────────
describe('DesktopController POST /set_layout', () => {
it('persists the new layout on the matching fsentry', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
expect(documents).not.toBeNull();
const { res, captured } = makeRes();
await callRoute(
'post',
'/set_layout',
makeReq({
body: { item_uid: documents!.uuid, layout: 'icons' },
actor,
}),
res,
);
expect(captured.body).toEqual({});
const refreshed = await server.stores.fsEntry.getEntryByUuid(
documents!.uuid,
);
expect(refreshed?.layout).toBe('icons');
});
it('resolves by item_path when item_uid is omitted', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const path = `/${username}/Pictures`;
const { res } = makeRes();
await callRoute(
'post',
'/set_layout',
makeReq({
body: { item_path: path, layout: 'list' },
actor,
}),
res,
);
const refreshed = await server.stores.fsEntry.getEntryByPath(path);
expect(refreshed?.layout).toBe('list');
});
it('throws 400 for an unknown layout value', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set_layout',
makeReq({
body: { item_uid: documents!.uuid, layout: 'gallery' },
actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when neither item_uid nor item_path is supplied', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set_layout',
makeReq({ body: { layout: 'icons' }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 404 when the fsentry cannot be resolved', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set_layout',
makeReq({
body: {
item_uid: '00000000-0000-0000-0000-000000000000',
layout: 'icons',
},
actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 404 });
});
it("throws 403 when the fsentry belongs to another user", async () => {
const owner = await makeUser();
const intruder = await makeUser();
const ownerUsername = owner.actor.user!.username!;
const ownerEntry = await server.stores.fsEntry.getEntryByPath(
`/${ownerUsername}/Documents`,
);
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set_layout',
makeReq({
body: { item_uid: ownerEntry!.uuid, layout: 'icons' },
actor: intruder.actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 403 });
// Owner's entry remains untouched.
const refreshed = await server.stores.fsEntry.getEntryByUuid(
ownerEntry!.uuid,
);
expect(refreshed?.layout).toBeFalsy();
});
});
// ── /set_sort_by ────────────────────────────────────────────────────
describe('DesktopController POST /set_sort_by', () => {
it('persists sort_by + sort_order on the matching fsentry', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
const { res, captured } = makeRes();
await callRoute(
'post',
'/set_sort_by',
makeReq({
body: {
item_uid: documents!.uuid,
sort_by: 'name',
sort_order: 'desc',
},
actor,
}),
res,
);
expect(captured.body).toEqual({});
const refreshed = await server.stores.fsEntry.getEntryByUuid(
documents!.uuid,
);
expect(refreshed?.sortBy).toBe('name');
expect(refreshed?.sortOrder).toBe('desc');
});
it('defaults sort_order to "asc" when omitted', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const pictures = await server.stores.fsEntry.getEntryByPath(
`/${username}/Pictures`,
);
const { res } = makeRes();
await callRoute(
'post',
'/set_sort_by',
makeReq({
body: { item_uid: pictures!.uuid, sort_by: 'modified' },
actor,
}),
res,
);
const refreshed = await server.stores.fsEntry.getEntryByUuid(
pictures!.uuid,
);
expect(refreshed?.sortBy).toBe('modified');
expect(refreshed?.sortOrder).toBe('asc');
});
it('throws 400 for an unknown sort_by value', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const docs = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set_sort_by',
makeReq({
body: { item_uid: docs!.uuid, sort_by: 'random' },
actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 for an invalid sort_order', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const docs = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set_sort_by',
makeReq({
body: {
item_uid: docs!.uuid,
sort_by: 'name',
sort_order: 'sideways',
},
actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
@@ -0,0 +1,112 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { DriverController } from './DriverController.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one PuterServer (in-memory sqlite + dynamo + s3 + mock redis).
// The DriverController under test is the same instance the live request
// pipeline uses, so its iface→driver registry is populated from real
// drivers (puter-kvstore, puter-apps, puter-subdomains, …). The HTTP
// handlers (`#handleCall`, `#handleListInterfaces`, `#handleXd`) are
// private — we exercise the public lookup API (`resolve` / list / get
// default) which the handlers themselves delegate to.
let server: PuterServer;
let controller: DriverController;
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.drivers as unknown as DriverController;
});
afterAll(async () => {
await server?.shutdown();
});
// ── Lookup API ──────────────────────────────────────────────────────
describe('DriverController.listInterfaces', () => {
it('exposes built-in interfaces', () => {
const interfaces = controller.listInterfaces();
// Several drivers ship by default — assert known ones rather
// than the exact set so adding a driver doesn't break this.
expect(interfaces).toEqual(
expect.arrayContaining([
'puter-kvstore',
'puter-apps',
'puter-subdomains',
'puter-notifications',
]),
);
});
});
describe('DriverController.listDrivers', () => {
it('returns every name registered for an interface', () => {
const drivers = controller.listDrivers('puter-kvstore');
expect(drivers).toContain('puter-kvstore');
});
it('returns [] for an unknown interface', () => {
expect(controller.listDrivers('nonexistent')).toEqual([]);
});
});
describe('DriverController.getDefault', () => {
it('returns the registered default driver name', () => {
// KVStoreDriver declares `isDefault = true`.
expect(controller.getDefault('puter-kvstore')).toBe('puter-kvstore');
});
it('returns undefined for an unknown interface', () => {
expect(controller.getDefault('nonexistent')).toBeUndefined();
});
});
describe('DriverController.resolve', () => {
it('returns the default-driver instance when no name is given', () => {
const driver = controller.resolve('puter-kvstore');
expect(driver).not.toBeNull();
// The KV driver exposes a `set` method per its interface.
expect(typeof (driver as Record<string, unknown>)?.set).toBe(
'function',
);
});
it('finds the same instance by explicit driver name', () => {
const byDefault = controller.resolve('puter-kvstore');
const byName = controller.resolve('puter-kvstore', 'puter-kvstore');
expect(byName).toBe(byDefault);
});
it('returns null for an unknown interface', () => {
expect(controller.resolve('nope')).toBeNull();
});
it('returns null for a known interface but unknown driver name', () => {
expect(
controller.resolve('puter-kvstore', 'no-such-driver'),
).toBeNull();
});
});
@@ -175,11 +175,6 @@ export class DriverController extends PuterController {
{ subdomain: 'api', requireAuth: true },
this.#handleXd,
);
router.get(
'/usage',
{ subdomain: 'api', requireAuth: true },
this.#handleUsage,
);
}
// ── Handlers ────────────────────────────────────────────────────
@@ -315,52 +310,6 @@ export class DriverController extends PuterController {
res.send(XD_HTML);
};
/** GET /drivers/usage — monthly driver usage for the authenticated actor. */
#handleUsage = async (req: Request, res: Response): Promise<void> => {
const actor = req.actor;
if (!actor?.user?.id)
throw new HttpError(401, 'Authentication required');
const userId = actor.user.id;
const db = this.clients.db;
// Per-user usage: aggregate today's counts from monthly_usage_counts
const userRows = await db.read(
`SELECT \`year\`, \`month\`, \`service\`, SUM(\`count\`) AS count, MAX(\`max\`) AS max
FROM \`monthly_usage_counts\`
WHERE \`user_id\` = ? AND \`app_id\` IS NULL
GROUP BY \`year\`, \`month\`, \`service\`
ORDER BY \`year\` DESC, \`month\` DESC
LIMIT 100`,
[userId],
);
// Per-app usage: aggregate by app
const appRows = await db.read(
`SELECT a.\`uid\` AS app_uid, a.\`name\` AS app_name,
m.\`year\`, m.\`month\`, m.\`service\`, SUM(m.\`count\`) AS count, MAX(m.\`max\`) AS max
FROM \`monthly_usage_counts\` m
LEFT JOIN \`apps\` a ON m.\`app_id\` = a.\`id\`
WHERE m.\`user_id\` = ? AND m.\`app_id\` IS NOT NULL
GROUP BY a.\`uid\`, a.\`name\`, m.\`year\`, m.\`month\`, m.\`service\`
ORDER BY m.\`year\` DESC, m.\`month\` DESC
LIMIT 500`,
[userId],
);
// Group app rows by app name
const apps: Record<string, Array<Record<string, unknown>>> = {};
for (const row of appRows) {
const name = String(
(row as Record<string, unknown>).app_name ?? 'unknown',
);
if (!apps[name]) apps[name] = [];
apps[name].push(row as Record<string, unknown>);
}
res.json({ user: userRows, apps });
};
// ── Internals ───────────────────────────────────────────────────
#buildIfaceMap(): void {
@@ -0,0 +1,480 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { FSController } from './FSController.js';
import type {
CompleteWriteRequest,
SignedWriteRequest,
SignedWriteResponse,
} from './requestTypes.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock redis).
// Each test creates its own user via `makeUser` and exercises the live
// FSController against the wired services / stores.
let server: PuterServer;
let controller: FSController;
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.fs as unknown as FSController;
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `fsc-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
interface CapturedResponse {
statusCode: number;
body: unknown;
}
const makeReq = <B>(init: {
body?: B;
query?: Record<string, unknown>;
headers?: Record<string, string>;
actor: Actor;
user?: { id: number; username: string };
}): Request => {
return {
body: init.body ?? ({} as B),
query: init.query ?? {},
headers: init.headers ?? {},
actor: init.actor,
// Some controller helpers fall back to `req.user` (set by the
// session middleware) for id / username before reading `req.actor`.
// Provide it so #getActorUserId / #getActorUsername resolve.
user: init.user ?? {
id: init.actor.user!.id!,
username: init.actor.user!.username!,
},
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
setHeader: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
// ── /startBatchWrite ────────────────────────────────────────────────
describe('FSController.startBatchWrites', () => {
it('returns [] for an empty/undefined body without creating sessions', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
const req = makeReq<SignedWriteRequest[]>({ body: undefined, actor });
await withActor(actor, () => controller.startBatchWrites(req, res));
expect(captured.body).toEqual([]);
});
it('creates a pending upload session per request and returns signed targets', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const body: SignedWriteRequest[] = [
{
fileMetadata: {
path: `/${username}/Documents/a.txt`,
size: 5,
},
},
{
fileMetadata: {
path: `/${username}/Documents/b.txt`,
size: 10,
},
},
];
const { res, captured } = makeRes();
const req = makeReq<SignedWriteRequest[]>({ body, actor });
await withActor(actor, () => controller.startBatchWrites(req, res));
const responses = captured.body as SignedWriteResponse[];
expect(responses).toHaveLength(2);
for (const r of responses) {
expect(r.sessionId).toEqual(expect.any(String));
expect(r.objectKey).toEqual(expect.any(String));
expect(r.bucket).toEqual(expect.any(String));
expect(r.uploadMode).toBe('single');
// In-memory mock S3 still returns a presigned-URL string for
// single-mode uploads — verify it's there but don't assert
// shape (varies by region/host config).
expect(typeof r.url).toBe('string');
}
// Pending sessions actually landed in the DB and point at the
// expected paths for the right user.
const sessions =
await server.stores.fsEntry.getPendingEntriesBySessionIds(
responses.map((r) => r.sessionId),
);
expect(sessions.map((s) => s?.targetPath).sort()).toEqual([
`/${username}/Documents/a.txt`,
`/${username}/Documents/b.txt`,
]);
for (const session of sessions) {
expect(session?.userId).toBe(userId);
expect(session?.status).toBe('pending');
}
});
it('expands `~/...` paths against the actor home before writing', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const body: SignedWriteRequest[] = [
{ fileMetadata: { path: '~/Documents/tilde.txt', size: 3 } },
];
const { res, captured } = makeRes();
const req = makeReq<SignedWriteRequest[]>({ body, actor });
await withActor(actor, () => controller.startBatchWrites(req, res));
const [response] = captured.body as SignedWriteResponse[];
const session = await server.stores.fsEntry.getPendingEntryBySessionId(
response!.sessionId,
);
expect(session?.targetPath).toBe(`/${username}/Documents/tilde.txt`);
});
it('materializes a directory entry when `directory: true`', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/new-batch-dir`;
const body: SignedWriteRequest[] = [
{
// `createMissingParents` lets the service materialize the
// target dir even though only `/Documents` exists in the
// newly-provisioned home tree.
fileMetadata: {
path: target,
size: 0,
createMissingParents: true,
},
directory: true,
},
];
const { res } = makeRes();
const req = makeReq<SignedWriteRequest[]>({ body, actor });
await withActor(actor, () => controller.startBatchWrites(req, res));
// Directory items aren't pending uploads — they're created
// immediately by the service. The fsentry should be queryable.
const created = await server.stores.fsEntry.getEntryByPath(target);
expect(created).not.toBeNull();
expect(created?.isDir).toBe(true);
});
it('rejects the batch when ACL denies any item', async () => {
const a = await makeUser();
const b = await makeUser();
// User a tries to drop a file inside user b's home.
const body: SignedWriteRequest[] = [
{
fileMetadata: {
path: `/${b.actor.user!.username}/Documents/intruder.txt`,
size: 1,
},
},
];
const { res } = makeRes();
const req = makeReq<SignedWriteRequest[]>({
body,
actor: a.actor,
});
const err = await withActor(a.actor, () =>
controller.startBatchWrites(req, res).then(
() => null,
(e: unknown) => e,
),
);
const status = (err as { statusCode?: number } | null)?.statusCode;
// 404 (can't see) or 403 (can see, can't write) are both valid
// denials per ACLService.getSafeAclError.
expect([403, 404]).toContain(status);
});
});
// ── /completeBatchWrite ────────────────────────────────────────────
describe('FSController.completeBatchWrites', () => {
it('returns [] for an empty body', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
const req = makeReq<CompleteWriteRequest[]>({
body: undefined,
actor,
});
await withActor(actor, () => controller.completeBatchWrites(req, res));
expect(captured.body).toEqual([]);
});
it('rejects an inline `data:` thumbnail with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
const req = makeReq<CompleteWriteRequest[]>({
body: [
{
uploadId: 'whatever',
thumbnailData: 'data:image/png;base64,AAA',
},
],
actor,
});
await expect(
withActor(actor, () => controller.completeBatchWrites(req, res)),
).rejects.toMatchObject({ statusCode: 400 });
});
it('finalizes pending sessions into real fsentries', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
// 1) Start two batched uploads. The signed-write flow inserts
// pending session rows and gives us back the upload IDs we'll
// feed to /completeBatchWrite.
const startBody: SignedWriteRequest[] = [
{
fileMetadata: {
path: `/${username}/Documents/c.txt`,
size: 5,
contentType: 'text/plain',
},
},
{
fileMetadata: {
path: `/${username}/Documents/d.txt`,
size: 7,
contentType: 'text/plain',
},
},
];
const startRes = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({ body: startBody, actor }),
startRes.res,
),
);
const startResponses = startRes.captured.body as SignedWriteResponse[];
expect(startResponses).toHaveLength(2);
// 2) Complete via the controller. Single-mode completion only
// needs the session row → it doesn't read the S3 object back,
// so we can skip the actual upload step in this test.
const { res, captured } = makeRes();
const completeBody: CompleteWriteRequest[] = startResponses.map(
(r) => ({ uploadId: r.sessionId }),
);
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: completeBody,
actor,
}),
res,
),
);
const responses = captured.body as Array<{
sessionId: string;
wasOverwrite: boolean;
fsEntry: { path: string; userId: number; isDir: boolean };
}>;
expect(responses.map((r) => r.fsEntry.path).sort()).toEqual([
`/${username}/Documents/c.txt`,
`/${username}/Documents/d.txt`,
]);
for (const response of responses) {
expect(response.wasOverwrite).toBe(false);
expect(response.fsEntry.userId).toBe(userId);
expect(response.fsEntry.isDir).toBe(false);
}
// The real fsentries were committed and are now resolvable.
for (const path of [
`/${username}/Documents/c.txt`,
`/${username}/Documents/d.txt`,
]) {
const entry = await server.stores.fsEntry.getEntryByPath(path);
expect(entry).not.toBeNull();
expect(entry?.userId).toBe(userId);
}
});
it('reports wasOverwrite=true when finalizing onto an existing entry', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/overwrite-me.txt`;
// First write — establishes an entry to overwrite.
const firstStart = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path: target, size: 1 } }],
actor,
}),
firstStart.res,
),
);
const [firstResponse] = firstStart.captured
.body as SignedWriteResponse[];
const firstComplete = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: firstResponse!.sessionId }],
actor,
}),
firstComplete.res,
),
);
// Second write with overwrite=true onto the same path.
const secondStart = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [
{
fileMetadata: {
path: target,
size: 2,
overwrite: true,
},
},
],
actor,
}),
secondStart.res,
),
);
const [secondResponse] = secondStart.captured
.body as SignedWriteResponse[];
const secondComplete = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: secondResponse!.sessionId }],
actor,
}),
secondComplete.res,
),
);
const [finalized] = secondComplete.captured.body as Array<{
wasOverwrite: boolean;
}>;
expect(finalized?.wasOverwrite).toBe(true);
});
it("rejects another user's session ids with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
// a starts a batch; b tries to complete it.
const startA = makeRes();
await withActor(a.actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [
{
fileMetadata: {
path: `/${a.actor.user!.username}/Documents/x.txt`,
size: 1,
},
},
],
actor: a.actor,
}),
startA.res,
),
);
const [aResponse] = startA.captured.body as SignedWriteResponse[];
const err = await withActor(b.actor, () =>
controller
.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: aResponse!.sessionId }],
actor: b.actor,
}),
makeRes().res,
)
.then(
() => null,
(e: unknown) => e,
),
);
const status = (err as { statusCode?: number } | null)?.statusCode;
// FSService.batchCompleteUrlWrite throws 403 on session/user
// mismatch (`Upload session access denied`).
expect([403, 404]).toContain(status);
});
});
@@ -0,0 +1,540 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { LegacyFSController } from './LegacyFSController.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer per test file (in-memory sqlite + dynamo +
// s3 + mock redis). Each test gets its own fresh user via `makeUser` so
// state doesn't leak between cases.
let server: PuterServer;
let controller: LegacyFSController;
beforeAll(async () => {
server = await setupTestServer();
// Pull the live controller off the wired server — same instance the
// request pipeline uses, so tests exercise the real services / stores.
controller = server.controllers.legacyFs as unknown as LegacyFSController;
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `lfs-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
// Provision /<username>, Trash, Documents, etc. Without this the
// resolveNode lookups for `/<username>/...` paths return 404.
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
// Express handlers only read `body` / `query` / `headers` / `actor` and
// write to `res.json` / `res.status` / `res.send`. A field bag plus a
// recorder for the response covers every code path in the controller.
interface CapturedResponse {
statusCode: number;
body: unknown;
sentText: string | undefined;
headers: Map<string, string>;
}
const makeReq = (init: {
body?: unknown;
query?: Record<string, unknown>;
headers?: Record<string, string>;
actor: Actor;
}): Request => {
return {
body: init.body ?? {},
query: init.query ?? {},
headers: init.headers ?? {},
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = {
statusCode: 200,
body: undefined,
sentText: undefined,
headers: new Map(),
};
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
send: vi.fn((value: unknown) => {
captured.sentText = String(value ?? '');
return res;
}),
setHeader: vi.fn((k: string, v: string) => {
captured.headers.set(k.toLowerCase(), String(v));
return res;
}),
};
return { res: res as unknown as Response, captured };
};
// `resolveV1Selector` reads the actor's username out of ALS — controllers
// do this implicitly because the request middleware sets it; tests have to
// run handlers inside `runWithContext` so tilde expansion lookups work.
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
// ── Tests ───────────────────────────────────────────────────────────
describe('LegacyFSController.df', () => {
it('reports zero used and a positive capacity for a fresh user', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
const req = makeReq({ body: {}, actor });
await withActor(actor, () => controller.df(req, res));
// Default test config has `is_storage_limited: false`, so `max`
// resolves from device-free-space rather than the user-row
// free_storage. Just sanity-check the shape and signs.
const body = captured.body as { used: number; capacity: number };
expect(body.used).toBe(0);
expect(body.capacity).toBeGreaterThan(0);
});
});
describe('LegacyFSController.mkdir', () => {
it('rejects a missing `path` with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
const req = makeReq({ body: {}, actor });
await expect(
withActor(actor, () => controller.mkdir(req, res)),
).rejects.toMatchObject({ statusCode: 400 });
});
it('creates a directory under the user home and persists it', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
const target = `/${actor.user!.username}/Documents/notes`;
const req = makeReq({
body: { path: target },
actor,
});
await withActor(actor, () => controller.mkdir(req, res));
const body = captured.body as Record<string, unknown>;
expect(body).toMatchObject({
path: target,
name: 'notes',
is_dir: true,
});
// Confirm the row landed in the DB rather than just trusting the
// controller's response.
const fetched = await server.stores.fsEntry.getEntryByPath(target);
expect(fetched?.path).toBe(target);
expect(fetched?.isDir).toBe(true);
});
it('joins a relative `path` onto the `parent` path', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const { res } = makeRes();
const req = makeReq({
body: {
parent: `/${username}/Documents`,
path: 'sub',
},
actor,
});
await withActor(actor, () => controller.mkdir(req, res));
const fetched = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/sub`,
);
expect(fetched).not.toBeNull();
expect(fetched?.isDir).toBe(true);
});
it("rejects writing into another user's home with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
const { res } = makeRes();
const req = makeReq({
body: {
path: `/${b.actor.user!.username}/Documents/intruder`,
},
actor: a.actor,
});
// ACLService maps "can't even `see`" to 404 (don't leak existence
// of sibling users' files) and "can see but not write" to 403 —
// either is a valid denial here, both block the mkdir.
await expect(
withActor(a.actor, () => controller.mkdir(req, res)),
).rejects.toMatchObject({
statusCode: expect.any(Number),
});
const err = await withActor(a.actor, () =>
controller.mkdir(req, res).then(
() => null,
(e: unknown) => e,
),
);
expect(err).toMatchObject({});
const status = (err as { statusCode?: number } | null)?.statusCode;
expect([403, 404]).toContain(status);
});
});
describe('LegacyFSController.stat', () => {
it('returns the legacy snake_case shape with type, owner, and is_dir', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
// Bootstrap a directory to stat.
const dirRes = makeRes();
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { path: `/${username}/Documents/folder` },
actor,
}),
dirRes.res,
),
);
const { res, captured } = makeRes();
const req = makeReq({
body: { path: `/${username}/Documents/folder` },
actor,
});
await withActor(actor, () => controller.stat(req, res));
const body = captured.body as Record<string, unknown>;
expect(body).toMatchObject({
path: `/${username}/Documents/folder`,
name: 'folder',
is_dir: true,
// Directories report `type: 'folder'` per the legacy contract.
type: 'folder',
});
expect(body.owner).toMatchObject({ username });
});
it('hydrates `size` for a directory when `return_size` is set', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const dirRes = makeRes();
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { path: `/${username}/Pictures/empty` },
actor,
}),
dirRes.res,
),
);
const { res, captured } = makeRes();
const req = makeReq({
body: {
path: `/${username}/Pictures/empty`,
return_size: true,
},
actor,
});
await withActor(actor, () => controller.stat(req, res));
const body = captured.body as Record<string, unknown>;
// No files in the dir → subtree size is 0.
expect(body.size).toBe(0);
});
it('throws 401 when the request has no actor', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
// Strip actor — controllers should refuse rather than fault on null.
const req = {
...makeReq({ body: { path: '/x' }, actor }),
actor: undefined,
} as unknown as Request;
await expect(controller.stat(req, res)).rejects.toMatchObject({
statusCode: 401,
});
});
});
describe('LegacyFSController.delete', () => {
it('removes a single entry by uid and returns `{ ok, uid }`', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/doomed`;
await withActor(actor, () =>
controller.mkdir(
makeReq({ body: { path: target }, actor }),
makeRes().res,
),
);
const before = await server.stores.fsEntry.getEntryByPath(target);
expect(before).not.toBeNull();
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.delete(
makeReq({ body: { uid: before!.uuid }, actor }),
res,
),
);
expect(captured.body).toEqual({ ok: true, uid: before!.uuid });
const after = await server.stores.fsEntry.getEntryByPath(target);
expect(after).toBeNull();
});
it('bulk-deletes via `paths` and returns one entry per path', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const a = `/${username}/Documents/a`;
const b = `/${username}/Documents/b`;
for (const p of [a, b]) {
await withActor(actor, () =>
controller.mkdir(
makeReq({ body: { path: p }, actor }),
makeRes().res,
),
);
}
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.delete(
makeReq({
body: { paths: [a, b], recursive: true },
actor,
}),
res,
),
);
const responseBody = captured.body as Array<Record<string, unknown>>;
expect(responseBody).toHaveLength(2);
expect(responseBody[0]?.path).toBe(a);
expect(responseBody[1]?.path).toBe(b);
expect(await server.stores.fsEntry.getEntryByPath(a)).toBeNull();
expect(await server.stores.fsEntry.getEntryByPath(b)).toBeNull();
});
});
describe('LegacyFSController.rename', () => {
it('rejects a missing `new_name` with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
const req = makeReq({
body: { path: `/${actor.user!.username}/Documents` },
actor,
});
await expect(
withActor(actor, () => controller.rename(req, res)),
).rejects.toMatchObject({ statusCode: 400 });
});
it('renames an existing entry and reports the new path', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const orig = `/${username}/Documents/orig`;
await withActor(actor, () =>
controller.mkdir(
makeReq({ body: { path: orig }, actor }),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.rename(
makeReq({
body: { path: orig, new_name: 'renamed' },
actor,
}),
res,
),
);
const body = captured.body as Record<string, unknown>;
expect(body.path).toBe(`/${username}/Documents/renamed`);
expect(body.name).toBe('renamed');
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/renamed`,
),
).not.toBeNull();
});
});
describe('LegacyFSController.touch', () => {
it('rejects touching at the root with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
const req = makeReq({ body: { path: '/foo' }, actor });
await expect(
withActor(actor, () => controller.touch(req, res)),
).rejects.toMatchObject({ statusCode: 400 });
});
it('creates a placeholder fsentry at the requested path', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/note.txt`;
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor,
}),
res,
),
);
// /touch returns an empty body — the side effect is the new entry.
expect(captured.sentText).toBe('');
const created = await server.stores.fsEntry.getEntryByPath(target);
expect(created).not.toBeNull();
});
});
describe('LegacyFSController.batch (json mode)', () => {
it('runs each op against the real fs and aggregates results on 200', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const { res, captured } = makeRes();
const req = makeReq({
body: {
operations: [
{
op: 'mkdir',
path: `/${username}/Documents`,
name: 'batch-folder',
},
],
},
headers: { 'content-type': 'application/json' },
actor,
});
await withActor(actor, () => controller.batch(req, res));
expect(captured.statusCode).toBe(200);
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/batch-folder`,
),
).not.toBeNull();
});
it('returns 218 with a serialized error when one op fails, but commits the others', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const { res, captured } = makeRes();
// First op deletes a path that doesn't exist (resolveV1Selector
// throws 404). Second op must still run and persist its mkdir.
const req = makeReq({
body: {
operations: [
{
op: 'delete',
path: `/${username}/Documents/does-not-exist`,
},
{
op: 'mkdir',
path: `/${username}/Documents`,
name: 'good',
},
],
},
headers: { 'content-type': 'application/json' },
actor,
});
await withActor(actor, () => controller.batch(req, res));
expect(captured.statusCode).toBe(218);
const body = captured.body as {
results: Array<Record<string, unknown>>;
};
expect(body.results).toHaveLength(2);
expect(body.results[0]).toMatchObject({ error: true });
// The second op still ran — verify by reading the resulting entry.
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/good`,
),
).not.toBeNull();
});
it('records a 400 per-op error for an unknown op-type', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
const req = makeReq({
body: {
operations: [{ op: 'evaporate', path: '/x' }],
},
headers: { 'content-type': 'application/json' },
actor,
});
await withActor(actor, () => controller.batch(req, res));
expect(captured.statusCode).toBe(218);
const body = captured.body as {
results: Array<Record<string, unknown>>;
};
expect(body.results[0]).toMatchObject({
error: true,
status: 400,
});
});
});
@@ -25,7 +25,7 @@ import type { FSService } from '../../services/fs/FSService.js';
import type { ACLService, AclMode } from '../../services/acl/ACLService.js';
import type { Actor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import type { EventClient } from '../../clients/EventClient.js';
import type { EventClient } from '../../clients/event/EventClient.js';
import { HttpError } from '../../core/http/HttpError.js';
import {
resolveNode,
@@ -0,0 +1,262 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock
// redis) and re-registers HomepageController's inline lambda routes
// onto a fresh PuterRouter so each handler is reachable. Tests
// exercise the live PuterHomepageService (it renders the shell HTML
// to res.send) and the real AppStore for /app/:name.
let server: PuterServer;
let router: PuterRouter;
beforeAll(async () => {
server = await setupTestServer();
router = new PuterRouter();
server.controllers.homepage.registerRoutes(router);
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `hpc-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
interface CapturedResponse {
statusCode: number;
body: unknown;
contentType?: string;
}
const makeReq = (init: {
params?: Record<string, unknown>;
path?: string;
actor?: Actor;
hostname?: string;
protocol?: string;
}): Request => {
return {
body: {},
query: {},
headers: {},
params: init.params ?? {},
path: init.path ?? '/',
hostname: init.hostname ?? 'test.local',
protocol: init.protocol ?? 'http',
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
send: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
set: vi.fn((key: string, value: string) => {
if (key.toLowerCase() === 'content-type') {
captured.contentType = value;
}
return res;
}),
setHeader: vi.fn((key: string, value: string) => {
if (key.toLowerCase() === 'content-type') {
captured.contentType = value;
}
return res;
}),
type: vi.fn((value: string) => {
captured.contentType = value;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
const findHandler = (method: string, path: string): RequestHandler | null => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
return route?.handler ?? null;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler = findHandler(method, path);
if (!handler) throw new Error(`No ${method.toUpperCase()} ${path} route`);
await handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
};
// ── Shell routes ────────────────────────────────────────────────────
describe('HomepageController shell routes', () => {
it('renders the live shell HTML on the root path', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/', makeReq({ path: '/' }), res);
// PuterHomepageService.send writes the rendered HTML via res.send.
expect(typeof captured.body).toBe('string');
const html = String(captured.body);
expect(html).toMatch(/<!DOCTYPE html>/i);
// The configured page title flows through the meta block.
expect(html).toContain('Puter');
});
it('still serves the shell when an authenticated actor is present', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute('get', '/', makeReq({ path: '/', actor }), res);
expect(typeof captured.body).toBe('string');
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
it('serves the shell on /settings, /dashboard, /action, /@:username', async () => {
for (const path of [
'/settings',
'/settings/*splat',
'/dashboard',
'/dashboard/',
'/action/*splat',
'/@:username',
]) {
const { res, captured } = makeRes();
await callRoute('get', path, makeReq({ path }), res);
expect(typeof captured.body).toBe('string');
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
}
});
});
// ── /app/:name ──────────────────────────────────────────────────────
describe('HomepageController GET /app/:name', () => {
it('returns 404 (still renders the shell) when the app is unknown', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/app/:name',
makeReq({
params: { name: 'no-such-app' },
path: '/app/no-such-app',
}),
res,
);
expect(captured.statusCode).toBe(404);
// The shell still renders so the client router can take over.
expect(typeof captured.body).toBe('string');
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
it('renders the shell with the app row in scope when the app exists', async () => {
const { userId } = await makeUser();
const name = `app-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.app.create(
{
name,
title: 'Cool App',
description: 'a real app row',
index_url: `https://example.com/${name}/`,
approved_for_listing: 1,
},
{ ownerUserId: userId },
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/app/:name',
makeReq({ params: { name }, path: `/app/${name}` }),
res,
);
expect(captured.statusCode).toBe(200);
// Shell payload is HTML; the app's title appears in the page meta.
const html = String(captured.body);
expect(html).toMatch(/<!DOCTYPE html>/i);
expect(html).toContain('Cool App');
});
});
// ── /show/* ─────────────────────────────────────────────────────────
describe('HomepageController GET /show/*splat', () => {
it('emits a launch_app explorer call with the post-/show path', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/show/*splat',
makeReq({ path: '/show/alice/Documents' }),
res,
);
// The launch payload is JSON-serialized into the rendered HTML.
// We assert the rendered shell included the explorer launch hint
// pointing at the expected (slashed) path.
const html = String(captured.body);
expect(html).toContain('launch_app');
expect(html).toContain('explorer');
expect(html).toContain('/alice/Documents');
});
});
@@ -0,0 +1,220 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { NotificationController } from './NotificationController.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one PuterServer with the live wired NotificationController.
// Tests seed real notification rows via the store, then drive the
// controller's `markAck` / `markRead` handlers with stub req/res
// objects. The controller's path through NotificationService updates
// the underlying row, so we verify behaviour by reading the store
// state back.
let server: PuterServer;
let controller: NotificationController;
beforeAll(async () => {
server = await setupTestServer();
controller =
server.controllers.notification as unknown as NotificationController;
});
afterAll(async () => {
await server?.shutdown();
});
interface CapturedResponse {
statusCode: number;
body: unknown;
}
const makeReq = (init: {
body?: unknown;
actor?: Actor;
}): Request => {
return {
body: init.body ?? {},
query: {},
headers: {},
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
setHeader: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `nc-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
// ── /notif/mark-ack ─────────────────────────────────────────────────
describe('NotificationController.markAck', () => {
it('sets `acknowledged` on the underlying notification row', async () => {
const { actor, userId } = await makeUser();
const created = await server.stores.notification.create({
userId,
value: { title: 't' },
});
const { res, captured } = makeRes();
await controller.markAck(
makeReq({ body: { uid: created.uid }, actor }),
res,
);
// Empty `{}` is the conventional success body for these routes.
expect(captured.body).toEqual({});
const after = await server.stores.notification.getByUid(
created.uid as string,
{ userId },
);
expect(after?.acknowledged).not.toBeNull();
});
it('rejects a missing uid with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
controller.markAck(makeReq({ body: {}, actor }), res),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a non-string uid with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
controller.markAck(makeReq({ body: { uid: 123 }, actor }), res),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 401 when there is no actor on the request', async () => {
const { res } = makeRes();
await expect(
controller.markAck(makeReq({ body: { uid: 'whatever' } }), res),
).rejects.toMatchObject({ statusCode: 401 });
});
it("does not flip another user's notification", async () => {
const a = await makeUser();
const b = await makeUser();
const created = await server.stores.notification.create({
userId: a.userId,
value: {},
});
const { res } = makeRes();
await controller.markAck(
makeReq({ body: { uid: created.uid }, actor: b.actor }),
res,
);
const after = await server.stores.notification.getByUid(
created.uid as string,
{ userId: a.userId },
);
// Store update is scoped by user_id — cross-user mutation is a
// silent no-op rather than an error from the controller.
expect(after?.acknowledged).toBeFalsy();
});
});
// ── /notif/mark-read ────────────────────────────────────────────────
describe('NotificationController.markRead', () => {
it('sets `shown` on the underlying notification row', async () => {
const { actor, userId } = await makeUser();
const created = await server.stores.notification.create({
userId,
value: {},
});
const { res, captured } = makeRes();
await controller.markRead(
makeReq({ body: { uid: created.uid }, actor }),
res,
);
expect(captured.body).toEqual({});
const after = await server.stores.notification.getByUid(
created.uid as string,
{ userId },
);
expect(after?.shown).not.toBeNull();
// Marking read should NOT also set acknowledged.
expect(after?.acknowledged).toBeFalsy();
});
it('rejects an empty uid string with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
controller.markRead(makeReq({ body: { uid: '' }, actor }), res),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 401 when there is no actor on the request', async () => {
const { res } = makeRes();
await expect(
controller.markRead(makeReq({ body: { uid: 'x' } }), res),
).rejects.toMatchObject({ statusCode: 401 });
});
});
@@ -0,0 +1,480 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import {
afterAll,
afterEach,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { runWithContext } from '../../core/context.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import type { OIDCService } from '../../services/auth/OIDCService.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
const TEST_ORIGIN = 'http://test.local';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer with a custom OIDC provider configured,
// then re-registers OIDCController's inline lambda routes onto a
// fresh PuterRouter so each handler is reachable. Tests run against
// the live wired OIDCService (real signState / verifyState, real
// findUserByProviderSub / linkProviderToUser, real createUserFromOIDC)
// and AuthService (real session token).
//
// The two methods that hit external HTTP — `exchangeCodeForTokens`
// and `getUserInfo` — are stubbed per-test with `vi.spyOn` so the
// callback flow can be exercised without standing up a fake IdP.
let server: PuterServer;
let router: PuterRouter;
beforeAll(async () => {
server = await setupTestServer({
origin: TEST_ORIGIN,
// A "custom" OIDC provider just needs static endpoints — no
// discovery fetch happens. The endpoint URLs aren't actually
// hit during the tests we keep here (we spy past them when
// necessary), but they must be set so `getProviderConfig`
// accepts the entry.
oidc: {
providers: {
custom: {
client_id: 'test-client',
client_secret: 'test-secret',
authorization_endpoint:
'https://idp.test.invalid/authorize',
token_endpoint: 'https://idp.test.invalid/token',
userinfo_endpoint: 'https://idp.test.invalid/userinfo',
},
},
},
} as never);
router = new PuterRouter();
server.controllers.oidc.registerRoutes(router);
});
afterAll(async () => {
await server?.shutdown();
});
afterEach(() => {
vi.restoreAllMocks();
});
interface CapturedResponse {
statusCode: number;
body: unknown;
redirectStatus?: number;
redirectUrl?: string;
headers: Record<string, string>;
cookies: Array<{ name: string; value: string; opts?: unknown }>;
contentType?: string;
}
const makeReq = (init: {
body?: unknown;
query?: Record<string, unknown>;
params?: Record<string, unknown>;
headers?: Record<string, string>;
method?: string;
}): Request => {
return {
body: init.body ?? {},
query: init.query ?? {},
params: init.params ?? {},
headers: init.headers ?? {},
method: init.method ?? 'GET',
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = {
statusCode: 200,
body: undefined,
headers: {},
cookies: [],
};
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
send: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
set: vi.fn((key: string, value: string) => {
if (typeof key === 'string') {
captured.headers[key.toLowerCase()] = value;
if (key.toLowerCase() === 'content-type') {
captured.contentType = value;
}
}
return res;
}),
setHeader: vi.fn(() => res),
redirect: vi.fn((status: number | string, url?: string) => {
if (typeof status === 'number' && typeof url === 'string') {
captured.redirectStatus = status;
captured.redirectUrl = url;
} else if (typeof status === 'string') {
captured.redirectStatus = 302;
captured.redirectUrl = status;
}
return res;
}),
cookie: vi.fn((name: string, value: string, opts?: unknown) => {
captured.cookies.push({ name, value, opts });
return res;
}),
type: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
const findHandler = (method: string, path: string): RequestHandler => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route.handler;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler = findHandler(method, path);
// OIDCService.createUserFromOIDC pulls `req` off the request
// context (for IP / signup-validate hooks). Express middleware
// sets it in production; the test driver mirrors that here.
await runWithContext({ req }, () =>
handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
}),
);
};
const oidc = (): OIDCService =>
server.services.oidc as unknown as OIDCService;
// ── GET /auth/oidc/providers ────────────────────────────────────────
describe('OIDCController GET /auth/oidc/providers', () => {
it('lists every provider whose config validates', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/providers',
makeReq({}),
res,
);
// We configured a single `custom` provider; the live service
// walks the config and returns its id.
expect(captured.body).toEqual({ providers: ['custom'] });
});
});
// ── GET /auth/oidc/:provider/start ──────────────────────────────────
describe('OIDCController GET /auth/oidc/:provider/start', () => {
it('throws 404 for an unconfigured provider', async () => {
const { res } = makeRes();
await expect(
callRoute(
'get',
'/auth/oidc/:provider/start',
makeReq({ params: { provider: 'no-such-idp' } }),
res,
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('redirects to the IdP authorization URL with a signed state', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/:provider/start',
makeReq({ params: { provider: 'custom' } }),
res,
);
expect(captured.redirectStatus).toBe(302);
const url = new URL(captured.redirectUrl ?? '');
expect(url.origin).toBe('https://idp.test.invalid');
expect(url.pathname).toBe('/authorize');
// Authorization URL carries the live-signed state token;
// pull it back out and confirm the OIDC service can verify it.
const state = url.searchParams.get('state');
expect(state).toBeTruthy();
const decoded = oidc().verifyState(state!);
expect(decoded).toMatchObject({ provider: 'custom' });
});
it('returns 400 for the revalidate flow without user_uuid', async () => {
const { res } = makeRes();
await expect(
callRoute(
'get',
'/auth/oidc/:provider/start',
makeReq({
params: { provider: 'custom' },
query: { flow: 'revalidate' },
}),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── /auth/oidc/callback/login ───────────────────────────────────────
describe('OIDCController login callback', () => {
it('redirects with auth_error=1 when the state is invalid', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'c', state: 'not-a-real-token' } }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
expect(captured.redirectUrl).toContain('action=login');
});
it('redirects with auth_error=1 when code or state is missing', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: {} }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
});
it('creates a fresh user and sets the session cookie on first sign-in', async () => {
// Sign a real state token so verifyState succeeds; spy past the
// two methods that hit external HTTP.
const state = oidc().signState({
provider: 'custom',
redirect_uri: 'http://test.local/',
});
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
const email = `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`;
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub,
email,
email_verified: true,
} as never);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'authcode', state } }),
res,
);
// Session cookie was set with the configured cookie name.
expect(captured.cookies).toHaveLength(1);
expect(captured.cookies[0]?.value).toBeTruthy();
// Same-origin redirect target is preserved.
expect(captured.redirectUrl).toBe('http://test.local/');
// The real OIDCService linked the new user — verify via the
// live store.
const linkedUser = await oidc().findUserByProviderSub(
'custom',
sub,
);
expect(linkedUser).not.toBeNull();
expect(linkedUser?.email).toBe(email);
});
it('clamps redirect_uri to the configured origin (rejects external)', async () => {
const state = oidc().signState({
provider: 'custom',
// External attacker-supplied target.
redirect_uri: 'https://evil.test/steal',
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub: `sub-${Math.random().toString(36).slice(2, 8)}`,
email: `oidc-${Math.random().toString(36).slice(2, 8)}@test.local`,
email_verified: true,
} as never);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'c', state } }),
res,
);
// Falls back to the configured origin since the requested URL
// is on a different host.
expect(captured.redirectUrl).toBe(TEST_ORIGIN);
});
it('refuses to link to an existing account with an unconfirmed email', async () => {
// Seed an existing UNCONFIRMED user with the OIDC-claimed email.
const email = `pending-${Math.random().toString(36).slice(2, 8)}@test.local`;
await server.stores.user.create({
username: `pending-${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'hashed',
email,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: true,
});
const state = oidc().signState({
provider: 'custom',
redirect_uri: 'http://test.local/',
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub: `sub-${Math.random().toString(36).slice(2, 8)}`,
email,
email_verified: true,
} as never);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
});
});
// ── /auth/oidc/callback/revalidate ──────────────────────────────────
describe('OIDCController revalidate callback', () => {
it('returns 400 on invalid state', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/revalidate',
makeReq({ query: { code: 'c', state: 'not-a-token' } }),
res,
);
expect(captured.statusCode).toBe(400);
});
it('returns 400 when the state has the wrong flow', async () => {
// verifyState succeeds, but state.flow !== 'revalidate'.
const state = oidc().signState({
provider: 'custom',
flow: 'login',
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub: 'whatever',
} as never);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/revalidate',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.statusCode).toBe(400);
});
it('returns 400 when no account exists for the OIDC sub', async () => {
const state = oidc().signState({
provider: 'custom',
flow: 'revalidate',
user_uuid: uuidv4(),
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub: `unlinked-${Math.random().toString(36).slice(2, 8)}`,
} as never);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/revalidate',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.statusCode).toBe(400);
});
});
// ── GET /auth/revalidate-done ───────────────────────────────────────
describe('OIDCController GET /auth/revalidate-done', () => {
it('renders the postMessage HTML body with the configured origin', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/revalidate-done',
makeReq({}),
res,
);
expect(captured.contentType).toContain('text/html');
const body = String(captured.body);
expect(body).toContain('puter-revalidate-done');
// The configured origin is JSON-stringified into the inline script.
expect(body).toContain(JSON.stringify(TEST_ORIGIN));
});
});
+181 -189
View File
@@ -201,197 +201,190 @@ export class OIDCController extends PuterController {
},
);
// ── GET /auth/oidc/callback/login ───────────────────────────
// ── /auth/oidc/callback/login (GET + POST) ────────────────
router.get(
'/auth/oidc/callback/login',
{
subdomain: '',
rateLimit: { scope: 'oidc-general', limit: 30, window: 60_000 },
},
async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
const result = await this.#processCallback(req, 'login');
if ('error' in result) {
console.warn(`OIDC login callback error: ${result.error}`);
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'login',
'other',
result.error,
),
);
}
const cbOpts = {
subdomain: '',
rateLimit: { scope: 'oidc-general', limit: 30, window: 60_000 },
};
const { provider, userinfo, stateDecoded } = result;
const resolved = await this.#resolveOrCreateOIDCUser(
provider,
userinfo,
const loginCb = async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
const result = await this.#processCallback(req, 'login');
if ('error' in result) {
console.warn(`OIDC login callback error: ${result.error}`);
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'login',
'other',
result.error,
),
);
if ('error' in resolved) {
console.warn(
`OIDC login user resolution error: ${resolved.error}`,
);
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'login',
'other',
resolved.error,
stateDecoded,
),
);
}
const user = resolved.user;
}
if (user.suspended) {
console.warn(
`Suspended user tried to login via oidc: ${user.username}`,
);
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'login',
'other',
'This account is suspended.',
stateDecoded,
),
);
}
await this.#finishLogin(res, user, stateDecoded);
},
);
// ── GET /auth/oidc/callback/signup ──────────────────────────
router.get(
'/auth/oidc/callback/signup',
{
subdomain: '',
rateLimit: { scope: 'oidc-general', limit: 30, window: 60_000 },
},
async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
const result = await this.#processCallback(req, 'signup');
if ('error' in result) {
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'unauthorized',
),
);
}
const { provider, userinfo, stateDecoded } = result;
const resolved = await this.#resolveOrCreateOIDCUser(
provider,
userinfo,
const { provider, userinfo, stateDecoded } = result;
const resolved = await this.#resolveOrCreateOIDCUser(
provider,
userinfo,
);
if ('error' in resolved) {
console.warn(
`OIDC login user resolution error: ${resolved.error}`,
);
if ('error' in resolved) {
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'unauthorized',
stateDecoded,
),
);
}
const user = resolved.user;
if (user.suspended) {
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'account_suspended',
stateDecoded,
),
);
}
// If we landed on an existing account (either via the
// provider_sub or via email match), signal the GUI so it can
// render a "signed in" flow rather than "account created".
const extra =
resolved.origin === 'created'
? undefined
: { oidc_switched: 'login' };
await this.#finishLogin(res, user, stateDecoded, extra);
},
);
// ── GET /auth/oidc/callback/revalidate ──────────────────────
router.get(
'/auth/oidc/callback/revalidate',
{
subdomain: '',
rateLimit: { scope: 'oidc-general', limit: 30, window: 60_000 },
},
async (req: Request, res: Response): Promise<void> => {
const result = await this.#processCallback(req, 'revalidate');
if ('error' in result) {
res.status(400).send(result.error);
return;
}
const { provider, userinfo, stateDecoded } = result;
if (
stateDecoded.flow !== 'revalidate' ||
typeof stateDecoded.user_uuid !== 'string' ||
stateDecoded.user_uuid.length === 0
) {
res.status(400).send('Invalid revalidate state.');
return;
}
const user = await this.services.oidc.findUserByProviderSub(
provider,
userinfo.sub,
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'login',
'other',
resolved.error,
stateDecoded,
),
);
if (!user) {
res.status(400).send('No account found.');
return;
}
if (user.uuid !== stateDecoded.user_uuid) {
res.status(403).send(
'Wrong account. Sign in with the account linked to this session.',
);
return;
}
}
const user = resolved.user;
const token = this.services.oidc.signRevalidation(user.uuid);
res.cookie(REVALIDATION_COOKIE_NAME, token, {
// Revalidation flow is same-site only — `lax` even on HTTPS.
...sessionCookieFlags(this.config, { crossSite: false }),
httpOnly: true,
maxAge: REVALIDATION_EXPIRY_SEC * 1000,
path: '/',
});
if (user.suspended) {
console.warn(
`Suspended user tried to login via oidc: ${user.username}`,
);
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'login',
'other',
'This account is suspended.',
stateDecoded,
),
);
}
const origin = (this.config.origin ?? '').replace(/\/$/, '');
const requested =
(stateDecoded.redirect_uri as string) ||
`${origin}/auth/revalidate-done`;
const target = isSameOrigin(requested, origin)
? requested
: `${origin}/auth/revalidate-done`;
res.redirect(302, target);
},
);
await this.#finishLogin(res, user, stateDecoded);
};
router.get('/auth/oidc/callback/login', cbOpts, loginCb);
router.post('/auth/oidc/callback/login', cbOpts, loginCb);
// ── /auth/oidc/callback/signup (GET + POST) ────────────────
const signupCb = async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
const result = await this.#processCallback(req, 'signup');
if ('error' in result) {
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'unauthorized',
),
);
}
const { provider, userinfo, stateDecoded } = result;
const resolved = await this.#resolveOrCreateOIDCUser(
provider,
userinfo,
);
if ('error' in resolved) {
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'unauthorized',
stateDecoded,
),
);
}
const user = resolved.user;
if (user.suspended) {
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'account_suspended',
stateDecoded,
),
);
}
// If we landed on an existing account (either via the
// provider_sub or via email match), signal the GUI so it can
// render a "signed in" flow rather than "account created".
const extra =
resolved.origin === 'created'
? undefined
: { oidc_switched: 'login' };
await this.#finishLogin(res, user, stateDecoded, extra);
};
router.get('/auth/oidc/callback/signup', cbOpts, signupCb);
router.post('/auth/oidc/callback/signup', cbOpts, signupCb);
// ── /auth/oidc/callback/revalidate (GET + POST) ────────────
const revalidateCb = async (
req: Request,
res: Response,
): Promise<void> => {
const result = await this.#processCallback(req, 'revalidate');
if ('error' in result) {
res.status(400).send(result.error);
return;
}
const { provider, userinfo, stateDecoded } = result;
if (
stateDecoded.flow !== 'revalidate' ||
typeof stateDecoded.user_uuid !== 'string' ||
stateDecoded.user_uuid.length === 0
) {
res.status(400).send('Invalid revalidate state.');
return;
}
const user = await this.services.oidc.findUserByProviderSub(
provider,
userinfo.sub,
);
if (!user) {
res.status(400).send('No account found.');
return;
}
if (user.uuid !== stateDecoded.user_uuid) {
res.status(403).send(
'Wrong account. Sign in with the account linked to this session.',
);
return;
}
const token = this.services.oidc.signRevalidation(user.uuid);
res.cookie(REVALIDATION_COOKIE_NAME, token, {
// Revalidation flow is same-site only — `lax` even on HTTPS.
...sessionCookieFlags(this.config, { crossSite: false }),
httpOnly: true,
maxAge: REVALIDATION_EXPIRY_SEC * 1000,
path: '/',
});
const origin = (this.config.origin ?? '').replace(/\/$/, '');
const requested =
(stateDecoded.redirect_uri as string) ||
`${origin}/auth/revalidate-done`;
const target = isSameOrigin(requested, origin)
? requested
: `${origin}/auth/revalidate-done`;
res.redirect(302, target);
};
router.get('/auth/oidc/callback/revalidate', cbOpts, revalidateCb);
router.post('/auth/oidc/callback/revalidate', cbOpts, revalidateCb);
// ── GET /auth/revalidate-done ───────────────────────────────
// Landing page after revalidation; posts to opener for popup flow.
@@ -500,15 +493,13 @@ if (window.opener) {
stateDecoded: Record<string, unknown>;
}
> {
// Apple uses response_mode=form_post, so params arrive in the body.
const src = req.method === 'POST' && req.body ? req.body : req.query;
const code = String(
Array.isArray(req.query.code)
? req.query.code[0]
: (req.query.code ?? ''),
Array.isArray(src.code) ? src.code[0] : (src.code ?? ''),
);
const state = String(
Array.isArray(req.query.state)
? req.query.state[0]
: (req.query.state ?? ''),
Array.isArray(src.state) ? src.state[0] : (src.state ?? ''),
);
if (!code || !state) return { error: 'Missing code or state.' };
@@ -531,6 +522,7 @@ if (window.opener) {
const userinfo = await this.services.oidc.getUserInfo(
provider,
tokens.access_token,
typeof tokens.id_token === 'string' ? tokens.id_token : undefined,
);
if (!userinfo || !userinfo.sub)
return { error: 'Could not get user info.' };
@@ -0,0 +1,313 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
import type { Request, Response } from 'express';
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import type { RouteDescriptor } from '../../core/http/types.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import { StaticAssetsController } from './StaticAssetsController.js';
// ── Test harness ────────────────────────────────────────────────────
//
// StaticAssetsController is config-gated: each branch
// (`client_libs_root`, `puterjs_root`, `gui_assets_root`,
// `builtin_apps`) is enabled by setting that root + having the right
// files on disk. Tests boot one real PuterServer to wire up the
// clients/stores/services that controller construction expects, then
// instantiate StaticAssetsController with per-test config overrides
// and inspect the routes it registers onto a fresh PuterRouter.
//
// Real temp directories are seeded with the files each branch
// expects, so the controller's `fs.existsSync` calls see real state
// instead of a mocked module.
let server: PuterServer;
let tmpRoots: string[] = [];
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
beforeEach(() => {
tmpRoots = [];
});
afterEach(() => {
for (const root of tmpRoots) {
try {
rmSync(root, { recursive: true, force: true });
} catch {
// Best-effort cleanup.
}
}
});
const makeTempDir = (): string => {
const dir = mkdtempSync(path.join(tmpdir(), 'puter-static-test-'));
tmpRoots.push(dir);
return dir;
};
const buildController = (configOverrides: Partial<IConfig> = {}) => {
// Reuse the live wired clients/stores/services/drivers from the
// booted test server — only the config differs per test.
const controller = new StaticAssetsController(
{ ...configOverrides } as IConfig,
server.clients,
server.stores,
server.services,
server.drivers,
);
const router = new PuterRouter();
controller.registerRoutes(router);
return router;
};
const findRoute = (
router: PuterRouter,
method: string,
routePath: string,
): RouteDescriptor | undefined =>
router.routes.find((r) => r.method === method && r.path === routePath);
const findUseRoute = (
router: PuterRouter,
routePath: string,
): RouteDescriptor | undefined =>
router.routes.find((r) => r.method === 'use' && r.path === routePath);
const callGetHandler = async (
router: PuterRouter,
routePath: string,
): Promise<{ filename?: string; root?: string }> => {
const route = findRoute(router, 'get', routePath);
if (!route) throw new Error(`No GET ${routePath} registered`);
let captured: { filename?: string; root?: string } = {};
const req = {} as Request;
const res = {
sendFile: vi.fn((filename: string, opts: { root: string }) => {
captured = { filename, root: opts.root };
}),
} as unknown as Response;
await route.handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
return captured;
};
// ── client_libs_root ────────────────────────────────────────────────
describe('StaticAssetsController client_libs_root', () => {
it('registers /puter.js/v1, /v2, /putility/v1 on the right subdomains', async () => {
const root = makeTempDir();
const router = buildController({ client_libs_root: root });
const v1Wild = findRoute(router, 'get', '/puter.js/v1');
const v2Wild = findRoute(router, 'get', '/puter.js/v2');
const v1Js = findRoute(router, 'get', '/v1');
const v2Js = findRoute(router, 'get', '/v2');
const putilityJs = findRoute(router, 'get', '/putility/v1');
// /puter.js/* routes are wildcard-subdomain (any host).
expect(v1Wild?.options.subdomain).toBe('*');
expect(v2Wild?.options.subdomain).toBe('*');
// Bare-version routes live on the `js` subdomain.
expect(v1Js?.options.subdomain).toBe('js');
expect(v2Js?.options.subdomain).toBe('js');
expect(putilityJs?.options.subdomain).toBe('js');
// The handler should hand off the right relative file path.
const sent = await callGetHandler(router, '/puter.js/v1');
expect(sent).toEqual({ filename: 'puter.js/v1.js', root });
const sent2 = await callGetHandler(router, '/v2');
expect(sent2).toEqual({ filename: 'puter.js/v2.js', root });
const sentPutility = await callGetHandler(router, '/putility/v1');
expect(sentPutility).toEqual({ filename: 'putility.js/v1.js', root });
});
it('does not register the client_libs routes when the root is unset', () => {
const router = buildController({});
expect(findRoute(router, 'get', '/puter.js/v1')).toBeUndefined();
expect(findRoute(router, 'get', '/v1')).toBeUndefined();
expect(findRoute(router, 'get', '/putility/v1')).toBeUndefined();
});
});
// ── puterjs_root ────────────────────────────────────────────────────
describe('StaticAssetsController puterjs_root', () => {
it('serves puter.js when puter.dev.js is missing (OSS-built repo)', async () => {
const root = makeTempDir();
// OSS repo ships only puter.js — no puter.dev.js artifact.
writeFileSync(path.join(root, 'puter.js'), '/* mock */');
const router = buildController({ puterjs_root: root });
// `/sdk/puter.dev.js` aliases to puter.js in this configuration.
const aliased = await callGetHandler(router, '/sdk/puter.dev.js');
expect(aliased.filename).toBe('puter.js');
expect(aliased.root).toBe(root);
// /sdk static mount must be registered on the empty subdomain.
const sdkUse = findUseRoute(router, '/sdk');
expect(sdkUse?.options.subdomain).toBe('');
// Without client_libs_root, /puter.js/v1 / /v1 routes also point
// at puterjs_root and serve puter.js.
const v1 = await callGetHandler(router, '/puter.js/v1');
expect(v1.filename).toBe('puter.js');
expect(v1.root).toBe(root);
const bareV1 = await callGetHandler(router, '/v1');
expect(bareV1.filename).toBe('puter.js');
});
it('serves puter.dev.js when present (dev webpack build)', async () => {
const root = makeTempDir();
writeFileSync(path.join(root, 'puter.dev.js'), '/* dev */');
// puter.js may also exist; presence of dev wins.
writeFileSync(path.join(root, 'puter.js'), '/* prod */');
const router = buildController({ puterjs_root: root });
const v1 = await callGetHandler(router, '/puter.js/v1');
expect(v1.filename).toBe('puter.dev.js');
// The /sdk/puter.dev.js alias is NOT registered in this case
// because express.static below it serves the file natively.
const aliasRoute = findRoute(router, 'get', '/sdk/puter.dev.js');
expect(aliasRoute).toBeUndefined();
});
it('skips bare /v1, /v2, /puter.js/* when client_libs_root is also set', async () => {
const libRoot = makeTempDir();
const sdkRoot = makeTempDir();
writeFileSync(path.join(sdkRoot, 'puter.js'), '/* mock */');
const router = buildController({
client_libs_root: libRoot,
puterjs_root: sdkRoot,
});
// The /puter.js/v1 handler routes through client_libs_root
// (registered first; PuterRouter is order-preserving). Verify
// by confirming the file the handler resolves comes from libRoot.
const sent = await callGetHandler(router, '/puter.js/v1');
expect(sent.root).toBe(libRoot);
// The /sdk mount should still be present from puterjs_root.
expect(findUseRoute(router, '/sdk')).toBeDefined();
});
it('does not register /sdk when puterjs_root is unset', () => {
const router = buildController({});
expect(findUseRoute(router, '/sdk')).toBeUndefined();
});
});
// ── gui_assets_root ─────────────────────────────────────────────────
describe('StaticAssetsController gui_assets_root', () => {
it('mounts /dist and /src on the empty subdomain', () => {
const root = makeTempDir();
// /assets requires public/ to also exist; create only dist+src here.
mkdirSync(path.join(root, 'dist'));
mkdirSync(path.join(root, 'src'));
const router = buildController({ gui_assets_root: root });
const distMount = findUseRoute(router, '/dist');
const srcMount = findUseRoute(router, '/src');
expect(distMount?.options.subdomain).toBe('');
expect(srcMount?.options.subdomain).toBe('');
});
it('mounts /assets only when public/ exists', () => {
const root = makeTempDir();
mkdirSync(path.join(root, 'dist'));
mkdirSync(path.join(root, 'src'));
// No public/ → /assets should NOT be registered.
const router = buildController({ gui_assets_root: root });
expect(findUseRoute(router, '/assets')).toBeUndefined();
});
it('mounts /assets when public/ exists', () => {
const root = makeTempDir();
mkdirSync(path.join(root, 'dist'));
mkdirSync(path.join(root, 'src'));
mkdirSync(path.join(root, 'public'));
const router = buildController({ gui_assets_root: root });
const assetsMount = findUseRoute(router, '/assets');
expect(assetsMount?.options.subdomain).toBe('');
});
});
// ── builtin_apps ────────────────────────────────────────────────────
describe('StaticAssetsController builtin_apps', () => {
it('registers a /builtin/<name> mount per existing dir', () => {
const editorRoot = makeTempDir();
const browserRoot = makeTempDir();
const router = buildController({
builtin_apps: {
editor: editorRoot,
browser: browserRoot,
} as unknown as IConfig['builtin_apps'],
});
expect(findUseRoute(router, '/builtin/editor')).toBeDefined();
expect(findUseRoute(router, '/builtin/browser')).toBeDefined();
});
it('skips entries whose dirPath is empty / nonexistent', () => {
const realRoot = makeTempDir();
const router = buildController({
builtin_apps: {
editor: realRoot,
missing: '/nonexistent/path',
empty: '',
} as unknown as IConfig['builtin_apps'],
});
expect(findUseRoute(router, '/builtin/editor')).toBeDefined();
expect(findUseRoute(router, '/builtin/missing')).toBeUndefined();
expect(findUseRoute(router, '/builtin/empty')).toBeUndefined();
});
it('does not register any /builtin mounts when builtin_apps is undefined', () => {
const router = buildController({});
for (const route of router.routes) {
expect(route.path).not.toMatch(/^\/builtin\//);
}
});
});
// ── No-config (everything off) ──────────────────────────────────────
describe('StaticAssetsController with no roots configured', () => {
it('registers no routes', () => {
const router = buildController({});
expect(router.routes).toHaveLength(0);
});
});
@@ -0,0 +1,420 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock
// redis) and re-registers StaticPagesController's inline lambda
// routes onto a fresh PuterRouter. Tests run against the live wired
// stores (user, group), DB client, and EventClient — no method spies
// or stub services. Each test seeds the data it needs (user rows,
// listed apps) directly through the real stores.
let server: PuterServer;
let router: PuterRouter;
beforeAll(async () => {
server = await setupTestServer();
router = new PuterRouter();
server.controllers.staticPages.registerRoutes(router);
});
afterAll(async () => {
await server?.shutdown();
});
interface CapturedResponse {
statusCode: number;
body: unknown;
contentType?: string;
}
const makeReq = (init: {
query?: Record<string, unknown>;
hostname?: string;
protocol?: string;
}): Request => {
return {
body: {},
query: init.query ?? {},
params: {},
headers: {},
hostname: init.hostname ?? 'test.local',
protocol: init.protocol ?? 'https',
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
send: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
type: vi.fn((value: string) => {
captured.contentType = value;
return res;
}),
setHeader: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
const findHandler = (method: string, path: string): RequestHandler => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route.handler;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler = findHandler(method, path);
await handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
};
interface TestUser {
id: number;
uuid: string;
username: string;
email: string;
email_confirm_token: string;
}
const makeUser = async (
overrides: {
email_confirmed?: 0 | 1;
unsubscribed?: 0 | 1;
} = {},
): Promise<TestUser> => {
const username = `spc-${Math.random().toString(36).slice(2, 10)}`;
const uuid = uuidv4();
const email = `${username}@test.local`;
const token = `tok-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid,
password: null,
email,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
// Layer the test-specific shape on top of the row (clean_email is
// populated by `create`; we just stamp the confirmation token here).
await server.stores.user.update(created.id, {
email_confirm_token: token,
email_confirmed: overrides.email_confirmed ?? 0,
unsubscribed: overrides.unsubscribed ?? 0,
});
return { id: created.id, uuid, username, email, email_confirm_token: token };
};
// ── /robots.txt ─────────────────────────────────────────────────────
describe('StaticPagesController GET /robots.txt', () => {
it('disallows known SEO bots and points to the sitemap', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/robots.txt',
makeReq({ protocol: 'https' }),
res,
);
expect(captured.contentType).toBe('text/plain');
const body = String(captured.body);
expect(body).toContain('User-agent: AhrefsBot');
expect(body).toContain('User-agent: SemrushBot');
expect(body).toContain('Disallow: /');
// Sitemap URL is built off the request protocol + configured domain
// (or req.hostname if domain is unset).
expect(body).toMatch(/Sitemap: https:\/\/[^/]+\/sitemap\.xml/);
});
});
// ── /sitemap.xml ────────────────────────────────────────────────────
describe('StaticPagesController GET /sitemap.xml', () => {
it('lists docs + each approved-for-listing app', async () => {
const { id: userId } = await makeUser();
// Seed an approved app via the real AppStore. `approved_for_listing`
// is in the store's READ_ONLY_COLUMNS (admin-controlled), so we
// flip it via a direct DB write after the row is created.
const name = `app-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.app.create(
{
name,
title: name,
description: '',
index_url: `https://example.com/${name}/`,
},
{ ownerUserId: userId },
);
await server.clients.db.write(
'UPDATE `apps` SET `approved_for_listing` = 1 WHERE `name` = ?',
[name],
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/sitemap.xml',
makeReq({ protocol: 'https' }),
res,
);
expect(captured.contentType).toBe('application/xml');
const body = String(captured.body);
expect(body).toContain(
'<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">',
);
// Docs subdomain entry is always present.
expect(body).toMatch(/<loc>https:\/\/docs\.[^<]+<\/loc>/);
// The seeded approved app appears.
expect(body).toContain(`/app/${name}`);
});
it('omits non-approved apps', async () => {
const { id: userId } = await makeUser();
const hidden = `hidden-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.app.create(
{
name: hidden,
title: hidden,
description: '',
index_url: `https://example.com/${hidden}/`,
approved_for_listing: 0,
},
{ ownerUserId: userId },
);
const { res, captured } = makeRes();
await callRoute('get', '/sitemap.xml', makeReq({}), res);
expect(String(captured.body)).not.toContain(`/app/${hidden}`);
});
});
// ── /unsubscribe ────────────────────────────────────────────────────
describe('StaticPagesController GET /unsubscribe', () => {
it('renders an error when user_uuid is missing', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/unsubscribe', makeReq({}), res);
expect(String(captured.body)).toContain('user_uuid is required');
});
it('renders an error when the user does not exist', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/unsubscribe',
makeReq({
query: { user_uuid: '00000000-0000-0000-0000-000000000000' },
}),
res,
);
expect(String(captured.body)).toContain('User not found');
});
it('flips unsubscribed=1 on the real user row', async () => {
const user = await makeUser({ unsubscribed: 0 });
const { res, captured } = makeRes();
await callRoute(
'get',
'/unsubscribe',
makeReq({ query: { user_uuid: user.uuid } }),
res,
);
expect(String(captured.body)).toContain(
'You have successfully unsubscribed',
);
const refreshed = await server.stores.user.getById(user.id);
// User store stores booleans as 1/0 in sqlite; both forms are accepted.
expect(Boolean(refreshed?.unsubscribed)).toBe(true);
});
it('reports already-unsubscribed without re-writing', async () => {
const user = await makeUser({ unsubscribed: 1 });
const { res, captured } = makeRes();
await callRoute(
'get',
'/unsubscribe',
makeReq({ query: { user_uuid: user.uuid } }),
res,
);
expect(String(captured.body)).toContain('already unsubscribed');
});
});
// ── /confirm-email-by-token ─────────────────────────────────────────
describe('StaticPagesController GET /confirm-email-by-token', () => {
it('renders an error when user_uuid is missing', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({ query: { token: 'whatever' } }),
res,
);
expect(String(captured.body)).toContain('user_uuid is required');
});
it('renders an error when token is missing', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({ query: { user_uuid: 'u-uuid' } }),
res,
);
expect(String(captured.body)).toContain('token is required');
});
it('renders an error when the user does not exist', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({
query: {
user_uuid: '00000000-0000-0000-0000-000000000000',
token: 'x',
},
}),
res,
);
expect(String(captured.body)).toContain('user not found');
});
it('rejects an invalid token without modifying the user', async () => {
const user = await makeUser({ email_confirmed: 0 });
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({
query: { user_uuid: user.uuid, token: 'wrong' },
}),
res,
);
expect(String(captured.body)).toContain('invalid token');
const refreshed = await server.stores.user.getById(user.id);
expect(Boolean(refreshed?.email_confirmed)).toBe(false);
});
it('reports already-confirmed without re-writing', async () => {
const user = await makeUser({ email_confirmed: 1 });
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({
query: {
user_uuid: user.uuid,
token: user.email_confirm_token,
},
}),
res,
);
expect(String(captured.body)).toContain('Email already confirmed');
});
it('confirms the email and clears the token on the real user row', async () => {
const user = await makeUser({ email_confirmed: 0 });
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({
query: {
user_uuid: user.uuid,
token: user.email_confirm_token,
},
}),
res,
);
expect(String(captured.body)).toContain('successfully confirmed');
const refreshed = await server.stores.user.getById(user.id);
expect(Boolean(refreshed?.email_confirmed)).toBe(true);
expect(refreshed?.email_confirm_token).toBeNull();
// requires_email_confirmation is also cleared by the controller.
expect(Boolean(refreshed?.requires_email_confirmation)).toBe(false);
});
it('rejects when the email is already confirmed on a different account', async () => {
// Duplicate-email gate fires only when the existing row is
// (a) email_confirmed=1 and (b) has a non-null password — the
// controller's EXISTS check requires both. `makeUser` writes
// password=null so we have to pin one in directly.
const ownerUser = await makeUser({ email_confirmed: 1 });
await server.clients.db.write(
'UPDATE `user` SET `password` = ? WHERE `id` = ?',
['hashed-pw', ownerUser.id],
);
// Second user with the same email; controller will refuse to
// confirm them because the original already owns the address.
const duplicateUuid = uuidv4();
const duplicateUsername = `dup-${Math.random().toString(36).slice(2, 8)}`;
await server.clients.db.write(
'INSERT INTO `user` (`uuid`, `username`, `email`, `clean_email`, `email_confirmed`, `password`, `email_confirm_token`) VALUES (?, ?, ?, ?, 0, NULL, ?)',
[
duplicateUuid,
duplicateUsername,
ownerUser.email,
ownerUser.email.toLowerCase(),
'tok-dup',
],
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/confirm-email-by-token',
makeReq({
query: {
user_uuid: duplicateUuid,
token: 'tok-dup',
},
}),
res,
);
expect(String(captured.body)).toContain(
'confirmed on a different account',
);
});
});
@@ -0,0 +1,277 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock
// redis) and re-registers SystemController's inline lambda routes
// onto a fresh PuterRouter so each handler is reachable. Tests then
// drive the captured handler with stub req/res — the underlying
// services (health, db, drivers) are the live wired ones.
let server: PuterServer;
let router: PuterRouter;
beforeAll(async () => {
server = await setupTestServer();
router = new PuterRouter();
server.controllers.system.registerRoutes(router);
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `sysc-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
interface CapturedResponse {
statusCode: number;
body: unknown;
}
const makeReq = (init: {
body?: unknown;
actor?: Actor;
}): Request => {
return {
body: init.body ?? {},
query: {},
headers: {},
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: CapturedResponse = { statusCode: 200, body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
send: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
setHeader: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
const findHandler = (method: string, path: string): RequestHandler => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route.handler;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler = findHandler(method, path);
await handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
};
// ── /healthcheck ────────────────────────────────────────────────────
describe('SystemController GET /healthcheck', () => {
it('returns the live ServerHealthService status payload', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/healthcheck', makeReq({}), res);
// Live status — boot is complete and the in-memory DB is up,
// so ok=true is the expected steady state for this harness.
expect(captured.body).toMatchObject({ ok: true });
expect(captured.statusCode).toBe(200);
});
});
// ── /version ────────────────────────────────────────────────────────
describe('SystemController GET /version', () => {
it('returns version-shape JSON with environment + deploy_timestamp', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/version', makeReq({}), res);
const body = captured.body as Record<string, unknown>;
// Default config has no `version` set — falls through to
// npm_package_version (set when running under vitest) or 'unknown'.
expect(typeof body.version).toBe('string');
// Default test config carries env='dev' from config.default.json.
expect(body.environment).toBe('dev');
expect(typeof body.deploy_timestamp).toBe('number');
});
});
// ── /contactUs ──────────────────────────────────────────────────────
describe('SystemController POST /contactUs', () => {
it('throws 400 when message is missing', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/contactUs',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when message is not a string', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/contactUs',
makeReq({ body: { message: 12345 }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when message exceeds 100,000 characters', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/contactUs',
makeReq({
body: { message: 'x'.repeat(100_001) },
actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('persists feedback into the DB on success', async () => {
const { actor, userId } = await makeUser();
const message = `hello ${Math.random().toString(36).slice(2)}`;
const { res, captured } = makeRes();
await callRoute(
'post',
'/contactUs',
makeReq({ body: { message }, actor }),
res,
);
expect(captured.body).toEqual({});
// The row landed in the real `feedback` table for the right user.
const rows = (await server.clients.db.read(
'SELECT `user_id`, `message` FROM `feedback` WHERE `user_id` = ? AND `message` = ?',
[userId, message],
)) as Array<{ user_id: number; message: string }>;
expect(rows).toHaveLength(1);
expect(rows[0]?.message).toBe(message);
});
});
// ── /whoarewe ───────────────────────────────────────────────────────
describe('SystemController GET /whoarewe', () => {
it('returns the configured Puter identity payload', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/whoarewe', makeReq({}), res);
expect(captured.body).toMatchObject({
name: 'Puter',
environment: 'dev',
});
});
});
// ── /lsmod ──────────────────────────────────────────────────────────
describe('SystemController GET /lsmod', () => {
it('lists wired drivers grouped by interface', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/lsmod', makeReq({}), res);
const body = captured.body as {
interfaces: Record<
string,
{ implementors: Record<string, { isDefault: boolean }> }
>;
};
expect(body.interfaces).toBeDefined();
// Test harness wires the full driver registry; at least one
// driver/interface pair must come through.
expect(Object.keys(body.interfaces).length).toBeGreaterThan(0);
for (const iface of Object.values(body.interfaces)) {
expect(Object.keys(iface.implementors).length).toBeGreaterThan(0);
}
});
});
// ── lifecycle ───────────────────────────────────────────────────────
describe('SystemController.onServerPrepareShutdown', () => {
it('flips the global drain flag', () => {
// Reset before the call so the assertion is meaningful even
// when earlier code in the same process already tripped it.
(
globalThis as unknown as { __puter_draining?: boolean }
).__puter_draining = false;
server.controllers.system.onServerPrepareShutdown();
expect(
(globalThis as unknown as { __puter_draining?: boolean })
.__puter_draining,
).toBe(true);
});
});
+37 -8
View File
@@ -18,24 +18,49 @@
*/
import type { puterClients } from '../clients';
import type { IExtensionClientInstances } from '../clients/types';
import type { PuterRouter } from '../core/http/PuterRouter';
import type { puterDrivers } from '../drivers';
import type { IExtensionDriverInstances } from '../drivers/types';
import type { puterServices } from '../services';
import type { IExtensionServiceInstances } from '../services/types';
import type { puterStores } from '../stores';
import type { IExtensionStoreInstances } from '../stores/types';
import type {
IConfig,
LayerInstances,
WithControllerRegistration,
} from '../types';
/**
* Extension-augmentable controller registry. Extensions add their own
* controller instance types via TypeScript declaration merging:
*
* declare module '@heyputer/backend/controllers/types' {
* interface IExtensionControllerInstances {
* myController: MyController;
* }
* }
*
* Augmentations flow into the `extension.import('controller')` proxy.
*/
export interface IExtensionControllerInstances {
/**
* Open index signature so reads of extension-only controller keys return
* `unknown` instead of a type error. Concrete declaration-merged keys
* override this for that name.
*/
[key: string]: unknown;
}
export type IPuterController<
T extends WithControllerRegistration = WithControllerRegistration,
> = new (
config: IConfig,
clients: LayerInstances<typeof puterClients>,
stores: LayerInstances<typeof puterStores>,
services: LayerInstances<typeof puterServices>,
drivers: LayerInstances<typeof puterDrivers>,
clients: LayerInstances<typeof puterClients> & IExtensionClientInstances,
stores: LayerInstances<typeof puterStores> & IExtensionStoreInstances,
services: LayerInstances<typeof puterServices> & IExtensionServiceInstances,
drivers: LayerInstances<typeof puterDrivers> & IExtensionDriverInstances,
) => T;
/**
@@ -49,10 +74,14 @@ export const PuterController =
class PuterController implements WithControllerRegistration {
constructor(
protected config: IConfig,
protected clients: LayerInstances<typeof puterClients>,
protected stores: LayerInstances<typeof puterStores>,
protected services: LayerInstances<typeof puterServices>,
protected drivers: LayerInstances<typeof puterDrivers>,
protected clients: LayerInstances<typeof puterClients> &
IExtensionClientInstances,
protected stores: LayerInstances<typeof puterStores> &
IExtensionStoreInstances,
protected services: LayerInstances<typeof puterServices> &
IExtensionServiceInstances,
protected drivers: LayerInstances<typeof puterDrivers> &
IExtensionDriverInstances,
) {}
public onServerStart() {
return;
@@ -82,9 +82,9 @@ async function buildRevalidateFields(
user: UserRow,
): Promise<Record<string, string> | undefined> {
const origin = (config.origin ?? '').replace(/\/$/, '');
const providers = await oidcService.getEnabledProviderIds();
const provider = providers && providers[0];
if (!provider || !origin) return undefined;
if (!origin) return undefined;
const provider = await oidcService.getLinkedProviderForUser(user.id);
if (!provider) return undefined;
return {
revalidate_url: `${origin}/auth/oidc/${provider}/start?flow=revalidate&user_uuid=${encodeURIComponent(user.uuid)}`,
};
@@ -54,6 +54,7 @@ import {
normalize_single_message,
} from './utils/Messages.js';
import { AIChatStream } from './utils/Streaming.js';
import { EventMap } from '../../clients/event/types.js';
const MAX_FALLBACKS = 4; // includes first attempt
@@ -163,7 +164,8 @@ export class ChatCompletionDriver extends PuterDriver {
.replaceAll('-', '')
.slice(0, 25);
const validateEvent: Record<string, unknown> = {
const validateEvent: EventMap['ai.prompt.validate'] = {
username: actor.user?.username || '',
actor,
completionId,
allow: true,
@@ -308,10 +310,8 @@ export class ChatCompletionDriver extends PuterDriver {
// Credits can be exhausted mid-fallback by parallel requests;
// re-check before another upstream hit. Same bail as the
// pre-flight above.
const fallbackUsageAllowed = await metering.hasEnoughCredits(
actor,
1,
);
const fallbackUsageAllowed =
await this.services.metering.hasEnoughCredits(actor, 1);
if (!fallbackUsageAllowed) {
throw new HttpError(402, 'No usage left for request.', {
legacyCode: 'insufficient_funds',
@@ -357,23 +357,25 @@ export class ChatCompletionDriver extends PuterDriver {
const init = res.init_chat_stream;
const cleanup = res.finally_fn;
// Intercept `chatStream.end(usage)` so we can fire the same
// complete + cost-calculated events the non-streaming branch
// emits. Providers always terminate streams via `.end(usage)`;
// if they skip it, we just lose the cost event (no worse than
// not emitting).
// Intercept `chatStream.end(usage)` to fire complete + cost events
// (mirrors the non-streaming branch). Clone usage so providers that
// meter after this call (e.g. Claude) don't pick up `usd_cents`.
const originalEnd = chatStream.end.bind(chatStream);
chatStream.end = (usage?: Record<string, number>) => {
const enrichedUsage = usage ? { ...usage } : usage;
if (enrichedUsage) {
this.#injectUsdCents(enrichedUsage, model);
}
this.clients.event.emit(
'ai.prompt.complete',
{
username,
username: username!,
completionId,
intended_service: intendedProvider,
parameters: args,
result: { usage, stream: true },
result: { usage: enrichedUsage, stream: true },
model_used: model.id,
service_used: model.provider,
service_used: model.provider!,
},
{},
);
@@ -386,7 +388,7 @@ export class ChatCompletionDriver extends PuterDriver {
intendedProvider,
});
}
return originalEnd(usage!);
return originalEnd(enrichedUsage!);
};
// Fire-and-forget — the stream writes happen async while the
@@ -423,18 +425,19 @@ export class ChatCompletionDriver extends PuterDriver {
this.clients.event.emit(
'ai.prompt.complete',
{
username,
username: username!,
completionId,
intended_service: intendedProvider,
parameters: args,
result: res,
model_used: model.id,
service_used: model.provider,
service_used: model.provider!,
},
{},
);
if ('usage' in res && res.usage) {
this.#injectUsdCents(res.usage, model);
this.#emitCostCalculated({
completionId,
username,
@@ -461,6 +464,116 @@ export class ChatCompletionDriver extends PuterDriver {
return { ...res, via_ai_chat_service: true };
}
// Compute per-token cost in microcents (1 cent = 1_000_000 microCents).
// Shape-agnostic: multiplies every usage key by its matching rate in
// `model.costs`. Returns `null` when cost data is unavailable.
#computeCost(
usage: Record<string, number>,
model: IChatModel,
): {
inputKey: string;
outputKey: string;
inputTokens: number;
outputTokens: number;
inputMicroCents: number;
outputMicroCents: number;
totalMicroCents: number;
} | null {
const inputKey =
(model.input_cost_key as string | undefined) ?? 'input_tokens';
const outputKey =
(model.output_cost_key as string | undefined) ?? 'output_tokens';
const costs = model.costs;
if (!costs) return null;
const outputRateRaw = costs[outputKey];
const outputRate =
typeof outputRateRaw === 'number' && Number.isFinite(outputRateRaw)
? outputRateRaw
: undefined;
const isOutputKey = (key: string) =>
key === outputKey ||
key === 'output_tokens' ||
key === 'completion_tokens' ||
key === 'thinking_tokens';
let inputMicroCents = 0;
let outputMicroCents = 0;
let sawAnyRate = false;
for (const [key, rawAmount] of Object.entries(usage)) {
if (typeof rawAmount !== 'number' || !Number.isFinite(rawAmount)) {
continue;
}
if (key === 'usd_cents') continue;
if (key === 'tokens') continue;
// thinking_tokens → output rate fallback
let rate = costs[key];
if (typeof rate !== 'number' || !Number.isFinite(rate)) {
if (key === 'thinking_tokens' && outputRate !== undefined) {
rate = outputRate;
} else {
continue;
}
}
const sub = rawAmount * rate;
sawAnyRate = true;
if (isOutputKey(key)) {
outputMicroCents += sub;
} else {
inputMicroCents += sub;
}
}
if (!sawAnyRate) return null;
inputMicroCents = Math.max(0, Math.round(inputMicroCents));
outputMicroCents = Math.max(0, Math.round(outputMicroCents));
const inputTokens = Number(
usage[inputKey] ?? usage.prompt_tokens ?? usage.input_tokens ?? 0,
);
const outputTokens = Number(
usage[outputKey] ??
usage.completion_tokens ??
usage.output_tokens ??
0,
);
return {
inputKey,
outputKey,
inputTokens,
outputTokens,
inputMicroCents,
outputMicroCents,
totalMicroCents: inputMicroCents + outputMicroCents,
};
}
// Add `usd_cents` to the usage object. Skips if the provider already
// set an authoritative value (e.g. OpenRouter's `usage.cost`).
// Sets `null` when cost data is unavailable for the model.
#injectUsdCents(usage: Record<string, number>, model: IChatModel): void {
if (
typeof usage.usd_cents === 'number' &&
Number.isFinite(usage.usd_cents)
) {
return;
}
const cost = this.#computeCost(usage, model);
if (!cost) {
(usage as Record<string, number | null>).usd_cents = null;
return;
}
usage.usd_cents = cost.totalMicroCents / 1_000_000;
}
// Compute per-token cost in microcents using the model's cost map,
// then emit `ai.prompt.cost-calculated` for listeners that persist
// billing/abuse rows keyed on the completion id.
@@ -474,42 +587,34 @@ export class ChatCompletionDriver extends PuterDriver {
const { completionId, username, usage, model, intendedProvider } =
params;
const cost = this.#computeCost(usage, model);
const inputKey =
(model.input_cost_key as string | undefined) ?? 'input_tokens';
const outputKey =
(model.output_cost_key as string | undefined) ?? 'output_tokens';
const inputCostPer = Number(model.costs?.[inputKey] ?? 0);
const outputCostPer = Number(model.costs?.[outputKey] ?? 0);
const inputTokens = Number(
usage[inputKey] ?? usage.prompt_tokens ?? usage.input_tokens ?? 0,
);
const outputTokens = Number(
usage[outputKey] ??
usage.completion_tokens ??
usage.output_tokens ??
0,
);
const inputUcents = Math.round(inputTokens * inputCostPer);
const outputUcents = Math.round(outputTokens * outputCostPer);
const inputTokens = cost?.inputTokens ?? 0;
const outputTokens = cost?.outputTokens ?? 0;
const inputMicroCents = cost?.inputMicroCents ?? 0;
const outputMicroCents = cost?.outputMicroCents ?? 0;
this.clients.event.emit(
'ai.prompt.cost-calculated',
{
completionId,
username,
username: username!,
usage,
input_tokens: inputTokens,
output_tokens: outputTokens,
input_ucents: inputUcents,
output_ucents: outputUcents,
total_ucents: inputUcents + outputUcents,
input_ucents: inputMicroCents,
output_ucents: outputMicroCents,
total_ucents: inputMicroCents + outputMicroCents,
costs_currency: model.costs_currency,
model_used: model.id,
service_used: model.provider,
service_used: model.provider!,
intended_service: intendedProvider,
model_details: {
id: model.id,
provider: model.provider,
provider: model.provider!,
input_cost_key: inputKey,
output_cost_key: outputKey,
costs: model.costs,
@@ -0,0 +1,70 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Claude provider.
*
* Hits the real Anthropic API with a tiny prompt against the cheapest
* model (Haiku) so the smoke check runs fast and doesn't accumulate
* cost. Skipped automatically when `PUTER_TEST_AI_CLAUDE_API_KEY` is
* not set; in CI, only triggered when the Claude provider source
* actually changes (see `.github/workflows/ai-provider-integration-tests.yaml`).
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { ClaudeProvider } from './ClaudeProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_CLAUDE_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))('ClaudeProvider (integration)', () => {
const buildProvider = () =>
new ClaudeProvider(
makeMeteringStub(),
// Stores / FS only consulted for `puter_path` uploads — text-only
// prompts never reach those code paths.
{ fsEntry: undefined as never, s3Object: undefined as never },
undefined as never,
{ apiKey: optionalEnv(ENV_VAR)! },
);
it('returns a non-empty completion from claude-haiku-4-5', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = buildProvider();
const result = await withTestActor(() =>
provider.complete({
model: 'claude-haiku-4-5-20251001',
messages: [{ role: 'user', content: 'Say hi in one word.' }],
max_tokens: 16,
}),
);
expect(result).toHaveProperty('message');
const content = (result as { message: { content: unknown } }).message
.content as Array<{ type: string; text?: string }>;
expect(Array.isArray(content)).toBe(true);
const text = content.find((c) => c.type === 'text')?.text ?? '';
expect(text.length).toBeGreaterThan(0);
});
});
@@ -0,0 +1,63 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the DeepSeek provider.
*
* Uses `deepseek-chat` (the cheap V3 chat model, provider default).
* Skipped when `PUTER_TEST_AI_DEEPSEEK_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { DeepSeekProvider } from './DeepSeekProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_DEEPSEEK_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'DeepSeekProvider (integration)',
() => {
it('returns a non-empty completion from deepseek-chat', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new DeepSeekProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'deepseek-chat',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
},
);
@@ -0,0 +1,63 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Gemini chat provider.
*
* Hits the real Google Gemini API with `gemini-2.0-flash-lite` (one
* of the cheapest variants). Skipped when `PUTER_TEST_AI_GEMINI_API_KEY`
* is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { GeminiChatProvider } from './GeminiChatProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_GEMINI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'GeminiChatProvider (integration)',
() => {
it('returns a non-empty completion from gemini-2.0-flash-lite', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new GeminiChatProvider(makeMeteringStub(), {
apiKey: optionalEnv(ENV_VAR)!,
});
const result = await withTestActor(() =>
provider.complete({
model: 'gemini-2.0-flash-lite',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
},
);
@@ -0,0 +1,59 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Groq provider.
*
* Uses `llama-3.1-8b-instant` (the provider's default and cheapest
* generally-available model). Skipped when `PUTER_TEST_AI_GROQ_API_KEY`
* is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { GroqAIProvider } from './GroqAIProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_GROQ_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))('GroqAIProvider (integration)', () => {
it('returns a non-empty completion from llama-3.1-8b-instant', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new GroqAIProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'llama-3.1-8b-instant',
messages: [{ role: 'user', content: 'Say hi in one word.' }],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
});
@@ -0,0 +1,77 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Mistral provider.
*
* Uses `mistral-small-2506` (provider default, cheapest tier). Skipped
* when `PUTER_TEST_AI_MISTRAL_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { MistralAIProvider } from './MistralAiProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_MISTRAL_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'MistralAIProvider (integration)',
() => {
it('returns a non-empty completion from mistral-small', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new MistralAIProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'mistral-small-2506',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: unknown } }).message
?.content;
// Mistral SDK may return string or array of content parts.
const asString =
typeof text === 'string'
? text
: Array.isArray(text)
? text
.map((p) =>
typeof p === 'string'
? p
: (p as { text?: string })?.text,
)
.filter(Boolean)
.join('')
: '';
expect(asString.length).toBeGreaterThan(0);
});
},
);
@@ -0,0 +1,63 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Moonshot provider.
*
* Uses `moonshot-v1-8k` (the cheapest 8K-context variant). Skipped
* when `PUTER_TEST_AI_MOONSHOT_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { MoonshotProvider } from './MoonshotProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_MOONSHOT_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'MoonshotProvider (integration)',
() => {
it('returns a non-empty completion from moonshot-v1-8k', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new MoonshotProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'moonshot-v1-8k',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
},
);
@@ -27,6 +27,7 @@ import type {
ICompleteArguments,
IChatCompleteResult,
} from '../../types.js';
import { inlineHttpImageUrls } from './imageHandling.js';
import { MOONSHOT_MODELS } from './models.js';
export class MoonshotProvider implements IChatProvider {
@@ -74,6 +75,13 @@ export class MoonshotProvider implements IChatProvider {
availableModels.find((m) =>
[m.id, ...(m.aliases || [])].includes(model),
) || availableModels.find((m) => m.id === this.getDefaultModel())!;
// Moonshot's vision API doesn't fetch http(s) URLs; inline them
// so callers can pass plain links like other vision providers.
if (modelUsed.modalities?.input?.includes('image')) {
await inlineHttpImageUrls(messages);
}
messages = await OpenAIUtil.process_input_messages(messages);
let completion;
try {
@@ -0,0 +1,208 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterEach, describe, expect, it, vi } from 'vitest';
vi.mock('../../../../util/secureHttp.js', () => ({
secureFetch: vi.fn(),
}));
import { secureFetch } from '../../../../util/secureHttp.js';
import { inlineHttpImageUrls, MAX_IMAGE_BYTES } from './imageHandling.js';
const mockedSecureFetch = vi.mocked(secureFetch);
const buildResponse = (
body: Buffer | ArrayBuffer,
{
status = 200,
contentType = 'image/png',
contentLength,
}: {
status?: number;
contentType?: string | null;
contentLength?: string;
} = {},
): Response => {
const buf = Buffer.isBuffer(body) ? body : Buffer.from(body);
const headers = new Headers();
if (contentType) headers.set('content-type', contentType);
if (contentLength) headers.set('content-length', contentLength);
return {
ok: status >= 200 && status < 300,
status,
headers,
arrayBuffer: async () =>
buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength),
} as unknown as Response;
};
describe('inlineHttpImageUrls', () => {
afterEach(() => {
mockedSecureFetch.mockReset();
});
it('rewrites http(s) image URLs to base64 data URIs', async () => {
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
mockedSecureFetch.mockResolvedValueOnce(
buildResponse(png, { contentType: 'image/png' }),
);
const messages = [
{
role: 'user',
content: [
{ type: 'text', text: 'what is this' },
{ image_url: { url: 'https://example.com/cat.png' } },
],
},
];
await inlineHttpImageUrls(messages);
expect(mockedSecureFetch).toHaveBeenCalledWith(
'https://example.com/cat.png',
);
const part = messages[0].content[1] as {
type?: string;
image_url?: { url?: string };
};
expect(part.type).toBe('image_url');
expect(part.image_url?.url).toBe(
`data:image/png;base64,${png.toString('base64')}`,
);
});
it('leaves data URIs untouched and skips fetching', async () => {
const messages = [
{
role: 'user',
content: [
{
type: 'image_url',
image_url: {
url: 'data:image/png;base64,AAAA',
},
},
],
},
];
await inlineHttpImageUrls(messages);
expect(mockedSecureFetch).not.toHaveBeenCalled();
const part = messages[0].content[0] as { image_url?: { url?: string } };
expect(part.image_url?.url).toBe('data:image/png;base64,AAAA');
});
it('replaces oversized images with a text error block', async () => {
const oversize = Buffer.alloc(10);
mockedSecureFetch.mockResolvedValueOnce(
buildResponse(oversize, {
contentType: 'image/jpeg',
contentLength: String(MAX_IMAGE_BYTES + 1),
}),
);
const messages = [
{
role: 'user',
content: [
{ image_url: { url: 'https://example.com/huge.jpg' } },
],
},
];
await inlineHttpImageUrls(messages);
const part = messages[0].content[0] as {
type?: string;
text?: string;
image_url?: unknown;
};
expect(part.type).toBe('text');
expect(part.image_url).toBeUndefined();
expect(part.text).toContain('exceeds maximum');
});
it('replaces non-image responses with a text error block', async () => {
mockedSecureFetch.mockResolvedValueOnce(
buildResponse(Buffer.from('<html/>'), {
contentType: 'text/html',
}),
);
const messages = [
{
role: 'user',
content: [
{ image_url: { url: 'https://example.com/page' } },
],
},
];
await inlineHttpImageUrls(messages);
const part = messages[0].content[0] as {
type?: string;
text?: string;
};
expect(part.type).toBe('text');
expect(part.text).toContain('expected an image');
});
it('replaces fetch failures with a text error block', async () => {
mockedSecureFetch.mockRejectedValueOnce(new Error('boom'));
const messages = [
{
role: 'user',
content: [
{ image_url: { url: 'https://example.com/x.png' } },
],
},
];
await inlineHttpImageUrls(messages);
const part = messages[0].content[0] as {
type?: string;
text?: string;
};
expect(part.type).toBe('text');
expect(part.text).toContain('boom');
});
it('ignores non-image-url parts and string content', async () => {
const messages = [
{ role: 'user', content: 'plain text' },
{
role: 'user',
content: [
{ type: 'text', text: 'still here' },
{ type: 'tool_use', id: 't', name: 'x', input: {} },
],
},
];
await inlineHttpImageUrls(messages);
expect(mockedSecureFetch).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,112 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { secureFetch } from '../../../../util/secureHttp.js';
// Matches the OpenAI Chat-Completions inline-upload cap.
export const MAX_IMAGE_BYTES = 5 * 1_000_000;
interface ImageContentPart {
type?: string;
text?: string;
image_url?: { url?: string };
}
interface MessageWithContent {
content?: unknown;
}
// Moonshot's vision API rejects http(s) image URLs and only accepts base64
// data URIs or file-id refs, so any web URL must be fetched and inlined.
// Failures become inline text-error parts (same shape as openai/fileUpload.ts).
export async function inlineHttpImageUrls(
messages: MessageWithContent[],
): Promise<void> {
const tasks: Array<Promise<void>> = [];
for (const message of messages) {
if (!Array.isArray(message.content)) continue;
for (const part of message.content as ImageContentPart[]) {
const url = part?.image_url?.url;
if (!url) continue;
if (!url.startsWith('http://') && !url.startsWith('https://')) {
continue;
}
tasks.push(inlineOne(part, url));
}
}
await Promise.all(tasks);
}
async function inlineOne(part: ImageContentPart, url: string): Promise<void> {
try {
const response = await secureFetch(url);
if (!response.ok) {
setTextError(
part,
`failed to fetch image (status ${response.status})`,
);
return;
}
const contentLength = Number(
response.headers.get('content-length') ?? NaN,
);
if (Number.isFinite(contentLength) && contentLength > MAX_IMAGE_BYTES) {
setTextError(
part,
`image exceeds maximum of ${MAX_IMAGE_BYTES} bytes`,
);
return;
}
const arrayBuf = await response.arrayBuffer();
if (arrayBuf.byteLength > MAX_IMAGE_BYTES) {
setTextError(
part,
`image exceeds maximum of ${MAX_IMAGE_BYTES} bytes`,
);
return;
}
const mimeType = (response.headers.get('content-type') ?? '')
.split(';')[0]
?.trim();
if (!mimeType || !mimeType.startsWith('image/')) {
setTextError(
part,
`expected an image, got ${mimeType || 'unknown MIME type'}`,
);
return;
}
const base64 = Buffer.from(arrayBuf).toString('base64');
part.type = 'image_url';
part.image_url = { url: `data:${mimeType};base64,${base64}` };
} catch (err) {
const message = (err as Error)?.message || 'failed to fetch image';
setTextError(part, message);
}
}
function setTextError(part: ImageContentPart, reason: string): void {
delete part.image_url;
part.type = 'text';
// Phrasing matches openai/fileUpload.ts so the model reads it as a
// system note, not user input.
part.text = `{error: ${reason}; the user did not write this message}`;
}
@@ -0,0 +1,71 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the OpenAI chat-completions provider.
*
* Hits the real OpenAI API with `gpt-4o-mini` — non-reasoning so
* `max_tokens=16` actually returns visible text (reasoning models like
* `gpt-5-nano` would burn the budget on thinking tokens before
* emitting any response). Skipped when `PUTER_TEST_AI_OPENAI_API_KEY`
* is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { OpenAiChatProvider } from './OpenAiChatCompletionsProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_OPENAI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'OpenAiChatCompletionsProvider (integration)',
() => {
const buildProvider = () =>
new OpenAiChatProvider(
makeMeteringStub(),
{ fsEntry: undefined as never, s3Object: undefined as never },
undefined as never,
{ apiKey: optionalEnv(ENV_VAR)! },
);
it('returns a non-empty completion from gpt-4o-mini', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = buildProvider();
const result = await withTestActor(() =>
provider.complete({
model: 'gpt-4o-mini',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
// OpenAIUtil returns the OpenAI choice object directly.
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
},
);
@@ -0,0 +1,70 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the OpenRouter aggregator.
*
* Routes through OpenRouter to a tiny upstream model
* (`google/gemini-2.0-flash-lite-001`). Skipped when
* `PUTER_TEST_AI_OPENROUTER_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { OpenRouterProvider } from './OpenRouterProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_OPENROUTER_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'OpenRouterProvider (integration)',
() => {
it('returns a non-empty completion via OpenRouter', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new OpenRouterProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
// OpenRouter's `complete` destructure types all params
// as `any` and lists all six explicitly, so TS demands
// every field — pass undefineds for the unused ones.
provider.complete({
model: 'openrouter:google/gemini-2.0-flash-lite-001',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
stream: undefined,
tools: undefined,
temperature: undefined,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
},
);
@@ -185,6 +185,8 @@ export class OpenRouterProvider implements IChatProvider {
modelUsed.id,
costOverwrites,
);
(trackedUsage as Record<string, number>).usd_cents =
usage.cost * 100;
return trackedUsage;
} else {
// custom open router logic because they're pricing are weird
@@ -0,0 +1,67 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Together AI provider.
*
* Uses `Qwen/Qwen2.5-7B-Instruct-Turbo` — non-Llama, small, cheap, and
* stays on Together's serverless tier. Llama variants on Together get
* rotated to dedicated endpoints often enough that they're not safe
* defaults. If Qwen also disappears, pick another live serverless
* model from https://api.together.ai/models?type=serverless. Skipped
* when `PUTER_TEST_AI_TOGETHER_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { TogetherAIProvider } from './TogetherAIProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_TOGETHER_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'TogetherAIProvider (integration)',
() => {
it('returns a non-empty completion from Qwen2.5 7B', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new TogetherAIProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'togetherai:Qwen/Qwen2.5-7B-Instruct-Turbo',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
},
);
@@ -0,0 +1,58 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the xAI (Grok) provider.
*
* Uses `grok-3-mini` — the cheapest small variant. Skipped when
* `PUTER_TEST_AI_XAI_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { XAIProvider } from './XAIProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_XAI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))('XAIProvider (integration)', () => {
it('returns a non-empty completion from grok-3-mini', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new XAIProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'grok-3-mini',
messages: [{ role: 'user', content: 'Say hi in one word.' }],
max_tokens: 16,
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
});
@@ -0,0 +1,63 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Z.AI (GLM) provider.
*
* Uses `glm-4.6` with `thinking: disabled` passed through `custom`.
* GLM models default to reasoning mode and route their tokens to a
* `reasoning_content` field, leaving `content` empty under tight
* budgets. Disabling thinking forces a plain text response so the
* usual `message.content` assertion works. Skipped when
* `PUTER_TEST_AI_ZAI_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { ZAIProvider } from './ZAIProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_ZAI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))('ZAIProvider (integration)', () => {
it('returns a non-empty completion from glm-4.6', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new ZAIProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.complete({
model: 'glm-4.6',
messages: [{ role: 'user', content: 'Say hi in one word.' }],
max_tokens: 16,
custom: { thinking: { type: 'disabled' } },
}),
);
const text = (result as { message?: { content?: string } }).message
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
});
@@ -63,7 +63,7 @@ const visionModel = (
maxTokens: number,
costs: IChatModel['costs'],
): IChatModel => ({
puterId: `z-ai:zai/${id}`,
puterId: `z-ai:z-ai/${id}`,
id,
name,
aliases: [`z-ai/${id}`, `zai/${id}`],
+1
View File
@@ -115,6 +115,7 @@ export interface IChatMessageResult {
stream?: never;
finally_fn?: never;
normalized?: boolean;
via_ai_chat_service?: boolean;
}
export type IChatCompleteResult = IChatStreamResult | IChatMessageResult;
@@ -0,0 +1,235 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
// @ts-expect-error — sibling JS module without an adjacent .d.ts
import {
make_claude_tools,
make_openai_tools,
normalize_json_schema,
normalize_tools_object,
} from './FunctionCalling.js';
// All four exports are pure data transforms — these tests just feed
// inputs and check the normalized output, no service mocks involved.
// ── normalize_json_schema ───────────────────────────────────────────
describe('normalize_json_schema', () => {
it('returns the schema unchanged when falsy', () => {
expect(normalize_json_schema(undefined)).toBeUndefined();
expect(normalize_json_schema(null)).toBeNull();
});
it('returns object schemas without properties unchanged', () => {
const schema = { type: 'object' };
const out = normalize_json_schema(schema);
// Same reference is returned — no clone is made.
expect(out).toBe(schema);
});
it('recursively normalizes object property schemas', () => {
const schema = {
type: 'object',
properties: {
items: { type: 'array' },
nested: {
type: 'object',
properties: { inner: { type: 'array' } },
},
},
};
const out = normalize_json_schema(schema);
// Empty `items` is filled in for every array branch reachable
// from the root.
expect(out.properties.items.items).toEqual({});
expect(out.properties.nested.properties.inner.items).toEqual({});
});
it('fills in `items: {}` for arrays that omit it', () => {
expect(normalize_json_schema({ type: 'array' })).toEqual({
type: 'array',
items: {},
});
});
it('recursively normalizes the items schema of arrays', () => {
const schema = {
type: 'array',
items: {
type: 'object',
properties: { sub: { type: 'array' } },
},
};
const out = normalize_json_schema(schema);
expect(out.items.properties.sub.items).toEqual({});
});
});
// ── normalize_tools_object ──────────────────────────────────────────
describe('normalize_tools_object', () => {
it('keeps OpenAI Responses web_search tools as-is', () => {
const tools = [{ type: 'web_search' }];
const out = normalize_tools_object(tools);
expect(out).toEqual([{ type: 'web_search' }]);
});
it('wraps a Claude-style {name, input_schema} tool into OpenAI shape', () => {
const out = normalize_tools_object([
{
name: 'lookup',
description: 'Search the docs',
input_schema: {
type: 'object',
properties: { q: { type: 'string' } },
},
},
]);
expect(out).toEqual([
{
type: 'function',
function: {
name: 'lookup',
description: 'Search the docs',
parameters: {
type: 'object',
properties: { q: { type: 'string' } },
},
},
},
]);
});
it('unwraps an OpenAI-style {type:"function", function:{...}} tool', () => {
const out = normalize_tools_object([
{
type: 'function',
function: {
name: 'lookup',
parameters: {
type: 'object',
properties: { q: { type: 'string' } },
},
},
},
]);
expect(out[0].type).toBe('function');
expect(out[0].function.name).toBe('lookup');
expect(out[0].function.parameters.properties.q).toEqual({
type: 'string',
});
});
it('defaults missing `parameters` to `{type: "object"}`', () => {
const out = normalize_tools_object([
{ type: 'function', function: { name: 'noargs' } },
]);
expect(out[0].function.parameters).toEqual({ type: 'object' });
});
it('infers `parameters.type = "object"` when missing', () => {
const out = normalize_tools_object([
{
type: 'function',
function: {
name: 'lookup',
parameters: { properties: { q: { type: 'string' } } },
},
},
]);
expect(out[0].function.parameters.type).toBe('object');
});
it('falls back to wrapping `tool` itself for bare/unknown shapes', () => {
// No `input_schema`, no `type === "function"` — accepted as the
// function definition itself.
const out = normalize_tools_object([
{
name: 'bare',
parameters: { type: 'object', properties: {} },
},
]);
expect(out[0].type).toBe('function');
expect(out[0].function.name).toBe('bare');
});
it('mutates the array in place and returns the same reference', () => {
const tools = [
{ name: 'lookup', input_schema: { type: 'object' } },
];
const out = normalize_tools_object(tools);
expect(out).toBe(tools);
});
});
// ── make_openai_tools ───────────────────────────────────────────────
describe('make_openai_tools', () => {
it('is the identity function (normalized format already matches OpenAI)', () => {
const tools = [
{
type: 'function',
function: {
name: 'lookup',
parameters: { type: 'object' },
},
},
];
expect(make_openai_tools(tools)).toBe(tools);
});
});
// ── make_claude_tools ───────────────────────────────────────────────
describe('make_claude_tools', () => {
it('returns undefined when tools is undefined', () => {
expect(make_claude_tools(undefined)).toBeUndefined();
});
it('returns [] when tools is an empty array', () => {
expect(make_claude_tools([])).toEqual([]);
});
it('flattens the OpenAI {function:{...}} wrapper into Claude shape', () => {
const out = make_claude_tools([
{
type: 'function',
function: {
name: 'lookup',
description: 'Search the docs',
parameters: {
type: 'object',
properties: { q: { type: 'string' } },
},
},
},
]);
expect(out).toEqual([
{
name: 'lookup',
description: 'Search the docs',
input_schema: {
type: 'object',
properties: { q: { type: 'string' } },
},
},
]);
});
});
@@ -0,0 +1,360 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
// @ts-expect-error — sibling JS module without an adjacent .d.ts
import {
extract_and_remove_system_messages,
extract_text,
normalize_messages,
normalize_single_message,
} from './Messages.js';
// All four exports are pure data transforms over arrays/objects, so
// these tests just feed inputs and assert on the output shape — no
// service mocks or method spies are needed.
// ── normalize_single_message ────────────────────────────────────────
describe('normalize_single_message', () => {
it('wraps a string into a single-content user message by default', () => {
const result = normalize_single_message('hello');
expect(result.role).toBe('user');
expect(result.content).toEqual([{ type: 'text', text: 'hello' }]);
});
it('honors a caller-supplied default role for string input', () => {
const result = normalize_single_message('greetings', {
role: 'system',
});
expect(result.role).toBe('system');
});
it('throws 400 when message is null/undefined/array', () => {
expect(() => normalize_single_message(null)).toThrow(
expect.objectContaining({ statusCode: 400 }),
);
expect(() => normalize_single_message(undefined)).toThrow(
expect.objectContaining({ statusCode: 400 }),
);
expect(() => normalize_single_message([])).toThrow(
expect.objectContaining({ statusCode: 400 }),
);
});
it('throws 400 when no content + no tool_calls (and not a tool message)', () => {
expect(() =>
normalize_single_message({ role: 'assistant' }),
).toThrow(expect.objectContaining({ statusCode: 400 }));
});
it('synthesizes content from tool_calls when content is missing', () => {
const result = normalize_single_message({
role: 'assistant',
tool_calls: [
{
id: 'call_1',
function: { name: 'lookup', arguments: { q: 'puter' } },
},
{
id: 'call_2',
function: { name: 'lookup', arguments: { q: 'fs' } },
},
],
});
expect(result.tool_calls).toBeUndefined();
expect(result.content).toEqual([
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: { q: 'puter' },
},
{
type: 'tool_use',
id: 'call_2',
name: 'lookup',
input: { q: 'fs' },
},
]);
});
it('coerces string tool content into a tool_result block', () => {
const result = normalize_single_message({
role: 'tool',
tool_call_id: 'call_1',
content: 'tool said hi',
});
expect(result.tool_use_id).toBe('call_1');
expect(result.content).toEqual([
{
type: 'tool_result',
tool_use_id: 'call_1',
content: 'tool said hi',
},
]);
});
it('JSON-serializes non-string tool content', () => {
const result = normalize_single_message({
role: 'tool',
tool_call_id: 'call_1',
content: { ok: true, data: [1, 2] },
});
expect(result.content[0].content).toBe(
JSON.stringify({ ok: true, data: [1, 2] }),
);
});
it('preserves the existing role when present', () => {
const result = normalize_single_message(
{ role: 'assistant', content: 'hi' },
{ role: 'system' },
);
expect(result.role).toBe('assistant');
});
it('upgrades string content blocks into typed text blocks', () => {
const result = normalize_single_message({
role: 'user',
content: ['hello', 'world'],
});
expect(result.content).toEqual([
{ type: 'text', text: 'hello' },
{ type: 'text', text: 'world' },
]);
});
it('infers type=text when only a `text` field is present', () => {
const result = normalize_single_message({
role: 'user',
content: [{ text: 'untyped' }],
});
expect(result.content[0].type).toBe('text');
});
it('throws 400 when a content item is not a string or object', () => {
expect(() =>
normalize_single_message({
role: 'user',
content: [42],
}),
).toThrow(expect.objectContaining({ statusCode: 400 }));
});
it('strips a stray `text` from tool_use blocks', () => {
const result = normalize_single_message({
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: {},
text: 'should be removed',
},
],
});
expect(result.content[0].text).toBeUndefined();
});
});
// ── normalize_messages ──────────────────────────────────────────────
describe('normalize_messages', () => {
it('normalizes each entry and merges consecutive same-role text messages', () => {
const result = normalize_messages([
'hi',
'there',
{ role: 'assistant', content: 'hello back' },
{ role: 'assistant', content: 'I can help' },
]);
// Two user text blocks merge; two assistant text blocks merge.
expect(result).toHaveLength(2);
expect(result[0].role).toBe('user');
expect(result[0].content).toEqual([
{ type: 'text', text: 'hi' },
{ type: 'text', text: 'there' },
]);
expect(result[1].role).toBe('assistant');
expect(result[1].content).toEqual([
{ type: 'text', text: 'hello back' },
{ type: 'text', text: 'I can help' },
]);
});
it('splits multi-block non-tool messages into one message per block', () => {
const result = normalize_messages([
{
role: 'user',
content: [
{ type: 'text', text: 'hello' },
{ type: 'image_url', image_url: 'https://x/a.png' },
],
},
]);
// After split the two user blocks collapse back into one merged
// message because both have role='user' and neither is a
// tool block.
expect(result).toHaveLength(1);
expect(result[0].content).toHaveLength(2);
});
it('keeps assistant tool_use blocks together (no per-block split)', () => {
const result = normalize_messages([
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: { q: 'a' },
},
{
type: 'tool_use',
id: 'call_2',
name: 'lookup',
input: { q: 'b' },
},
],
},
]);
expect(result).toHaveLength(1);
expect(result[0].content).toHaveLength(2);
// Both tool_use blocks live under the same assistant message
// so OpenAI's tool-call ordering is preserved on the wire.
expect(result[0].content[0].id).toBe('call_1');
expect(result[0].content[1].id).toBe('call_2');
});
it('does not merge across tool_use / tool_result boundaries', () => {
const result = normalize_messages([
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: {},
},
],
},
{
role: 'user',
content: [
{
type: 'tool_result',
tool_use_id: 'call_1',
content: 'ok',
},
],
},
{ role: 'user', content: 'and another follow-up' },
]);
// tool_use, tool_result, and the trailing user text must remain
// as separate messages — merging would clobber tool ordering.
expect(result).toHaveLength(3);
expect(result[0].content[0].type).toBe('tool_use');
expect(result[1].content[0].type).toBe('tool_result');
expect(result[2].content[0]).toEqual({
type: 'text',
text: 'and another follow-up',
});
});
});
// ── extract_and_remove_system_messages ─────────────────────────────
describe('extract_and_remove_system_messages', () => {
it('returns [system, non-system] preserving relative order', () => {
const messages = [
{ role: 'system', content: 'sys-1' },
{ role: 'user', content: 'u-1' },
{ role: 'system', content: 'sys-2' },
{ role: 'assistant', content: 'a-1' },
{ role: 'user', content: 'u-2' },
];
const [systems, others] = extract_and_remove_system_messages(messages);
expect(systems.map((m: Record<string, unknown>) => m.content)).toEqual([
'sys-1',
'sys-2',
]);
expect(others.map((m: Record<string, unknown>) => m.content)).toEqual([
'u-1',
'a-1',
'u-2',
]);
});
it('returns empty arrays for an empty input', () => {
const [systems, others] = extract_and_remove_system_messages([]);
expect(systems).toEqual([]);
expect(others).toEqual([]);
});
});
// ── extract_text ────────────────────────────────────────────────────
describe('extract_text', () => {
it('joins string messages with a single space', () => {
expect(extract_text(['hello', 'world'])).toBe('hello world');
});
it('skips falsy / non-object entries by emitting an empty string', () => {
// null / array entries collapse to '' and don't crash.
expect(extract_text([null, undefined, ['nope'], 'kept'])).toBe(
' kept',
);
});
it('joins content arrays of {text} blocks with spaces, then space-joins messages', () => {
const result = extract_text([
{ content: [{ text: 'a' }, { text: 'b' }] },
{ content: [{ text: 'c' }] },
]);
expect(result).toBe('a b c');
});
it('passes through messages whose content is a plain string', () => {
expect(extract_text([{ content: 'plain' }])).toBe('plain');
});
it('reads single-block content objects with type=text', () => {
expect(extract_text([{ content: { type: 'text', text: 'one' } }])).toBe(
'one',
);
});
it('returns "" for non-text typed single-block content', () => {
expect(
extract_text([{ content: { type: 'image_url', image_url: 'x' } }]),
).toBe('');
});
it('throws 400 when a typed text block has a non-string text field', () => {
expect(() =>
extract_text([{ content: { type: 'text', text: 42 } }]),
).toThrow(expect.objectContaining({ statusCode: 400 }));
});
});
@@ -0,0 +1,796 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Writable } from 'node:stream';
import { describe, expect, it, vi } from 'vitest';
// @ts-expect-error — sibling JS module without an adjacent .d.ts
import {
create_chat_stream_handler,
create_chat_stream_handler_responses_api,
create_usage_calculator,
extractMeteredUsage,
handle_completion_output,
handle_completion_output_responses_api,
process_input_messages,
process_input_messages_responses_api,
} from './OpenAIUtil.js';
// @ts-expect-error — sibling JS module without an adjacent .d.ts
import { AIChatStream } from './Streaming.js';
// ── Stream test harness ─────────────────────────────────────────────
//
// These stream handlers (and AIChatStream itself) write
// newline-delimited JSON into a Writable. Tests use a real
// `AIChatStream` wired to a buffering Writable, then parse the
// captured chunks back out so assertions can inspect the live
// shape — no method spies on the stream classes.
const makeCapturingChatStream = () => {
const chunks: string[] = [];
const sink = new Writable({
write(chunk, _enc, cb) {
chunks.push(chunk.toString('utf8'));
cb();
},
});
const chatStream = new AIChatStream({ stream: sink });
return {
chatStream,
events: () =>
chunks
.join('')
.split('\n')
.filter(Boolean)
.map((line) => JSON.parse(line)),
};
};
const asAsyncIterable = <T>(items: T[]): AsyncIterable<T> => ({
async *[Symbol.asyncIterator]() {
for (const item of items) {
yield item;
}
},
});
// ── process_input_messages ──────────────────────────────────────────
describe('process_input_messages', () => {
it('infers `image_url` and `video_url` types when missing', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'user',
content: [
{ image_url: 'https://cdn.test/a.png' },
{ video_url: 'https://cdn.test/a.mp4' },
{ type: 'text', text: 'hello' },
],
},
];
const result = (await process_input_messages(messages)) as Array<
Record<string, unknown>
>;
const content = (result[0]!.content ?? []) as Array<
Record<string, unknown>
>;
expect(content[0]?.type).toBe('image_url');
expect(content[1]?.type).toBe('video_url');
// Existing typed blocks are left alone.
expect(content[2]?.type).toBe('text');
});
it('hoists tool_use blocks into top-level tool_calls and clears content', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: { q: 'puter' },
},
],
},
];
const [out] = (await process_input_messages(messages)) as Array<
Record<string, unknown>
>;
// tool_calls flipped to OpenAI shape; content is null when this
// message was originally just a tool call.
expect(out!.content).toBeNull();
expect(out!.tool_calls).toEqual([
{
id: 'call_1',
type: 'function',
function: {
name: 'lookup',
// input is JSON-stringified for the OpenAI wire format.
arguments: JSON.stringify({ q: 'puter' }),
},
},
]);
});
it('preserves extra_content on hoisted tool calls', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_2',
name: 'lookup',
input: {},
extra_content: { hint: 'metadata' },
},
],
},
];
const [out] = (await process_input_messages(messages)) as Array<
Record<string, unknown>
>;
const calls = out!.tool_calls as Array<Record<string, unknown>>;
expect(calls[0]?.extra_content).toEqual({ hint: 'metadata' });
});
it('coerces tool_result blocks into a top-level tool message', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'user',
content: [
{
type: 'tool_result',
tool_use_id: 'call_1',
content: 'result body',
},
],
},
];
const [out] = (await process_input_messages(messages)) as Array<
Record<string, unknown>
>;
expect(out!.role).toBe('tool');
expect(out!.tool_call_id).toBe('call_1');
expect(out!.content).toBe('result body');
});
it('skips messages with falsy or non-object content', async () => {
const messages = [
{ role: 'system', content: 'hello' },
{ role: 'user', content: null },
];
const out = (await process_input_messages(messages)) as Array<
Record<string, unknown>
>;
// Strings + null are passed through unchanged.
expect(out[0]?.content).toBe('hello');
expect(out[1]?.content).toBeNull();
});
});
// ── process_input_messages_responses_api ────────────────────────────
describe('process_input_messages_responses_api', () => {
it('rewrites tool messages into function_call_output', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'tool',
tool_call_id: 'call_1',
content: 'tool said hi',
},
];
const [out] = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
expect(out!.type).toBe('function_call_output');
expect(out!.call_id).toBe('call_1');
expect(out!.output).toBe('tool said hi');
// Original tool-shape fields are stripped.
expect(out!.role).toBeUndefined();
expect(out!.content).toBeUndefined();
expect(out!.tool_call_id).toBeUndefined();
});
it('serializes complex tool content into the output string', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'tool',
tool_call_id: 'call_1',
content: [
{ text: 'part-a' },
'part-b',
{ content: 'part-c' },
],
},
];
const [out] = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
expect(out!.output).toBe('part-apart-bpart-c');
});
it('upgrades user/system text blocks to input_text', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'user',
content: [{ type: 'text', text: 'hello' }],
},
{
role: 'system',
content: [{ type: 'text', text: 'sys' }],
},
];
const out = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
const userBlocks = out[0]!.content as Array<Record<string, unknown>>;
const sysBlocks = out[1]!.content as Array<Record<string, unknown>>;
expect(userBlocks[0]?.type).toBe('input_text');
expect(sysBlocks[0]?.type).toBe('input_text');
});
it('upgrades assistant text blocks to output_text', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [{ type: 'text', text: 'hi from gpt' }],
},
];
const out = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
const blocks = out[0]!.content as Array<Record<string, unknown>>;
expect(blocks[0]?.type).toBe('output_text');
});
it('hoists a single assistant tool_use into a top-level function_call', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
canonical_id: 'fc_1',
name: 'lookup',
input: { q: 'puter' },
},
],
},
];
const [out] = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
expect(out!.type).toBe('function_call');
expect(out!.call_id).toBe('call_1');
expect(out!.id).toBe('fc_1');
expect(out!.name).toBe('lookup');
expect(out!.arguments).toBe(JSON.stringify({ q: 'puter' }));
expect(out!.role).toBeUndefined();
expect(out!.content).toBeUndefined();
});
it('rewrites tool_result blocks into function_call_output', async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'user',
content: [
{
type: 'tool_result',
tool_use_id: 'call_1',
content: 'result',
},
],
},
];
const [out] = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
expect(out!.type).toBe('function_call_output');
expect(out!.call_id).toBe('call_1');
expect(out!.output).toBe('result');
});
});
// ── create_usage_calculator ─────────────────────────────────────────
describe('create_usage_calculator', () => {
it('emits prompt + completion token rows priced by model_details.cost', () => {
const calc = create_usage_calculator({
model_details: {
id: 'gpt-test',
cost: { input: 0.01, output: 0.02 },
},
});
const tokens = calc({
usage: { prompt_tokens: 100, completion_tokens: 50 },
});
expect(tokens).toEqual([
{
type: 'prompt',
model: 'gpt-test',
amount: 100,
cost: 100 * 0.01,
},
{
type: 'completion',
model: 'gpt-test',
amount: 50,
cost: 50 * 0.02,
},
]);
});
});
// ── extractMeteredUsage ─────────────────────────────────────────────
describe('extractMeteredUsage', () => {
it('extracts prompt/completion/cached counts with safe defaults', () => {
expect(
extractMeteredUsage({
prompt_tokens: 10,
completion_tokens: 5,
prompt_tokens_details: { cached_tokens: 3 },
}),
).toEqual({
prompt_tokens: 10,
completion_tokens: 5,
cached_tokens: 3,
});
});
it('defaults missing fields to 0', () => {
expect(extractMeteredUsage({})).toEqual({
prompt_tokens: 0,
completion_tokens: 0,
cached_tokens: 0,
});
});
});
// ── create_chat_stream_handler ──────────────────────────────────────
describe('create_chat_stream_handler', () => {
it('streams text deltas through to a `text` content block', async () => {
const completion = asAsyncIterable([
{ choices: [{ delta: { content: 'hel' } }] },
{ choices: [{ delta: { content: 'lo' } }] },
{ choices: [{ delta: {} }], usage: { prompt_tokens: 1 } },
]);
const init = create_chat_stream_handler({
deviations: undefined,
completion,
usage_calculator: ({ usage }: { usage: unknown }) => ({
forwarded: usage,
}),
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const events = harness.events();
// Deltas land as separate text events.
const textEvents = events.filter((e) => e.type === 'text');
expect(textEvents.map((e) => e.text)).toEqual(['hel', 'lo']);
// Final usage event uses the calculator output.
const usageEvent = events.find((e) => e.type === 'usage');
expect(usageEvent?.usage).toEqual({ forwarded: { prompt_tokens: 1 } });
});
it('emits a separate reasoning event for `reasoning_content`', async () => {
const completion = asAsyncIterable([
{ choices: [{ delta: { reasoning_content: 'thinking…' } }] },
{ choices: [{ delta: { content: 'done' } }] },
{ choices: [{ delta: {} }], usage: { prompt_tokens: 1 } },
]);
const init = create_chat_stream_handler({
deviations: undefined,
completion,
usage_calculator: () => ({}),
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const events = harness.events();
expect(events.some((e) => e.type === 'reasoning')).toBe(true);
const reasoning = events.find((e) => e.type === 'reasoning');
expect(reasoning?.reasoning).toBe('thinking…');
});
it('builds a tool_use block from streamed function-call deltas', async () => {
const completion = asAsyncIterable([
{
choices: [
{
delta: {
tool_calls: [
{
index: 0,
id: 'call_1',
function: {
name: 'lookup',
arguments: '{"q":',
},
},
],
},
},
],
},
{
choices: [
{
delta: {
tool_calls: [
{
index: 0,
function: { arguments: '"puter"}' },
},
],
},
},
],
},
{ choices: [{ delta: {} }], usage: { prompt_tokens: 2 } },
]);
const init = create_chat_stream_handler({
deviations: undefined,
completion,
usage_calculator: () => ({}),
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const events = harness.events();
const toolEvent = events.find((e) => e.type === 'tool_use');
expect(toolEvent).toBeDefined();
expect(toolEvent?.id).toBe('call_1');
expect(toolEvent?.name).toBe('lookup');
// Buffered partial-JSON is parsed once on `.end()`.
expect(toolEvent?.input).toEqual({ q: 'puter' });
});
it('honors the deviations.chunk_but_like_actually unwrap', async () => {
// Mistral wraps each chunk under `.data`.
const completion = asAsyncIterable([
{ data: { choices: [{ delta: { content: 'hi' } }] } },
{
data: {
choices: [{ delta: {} }],
usage: { prompt_tokens: 1 },
},
},
]);
const init = create_chat_stream_handler({
deviations: {
chunk_but_like_actually: (
chunk: { data: Record<string, unknown> },
) => chunk.data,
},
completion,
usage_calculator: ({ usage }: { usage: unknown }) => usage,
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const events = harness.events();
expect(events.some((e) => e.type === 'text' && e.text === 'hi')).toBe(
true,
);
const usage = events.find((e) => e.type === 'usage');
expect(usage?.usage).toEqual({ prompt_tokens: 1 });
});
});
// ── create_chat_stream_handler_responses_api ────────────────────────
describe('create_chat_stream_handler_responses_api', () => {
it('streams text from response.output_text.delta chunks', async () => {
const completion = asAsyncIterable([
{ type: 'response.output_text.delta', delta: 'hel' },
{ type: 'response.output_text.delta', delta: 'lo' },
{
type: 'response.completed',
response: { usage: { input_tokens: 1, output_tokens: 2 } },
},
]);
const init = create_chat_stream_handler_responses_api({
deviations: undefined,
completion,
usage_calculator: ({ usage }: { usage: unknown }) => ({
forwarded: usage,
}),
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const events = harness.events();
const textEvents = events.filter((e) => e.type === 'text');
expect(textEvents.map((e) => e.text)).toEqual(['hel', 'lo']);
const usage = events.find((e) => e.type === 'usage');
expect(usage?.usage).toEqual({
forwarded: { input_tokens: 1, output_tokens: 2 },
});
});
it('emits a tool_use block when a function_call output_item completes', async () => {
const completion = asAsyncIterable([
{
type: 'response.output_item.done',
item: {
type: 'function_call',
id: 'fc_1',
call_id: 'call_1',
name: 'lookup',
arguments: '{"q":"puter"}',
},
},
{
type: 'response.completed',
response: { usage: { input_tokens: 1, output_tokens: 2 } },
},
]);
const init = create_chat_stream_handler_responses_api({
deviations: undefined,
completion,
usage_calculator: () => ({}),
});
const harness = makeCapturingChatStream();
await init({ chatStream: harness.chatStream });
const events = harness.events();
const tool = events.find((e) => e.type === 'tool_use');
expect(tool).toBeDefined();
expect(tool?.id).toBe('call_1');
expect(tool?.canonical_id).toBe('fc_1');
expect(tool?.name).toBe('lookup');
expect(tool?.input).toEqual({ q: 'puter' });
});
});
// ── handle_completion_output (non-stream) ───────────────────────────
describe('handle_completion_output non-stream', () => {
it('returns the first choice with usage from the calculator', async () => {
const completion = {
choices: [
{
message: { content: 'hello there' },
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 10, completion_tokens: 5 },
};
const result = await handle_completion_output({
deviations: undefined,
stream: false,
completion,
moderate: undefined,
usage_calculator: ({ usage }: { usage: unknown }) => ({
forwarded: usage,
}),
finally_fn: undefined,
});
expect(result.message.content).toBe('hello there');
expect(result.usage).toEqual({
forwarded: { prompt_tokens: 10, completion_tokens: 5 },
});
});
it('falls back to a raw input/output token shape when no calculator', async () => {
const completion = {
choices: [{ message: { content: 'ok' } }],
usage: { prompt_tokens: 1, completion_tokens: 2 },
};
const result = await handle_completion_output({
deviations: undefined,
stream: false,
completion,
});
expect(result.usage).toEqual({ input_tokens: 1, output_tokens: 2 });
});
it('throws 400 when moderation flags the completion text', async () => {
const completion = {
choices: [{ message: { content: 'banned content' } }],
usage: { prompt_tokens: 1, completion_tokens: 1 },
};
const moderate = vi.fn(async () => ({ flagged: true }));
await expect(
handle_completion_output({
deviations: undefined,
stream: false,
completion,
moderate,
}),
).rejects.toMatchObject({ statusCode: 400 });
expect(moderate).toHaveBeenCalledWith('banned content');
});
it('skips moderation when the completion content is null', async () => {
const completion = {
choices: [{ message: { content: null } }],
usage: { prompt_tokens: 1, completion_tokens: 0 },
};
const moderate = vi.fn(async () => ({ flagged: true }));
const result = await handle_completion_output({
deviations: undefined,
stream: false,
completion,
moderate,
});
expect(moderate).not.toHaveBeenCalled();
expect(result.message.content).toBeNull();
});
it('runs `finally_fn` before returning on the non-stream path', async () => {
const completion = {
choices: [{ message: { content: 'ok' } }],
usage: { prompt_tokens: 1, completion_tokens: 1 },
};
const finally_fn = vi.fn(async () => {});
await handle_completion_output({
deviations: undefined,
stream: false,
completion,
finally_fn,
});
expect(finally_fn).toHaveBeenCalled();
});
it('returns a stream init descriptor when stream=true (does not call finally_fn yet)', async () => {
const completion = asAsyncIterable([]);
const finally_fn = vi.fn(async () => {});
const result = await handle_completion_output({
deviations: undefined,
stream: true,
completion,
finally_fn,
});
expect(result.stream).toBe(true);
expect(typeof result.init_chat_stream).toBe('function');
// finally_fn is forwarded for the caller to invoke after streaming.
expect(result.finally_fn).toBe(finally_fn);
expect(finally_fn).not.toHaveBeenCalled();
});
it('honors deviations.coerce_completion_usage (Mistral shape)', async () => {
const completion = {
choices: [{ message: { content: 'ok' } }],
// Mistral wraps usage in a wrapper object — coerce drills in.
wrapper: { prompt_tokens: 7, completion_tokens: 3 },
};
const result = await handle_completion_output({
deviations: {
coerce_completion_usage: (
c: { wrapper: Record<string, unknown> },
) => c.wrapper,
},
stream: false,
completion,
});
expect(result.usage).toEqual({ input_tokens: 7, output_tokens: 3 });
});
});
// ── handle_completion_output_responses_api (non-stream) ─────────────
describe('handle_completion_output_responses_api non-stream', () => {
it('shapes responses.output_text into a v1 chat-completion message', async () => {
const completion = {
output: [{ role: 'assistant', type: 'message' }],
output_text: 'hi',
usage: { input_tokens: 1, output_tokens: 2 },
};
const result = await handle_completion_output_responses_api({
deviations: undefined,
stream: false,
completion,
});
expect(result.finish_reason).toBe('stop');
expect(result.message.content).toBe('hi');
expect(result.role).toBe('assistant');
expect(result.usage).toEqual({ input_tokens: 1, output_tokens: 2 });
// Sanity: no leftover `type` field bleeds into the response.
expect(result.type).toBeUndefined();
});
it('surfaces tool_calls from output[type=function_call] entries', async () => {
const completion = {
output: [
{
type: 'function_call',
id: 'fc_1',
call_id: 'call_1',
name: 'lookup',
arguments: '{"q":"puter"}',
},
],
output_text: '',
usage: { input_tokens: 1, output_tokens: 2 },
};
const result = await handle_completion_output_responses_api({
deviations: undefined,
stream: false,
completion,
});
expect(result.message.tool_calls).toEqual([
{
id: 'call_1',
type: 'function',
function: { name: 'lookup', arguments: '{"q":"puter"}' },
canonical_id: 'fc_1',
},
]);
});
it('throws 400 when output_text is empty AND there are no tool calls', async () => {
const completion = {
output: [],
output_text: ' ',
usage: { input_tokens: 1, output_tokens: 0 },
};
await expect(
handle_completion_output_responses_api({
deviations: undefined,
stream: false,
completion,
}),
).rejects.toMatchObject({ statusCode: 400 });
});
it('runs moderation against output_text when a moderate fn is supplied', async () => {
const completion = {
output: [{ role: 'assistant' }],
output_text: 'questionable content',
usage: { input_tokens: 1, output_tokens: 2 },
};
const moderate = vi.fn(async () => ({ flagged: true }));
await expect(
handle_completion_output_responses_api({
deviations: undefined,
stream: false,
completion,
moderate,
}),
).rejects.toMatchObject({ statusCode: 400 });
expect(moderate).toHaveBeenCalledWith('questionable content');
});
it('returns a stream init descriptor when stream=true', async () => {
const completion = asAsyncIterable([]);
const result = await handle_completion_output_responses_api({
deviations: undefined,
stream: true,
completion,
});
expect(result.stream).toBe(true);
expect(typeof result.init_chat_stream).toBe('function');
});
});
@@ -0,0 +1,239 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Writable } from 'node:stream';
import { describe, expect, it } from 'vitest';
// @ts-expect-error — sibling JS module without an adjacent .d.ts
import Streaming, {
AIChatMessageStream,
AIChatStream,
AIChatTextStream,
AIChatToolUseStream,
} from './Streaming.js';
// AIChatStream + friends emit newline-delimited JSON to an underlying
// Writable. Tests run them against a real buffering Writable and parse
// the captured chunks back, so assertions read the live wire shape —
// no method-level spies on the stream classes.
const makeHarness = () => {
const chunks: string[] = [];
let ended = false;
const sink = new Writable({
write(chunk, _enc, cb) {
chunks.push(chunk.toString('utf8'));
cb();
},
final(cb) {
ended = true;
cb();
},
});
const chatStream = new AIChatStream({ stream: sink });
return {
chatStream,
sink,
rawChunks: () => chunks.slice(),
events: () =>
chunks
.join('')
.split('\n')
.filter(Boolean)
.map((line) => JSON.parse(line)),
isEnded: () => ended,
};
};
// ── AIChatStream ────────────────────────────────────────────────────
describe('AIChatStream', () => {
it('exposes a Streaming default with AIChatStream attached', () => {
expect(Streaming.AIChatStream).toBe(AIChatStream);
});
it('writes a `usage` event and ends the underlying stream on .end()', () => {
const h = makeHarness();
h.chatStream.end({ tokens: 42 });
const events = h.events();
expect(events).toEqual([
{ type: 'usage', usage: { tokens: 42 } },
]);
expect(h.isEnded()).toBe(true);
});
it('forwards .write(...) calls straight to the underlying stream', () => {
const h = makeHarness();
h.chatStream.write('raw chunk\n');
// .write is a passthrough — the raw bytes hit the sink without
// being wrapped in an event envelope.
expect(h.rawChunks()).toEqual(['raw chunk\n']);
});
it('returns a fresh AIChatMessageStream from .message()', () => {
const h = makeHarness();
const m = h.chatStream.message();
expect(m).toBeInstanceOf(AIChatMessageStream);
});
});
// ── AIChatMessageStream / AIChatTextStream ─────────────────────────
describe('AIChatTextStream (via message().contentBlock)', () => {
it('emits a text event for addText with no extra_content', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({ type: 'text' });
block.addText('hello');
expect(h.events()).toEqual([{ type: 'text', text: 'hello' }]);
});
it('attaches extra_content when provided', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({ type: 'text' });
block.addText('hello', { meta: 1 });
expect(h.events()).toEqual([
{ type: 'text', text: 'hello', extra_content: { meta: 1 } },
]);
});
it('emits a separate reasoning event from addReasoning', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({ type: 'text' });
block.addReasoning('thinking…');
expect(h.events()).toEqual([
{ type: 'reasoning', reasoning: 'thinking…' },
]);
});
it('emits an extra_content event from addExtraContent', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({ type: 'text' });
block.addExtraContent({ tag: 'gemini-meta' });
expect(h.events()).toEqual([
{ type: 'extra_content', extra_content: { tag: 'gemini-meta' } },
]);
});
it('exposes AIChatTextStream as the constructor for type=text', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({ type: 'text' });
expect(block).toBeInstanceOf(AIChatTextStream);
});
});
// ── AIChatToolUseStream ────────────────────────────────────────────
describe('AIChatToolUseStream (via message().contentBlock)', () => {
it('exposes AIChatToolUseStream as the constructor for type=tool_use', () => {
const h = makeHarness();
const block = h.chatStream
.message()
.contentBlock({ type: 'tool_use', id: 'call_1', name: 'lookup' });
expect(block).toBeInstanceOf(AIChatToolUseStream);
});
it('parses buffered partial JSON arguments on .end()', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({
type: 'tool_use',
id: 'call_1',
name: 'lookup',
});
block.addPartialJSON('{"q":');
block.addPartialJSON('"puter"}');
block.end();
expect(h.events()).toEqual([
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: { q: 'puter' },
text: '',
},
]);
});
it('forwards extra_content when supplied on the contentBlock spec', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({
type: 'tool_use',
id: 'call_2',
name: 'lookup',
extra_content: { hint: 'metadata' },
});
block.addPartialJSON('{}');
block.end();
const [event] = h.events();
expect(event.extra_content).toEqual({ hint: 'metadata' });
});
it('falls back to {} when nothing was buffered', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({
type: 'tool_use',
id: 'call_3',
name: 'lookup',
});
block.end();
const [event] = h.events();
expect(event.input).toEqual({});
});
it('omits the empty-text suffix when contentBlock already has text', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({
type: 'tool_use',
id: 'call_4',
name: 'lookup',
text: 'preserved',
});
block.addPartialJSON('{}');
block.end();
const [event] = h.events();
// The trailing-text empty-fill only happens when no `text` is
// already present on the spec.
expect(event.text).toBe('preserved');
});
it('throws when the buffered partial JSON is malformed', () => {
const h = makeHarness();
const block = h.chatStream.message().contentBlock({
type: 'tool_use',
id: 'call_5',
name: 'lookup',
});
block.addPartialJSON('not-json');
// .end() runs JSON.parse on the buffer — bad JSON surfaces here.
expect(() => block.end()).toThrow(SyntaxError);
});
});
// ── unknown content block type ──────────────────────────────────────
describe('AIChatMessageStream.contentBlock', () => {
it('throws on an unknown content block type', () => {
const h = makeHarness();
expect(() =>
h.chatStream.message().contentBlock({ type: 'audio' }),
).toThrow(/Unknown content block type/);
});
});
@@ -135,6 +135,9 @@ export class ImageGenerationDriver extends PuterDriver {
throw new HttpError(500, `No provider found for model ${model.id}`);
}
// `width`/`height` or `aspect_ratio` -> `ratio: {w,h}`
this.#normalizeRatio(args);
// Audit log for abuse / billing. Fired before the upstream call
// so a failed generate still shows up in the log (prompt_block
// uses this to track user-by-user image prompts).
@@ -159,6 +162,34 @@ export class ImageGenerationDriver extends PuterDriver {
});
}
#normalizeRatio(parameters: IGenerateParams) {
if (parameters.ratio) return;
const w = parameters.width as number | undefined;
const h = parameters.height as number | undefined;
if (typeof w === 'number' && typeof h === 'number') {
parameters.ratio = { w, h };
delete parameters.width;
delete parameters.height;
return;
}
const ar = parameters.aspect_ratio as string | undefined;
if (typeof ar === 'string' && ar.includes(':')) {
const [aw, ah] = ar.split(':').map(Number);
if (
Number.isFinite(aw) &&
Number.isFinite(ah) &&
aw > 0 &&
ah > 0
) {
parameters.ratio = { w: aw, h: ah };
delete parameters.aspect_ratio;
return;
}
}
}
#registerProviders() {
const providers = this.config.providers ?? {};
const m = this.services.metering;
@@ -0,0 +1,61 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the Gemini image generation provider.
*
* Uses `imagen-4.0-fast-generate-001` ($0.02/image — cheapest
* Gemini imagen variant). Skipped when `PUTER_TEST_AI_GEMINI_API_KEY`
* is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { GeminiImageProvider } from './GeminiImageProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_GEMINI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'GeminiImageProvider (integration)',
() => {
it('returns image data from imagen-4.0-fast', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new GeminiImageProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.generate({
model: 'imagen-4.0-fast-generate-001',
prompt: 'a tiny red dot on a white background',
ratio: { w: 1, h: 1 },
}),
);
expect(typeof result).toBe('string');
expect((result as string).length).toBeGreaterThan(0);
});
},
);
@@ -0,0 +1,61 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the OpenAI image generation provider.
*
* Uses `dall-e-2` at 256x256 — the cheapest OpenAI image
* configuration ($0.016/image). Skipped when
* `PUTER_TEST_AI_OPENAI_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { OpenAiImageProvider } from './OpenAiImageProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_OPENAI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'OpenAiImageProvider (integration)',
() => {
it('returns an image url/data from dall-e-2 at 256x256', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new OpenAiImageProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = await withTestActor(() =>
provider.generate({
model: 'dall-e-2',
prompt: 'a tiny red dot on a white background',
ratio: { w: 256, h: 256 },
}),
);
expect(typeof result).toBe('string');
expect((result as string).length).toBeGreaterThan(0);
});
},
);
@@ -33,20 +33,19 @@ import {
const DEFAULT_MODEL = 'black-forest-labs/flux-schnell';
const DEFAULT_RATIO = { w: 1024, h: 1024 };
type ReplicateGenerateParams = IGenerateParams & {
go_fast?: boolean;
seed?: number;
steps?: number;
guidance?: number;
output_quality?: number;
output_megapixels?: string;
prompt_strength?: number;
negative_prompt?: string;
response_format?: string;
disable_safety_checker?: boolean;
};
export class ReplicateImageGenerationProvider implements IImageProvider {
static readonly #CORE_PARAMS: readonly string[] = [
'prompt',
'model',
'ratio',
'quality',
'provider',
'test_mode',
'input_image',
'input_image_mime_type',
'input_images',
];
#client: Replicate;
#meteringService: MeteringService;
@@ -67,11 +66,10 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
}
async generate(params: IGenerateParams): Promise<string> {
const extra = params as ReplicateGenerateParams;
const { prompt, test_mode } = extra;
const { prompt, test_mode } = params;
const selectedModel = this.#getModel(extra.model);
const ratio = this.#normalizeRatio(extra.ratio);
const selectedModel = this.#getModel(params.model);
const ratio = this.#normalizeRatio(params.ratio);
if (test_mode) {
return 'https://puter-sample-data.puter.site/image_example.png';
@@ -90,34 +88,61 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
});
}
const goFast = selectedModel.supportsGoFast
? extra.go_fast !== undefined
? !!extra.go_fast
: (selectedModel.goFastDefault ?? false)
: false;
const filtered = this.#filterAllowedParams(params, selectedModel);
const aliased = this.#applyParamAliases(filtered, selectedModel);
const transformed = this.#applyTransforms(aliased, selectedModel);
const goFast = !!transformed.go_fast;
const generationMode =
typeof transformed.generation_mode === 'string'
? transformed.generation_mode
: undefined;
const inputImages: string[] = [];
if (selectedModel.imageInputKey) {
if (extra.input_image) inputImages.push(extra.input_image);
if (extra.input_images?.length)
inputImages.push(...extra.input_images);
if (params.input_image) inputImages.push(params.input_image);
if (params.input_images?.length)
inputImages.push(...params.input_images);
if (inputImages.length === 0) {
const nativeVal = (params as Record<string, unknown>)[
selectedModel.imageInputKey
];
if (typeof nativeVal === 'string') {
inputImages.push(nativeVal);
} else if (Array.isArray(nativeVal)) {
for (const v of nativeVal) {
if (typeof v === 'string') inputImages.push(v);
}
}
}
}
let singleImage: string | undefined;
if (selectedModel.singleImageInputKey) {
if (typeof params.input_image === 'string') {
singleImage = params.input_image;
} else {
const nativeVal = (params as Record<string, unknown>)[
selectedModel.singleImageInputKey
];
if (typeof nativeVal === 'string') singleImage = nativeVal;
}
}
const singleImage = selectedModel.singleImageInputKey
? extra.input_image
: undefined;
const allInputUrls = singleImage ? [singleImage] : inputImages;
const inputMp =
allInputUrls.length > 0
? await this.#measureInputMegapixels(allInputUrls)
: 0;
const outputMp = this.#resolveOutputMegapixels(extra.output_megapixels);
const outputMp = this.#resolveOutputMegapixels(
params.output_megapixels as string | undefined,
);
const totalCostMicroCents = this.#estimateCost(
selectedModel,
outputMp,
goFast,
inputMp,
generationMode,
);
if (totalCostMicroCents <= 0) {
throw new HttpError(
@@ -140,42 +165,13 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
);
}
const input: Record<string, unknown> = {
const input = this.#buildRequest(selectedModel, {
prompt,
aspect_ratio: this.#toAspectRatio(ratio),
disable_safety_checker: !!extra.disable_safety_checker,
};
if (selectedModel.supportsGoFast) {
input.go_fast = goFast;
}
if (inputImages.length && selectedModel.imageInputKey) {
input[selectedModel.imageInputKey] = inputImages;
} else if (singleImage && selectedModel.singleImageInputKey) {
input[selectedModel.singleImageInputKey] = singleImage;
}
if (Number.isFinite(extra.seed))
input.seed = Math.round(extra.seed as number);
if (Number.isFinite(extra.steps))
input.num_inference_steps = Math.round(extra.steps as number);
if (Number.isFinite(extra.guidance)) input.guidance = extra.guidance;
if (Number.isFinite(extra.output_quality))
input.output_quality = Math.round(extra.output_quality as number);
if (
typeof extra.output_megapixels === 'string' &&
selectedModel.resolutionInputKey
) {
input[selectedModel.resolutionInputKey] =
extra.output_megapixels +
(selectedModel.resolutionSuffix ?? '');
} else if (typeof extra.output_megapixels === 'string') {
input.megapixels = extra.output_megapixels;
}
if (Number.isFinite(extra.prompt_strength))
input.prompt_strength = extra.prompt_strength;
if (typeof extra.negative_prompt === 'string')
input.negative_prompt = extra.negative_prompt;
if (typeof extra.response_format === 'string')
input.output_format = extra.response_format;
ratio,
transformed,
inputImages,
singleImage,
});
const output = await this.#client.run(
selectedModel.replicateId as `${string}/${string}`,
@@ -191,7 +187,14 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
);
}
this.#recordUsage(actor, selectedModel, outputMp, goFast, inputMp);
this.#recordUsage(
actor,
selectedModel,
outputMp,
goFast,
inputMp,
generationMode,
);
return url;
}
@@ -204,6 +207,130 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
return found ?? models.find((m) => m.id === DEFAULT_MODEL)!;
}
/**
* Builds the Replicate API input payload from already-aliased+transformed
* params. Image inputs and prompt/ratio are placed explicitly; everything
* else is spread verbatim so newly-allowed keys flow through without
* needing a code change here.
*/
#buildRequest(
model: ReplicateImageModel,
ctx: {
prompt: string;
ratio: { w: number; h: number };
transformed: Record<string, unknown>;
inputImages: string[];
singleImage?: string;
},
): Record<string, unknown> {
const { prompt, ratio, transformed, inputImages, singleImage } = ctx;
const input: Record<string, unknown> = {
prompt,
aspect_ratio: this.#toAspectRatio(ratio),
};
const handled = new Set<string>(
ReplicateImageGenerationProvider.#CORE_PARAMS,
);
if (model.imageInputKey) handled.add(model.imageInputKey);
if (model.singleImageInputKey) handled.add(model.singleImageInputKey);
if (inputImages.length && model.imageInputKey) {
input[model.imageInputKey] = inputImages;
} else if (singleImage && model.singleImageInputKey) {
input[model.singleImageInputKey] = singleImage;
}
for (const [key, value] of Object.entries(transformed)) {
if (handled.has(key)) continue;
if (value === undefined || value === null) continue;
input[key] = value;
}
return input;
}
/**
* Drops params not in `model.allowed_params` (plus `#CORE_PARAMS` and any
* alias targets, so the native key is also accepted).
*/
#filterAllowedParams(
params: IGenerateParams,
model: ReplicateImageModel,
): IGenerateParams {
const allowedSet = model.allowed_params;
if (!allowedSet) return params;
const aliasTargets = model.param_aliases
? Object.values(model.param_aliases)
: [];
const nativeImageKeys: string[] = [];
if (model.imageInputKey) nativeImageKeys.push(model.imageInputKey);
if (model.singleImageInputKey)
nativeImageKeys.push(model.singleImageInputKey);
const filtered: Record<string, unknown> = {};
for (const key of Object.keys(params)) {
if (
ReplicateImageGenerationProvider.#CORE_PARAMS.includes(key) ||
allowedSet.includes(key) ||
aliasTargets.includes(key) ||
nativeImageKeys.includes(key)
) {
filtered[key] = params[key];
}
}
return filtered as IGenerateParams;
}
/**
* Renames canonical keys to the model's native API names per
* `model.param_aliases` (e.g. `steps` → `num_inference_steps`).
*/
#applyParamAliases(
params: IGenerateParams,
model: ReplicateImageModel,
): Record<string, unknown> {
const aliases = model.param_aliases;
if (!aliases) return params as Record<string, unknown>;
const result: Record<string, unknown> = {};
for (const [key, value] of Object.entries(params)) {
const nativeKey = aliases[key] ?? key;
result[nativeKey] = value;
}
return result;
}
/**
* Applies `param_transforms` on top of the aliased map: injects defaults
* for missing keys, then appends any configured string suffix to the
* value. Returns the original map unchanged when the model declares no
* transforms.
*/
#applyTransforms(
aliased: Record<string, unknown>,
model: ReplicateImageModel,
): Record<string, unknown> {
const transforms = model.param_transforms;
if (!transforms) return aliased;
const result = { ...aliased };
for (const [key, cfg] of Object.entries(transforms)) {
let value = result[key];
if (value === undefined && cfg.default !== undefined) {
value = cfg.default;
}
if (value === undefined) continue;
if (cfg.suffix !== undefined && typeof value === 'string') {
value = value + cfg.suffix;
}
result[key] = value;
}
return result;
}
#normalizeRatio(ratio?: { w: number; h: number }) {
const w = Number(ratio?.w);
const h = Number(ratio?.h);
@@ -253,10 +380,16 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
#resolveCosts(
model: ReplicateImageModel,
goFast: boolean,
generationMode?: string,
): Record<string, number> {
return goFast && model.costs_go_fast
? model.costs_go_fast
: model.costs;
if (goFast && model.costs_go_fast) return model.costs_go_fast;
if (
generationMode &&
model.costs_by_generation_mode?.[generationMode]
) {
return model.costs_by_generation_mode[generationMode];
}
return model.costs;
}
#estimateCost(
@@ -264,8 +397,9 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
outputMp: number,
goFast: boolean,
inputMp: number,
generationMode?: string,
): number {
const costs = this.#resolveCosts(model, goFast);
const costs = this.#resolveCosts(model, goFast, generationMode);
if (model.billingScheme === 'per-image') {
const cents = costs.output;
@@ -300,9 +434,10 @@ export class ReplicateImageGenerationProvider implements IImageProvider {
outputMp: number,
goFast: boolean,
inputMp: number,
generationMode?: string,
) {
const prefix = `replicate:${model.id}`;
const costs = this.#resolveCosts(model, goFast);
const costs = this.#resolveCosts(model, goFast, generationMode);
if (model.billingScheme === 'per-image') {
const cents = costs.output;
@@ -24,15 +24,23 @@ export type ReplicateBillingScheme = 'per-image' | 'megapixel';
export type ReplicateImageModel = IImageModel & {
replicateId: string;
billingScheme: ReplicateBillingScheme;
imageInputKey?: string;
singleImageInputKey?: string;
supportsGoFast?: boolean;
goFastDefault?: boolean;
imageInputKey?: string; // our `input_images`
singleImageInputKey?: string; // our `input_image`
/** Cost map used when `go_fast: true` (e.g. flux-2-dev fast mode). */
costs_go_fast?: Record<string, number>;
resolutionInputKey?: string;
resolutionSuffix?: string;
/** Cost maps keyed by Leonardo `generation_mode`; falls back to `costs`. */
costs_by_generation_mode?: Record<string, Record<string, number>>;
/** Whitelist of caller-supplied params (canonical names) that are forwarded to Replicate. */
allowed_params?: string[];
/** Renames canonical param keys to the model's native API names (e.g. `steps` → `num_inference_steps`). */
param_aliases?: Record<string, string>;
/** Per-key value transforms (default + suffix) applied after `param_aliases`. */
param_transforms?: Record<string, { suffix?: string; default?: unknown }>;
};
const ALIAS_FORMAT = { response_format: 'output_format' };
const ALIAS_FORMAT_STEPS = { ...ALIAS_FORMAT, steps: 'num_inference_steps' };
// Costs are in USD cents.
// Megapixel models: `output_mp` = cost per output megapixel, `input_mp` =
// cost per input megapixel (img2img). Some also carry a flat `run` cost.
@@ -50,8 +58,15 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
costs: { run: 1.5, input_mp: 1.5, output_mp: 1.5 },
billingScheme: 'megapixel',
imageInputKey: 'input_images',
resolutionInputKey: 'resolution',
resolutionSuffix: ' MP',
allowed_params: [
'seed',
'response_format',
'output_quality',
'output_megapixels',
'safety_tolerance',
],
param_aliases: { ...ALIAS_FORMAT, output_megapixels: 'resolution' },
param_transforms: { resolution: { suffix: ' MP' } },
},
{
id: 'black-forest-labs/flux-2-dev',
@@ -65,9 +80,15 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
costs_go_fast: { input_mp: 1.2, output_mp: 1.2 },
billingScheme: 'megapixel',
imageInputKey: 'input_images',
supportsGoFast: true,
goFastDefault: true,
resolutionInputKey: 'output_megapixels',
allowed_params: [
'seed',
'response_format',
'output_quality',
'disable_safety_checker',
'go_fast',
],
param_aliases: ALIAS_FORMAT,
param_transforms: { go_fast: { default: true } },
},
{
id: 'black-forest-labs/flux-2-klein-9b-base',
@@ -80,7 +101,15 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
costs: { input_mp: 1.1, output_mp: 1.1 },
billingScheme: 'megapixel',
imageInputKey: 'images',
resolutionInputKey: 'output_megapixels',
allowed_params: [
'seed',
'guidance',
'response_format',
'output_quality',
'disable_safety_checker',
'output_megapixels',
],
param_aliases: ALIAS_FORMAT,
},
{
id: 'black-forest-labs/flux-2-klein-4b',
@@ -93,7 +122,14 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
costs: { input_mp: 0.1, output_mp: 0.1 },
billingScheme: 'megapixel',
imageInputKey: 'images',
resolutionInputKey: 'output_megapixels',
allowed_params: [
'seed',
'response_format',
'output_quality',
'disable_safety_checker',
'output_megapixels',
],
param_aliases: ALIAS_FORMAT,
},
// Black Forest Labs FLUX.1
@@ -107,6 +143,18 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
index_cost_key: 'output',
costs: { output: 0.3 },
billingScheme: 'per-image',
allowed_params: [
'seed',
'steps',
'response_format',
'output_quality',
'disable_safety_checker',
'output_megapixels',
],
param_aliases: {
...ALIAS_FORMAT_STEPS,
output_megapixels: 'megapixels',
},
},
{
id: 'black-forest-labs/flux-1.1-pro',
@@ -119,6 +167,14 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
costs: { output: 4 },
billingScheme: 'per-image',
singleImageInputKey: 'image_prompt',
allowed_params: [
'seed',
'response_format',
'output_quality',
'safety_tolerance',
'prompt_upsampling',
],
param_aliases: ALIAS_FORMAT,
},
// Leonardo AI
@@ -130,8 +186,20 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
name: 'Lucid Origin',
costs_currency: 'usd-cents',
index_cost_key: 'output',
costs: { output: 1.65 },
costs: {
output: 1.65, // standard: 11 units * $0.0015/unit = $0.0165
},
costs_by_generation_mode: {
standard: { output: 1.65 },
ultra: { output: 7.65 }, // 51 units * $0.0015/unit = $0.0765
},
billingScheme: 'per-image',
allowed_params: [
'style',
'contrast',
'prompt_enhance',
'generation_mode',
],
},
{
id: 'leonardoai/phoenix-1.0',
@@ -141,7 +209,20 @@ export const REPLICATE_IMAGE_GENERATION_MODELS: ReplicateImageModel[] = [
name: 'Phoenix 1.0',
costs_currency: 'usd-cents',
index_cost_key: 'output',
costs: { output: 3.75 },
costs: {
output: 3.75, // quality default: 25 units * $0.0015/unit = $0.0375
},
costs_by_generation_mode: {
fast: { output: 1.8 }, // 12 units * $0.0015/unit = $0.018
quality: { output: 3.75 }, // 25 units * $0.0015/unit = $0.0375
ultra: { output: 7.5 }, // 50 units * $0.0015/unit = $0.075
},
billingScheme: 'per-image',
allowed_params: [
'style',
'contrast',
'prompt_enhance',
'generation_mode',
],
},
];
+1
View File
@@ -63,6 +63,7 @@ export interface IGenerateParams {
input_image?: string;
input_image_mime_type?: string;
input_images?: string[];
[key: string]: unknown;
}
export interface IImageProvider {
+1
View File
@@ -273,6 +273,7 @@ export class OCRDriver extends PuterDriver {
'MERGED_CELL',
'LAYOUT_FIGURE',
'LAYOUT_TEXT',
'WORD',
].includes(block.BlockType ?? '')
)
continue;
@@ -0,0 +1,64 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the ElevenLabs TTS provider.
*
* Uses `eleven_flash_v2_5` (the cheapest tier) with a tiny input.
* Skipped when `PUTER_TEST_AI_ELEVENLABS_API_KEY` is unset.
*/
import { Readable } from 'node:stream';
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { ElevenLabsTTSProvider } from './ElevenLabsTTSProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_ELEVENLABS_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'ElevenLabsTTSProvider (integration)',
() => {
it('returns an audio stream from eleven_flash_v2_5', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new ElevenLabsTTSProvider(makeMeteringStub(), {
apiKey: optionalEnv(ENV_VAR)!,
});
const result = (await withTestActor(() =>
provider.synthesize({
text: 'hi',
model: 'eleven_flash_v2_5',
}),
)) as { stream: Readable; content_type: string };
expect(result.stream).toBeInstanceOf(Readable);
const chunks: Buffer[] = [];
for await (const chunk of result.stream) {
chunks.push(chunk as Buffer);
}
const total = chunks.reduce((n, c) => n + c.length, 0);
expect(total).toBeGreaterThan(0);
});
},
);
@@ -0,0 +1,73 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the OpenAI TTS provider.
*
* Uses `tts-1` (the cheapest OpenAI TTS model) with a 2-character
* input to keep cost negligible. Skipped when
* `PUTER_TEST_AI_OPENAI_API_KEY` is unset.
*/
import { Readable } from 'node:stream';
import { describe, expect, it } from 'vitest';
import {
INTEGRATION_TEST_TIMEOUT_MS,
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { OpenAITTSProvider } from './OpenAITTSProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_OPENAI_API_KEY';
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'OpenAITTSProvider (integration)',
() => {
it('returns an audio stream from tts-1', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new OpenAITTSProvider(makeMeteringStub(), {
apiKey: optionalEnv(ENV_VAR)!,
});
const result = (await withTestActor(() =>
provider.synthesize({
text: 'hi',
model: 'tts-1',
voice: 'alloy',
response_format: 'mp3',
}),
)) as { stream: Readable; content_type: string };
expect(result).toMatchObject({
content_type: expect.stringContaining('audio'),
});
expect(result.stream).toBeInstanceOf(Readable);
// Drain the stream so we know real bytes came back, not an
// empty placeholder.
const chunks: Buffer[] = [];
for await (const chunk of result.stream) {
chunks.push(chunk as Buffer);
}
const total = chunks.reduce((n, c) => n + c.length, 0);
expect(total).toBeGreaterThan(0);
});
},
);
+40 -12
View File
@@ -103,7 +103,9 @@ export class AppDriver extends PuterDriver {
async create({ object, options } = {}) {
if (!object || typeof object !== 'object') {
throw new HttpError(400, 'Missing or invalid `object`');
throw new HttpError(400, 'Missing or invalid `object`', {
legacyCode: 'bad_request',
});
}
const actor = this.#requireActor();
this.#requireUserOrAppActor(actor);
@@ -140,7 +142,9 @@ export class AppDriver extends PuterDriver {
do {
candidate = `${fields.name}-${++n}`;
if (n > 50)
throw new HttpError(400, 'Failed to dedupe app name');
throw new HttpError(400, 'Failed to dedupe app name', {
legacyCode: 'bad_request',
});
} while (await this.appStore.existsByName(candidate));
fields.name = candidate;
} else {
@@ -174,7 +178,10 @@ export class AppDriver extends PuterDriver {
async read({ uid, id, params = {}, ...rest } = {}) {
const actor = this.#requireActor();
const app = await this.#resolve({ uid, id });
if (!app) throw new HttpError(404, 'App not found');
if (!app)
throw new HttpError(404, 'App not found', {
legacyCode: 'not_found',
});
await this.#checkReadAccess(app, actor);
@@ -279,13 +286,18 @@ export class AppDriver extends PuterDriver {
async update({ uid, id, object } = {}) {
if (!object || typeof object !== 'object') {
throw new HttpError(400, 'Missing or invalid `object`');
throw new HttpError(400, 'Missing or invalid `object`', {
legacyCode: 'bad_request',
});
}
const actor = this.#requireActor();
this.#requireUserOrAppActor(actor);
const app = await this.#resolve({ uid, id });
if (!app) throw new HttpError(404, 'App not found');
if (!app)
throw new HttpError(404, 'App not found', {
legacyCode: 'not_found',
});
await this.#checkWriteAccess(app, actor);
@@ -322,8 +334,9 @@ export class AppDriver extends PuterDriver {
if (fields.name && fields.name !== app.name) {
if (await this.appStore.existsByName(fields.name)) {
throw new HttpError(
400,
409,
'An app with this name already exists',
{ legacyCode: 'conflict' },
);
}
}
@@ -359,10 +372,15 @@ export class AppDriver extends PuterDriver {
this.#requireUserOrAppActor(actor);
const app = await this.#resolve({ uid, id });
if (!app) throw new HttpError(404, 'App not found');
if (!app)
throw new HttpError(404, 'App not found', {
legacyCode: 'not_found',
});
if (app.protected) {
throw new HttpError(403, 'Cannot delete a protected app');
throw new HttpError(403, 'Cannot delete a protected app', {
legacyCode: 'forbidden',
});
}
await this.#checkWriteAccess(app, actor);
@@ -495,12 +513,14 @@ export class AppDriver extends PuterDriver {
throw new HttpError(
400,
'`icon` data URL must use an image MIME type',
{ legacyCode: 'bad_request' },
);
}
} else if (!isAppIconEndpointUrl(iconStr, this.config)) {
throw new HttpError(
400,
'`icon` must be base64, a data:image/… URL, or an app-icon endpoint URL',
{ legacyCode: 'bad_request' },
);
}
}
@@ -542,12 +562,18 @@ export class AppDriver extends PuterDriver {
#requireActor() {
const actor = Context.get('actor');
if (!actor) throw new HttpError(401, 'Authentication required');
if (!actor)
throw new HttpError(401, 'Authentication required', {
legacyCode: 'unauthorized',
});
return actor;
}
#requireUserOrAppActor(actor) {
if (!actor.user) throw new HttpError(403, 'User actor required');
if (!actor.user)
throw new HttpError(403, 'User actor required', {
legacyCode: 'forbidden',
});
}
async #resolve({ uid, id }) {
@@ -605,7 +631,7 @@ export class AppDriver extends PuterDriver {
async #checkReadAccess(app, actor) {
if (await this.#canReadApp(app, actor)) return;
throw new HttpError(403, 'Access denied');
throw new HttpError(403, 'Access denied', { legacyCode: 'forbidden' });
}
async #checkWriteAccess(app, actor) {
@@ -624,7 +650,9 @@ export class AppDriver extends PuterDriver {
);
}
if (!hasAccess) {
throw new HttpError(403, 'Access denied');
throw new HttpError(403, 'Access denied', {
legacyCode: 'forbidden',
});
}
}
+496
View File
@@ -0,0 +1,496 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one PuterServer (in-memory sqlite + dynamo + s3 + mock redis)
// and exercises the live AppDriver (`puter-apps`) against the real
// AppStore. Each test makes its own user via `makeUser` so app rows
// from one test don't pollute another's `select` results.
let server: PuterServer;
// AppDriver is a JS module without an exported class type; treat as a
// generic CRUD-Q surface so we don't fight TS over private internals.
type CrudQDriver = {
create: (args: Record<string, unknown>) => Promise<Record<string, unknown>>;
read: (args: Record<string, unknown>) => Promise<Record<string, unknown>>;
select: (args: Record<string, unknown>) => Promise<unknown[]>;
update: (args: Record<string, unknown>) => Promise<Record<string, unknown>>;
upsert: (args: Record<string, unknown>) => Promise<Record<string, unknown>>;
delete: (args: Record<string, unknown>) => Promise<{ success: boolean; uid: string }>;
isNameAvailable: (name: string) => Promise<boolean>;
};
let driver: CrudQDriver;
beforeAll(async () => {
server = await setupTestServer();
driver = server.drivers.apps as unknown as CrudQDriver;
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `ad-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
const uniqueName = (prefix: string) =>
`${prefix}-${Math.random().toString(36).slice(2, 10)}`;
const uniqueIndexUrl = () =>
`https://example-${Math.random().toString(36).slice(2, 10)}.test/`;
// ── create ──────────────────────────────────────────────────────────
describe('AppDriver.create', () => {
it('creates an app and stamps the actor as owner', async () => {
const { actor, userId } = await makeUser();
const name = uniqueName('app');
const result = await withActor(actor, () =>
driver.create({
object: {
name,
title: 'My App',
description: 'desc',
index_url: uniqueIndexUrl(),
},
}),
);
expect(result.uid).toEqual(expect.any(String));
expect(result.name).toBe(name);
expect(result.title).toBe('My App');
// `owner` is only attached when the actor is the owner.
expect(result.owner).toMatchObject({ username: actor.user!.username });
// Confirm DB-level ownership.
const stored = await server.stores.app.getByUid(result.uid as string);
expect(stored?.owner_user_id).toBe(userId);
});
it('rejects an invalid app name with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
name: 'has spaces',
title: 'x',
index_url: uniqueIndexUrl(),
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a missing index_url with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: { name: uniqueName('no-url'), title: 't' },
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a duplicate app name with 400', async () => {
const a = await makeUser();
const b = await makeUser();
const name = uniqueName('dup');
await withActor(a.actor, () =>
driver.create({
object: { name, title: 'a', index_url: uniqueIndexUrl() },
}),
);
await expect(
withActor(b.actor, () =>
driver.create({
object: {
name,
title: 'b',
index_url: uniqueIndexUrl(),
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('dedupes a colliding name when `dedupe_name` is true', async () => {
const { actor } = await makeUser();
const name = uniqueName('dedup');
await withActor(actor, () =>
driver.create({
object: { name, title: 't', index_url: uniqueIndexUrl() },
}),
);
const second = await withActor(actor, () =>
driver.create({
object: { name, title: 't', index_url: uniqueIndexUrl() },
options: { dedupe_name: true },
}),
);
expect(second.name).not.toBe(name);
expect(String(second.name).startsWith(name)).toBe(true);
});
it('rejects a non-image data: icon with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
name: uniqueName('bad-icon'),
title: 't',
index_url: uniqueIndexUrl(),
icon: 'data:text/plain;base64,AAAA',
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 401 with no actor in context', async () => {
await expect(
driver.create({
object: {
name: uniqueName('noctx'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
).rejects.toMatchObject({ statusCode: 401 });
});
});
// ── read ────────────────────────────────────────────────────────────
describe('AppDriver.read', () => {
it('reads a public app for any actor', async () => {
const a = await makeUser();
const b = await makeUser();
const created = await withActor(a.actor, () =>
driver.create({
object: {
name: uniqueName('public'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
const fetched = await withActor(b.actor, () =>
driver.read({ uid: created.uid }),
);
expect(fetched.uid).toBe(created.uid);
// Owner block is NOT exposed to non-owners.
expect(fetched.owner).toBeUndefined();
});
it('reads via id object with `{ name }`', async () => {
const { actor } = await makeUser();
const name = uniqueName('by-name');
await withActor(actor, () =>
driver.create({
object: { name, title: 't', index_url: uniqueIndexUrl() },
}),
);
const fetched = await withActor(actor, () =>
driver.read({ id: { name } }),
);
expect(fetched.name).toBe(name);
});
it('returns 404 for a missing app', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () => driver.read({ uid: 'app-nonexistent' })),
).rejects.toMatchObject({ statusCode: 404 });
});
});
// ── select ──────────────────────────────────────────────────────────
describe('AppDriver.select', () => {
it('returns visible apps including those owned by other users', async () => {
const a = await makeUser();
const b = await makeUser();
const aName = uniqueName('a');
const bName = uniqueName('b');
await withActor(a.actor, () =>
driver.create({
object: {
name: aName,
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
await withActor(b.actor, () =>
driver.create({
object: {
name: bName,
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
const result = (await withActor(a.actor, () =>
driver.select({}),
)) as Array<Record<string, unknown>>;
const names = result.map((r) => r.name);
expect(names).toContain(aName);
expect(names).toContain(bName);
});
it('predicate `user-can-edit` filters to actor-owned apps only', async () => {
const a = await makeUser();
const b = await makeUser();
const mine = uniqueName('mine');
await withActor(a.actor, () =>
driver.create({
object: {
name: mine,
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
await withActor(b.actor, () =>
driver.create({
object: {
name: uniqueName('theirs'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
const result = (await withActor(a.actor, () =>
driver.select({ predicate: ['user-can-edit'] }),
)) as Array<Record<string, unknown>>;
// `select` only returns one row in this slice — the actor-owned
// one. Caller filters server-side via `owner_user_id`.
const names = result.map((r) => r.name);
expect(names).toContain(mine);
for (const row of result) {
expect(row.owner).toMatchObject({
username: a.actor.user!.username,
});
}
});
});
// ── update / delete ─────────────────────────────────────────────────
describe('AppDriver.update', () => {
it('updates editable fields on an owned app', async () => {
const { actor } = await makeUser();
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('upd'),
title: 'Old',
index_url: uniqueIndexUrl(),
},
}),
);
const updated = await withActor(actor, () =>
driver.update({
uid: created.uid,
object: { title: 'New', description: 'now with desc' },
}),
);
expect(updated.title).toBe('New');
expect(updated.description).toBe('now with desc');
});
it("rejects updating another user's app with 403", async () => {
const a = await makeUser();
const b = await makeUser();
const created = await withActor(a.actor, () =>
driver.create({
object: {
name: uniqueName('cross'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
await expect(
withActor(b.actor, () =>
driver.update({
uid: created.uid,
object: { title: 'hacked' },
}),
),
).rejects.toMatchObject({ statusCode: 403 });
});
});
describe('AppDriver.delete', () => {
it('deletes an owned app and reports `{ success, uid }`', async () => {
const { actor } = await makeUser();
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('del'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
const result = await withActor(actor, () =>
driver.delete({ uid: created.uid }),
);
expect(result).toEqual({ success: true, uid: created.uid });
expect(
await server.stores.app.getByUid(created.uid as string),
).toBeNull();
});
it("refuses to delete another user's app with 403", async () => {
const a = await makeUser();
const b = await makeUser();
const created = await withActor(a.actor, () =>
driver.create({
object: {
name: uniqueName('cross-del'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
await expect(
withActor(b.actor, () => driver.delete({ uid: created.uid })),
).rejects.toMatchObject({ statusCode: 403 });
});
it('returns 404 for a non-existent uid', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () => driver.delete({ uid: 'app-nonexistent' })),
).rejects.toMatchObject({ statusCode: 404 });
});
});
// ── upsert ──────────────────────────────────────────────────────────
describe('AppDriver.upsert', () => {
it('creates when no row matches', async () => {
const { actor } = await makeUser();
const result = await withActor(actor, () =>
driver.upsert({
object: {
name: uniqueName('ups'),
title: 't',
index_url: uniqueIndexUrl(),
},
}),
);
expect(result.uid).toEqual(expect.any(String));
});
it('updates when a row already exists at the resolved uid', async () => {
const { actor } = await makeUser();
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('ups-existing'),
title: 'first',
index_url: uniqueIndexUrl(),
},
}),
);
const result = await withActor(actor, () =>
driver.upsert({
uid: created.uid,
object: { title: 'second' },
}),
);
expect(result.title).toBe('second');
});
});
// ── isNameAvailable ────────────────────────────────────────────────
describe('AppDriver.isNameAvailable', () => {
it('returns true for an unused name', async () => {
const result = await driver.isNameAvailable(uniqueName('avail'));
expect(result).toBe(true);
});
it('returns false once an app has claimed the name', async () => {
const { actor } = await makeUser();
const name = uniqueName('claimed');
await withActor(actor, () =>
driver.create({
object: { name, title: 't', index_url: uniqueIndexUrl() },
}),
);
const result = await driver.isNameAvailable(name);
expect(result).toBe(false);
});
it('rejects an invalid name format with 400', async () => {
await expect(driver.isNameAvailable('has spaces')).rejects.toMatchObject(
{ statusCode: 400 },
);
});
});
@@ -0,0 +1,86 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Shared utilities for AI provider integration tests.
*
* Each test reads its credentials from `PUTER_TEST_AI_*` env vars
* (loaded by the vitest config's `PUTER_` prefix) and skips itself
* when the var is missing — tests run only on developer machines and
* in CI environments that supply the right secrets.
*
* Filename intentionally omits `.test.` so vitest does not treat this
* helper as a test file.
*/
import type { Actor } from '../core/actor.js';
import { SYSTEM_ACTOR } from '../core/actor.js';
import { runWithContext } from '../core/context.js';
import type { MeteringService } from '../services/metering/MeteringService.js';
/**
* Returns the env var value, or `undefined` if missing/empty.
* Used as the gate for `describe.skipIf` blocks.
*/
export const optionalEnv = (name: string): string | undefined => {
const v = process.env[name];
return v && v.length > 0 ? v : undefined;
};
/**
* Returns true when the env var is unset, signaling the test block
* should be skipped. Pair with `describe.skipIf(skipUnlessEnv(...))`.
*/
export const skipUnlessEnv = (name: string): boolean => !optionalEnv(name);
/**
* Per-test timeout for provider integration tests. The default 5s
* vitest timeout is way too short for real API calls — image
* generation in particular routinely takes 15–30s. Pass this as the
* third argument to `it(...)`.
*/
export const INTEGRATION_TEST_TIMEOUT_MS = 90_000;
/**
* Returns a no-op MeteringService stub. Real metering would write to
* DynamoDB / Redis, which integration tests for AI providers don't
* care about — we just need the provider's metering calls to not
* throw and to short-circuit credit checks.
*/
export const makeMeteringStub = (): MeteringService =>
({
utilRecordUsageObject: () => Promise.resolve([] as never),
incrementUsage: () => Promise.resolve({} as never),
batchIncrementUsages: () => Promise.resolve([] as never),
hasEnoughCredits: () => Promise.resolve(true),
getReportedCosts: () => [],
}) as unknown as MeteringService;
/**
* Run `fn` inside a request-scoped context with `SYSTEM_ACTOR` set,
* which is what providers expect (`Context.get('actor')`). The system
* actor bypasses metering / quota gates by design.
*/
export const withTestActor = <T>(
fn: () => T | Promise<T>,
actor: Actor = SYSTEM_ACTOR,
): Promise<T> =>
Promise.resolve(
runWithContext({ actor, requestId: 'integration-test' }, fn),
);
@@ -0,0 +1,799 @@
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
} from 'vitest';
import { Actor } from '../../core/actor.ts';
import { runWithContext } from '../../core/context.ts';
import { PuterServer } from '../../server.ts';
import { setupTestServer } from '../../testUtil.ts';
import { KV_COSTS } from './costs.ts';
import type { KVStoreDriver } from './KVStoreDriver.ts';
describe('KVStoreDriver', () => {
let server: PuterServer;
let target: KVStoreDriver;
beforeAll(async () => {
server = await setupTestServer();
target = server.drivers.kvStore;
});
afterAll(async () => {
await server.shutdown();
});
// Each test runs against a unique actor namespace so state from one test
// never leaks into another. Mirrors the pattern used by SystemKVStore.test.
let actor: Actor;
const makeActor = (overrides: Partial<Actor> = {}): Actor => ({
user: {
uuid: `test-user-${Math.random().toString(36).slice(2)}`,
id: 1,
username: 'test-user',
email: '[email protected]',
email_confirmed: true,
},
app: { uid: 'test-app', id: 1 },
...overrides,
});
beforeEach(() => {
actor = makeActor();
});
const inCtx = <T>(fn: () => T | Promise<T>, withActor: Actor = actor) =>
runWithContext({ actor: withActor }, fn);
describe('get', () => {
it('returns the value previously stored under the same key', async () => {
const res = await inCtx(async () => {
await target.set({ key: 'k', value: 'v' });
return target.get({ key: 'k' });
});
expect(res).toBe('v');
});
it('returns null for a missing key', async () => {
const res = await inCtx(() => target.get({ key: 'absent' }));
expect(res).toBeNull();
});
it('returns an array of values when called with an array of keys', async () => {
const res = await inCtx(async () => {
await target.set({ key: 'a', value: 1 });
await target.set({ key: 'b', value: 2 });
return target.get({ key: ['a', 'b', 'missing'] });
});
expect(res).toEqual([1, 2, null]);
});
it('returns [] for an empty array of keys without hitting the store', async () => {
const res = await inCtx(() => target.get({ key: [] }));
expect(res).toEqual([]);
});
it('coerces a non-string key to a string before lookup', async () => {
const res = await inCtx(async () => {
await target.set({ key: 123 as unknown as string, value: 'numeric' });
return target.get({ key: '123' });
});
expect(res).toBe('numeric');
});
it('rejects when key is undefined', async () => {
await expect(
inCtx(() => target.get({ key: undefined })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects when key is null', async () => {
await expect(
inCtx(() => target.get({ key: null })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects when any key in an array is empty', async () => {
await expect(
inCtx(() => target.get({ key: ['ok', ''] })),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('set', () => {
it('returns true on success', async () => {
const res = await inCtx(() => target.set({ key: 'k', value: 'v' }));
expect(res).toBe(true);
});
it('overwrites a previously-set value', async () => {
const res = await inCtx(async () => {
await target.set({ key: 'k', value: 'first' });
await target.set({ key: 'k', value: 'second' });
return target.get({ key: 'k' });
});
expect(res).toBe('second');
});
it('stores complex object values', async () => {
const value = { nested: { count: 1 }, items: [1, 2, 3] };
const res = await inCtx(async () => {
await target.set({ key: 'obj', value });
return target.get({ key: 'obj' });
});
expect(res).toEqual(value);
});
it('stores null as a real value (distinct from missing)', async () => {
const res = await inCtx(async () => {
await target.set({ key: 'nullable', value: null });
return target.get({ key: 'nullable' });
});
expect(res).toBeNull();
});
it('honours expireAt — past timestamps make the value invisible', async () => {
const past = Math.floor(Date.now() / 1000) - 10;
const res = await inCtx(async () => {
await target.set({ key: 'gone', value: 'soon', expireAt: past });
return target.get({ key: 'gone' });
});
expect(res).toBeNull();
});
it('rejects an empty key', async () => {
await expect(
inCtx(() => target.set({ key: '', value: 'v' })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a missing key', async () => {
await expect(
inCtx(() => target.set({ key: undefined, value: 'v' })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects when value is undefined', async () => {
await expect(
inCtx(() => target.set({ key: 'k', value: undefined })),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('batchPut', () => {
it('writes multiple items and they read back', async () => {
const res = await inCtx(async () => {
await target.batchPut({
items: [
{ key: 'bp1', value: 'v1' },
{ key: 'bp2', value: 'v2' },
{ key: 'bp3', value: { nested: true } },
],
});
return target.get({ key: ['bp1', 'bp2', 'bp3'] });
});
expect(res).toEqual(['v1', 'v2', { nested: true }]);
});
it('coerces non-string keys', async () => {
const res = await inCtx(async () => {
await target.batchPut({
items: [
{ key: 1 as unknown as string, value: 'one' },
{ key: 2 as unknown as string, value: 'two' },
],
});
return target.get({ key: ['1', '2'] });
});
expect(res).toEqual(['one', 'two']);
});
it('rejects a missing items array', async () => {
await expect(
inCtx(() =>
target.batchPut({
items: undefined as unknown as [],
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an empty items array', async () => {
await expect(
inCtx(() => target.batchPut({ items: [] })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects when any item has an empty key', async () => {
await expect(
inCtx(() =>
target.batchPut({
items: [
{ key: 'ok', value: 1 },
{ key: '', value: 2 },
],
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('del', () => {
it('removes a previously-set key', async () => {
const res = await inCtx(async () => {
await target.set({ key: 'gone', value: 'bye' });
await target.del({ key: 'gone' });
return target.get({ key: 'gone' });
});
expect(res).toBeNull();
});
it('returns true even when the key never existed', async () => {
const res = await inCtx(() =>
target.del({ key: 'never-existed' }),
);
expect(res).toBe(true);
});
it('rejects a missing key', async () => {
await expect(
inCtx(() => target.del({ key: undefined })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an empty key', async () => {
await expect(
inCtx(() => target.del({ key: '' })),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('list', () => {
const seed = () =>
target.batchPut({
items: [
{ key: 'fruit:apple', value: 'red' },
{ key: 'fruit:banana', value: 'yellow' },
{ key: 'veg:carrot', value: 'orange' },
],
});
it('returns key/value entries by default', async () => {
const res = (await inCtx(async () => {
await seed();
return target.list({});
})) as { key: string; value: unknown }[];
expect(res).toEqual(
expect.arrayContaining([
{ key: 'fruit:apple', value: 'red' },
{ key: 'fruit:banana', value: 'yellow' },
{ key: 'veg:carrot', value: 'orange' },
]),
);
});
it('returns just keys when as=keys', async () => {
const res = (await inCtx(async () => {
await seed();
return target.list({ as: 'keys' });
})) as string[];
expect(res).toEqual(
expect.arrayContaining([
'fruit:apple',
'fruit:banana',
'veg:carrot',
]),
);
});
it('returns just values when as=values', async () => {
const res = (await inCtx(async () => {
await seed();
return target.list({ as: 'values' });
})) as unknown[];
expect(res).toEqual(
expect.arrayContaining(['red', 'yellow', 'orange']),
);
});
it('filters by wildcard prefix pattern', async () => {
const res = (await inCtx(async () => {
await seed();
return target.list({ as: 'keys', pattern: 'fruit:*' });
})) as string[];
expect(res).toEqual(
expect.arrayContaining(['fruit:apple', 'fruit:banana']),
);
expect(res).not.toContain('veg:carrot');
});
it('returns a paginated envelope with cursor when limit is supplied', async () => {
const res = (await inCtx(async () => {
await seed();
return target.list({ limit: 1 });
})) as { items: unknown[]; cursor?: string };
expect(res.items.length).toBe(1);
expect(typeof res.cursor).toBe('string');
});
it('paginates across pages using the returned cursor', async () => {
const collected = await inCtx(async () => {
await seed();
const page1 = (await target.list({ limit: 2 })) as {
items: { key: string }[];
cursor?: string;
};
const page2 = (await target.list({
limit: 2,
cursor: page1.cursor,
})) as { items: { key: string }[]; cursor?: string };
return [...page1.items, ...page2.items].map((e) => e.key);
});
expect(collected.sort()).toEqual([
'fruit:apple',
'fruit:banana',
'veg:carrot',
]);
});
it('rejects an unsupported as value', async () => {
await expect(
inCtx(() =>
target.list({
as: 'bogus' as 'keys',
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('flush', () => {
it('removes every key in the actor namespace', async () => {
const res = (await inCtx(async () => {
await target.batchPut({
items: [
{ key: 'f1', value: 1 },
{ key: 'f2', value: 2 },
],
});
await target.flush({});
return target.list({ as: 'keys' });
})) as string[];
expect(res).toEqual([]);
});
it('only flushes the calling actor namespace', async () => {
const otherActor = makeActor();
await inCtx(() => target.set({ key: 'mine', value: 1 }));
await inCtx(
() => target.set({ key: 'theirs', value: 2 }),
otherActor,
);
await inCtx(() => target.flush({}));
const mine = await inCtx(() => target.get({ key: 'mine' }));
const theirs = await inCtx(
() => target.get({ key: 'theirs' }),
otherActor,
);
expect(mine).toBeNull();
expect(theirs).toBe(2);
});
});
describe('incr / decr', () => {
it('increments a top-level numeric counter from zero', async () => {
const res = await inCtx(() =>
target.incr({ key: 'c', pathAndAmountMap: { hits: 1 } }),
);
expect(res).toMatchObject({ hits: 1 });
});
it('accumulates across calls', async () => {
const res = await inCtx(async () => {
await target.incr({ key: 'c', pathAndAmountMap: { hits: 2 } });
return target.incr({ key: 'c', pathAndAmountMap: { hits: 3 } });
});
expect(res).toMatchObject({ hits: 5 });
});
it('decr subtracts via the same machinery', async () => {
const res = await inCtx(async () => {
await target.incr({ key: 'c', pathAndAmountMap: { hits: 10 } });
return target.decr({ key: 'c', pathAndAmountMap: { hits: 3 } });
});
expect(res).toMatchObject({ hits: 7 });
});
it('coerces non-string keys', async () => {
const res = await inCtx(() =>
target.incr({
key: 7 as unknown as string,
pathAndAmountMap: { n: 1 },
}),
);
expect(res).toMatchObject({ n: 1 });
});
it.each([
['incr' as const],
['decr' as const],
])('%s rejects a missing key', async (op) => {
await expect(
inCtx(() =>
target[op]({
key: undefined,
pathAndAmountMap: { n: 1 },
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it.each([
['incr' as const],
['decr' as const],
])('%s rejects a missing pathAndAmountMap', async (op) => {
await expect(
inCtx(() =>
target[op]({
key: 'k',
pathAndAmountMap: undefined as unknown as Record<
string,
number
>,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it.each([
['incr' as const],
['decr' as const],
])('%s rejects a non-object pathAndAmountMap', async (op) => {
await expect(
inCtx(() =>
target[op]({
key: 'k',
pathAndAmountMap: 'nope' as unknown as Record<
string,
number
>,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('expireAt / expire', () => {
it('expireAt makes a key invisible once the timestamp has passed', async () => {
const past = Math.floor(Date.now() / 1000) - 5;
const res = await inCtx(async () => {
await target.set({ key: 'fade', value: 'soon' });
await target.expireAt({ key: 'fade', timestamp: past });
return target.get({ key: 'fade' });
});
expect(res).toBeNull();
});
it('expire computes the TTL relative to now (negative TTL = expired)', async () => {
const res = await inCtx(async () => {
await target.set({ key: 'fade2', value: 'soon' });
await target.expire({ key: 'fade2', ttl: -10 });
return target.get({ key: 'fade2' });
});
expect(res).toBeNull();
});
it('expireAt rejects a non-number timestamp', async () => {
await expect(
inCtx(() =>
target.expireAt({
key: 'k',
timestamp: 'soon' as unknown as number,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('expire rejects a non-number ttl', async () => {
await expect(
inCtx(() =>
target.expire({
key: 'k',
ttl: 'soon' as unknown as number,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it.each([
['expireAt' as const, { timestamp: 0 }],
['expire' as const, { ttl: 0 }],
])('%s rejects an empty key', async (op, args) => {
await expect(
inCtx(() =>
(target[op] as (a: unknown) => Promise<unknown>)({
key: '',
...args,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('update', () => {
it('sets a top-level path on a fresh key', async () => {
const res = await inCtx(() =>
target.update({
key: 'doc',
pathAndValueMap: { name: 'puter' },
}),
);
expect(res).toMatchObject({ name: 'puter' });
});
it('writes nested paths and creates intermediate maps', async () => {
const res = await inCtx(() =>
target.update({
key: 'doc',
pathAndValueMap: { 'profile.email': '[email protected]' },
}),
);
expect(res).toMatchObject({ profile: { email: '[email protected]' } });
});
it('preserves untouched fields when updating a single path', async () => {
const res = await inCtx(async () => {
await target.update({
key: 'doc',
pathAndValueMap: { name: 'first', age: 1 },
});
return target.update({
key: 'doc',
pathAndValueMap: { age: 2 },
});
});
expect(res).toMatchObject({ name: 'first', age: 2 });
});
it('applies a TTL when ttl is supplied', async () => {
const res = await inCtx(async () => {
await target.update({
key: 'doc',
pathAndValueMap: { name: 'temp' },
ttl: -10,
});
return target.get({ key: 'doc' });
});
expect(res).toBeNull();
});
it('rejects a missing key', async () => {
await expect(
inCtx(() =>
target.update({
key: undefined,
pathAndValueMap: { x: 1 },
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a missing pathAndValueMap', async () => {
await expect(
inCtx(() =>
target.update({
key: 'k',
pathAndValueMap: undefined as unknown as Record<
string,
unknown
>,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a non-object pathAndValueMap', async () => {
await expect(
inCtx(() =>
target.update({
key: 'k',
pathAndValueMap: 'bogus' as unknown as Record<
string,
unknown
>,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('add', () => {
it('appends a single element to an empty path, creating a new list', async () => {
const res = await inCtx(() =>
target.add({
key: 'list',
pathAndValueMap: { items: 'a' },
}),
);
expect(res).toMatchObject({ items: ['a'] });
});
it('appends an array to an existing list', async () => {
const res = await inCtx(async () => {
await target.add({
key: 'list',
pathAndValueMap: { items: ['a'] },
});
return target.add({
key: 'list',
pathAndValueMap: { items: ['b', 'c'] },
});
});
expect(res).toMatchObject({ items: ['a', 'b', 'c'] });
});
it('rejects a missing pathAndValueMap', async () => {
await expect(
inCtx(() =>
target.add({
key: 'k',
pathAndValueMap: undefined as unknown as Record<
string,
unknown
>,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an empty key', async () => {
await expect(
inCtx(() =>
target.add({ key: '', pathAndValueMap: { x: 1 } }),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('remove', () => {
it('removes a path that exists', async () => {
const res = await inCtx(async () => {
await target.update({
key: 'doc',
pathAndValueMap: { keep: 1, drop: 2 },
});
return target.remove({ key: 'doc', paths: ['drop'] });
});
expect(res).toMatchObject({ keep: 1 });
expect(res).not.toHaveProperty('drop');
});
it('rejects a missing paths array', async () => {
await expect(
inCtx(() =>
target.remove({
key: 'k',
paths: undefined as unknown as string[],
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an empty paths array', async () => {
await expect(
inCtx(() => target.remove({ key: 'k', paths: [] })),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a missing key', async () => {
await expect(
inCtx(() =>
target.remove({ key: undefined, paths: ['x'] }),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('actor scoping', () => {
it('isolates values between actors with different user uuids', async () => {
const otherActor = makeActor();
await inCtx(() => target.set({ key: 'shared', value: 'mine' }));
const otherSees = await inCtx(
() => target.get({ key: 'shared' }),
otherActor,
);
expect(otherSees).toBeNull();
});
it('isolates values between two app actors with the same user but different apps', async () => {
const baseUser = `user-${Math.random().toString(36).slice(2)}`;
const appA: Actor = {
user: { uuid: baseUser },
app: { uid: 'app-A', id: 100 },
};
const appB: Actor = {
user: { uuid: baseUser },
app: { uid: 'app-B', id: 200 },
};
await inCtx(() => target.set({ key: 'k', value: 'A' }), appA);
const fromB = await inCtx(() => target.get({ key: 'k' }), appB);
expect(fromB).toBeNull();
const fromA = await inCtx(() => target.get({ key: 'k' }), appA);
expect(fromA).toBe('A');
});
it('ignores optConfig.appUuid when the actor already has an app uid', async () => {
// App-actor sets a value, then tries to read with an appUuid override
// pointing somewhere else — driver must scrub the override.
const baseUser = `user-${Math.random().toString(36).slice(2)}`;
const appActor: Actor = {
user: { uuid: baseUser },
app: { uid: 'real-app', id: 1 },
};
await inCtx(
() => target.set({ key: 'k', value: 'real' }),
appActor,
);
const res = await inCtx(
() =>
target.get({
key: 'k',
optConfig: { appUuid: 'spoof-app' },
}),
appActor,
);
expect(res).toBe('real');
});
it('uses optConfig.appUuid for a user-only (root) actor', async () => {
// User-only actor is allowed to scope reads/writes to a target
// app namespace via optConfig.appUuid. Verify by reading the same
// entry via a real app-actor for that app.
const baseUser = `user-${Math.random().toString(36).slice(2)}`;
const userOnly: Actor = { user: { uuid: baseUser } };
const asApp: Actor = {
user: { uuid: baseUser },
app: { uid: 'target-app', id: 1 },
};
await inCtx(
() =>
target.set({
key: 'k',
value: 'set-by-root',
optConfig: { appUuid: 'target-app' },
}),
userOnly,
);
const res = await inCtx(() => target.get({ key: 'k' }), asApp);
expect(res).toBe('set-by-root');
});
});
describe('getReportedCosts', () => {
it('reports a row per KV usage type with the configured rate', () => {
const rows = target.getReportedCosts();
expect(rows).toEqual(
expect.arrayContaining([
{
usageType: 'kv:read',
ucentsPerUnit: KV_COSTS['kv:read'],
unit: 'capacity-unit',
source: 'driver:kvStore',
},
{
usageType: 'kv:write',
ucentsPerUnit: KV_COSTS['kv:write'],
unit: 'capacity-unit',
source: 'driver:kvStore',
},
]),
);
expect(rows.length).toBe(Object.keys(KV_COSTS).length);
});
});
});
@@ -0,0 +1,335 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { NotificationDriver } from './NotificationDriver.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one PuterServer (in-memory sqlite + dynamo + s3 + mock redis)
// and exercises the live NotificationDriver against the wired stores.
// Each test allocates its own user via `makeUser` so notification rows
// from one test don't leak into another's `select` results.
let server: PuterServer;
let driver: NotificationDriver;
beforeAll(async () => {
server = await setupTestServer();
driver = server.drivers.notifications as unknown as NotificationDriver;
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `nd-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
// ── create ──────────────────────────────────────────────────────────
describe('NotificationDriver.create', () => {
it('creates a notification row scoped to the actor', async () => {
const { actor, userId } = await makeUser();
const result = (await withActor(actor, () =>
driver.create({
object: { value: { title: 'hi' } },
}),
)) as Record<string, unknown> | null;
expect(result?.uid).toEqual(expect.any(String));
expect(result?.value).toEqual({ title: 'hi' });
// shown / acknowledged are unset on creation.
expect(result?.shown).toBeNull();
expect(result?.acknowledged).toBeNull();
const row = await server.stores.notification.getByUid(
result!.uid as string,
{ userId },
);
expect(row).not.toBeNull();
});
it('defaults `value` to {} when omitted', async () => {
const { actor } = await makeUser();
const result = (await withActor(actor, () =>
driver.create({ object: {} }),
)) as Record<string, unknown> | null;
expect(result?.value).toEqual({});
});
it('rejects a missing object body with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({} as Record<string, unknown>),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an app-actor with 403', async () => {
const { actor } = await makeUser();
const appActor: Actor = {
...actor,
app: { uid: 'some-app', id: 1 },
};
await expect(
withActor(appActor, () =>
driver.create({ object: { value: { title: 'app' } } }),
),
).rejects.toMatchObject({ statusCode: 403 });
});
it('throws 401 with no actor in context', async () => {
await expect(
driver.create({ object: { value: { title: 'noctx' } } }),
).rejects.toMatchObject({ statusCode: 401 });
});
});
// ── read ────────────────────────────────────────────────────────────
describe('NotificationDriver.read', () => {
it('reads a notification by uid for its owner', async () => {
const { actor } = await makeUser();
const created = (await withActor(actor, () =>
driver.create({ object: { value: { title: 'a' } } }),
)) as Record<string, unknown>;
const fetched = (await withActor(actor, () =>
driver.read({ uid: created.uid }),
)) as Record<string, unknown> | null;
expect(fetched?.uid).toBe(created.uid);
expect(fetched?.value).toEqual({ title: 'a' });
});
it('accepts `id` as an alias for `uid`', async () => {
const { actor } = await makeUser();
const created = (await withActor(actor, () =>
driver.create({ object: { value: {} } }),
)) as Record<string, unknown>;
const fetched = (await withActor(actor, () =>
driver.read({ id: created.uid }),
)) as Record<string, unknown> | null;
expect(fetched?.uid).toBe(created.uid);
});
it("returns 404 for another user's notification uid", async () => {
const a = await makeUser();
const b = await makeUser();
const created = (await withActor(a.actor, () =>
driver.create({ object: { value: { hidden: true } } }),
)) as Record<string, unknown>;
await expect(
withActor(b.actor, () => driver.read({ uid: created.uid })),
).rejects.toMatchObject({ statusCode: 404 });
});
it('rejects a missing uid with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () => driver.read({})),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── select / predicates ─────────────────────────────────────────────
describe('NotificationDriver.select', () => {
it('returns the actor-owned notifications', async () => {
const { actor } = await makeUser();
await withActor(actor, () =>
driver.create({ object: { value: { i: 1 } } }),
);
await withActor(actor, () =>
driver.create({ object: { value: { i: 2 } } }),
);
const result = (await withActor(actor, () =>
driver.select({}),
)) as Array<Record<string, unknown>>;
// SQLite's `created_at` is second-precision so two rapid inserts
// can tie on the ORDER BY column — assert membership, not order.
expect(result.length).toBe(2);
const values = result.map(
(r) => (r.value as { i: number }).i,
);
expect(values.sort()).toEqual([1, 2]);
});
it('does not leak other users\' notifications', async () => {
const a = await makeUser();
const b = await makeUser();
await withActor(a.actor, () =>
driver.create({ object: { value: { who: 'a' } } }),
);
const result = (await withActor(b.actor, () =>
driver.select({}),
)) as Array<Record<string, unknown>>;
expect(result).toEqual([]);
});
it('predicate `unseen` filters out shown notifications', async () => {
const { actor, userId } = await makeUser();
const seen = (await withActor(actor, () =>
driver.create({ object: { value: { i: 'seen' } } }),
)) as Record<string, unknown>;
const unseen = (await withActor(actor, () =>
driver.create({ object: { value: { i: 'unseen' } } }),
)) as Record<string, unknown>;
await server.stores.notification.markShown(
seen.uid as string,
userId,
);
const result = (await withActor(actor, () =>
driver.select({ predicate: 'unseen' }),
)) as Array<Record<string, unknown>>;
const uids = result.map((r) => r.uid);
expect(uids).toContain(unseen.uid);
expect(uids).not.toContain(seen.uid);
});
it('predicate `acknowledged` returns only acked rows', async () => {
const { actor, userId } = await makeUser();
const ack = (await withActor(actor, () =>
driver.create({ object: { value: { i: 'ack' } } }),
)) as Record<string, unknown>;
await withActor(actor, () =>
driver.create({ object: { value: { i: 'pending' } } }),
);
await server.stores.notification.markAcknowledged(
ack.uid as string,
userId,
);
const result = (await withActor(actor, () =>
driver.select({ predicate: 'acknowledged' }),
)) as Array<Record<string, unknown>>;
expect(result.map((r) => r.uid)).toEqual([ack.uid]);
});
it('caps `limit` at the driver max even when overridden by the caller', async () => {
const { actor } = await makeUser();
// Verify shape, not exact upper bound — keep test fast.
const result = (await withActor(actor, () =>
driver.select({ limit: 100_000 }),
)) as unknown[];
expect(Array.isArray(result)).toBe(true);
});
});
// ── mark_shown / mark_acknowledged ─────────────────────────────────
describe('NotificationDriver.mark_shown / mark_acknowledged', () => {
it('mark_shown sets `shown` and reports success', async () => {
const { actor, userId } = await makeUser();
const created = (await withActor(actor, () =>
driver.create({ object: { value: {} } }),
)) as Record<string, unknown>;
const result = (await withActor(actor, () =>
driver.mark_shown({ uid: created.uid }),
)) as { success: boolean };
expect(result.success).toBe(true);
const row = await server.stores.notification.getByUid(
created.uid as string,
{ userId },
);
expect(row?.shown).not.toBeNull();
});
it('mark_acknowledged sets `acknowledged` and reports success', async () => {
const { actor, userId } = await makeUser();
const created = (await withActor(actor, () =>
driver.create({ object: { value: {} } }),
)) as Record<string, unknown>;
const result = (await withActor(actor, () =>
driver.mark_acknowledged({ uid: created.uid }),
)) as { success: boolean };
expect(result.success).toBe(true);
const row = await server.stores.notification.getByUid(
created.uid as string,
{ userId },
);
expect(row?.acknowledged).not.toBeNull();
});
it('mark_shown rejects missing uid with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () => driver.mark_shown({})),
).rejects.toMatchObject({ statusCode: 400 });
});
it("mark_shown returns success=false for another user's uid", async () => {
const a = await makeUser();
const b = await makeUser();
const created = (await withActor(a.actor, () =>
driver.create({ object: { value: {} } }),
)) as Record<string, unknown>;
const result = (await withActor(b.actor, () =>
driver.mark_shown({ uid: created.uid }),
)) as { success: boolean };
// Store update is scoped by user_id, so cross-user mutation is a
// silent no-op. The driver reports the store's `affected = 0`
// verbatim as `success: false`.
expect(result.success).toBe(false);
});
});
@@ -0,0 +1,493 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { SubdomainDriver } from './SubdomainDriver.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one PuterServer (in-memory sqlite + dynamo + s3 + mock redis)
// and exercises the live SubdomainDriver against the real wired stores.
// Each test makes its own user via `makeUser` so subdomain rows / quota
// counts don't leak across cases.
let server: PuterServer;
let driver: SubdomainDriver;
beforeAll(async () => {
server = await setupTestServer();
driver = server.drivers.subdomains as unknown as SubdomainDriver;
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `sd-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
// Driver checks ACL on `root_dir`, which requires the home tree to
// exist — without provisioning, every create call would 400.
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
const uniqueSubdomain = (prefix: string) =>
`${prefix}-${Math.random().toString(36).slice(2, 10)}`;
// ── create ──────────────────────────────────────────────────────────
describe('SubdomainDriver.create', () => {
it('creates a subdomain pointing at an owned fs path', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const sub = uniqueSubdomain('site');
const result = (await withActor(actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${username}/Public`,
},
}),
)) as Record<string, unknown> | null;
expect(result).not.toBeNull();
expect(result?.subdomain).toBe(sub);
// Owner is hydrated as `{ username, uuid }`, not a numeric id.
expect(result?.owner).toMatchObject({ username });
const row =
await server.stores.subdomain.getBySubdomain(sub);
expect(row?.user_id).toBe(userId);
});
it('expands `~/Public` against the actor home before resolving root_dir', async () => {
const { actor } = await makeUser();
const sub = uniqueSubdomain('tilde');
await withActor(actor, () =>
driver.create({
object: { subdomain: sub, root_dir: '~/Public' },
}),
);
const row =
await server.stores.subdomain.getBySubdomain(sub);
expect(row).not.toBeNull();
});
it('rejects an invalid subdomain format with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
subdomain: 'NOT_VALID!',
root_dir: `/${actor.user!.username}/Public`,
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a reserved subdomain word with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
subdomain: 'admin',
root_dir: `/${actor.user!.username}/Public`,
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a duplicate subdomain with 409', async () => {
const a = await makeUser();
const b = await makeUser();
const sub = uniqueSubdomain('dup');
await withActor(a.actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${a.actor.user!.username}/Public`,
},
}),
);
await expect(
withActor(b.actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${b.actor.user!.username}/Public`,
},
}),
),
).rejects.toMatchObject({ statusCode: 409 });
});
it('rejects when root_dir does not exist', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
subdomain: uniqueSubdomain('missing'),
root_dir: `/${actor.user!.username}/does-not-exist`,
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it("rejects pointing root_dir at another user's tree", async () => {
const a = await makeUser();
const b = await makeUser();
await expect(
withActor(a.actor, () =>
driver.create({
object: {
subdomain: uniqueSubdomain('intruder'),
root_dir: `/${b.actor.user!.username}/Public`,
},
}),
),
).rejects.toMatchObject({
statusCode: expect.any(Number),
});
});
it('rejects a missing object body with 400', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({} as Record<string, unknown>),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 401 with no actor in context', async () => {
await expect(
driver.create({
object: {
subdomain: uniqueSubdomain('noctx'),
root_dir: '/x',
},
}),
).rejects.toMatchObject({ statusCode: 401 });
});
});
// ── read / select ───────────────────────────────────────────────────
describe('SubdomainDriver.read / select', () => {
it('reads a subdomain by uid for its owner', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const sub = uniqueSubdomain('read');
const created = (await withActor(actor, () =>
driver.create({
object: { subdomain: sub, root_dir: `/${username}/Public` },
}),
)) as Record<string, unknown>;
const fetched = (await withActor(actor, () =>
driver.read({ uid: created.uid }),
)) as Record<string, unknown> | null;
expect(fetched?.subdomain).toBe(sub);
});
it('reads via id object with `{ subdomain }`', async () => {
const { actor } = await makeUser();
const sub = uniqueSubdomain('read-by-name');
await withActor(actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${actor.user!.username}/Public`,
},
}),
);
const fetched = (await withActor(actor, () =>
driver.read({ id: { subdomain: sub } }),
)) as Record<string, unknown> | null;
expect(fetched?.subdomain).toBe(sub);
});
it("rejects reading another user's subdomain with 403", async () => {
const a = await makeUser();
const b = await makeUser();
const sub = uniqueSubdomain('private');
const created = (await withActor(a.actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${a.actor.user!.username}/Public`,
},
}),
)) as Record<string, unknown>;
await expect(
withActor(b.actor, () => driver.read({ uid: created.uid })),
).rejects.toMatchObject({ statusCode: 403 });
});
it('returns 404 when reading a missing subdomain', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.read({ uid: 'nonexistent-uuid' }),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('select returns only the actor-owned subdomains', async () => {
const a = await makeUser();
const b = await makeUser();
await withActor(a.actor, () =>
driver.create({
object: {
subdomain: uniqueSubdomain('mine'),
root_dir: `/${a.actor.user!.username}/Public`,
},
}),
);
await withActor(b.actor, () =>
driver.create({
object: {
subdomain: uniqueSubdomain('theirs'),
root_dir: `/${b.actor.user!.username}/Public`,
},
}),
);
const result = (await withActor(a.actor, () =>
driver.select({}),
)) as Array<Record<string, unknown>>;
// Owners surface as `{ username, uuid }`; assert we only see a's.
for (const row of result) {
expect((row.owner as { username: string }).username).toBe(
a.actor.user!.username,
);
}
});
});
// ── update ──────────────────────────────────────────────────────────
describe('SubdomainDriver.update', () => {
it('updates root_dir to another owned path', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const sub = uniqueSubdomain('upd');
const created = (await withActor(actor, () =>
driver.create({
object: { subdomain: sub, root_dir: `/${username}/Public` },
}),
)) as Record<string, unknown>;
const updated = (await withActor(actor, () =>
driver.update({
uid: created.uid,
object: { root_dir: `/${username}/Documents` },
}),
)) as Record<string, unknown> | null;
expect(updated).not.toBeNull();
const rootDir = updated!.root_dir as Record<string, unknown> | null;
expect(rootDir?.path).toBe(`/${username}/Documents`);
});
it('refuses to update a subdomain owned by another user with 403', async () => {
const a = await makeUser();
const b = await makeUser();
const sub = uniqueSubdomain('cross-upd');
const created = (await withActor(a.actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${a.actor.user!.username}/Public`,
},
}),
)) as Record<string, unknown>;
await expect(
withActor(b.actor, () =>
driver.update({
uid: created.uid,
object: {
root_dir: `/${b.actor.user!.username}/Public`,
},
}),
),
).rejects.toMatchObject({ statusCode: 403 });
});
it('returns 404 for a missing object body', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.update({ uid: 'whatever' } as Record<string, unknown>),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── upsert ──────────────────────────────────────────────────────────
describe('SubdomainDriver.upsert', () => {
it('creates when no row matches the args', async () => {
const { actor } = await makeUser();
const sub = uniqueSubdomain('ups');
const result = (await withActor(actor, () =>
driver.upsert({
object: {
subdomain: sub,
root_dir: `/${actor.user!.username}/Public`,
},
}),
)) as Record<string, unknown> | null;
expect(result?.subdomain).toBe(sub);
});
it('updates when an existing row resolves via id.subdomain', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const sub = uniqueSubdomain('ups-existing');
await withActor(actor, () =>
driver.create({
object: { subdomain: sub, root_dir: `/${username}/Public` },
}),
);
const result = (await withActor(actor, () =>
driver.upsert({
id: { subdomain: sub },
object: { root_dir: `/${username}/Documents` },
}),
)) as Record<string, unknown> | null;
const rootDir = result!.root_dir as Record<string, unknown> | null;
expect(rootDir?.path).toBe(`/${username}/Documents`);
});
});
// ── delete ──────────────────────────────────────────────────────────
describe('SubdomainDriver.delete', () => {
it('deletes an owned subdomain and reports success', async () => {
const { actor } = await makeUser();
const sub = uniqueSubdomain('del');
const created = (await withActor(actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${actor.user!.username}/Public`,
},
}),
)) as Record<string, unknown>;
const result = (await withActor(actor, () =>
driver.delete({ uid: created.uid }),
)) as { success: boolean; uid: string };
expect(result.success).toBe(true);
expect(result.uid).toBe(created.uid);
expect(
await server.stores.subdomain.getBySubdomain(sub),
).toBeNull();
});
it("refuses to delete another user's subdomain with 403", async () => {
const a = await makeUser();
const b = await makeUser();
const sub = uniqueSubdomain('cross-del');
const created = (await withActor(a.actor, () =>
driver.create({
object: {
subdomain: sub,
root_dir: `/${a.actor.user!.username}/Public`,
},
}),
)) as Record<string, unknown>;
await expect(
withActor(b.actor, () => driver.delete({ uid: created.uid })),
).rejects.toMatchObject({ statusCode: 403 });
// a's row is still there.
expect(
await server.stores.subdomain.getBySubdomain(sub),
).not.toBeNull();
});
it('returns 404 for a non-existent subdomain', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.delete({ uid: 'nonexistent-uuid' }),
),
).rejects.toMatchObject({ statusCode: 404 });
});
});
+36 -6
View File
@@ -18,16 +18,43 @@
*/
import type { puterClients } from '../clients';
import type { IExtensionClientInstances } from '../clients/types';
import type { puterServices } from '../services';
import type { IExtensionServiceInstances } from '../services/types';
import type { puterStores } from '../stores';
import type { IExtensionStoreInstances } from '../stores/types';
import type { IConfig, LayerInstances, WithCostsReporting } from '../types';
/**
* Extension-augmentable driver registry. Extensions add their own driver
* instance types via TypeScript declaration merging:
*
* declare module '@heyputer/backend/drivers/types' {
* interface IExtensionDriverInstances {
* myDriver: MyDriver;
* }
* }
*
* Augmentations flow into `this.drivers` (PuterController) and into the
* `extension.import('driver')` proxy.
*/
export interface IExtensionDriverInstances {
/**
* Open index signature so reads of extension-only driver keys return
* `unknown` instead of a type error. Concrete declaration-merged keys
* override this for that name.
*/
[key: string]: unknown;
}
export type IPuterDriver<T extends WithCostsReporting = WithCostsReporting> =
new (
config: IConfig,
clients: LayerInstances<typeof puterClients>,
stores: LayerInstances<typeof puterStores>,
services: LayerInstances<typeof puterServices>,
clients: LayerInstances<typeof puterClients> &
IExtensionClientInstances,
stores: LayerInstances<typeof puterStores> & IExtensionStoreInstances,
services: LayerInstances<typeof puterServices> &
IExtensionServiceInstances,
) => T;
/**
@@ -65,9 +92,12 @@ export const PuterDriver = class PuterDriver implements WithCostsReporting {
constructor(
protected config: IConfig,
protected clients: LayerInstances<typeof puterClients>,
protected stores: LayerInstances<typeof puterStores>,
protected services: LayerInstances<typeof puterServices>,
protected clients: LayerInstances<typeof puterClients> &
IExtensionClientInstances,
protected stores: LayerInstances<typeof puterStores> &
IExtensionStoreInstances,
protected services: LayerInstances<typeof puterServices> &
IExtensionServiceInstances,
) {}
public onServerStart() {
return;
+377
View File
@@ -0,0 +1,377 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import { inferFilenameFromUrlOrPath, loadFileInput } from './fileInput.js';
// ── Test harness ────────────────────────────────────────────────────
//
// Boots one real PuterServer (in-memory sqlite + dynamo + s3 + mock
// redis) and exercises `loadFileInput` against the live wired stores
// and FSService. Each test makes its own user via `makeUser` and,
// where the FS-resolution path is being exercised, writes a real
// file through FSService.write so there's an actual fsentry +
// in-memory S3 object behind the path / uuid we hand to loadFileInput.
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `fic-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
const writeFile = async (
userId: number,
path: string,
body: Buffer,
contentType = 'application/octet-stream',
) => {
const result = await server.services.fs.write(userId, {
fileMetadata: {
path,
size: body.byteLength,
contentType,
},
fileContent: body,
});
return result.fsEntry;
};
const callLoadFileInput = (
actor: Actor,
input: unknown,
options?: Parameters<typeof loadFileInput>[4],
) =>
withActor(actor, () =>
loadFileInput(
server.stores,
server.services.fs,
actor,
input,
options,
),
);
// ── inferFilenameFromUrlOrPath ─────────────────────────────────────
describe('inferFilenameFromUrlOrPath', () => {
it('returns the basename of a URL pathname', () => {
expect(inferFilenameFromUrlOrPath('https://cdn.test/a/b/photo.png')).toBe(
'photo.png',
);
});
it('returns the basename of a posix-style path string', () => {
expect(inferFilenameFromUrlOrPath('/alice/Music/song.mp3')).toBe(
'song.mp3',
);
});
it('falls back to the supplied default when there is no basename', () => {
// Empty string has no URL form and no posix basename, so the
// explicit fallback wins.
expect(inferFilenameFromUrlOrPath('', 'fallback-name')).toBe(
'fallback-name',
);
});
it('uses the literal `input` as fallback when no override is passed', () => {
expect(inferFilenameFromUrlOrPath('')).toBe('input');
});
it('handles bare filenames (no slashes, not a URL)', () => {
expect(inferFilenameFromUrlOrPath('plain.txt')).toBe('plain.txt');
});
});
// ── loadFileInput — argument validation ─────────────────────────────
describe('loadFileInput validation', () => {
it('throws 400 when input is empty/falsy', async () => {
const { actor } = await makeUser();
await expect(callLoadFileInput(actor, undefined)).rejects.toMatchObject(
{ statusCode: 400 },
);
await expect(callLoadFileInput(actor, null)).rejects.toMatchObject({
statusCode: 400,
});
await expect(callLoadFileInput(actor, '')).rejects.toMatchObject({
statusCode: 400,
});
});
it('throws 401 when actor.user.id is not a finite number', async () => {
// Missing actor entirely
await expect(
loadFileInput(
server.stores,
server.services.fs,
undefined as unknown as Actor,
'data:text/plain,hi',
),
).rejects.toMatchObject({ statusCode: 401 });
// Actor with no user
await expect(
loadFileInput(
server.stores,
server.services.fs,
{} as Actor,
'data:text/plain,hi',
),
).rejects.toMatchObject({ statusCode: 401 });
// Actor with non-numeric user.id
await expect(
loadFileInput(
server.stores,
server.services.fs,
{ user: { id: 'nope' } } as unknown as Actor,
'data:text/plain,hi',
),
).rejects.toMatchObject({ statusCode: 401 });
});
});
// ── loadFileInput — data URL path ───────────────────────────────────
describe('loadFileInput data URL', () => {
it('decodes a base64 data URL and reports the declared MIME', async () => {
const { actor } = await makeUser();
const payload = Buffer.from('hello world');
const dataUrl = `data:text/plain;base64,${payload.toString('base64')}`;
const result = await callLoadFileInput(actor, dataUrl);
expect(result.buffer.equals(payload)).toBe(true);
expect(result.mimeType).toBe('text/plain');
expect(result.fsEntry).toBeNull();
// Filename derives from the MIME subtype.
expect(result.filename).toBe('input.plain');
});
it('decodes a non-base64 (URL-encoded) data URL', async () => {
const { actor } = await makeUser();
// Plain (no `;base64`) → URL-decoded payload.
const result = await callLoadFileInput(
actor,
'data:text/plain,hello%20world',
);
expect(result.buffer.toString('utf8')).toBe('hello world');
expect(result.mimeType).toBe('text/plain');
});
it('infers MIME-derived filename for compound types like svg+xml', async () => {
const { actor } = await makeUser();
const result = await callLoadFileInput(
actor,
'data:image/svg+xml;base64,PHN2Zy8+',
);
// `image/svg+xml` → input.svg (subtype, before the `+`).
expect(result.filename).toBe('input.svg');
expect(result.mimeType).toBe('image/svg+xml');
});
it('defaults MIME to application/octet-stream when omitted', async () => {
const { actor } = await makeUser();
const result = await callLoadFileInput(actor, 'data:;base64,QUJD');
expect(result.mimeType).toBe('application/octet-stream');
expect(result.buffer.toString('utf8')).toBe('ABC');
});
it('throws 400 on a malformed data URL', async () => {
const { actor } = await makeUser();
// Missing comma → DATA_URL_PATTERN.exec returns null.
await expect(
callLoadFileInput(actor, 'data:not-a-url'),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an oversize data URL with 413 + storage_limit_reached', async () => {
const { actor } = await makeUser();
const payload = Buffer.alloc(64);
const dataUrl = `data:application/octet-stream;base64,${payload.toString('base64')}`;
await expect(
callLoadFileInput(actor, dataUrl, { maxBytes: 32 }),
).rejects.toMatchObject({
statusCode: 413,
legacyCode: 'storage_limit_reached',
});
});
it('accepts a data URL exactly at the maxBytes threshold', async () => {
const { actor } = await makeUser();
const payload = Buffer.alloc(16, 0x41); // 16 bytes of 'A'
const dataUrl = `data:application/octet-stream;base64,${payload.toString('base64')}`;
const result = await callLoadFileInput(actor, dataUrl, {
maxBytes: 16,
});
expect(result.buffer.byteLength).toBe(16);
});
});
// ── loadFileInput — FS path ─────────────────────────────────────────
describe('loadFileInput FS path', () => {
it('reads bytes back through a real fsentry written via FSService', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const path = `/${username}/Documents/sample.txt`;
const body = Buffer.from('hello from sql + s3');
const entry = await withActor(actor, () =>
writeFile(userId, path, body, 'text/plain'),
);
const result = await callLoadFileInput(actor, path);
expect(result.buffer.equals(body)).toBe(true);
expect(result.filename).toBe('sample.txt');
// FSService stamps the contentType into S3 metadata; loadFileInput
// returns it on the way out.
expect(result.mimeType).toBe('text/plain');
expect(result.fsEntry?.uuid).toBe(entry.uuid);
expect(result.fsEntry?.path).toBe(path);
});
it('also accepts a `{ uuid }` object reference', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const path = `/${username}/Documents/by-uuid.bin`;
const body = Buffer.from([0x01, 0x02, 0x03, 0x04]);
const entry = await withActor(actor, () =>
writeFile(userId, path, body),
);
const result = await callLoadFileInput(actor, { uuid: entry.uuid });
expect(result.buffer.equals(body)).toBe(true);
expect(result.fsEntry?.uuid).toBe(entry.uuid);
});
it('expands `~/...` paths against the actor home before resolving', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const body = Buffer.from('tilde');
await withActor(actor, () =>
writeFile(userId, `/${username}/Documents/tilde.txt`, body),
);
const result = await callLoadFileInput(actor, '~/Documents/tilde.txt');
expect(result.buffer.equals(body)).toBe(true);
});
it('throws 404 when the fsentry cannot be resolved', async () => {
const { actor } = await makeUser();
await expect(
callLoadFileInput(actor, {
uuid: '00000000-0000-0000-0000-000000000000',
}),
).rejects.toMatchObject({ statusCode: 404 });
});
it('rejects directory entries with 400', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
// The Documents folder is a real fsentry directory created by
// generateDefaultFsentries.
await expect(
callLoadFileInput(actor, `/${username}/Documents`),
).rejects.toMatchObject({ statusCode: 400 });
});
it("rejects when the FS access check denies the actor", async () => {
// Owner writes a file; intruder tries to read it. The real
// FSService.checkFSAccess walks the ACL and refuses.
const owner = await makeUser();
const intruder = await makeUser();
const ownerName = owner.actor.user!.username!;
const path = `/${ownerName}/Documents/private.txt`;
await withActor(owner.actor, () =>
writeFile(owner.userId, path, Buffer.from('owned')),
);
const err = await callLoadFileInput(intruder.actor, path).then(
() => null,
(e: unknown) => e,
);
const status = (err as { statusCode?: number } | null)?.statusCode;
// Access denied lands as 403; "can't see" lands as 404 — both
// are valid refusals from ACLService.getSafeAclError.
expect([403, 404]).toContain(status);
});
it('rejects up-front when contentLength exceeds maxBytes', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const path = `/${username}/Documents/big.bin`;
const body = Buffer.alloc(2048, 0x42);
await withActor(actor, () =>
writeFile(userId, path, body, 'application/octet-stream'),
);
await expect(
callLoadFileInput(actor, path, { maxBytes: 64 }),
).rejects.toMatchObject({
statusCode: 413,
legacyCode: 'storage_limit_reached',
});
});
});
+41 -14
View File
@@ -19,9 +19,15 @@
import type { RequestHandler } from 'express';
import type { puterClients } from './clients';
import type { IPuterClientRegistry } from './clients/types';
import type {
IExtensionClientInstances,
IPuterClientRegistry,
} from './clients/types';
import type { puterControllers } from './controllers';
import type { IPuterControllerRegistry } from './controllers/types';
import type {
IExtensionControllerInstances,
IPuterControllerRegistry,
} from './controllers/types';
import type {
RouteDescriptor,
RouteMethod,
@@ -29,7 +35,10 @@ import type {
RoutePath,
} from './core/http/types';
import type { puterDrivers } from './drivers';
import type { IPuterDriverRegistry } from './drivers/types';
import type {
IExtensionDriverInstances,
IPuterDriverRegistry,
} from './drivers/types';
import {
clientsContainers,
configContainer,
@@ -39,9 +48,15 @@ import {
storesContainers,
} from './exports';
import type { puterServices } from './services';
import type { IPuterServiceRegistry } from './services/types';
import type {
IExtensionServiceInstances,
IPuterServiceRegistry,
} from './services/types';
import type { puterStores } from './stores';
import type { IPuterStoreRegistry } from './stores/types';
import type {
IExtensionStoreInstances,
IPuterStoreRegistry,
} from './stores/types';
import type { IConfig, LayerInstances } from './types';
/**
@@ -56,8 +71,10 @@ export const extensionStore = {
controllers: {} as IPuterControllerRegistry,
drivers: {} as IPuterDriverRegistry,
globalMiddlewares: [] as RequestHandler[],
// eslint-disable-next-line @typescript-eslint/no-explicit-any
events: {} as Record<string, ((...args: any[]) => void)[]>,
events: {} as Record<
string,
((key: string, data: unknown, meta: unknown) => void)[]
>,
/**
* Extension-declared routes. Shape matches the controller-layer
* `RouteDescriptor`, so both flow through the same materializer
@@ -192,8 +209,11 @@ export const extension = {
// ── Event subscription ───────────────────────────────────────────
// eslint-disable-next-line @typescript-eslint/no-explicit-any
on: (event: string, handler: (...args: any[]) => void) => {
on: (
event: string,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
handler: (key: string, data: any, meta: any) => void,
) => {
if (!extensionStore.events[event]) {
extensionStore.events[event] = [];
}
@@ -262,15 +282,17 @@ export const extension = {
import: <S extends string>(
name: S,
): S extends 'client'
? LayerInstances<typeof puterClients>
? LayerInstances<typeof puterClients> & IExtensionClientInstances
: S extends 'store'
? LayerInstances<typeof puterStores>
? LayerInstances<typeof puterStores> & IExtensionStoreInstances
: S extends 'service'
? LayerInstances<typeof puterServices>
? LayerInstances<typeof puterServices> & IExtensionServiceInstances
: S extends 'controller'
? LayerInstances<typeof puterControllers>
? LayerInstances<typeof puterControllers> &
IExtensionControllerInstances
: S extends 'driver'
? LayerInstances<typeof puterDrivers>
? LayerInstances<typeof puterDrivers> &
IExtensionDriverInstances
: never => {
switch (name) {
case 'client': {
@@ -281,6 +303,7 @@ export const extension = {
const proxyProxyHandler = {
get: (_target2: object, prop2: string) => {
const proxiedObj2 =
// @ts-expect-error any type needed
clientsContainers[prop][prop2];
if (!proxiedObj2) {
throw new Error(
@@ -307,6 +330,7 @@ export const extension = {
const proxyProxyHandler = {
get: (_target2: object, prop2: string) => {
const proxiedObj2 =
// @ts-expect-error any type needed
clientsContainers[prop][prop2];
if (!proxiedObj2) {
throw new Error(
@@ -333,6 +357,7 @@ export const extension = {
const proxyProxyHandler = {
get: (_target2: object, prop2: string) => {
const proxiedObj2 =
// @ts-expect-error any type needed
clientsContainers[prop][prop2];
if (!proxiedObj2) {
throw new Error(
@@ -359,6 +384,7 @@ export const extension = {
const proxyProxyHandler = {
get: (_target2: object, prop2: string) => {
const proxiedObj2 =
// @ts-expect-error any type needed
clientsContainers[prop][prop2];
if (!proxiedObj2) {
throw new Error(
@@ -385,6 +411,7 @@ export const extension = {
const proxyProxyHandler = {
get: (_target2: object, prop2: string) => {
const proxiedObj2 =
// @ts-expect-error any type needed
clientsContainers[prop][prop2];
if (!proxiedObj2) {
throw new Error(
+3 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@heyputer/backend",
"type": "module",
"type": "commonjs",
"version": "2.5.1",
"description": "Backend/Kernel for Puter",
"main": "exports.ts",
@@ -63,7 +63,7 @@
"parse-domain": "^8.2.2",
"prompt-sync": "^4.2.0",
"replicate": "^1.0.0",
"sharp": "^0.34.3",
"sharp": "^0.34.5",
"socket.io": "^4.8.3",
"svg-captcha": "^1.4.0",
"together-ai": "^0.33.0",
@@ -73,6 +73,7 @@
"validator": "^13.15.35"
},
"devDependencies": {
"@types/busboy": "^1.5.4",
"@types/node": "^24.0.0",
"chai": "^4.3.7",
"nodemon": "^3.1.0",
+52 -63
View File
@@ -93,11 +93,11 @@ export class PuterServer {
constructor(
config: IConfig,
clients: typeof puterClients,
stores: typeof puterStores,
services: typeof puterServices,
controllers: typeof puterControllers,
drivers: typeof puterDrivers,
clients: typeof puterClients = puterClients,
stores: typeof puterStores = puterStores,
services: typeof puterServices = puterServices,
controllers: typeof puterControllers = puterControllers,
drivers: typeof puterDrivers = puterDrivers,
) {
this.#config = config;
// Expose config to the extension API (extension.config)
@@ -124,24 +124,29 @@ export class PuterServer {
this.clients = {} as typeof this.clients;
for (const [clientName, ClientClass] of Object.entries(clients)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.clients[clientName] =
typeof ClientClass === 'object'
? ClientClass
: (new (ClientClass as any)(this.#config) as any);
// @ts-expect-error implicit any casting to avoid overly complex or circular types
clientsContainers[clientName] = this.clients[clientName];
}
for (const [clientName, ClientClass] of Object.entries(
extensionStore.clients,
)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.clients[clientName] =
typeof ClientClass === 'object'
? ClientClass
: (new (ClientClass as any)(this.#config) as any);
// @ts-expect-error implicit any casting to avoid overly complex or circular types
clientsContainers[clientName] = this.clients[clientName];
}
this.stores = {} as typeof this.stores;
for (const [storeName, StoreClass] of Object.entries(stores)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.stores[storeName] =
typeof StoreClass === 'object'
? StoreClass
@@ -150,11 +155,13 @@ export class PuterServer {
this.clients,
this.stores,
) as any);
// @ts-expect-error implicit any casting to avoid overly complex or circular types
storesContainers[storeName] = this.stores[storeName];
}
for (const [storeName, StoreClass] of Object.entries(
extensionStore.stores,
)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.stores[storeName] =
typeof StoreClass === 'object'
? StoreClass
@@ -163,11 +170,13 @@ export class PuterServer {
this.clients,
this.stores,
) as any);
// @ts-expect-error implicit any casting to avoid overly complex or circular types
storesContainers[storeName] = this.stores[storeName];
}
this.services = {} as typeof this.services;
for (const [serviceName, ServiceClass] of Object.entries(services)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.services[serviceName] =
typeof ServiceClass === 'object'
? ServiceClass
@@ -177,11 +186,13 @@ export class PuterServer {
this.stores,
this.services,
) as any);
// @ts-expect-error implicit any casting to avoid overly complex or circular types
servicesContainers[serviceName] = this.services[serviceName];
}
for (const [serviceName, ServiceClass] of Object.entries(
extensionStore.services,
)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.services[serviceName] =
typeof ServiceClass === 'object'
? ServiceClass
@@ -191,6 +202,7 @@ export class PuterServer {
this.stores,
this.services,
) as any);
// @ts-expect-error implicit any casting to avoid overly complex or circular types
servicesContainers[serviceName] = this.services[serviceName];
}
@@ -235,6 +247,7 @@ export class PuterServer {
this.stores,
this.services,
) as any);
// @ts-expect-error as any casting to avoid overly complex or circular types
this.drivers[driverKey] = instance;
driversContainers[driverKey] = instance;
}
@@ -243,6 +256,7 @@ export class PuterServer {
for (const [controllerName, ControllerClass] of Object.entries(
controllers,
)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.controllers[controllerName] =
typeof ControllerClass === 'object'
? ControllerClass
@@ -255,14 +269,17 @@ export class PuterServer {
) as any);
this.#registerControllerRoutes(
controllerName,
// @ts-expect-error as any casting to avoid overly complex or circular types
this.controllers[controllerName],
);
controllersContainers[controllerName] =
// @ts-expect-error as any casting to avoid overly complex or circular types
this.controllers[controllerName];
}
for (const [controllerName, ControllerClass] of Object.entries(
extensionStore.controllers,
)) {
// @ts-expect-error as any casting to avoid overly complex or circular types
this.controllers[controllerName] =
typeof ControllerClass === 'object'
? ControllerClass
@@ -275,26 +292,14 @@ export class PuterServer {
) as any);
this.#registerControllerRoutes(
controllerName,
// @ts-expect-error as any casting to avoid overly complex or circular types
this.controllers[controllerName],
);
controllersContainers[controllerName] =
// @ts-expect-error as any casting to avoid overly complex or circular types
this.controllers[controllerName];
}
// Register extension event listeners. Extensions opted for a
// 2-arg `(data, meta)` handler shape; EventClient calls with
// `(key, data, meta)`. Drop `key` in the adapter so extension
// code stays stable.
Object.entries(extensionStore.events).forEach(([event, handlers]) => {
handlers.forEach((handler) => {
this.clients.event.on(
event,
(_key: string, data: unknown, meta: object) =>
handler(data, meta),
);
});
});
// Extension routes are shaped as `RouteDescriptor`s too, so they
// flow through the same materializer as controller routes — same
// option → middleware translation (subdomain, auth, body parsers, …).
@@ -617,12 +622,7 @@ export class PuterServer {
res.setHeader('Access-Control-Allow-Methods', allowedMethods);
res.setHeader('Access-Control-Allow-Headers', allowedHeaders);
// Private Network Access: grant public origins permission to
// reach loopback/private addresses (e.g. self-hosted Puter on
// localhost, or api.puter.com pointed at a local IP via hosts).
if (req.headers['access-control-request-private-network']) {
res.setHeader('Access-Control-Allow-Private-Network', 'true');
}
res.setHeader('Access-Control-Allow-Private-Network', 'true');
// Disable iframes on the main domain
if (req.hostname === config.domain) {
@@ -1048,41 +1048,7 @@ export class PuterServer {
'************************************************************\n',
);
for (const client of Object.values(
this.clients,
) as WithLifecycle[]) {
if (client.onServerStart) {
await client.onServerStart();
}
}
for (const store of Object.values(
this.stores,
) as WithLifecycle[]) {
if (store.onServerStart) {
await store.onServerStart();
}
}
for (const service of Object.values(
this.services,
) as WithLifecycle[]) {
if (service.onServerStart) {
await service.onServerStart();
}
}
for (const controller of Object.values(
this.controllers,
) as WithLifecycle[]) {
if (controller.onServerStart) {
await controller.onServerStart();
}
}
for (const driver of Object.values(
this.drivers,
) as WithLifecycle[]) {
if (driver.onServerStart) {
await driver.onServerStart();
}
}
await this.#fireOnServerStart();
console.log('PuterServer has fully booted.');
// Auto-launch the browser on dev boot (matches v1 WebServerService).
// Opt out via `no_browser_launch: true` in config.
@@ -1100,16 +1066,39 @@ export class PuterServer {
});
} else {
this.#server = {
close: (cb) => {
close: (cb: (error?: Error) => void | undefined) => {
console.debug('PuterServer mock close called');
cb();
cb?.();
},
closeAllConnections: () => {
console.debug(
'PuterServer mock closeAllConnections called',
);
},
};
} as unknown as http.Server;
// Tests still need onServerStart to fire so stores can
// bootstrap (e.g. SystemKVStore creates its dynalite table).
await this.#fireOnServerStart();
}
}
async #fireOnServerStart() {
for (const client of Object.values(this.clients) as WithLifecycle[]) {
if (client.onServerStart) await client.onServerStart();
}
for (const store of Object.values(this.stores) as WithLifecycle[]) {
if (store.onServerStart) await store.onServerStart();
}
for (const service of Object.values(this.services) as WithLifecycle[]) {
if (service.onServerStart) await service.onServerStart();
}
for (const controller of Object.values(
this.controllers,
) as WithLifecycle[]) {
if (controller.onServerStart) await controller.onServerStart();
}
for (const driver of Object.values(this.drivers) as WithLifecycle[]) {
if (driver.onServerStart) await driver.onServerStart();
}
}
+124 -18
View File
@@ -25,10 +25,14 @@ import { cleanEmail, isBlockedEmail } from '../../util/email.js';
import { generate_identifier } from '../../util/identifier.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import { Context } from '../../core';
import crypto from 'node:crypto';
const GOOGLE_DISCOVERY_URL =
'https://accounts.google.com/.well-known/openid-configuration';
const APPLE_DISCOVERY_URL =
'https://appleid.apple.com/.well-known/openid-configuration';
const GOOGLE_SCOPES = 'openid email profile';
const APPLE_SCOPES = 'openid email';
const STATE_EXPIRY_SEC = 600; // 10 minutes
const VALID_OIDC_FLOWS = ['login', 'signup', 'revalidate'] as const;
const REVALIDATION_EXPIRY_SEC = 300; // 5 minutes
@@ -40,6 +44,7 @@ interface ProviderConfig {
token_endpoint: string;
userinfo_endpoint: string;
scopes: string;
response_mode?: string;
}
interface OIDCUserInfo {
@@ -62,7 +67,7 @@ interface OIDCUserInfo {
export class OIDCService extends PuterService {
declare protected services: LayerInstances<typeof puterServices>;
#googleDiscovery: Record<string, string> | null = null;
#discoveryCache: Map<string, Record<string, string>> = new Map();
#providers: Record<string, Record<string, string>> = {};
override onServerStart(): void {
@@ -79,10 +84,39 @@ export class OIDCService extends PuterService {
providerId: string,
): Promise<ProviderConfig | null> {
const raw = this.#providers[providerId];
if (!raw || !raw.client_id || !raw.client_secret) return null;
if (!raw?.client_id) return null;
if (providerId === 'apple') {
if (!raw.team_id || !raw.key_id || !raw.private_key) return null;
const discovery = await this.#fetchDiscovery(
APPLE_DISCOVERY_URL,
'Apple',
);
if (!discovery) return null;
return {
client_id: raw.client_id,
client_secret: this.#generateAppleClientSecret(
raw.team_id,
raw.client_id,
raw.key_id,
raw.private_key,
),
authorization_endpoint: discovery.authorization_endpoint,
token_endpoint: discovery.token_endpoint,
userinfo_endpoint: '',
scopes: raw.scopes ?? APPLE_SCOPES,
response_mode: 'form_post',
};
}
// Google and custom providers require a static client_secret.
if (!raw.client_secret) return null;
if (providerId === 'google') {
const discovery = await this.#fetchGoogleDiscovery();
const discovery = await this.#fetchDiscovery(
GOOGLE_DISCOVERY_URL,
'Google',
);
if (!discovery) return null;
return {
client_id: raw.client_id,
@@ -148,6 +182,9 @@ export class OIDCService extends PuterService {
scope: config.scopes,
state,
});
if (config.response_mode) {
params.set('response_mode', config.response_mode);
}
return `${config.authorization_endpoint}?${params.toString()}`;
}
@@ -223,15 +260,24 @@ export class OIDCService extends PuterService {
async getUserInfo(
providerId: string,
accessToken: string,
idToken?: string,
): Promise<OIDCUserInfo | null> {
const config = await this.getProviderConfig(providerId);
if (!config?.userinfo_endpoint) return null;
const res = await fetch(config.userinfo_endpoint, {
headers: { Authorization: `Bearer ${accessToken}` },
});
if (!res.ok) return null;
return (await res.json()) as OIDCUserInfo;
if (config?.userinfo_endpoint) {
const res = await fetch(config.userinfo_endpoint, {
headers: { Authorization: `Bearer ${accessToken}` },
});
if (!res.ok) return null;
return (await res.json()) as OIDCUserInfo;
}
// No userinfo endpoint — decode claims from the id_token (e.g. Apple).
if (idToken) {
return this.#decodeIdToken(idToken);
}
return null;
}
// ── User lookup / creation ──────────────────────────────────────
@@ -248,6 +294,12 @@ export class OIDCService extends PuterService {
return this.stores.user.getById(link.user_id as number);
}
async getLinkedProviderForUser(userId: number): Promise<string | null> {
const links = await this.stores.oidc.listByUserId(userId);
if (!links || links.length === 0) return null;
return links[0].provider as string;
}
/**
* Find an existing Puter user by the email claimed by the OIDC provider.
*
@@ -355,6 +407,7 @@ export class OIDCService extends PuterService {
no_temp_user: false,
requires_email_confirmation: false,
message: null as string | null,
code: null as string | null,
};
try {
await this.clients.event?.emitAndWait(
@@ -369,6 +422,7 @@ export class OIDCService extends PuterService {
return {
success: false,
error: validateEvent.message ?? 'Signup blocked',
...(validateEvent.code ? { code: validateEvent.code } : {}),
};
}
@@ -511,18 +565,70 @@ export class OIDCService extends PuterService {
// ── Internals ───────────────────────────────────────────────────
async #fetchGoogleDiscovery(): Promise<Record<string, string> | null> {
if (this.#googleDiscovery) return this.#googleDiscovery;
async #fetchDiscovery(
url: string,
label: string,
): Promise<Record<string, string> | null> {
const cached = this.#discoveryCache.get(url);
if (cached) return cached;
try {
const res = await fetch(GOOGLE_DISCOVERY_URL);
const res = await fetch(url);
if (!res.ok) return null;
this.#googleDiscovery = (await res.json()) as Record<
string,
string
>;
return this.#googleDiscovery;
const data = (await res.json()) as Record<string, string>;
this.#discoveryCache.set(url, data);
return data;
} catch (e) {
console.warn('[oidc] Google discovery fetch failed', e);
console.warn(`[oidc] ${label} discovery fetch failed`, e);
return null;
}
}
#generateAppleClientSecret(
teamId: string,
clientId: string,
keyId: string,
privateKey: string,
): string {
const header = { alg: 'ES256', kid: keyId, typ: 'JWT' };
const now = Math.floor(Date.now() / 1000);
const payload = {
iss: teamId,
sub: clientId,
aud: 'https://appleid.apple.com',
iat: now,
exp: now + 15777000, // ~6 months
};
const headerB64 = Buffer.from(JSON.stringify(header)).toString(
'base64url',
);
const payloadB64 = Buffer.from(JSON.stringify(payload)).toString(
'base64url',
);
const signingInput = `${headerB64}.${payloadB64}`;
const key = crypto.createPrivateKey(privateKey);
const signature = crypto.sign('sha256', Buffer.from(signingInput), {
key,
dsaEncoding: 'ieee-p1363',
});
return `${signingInput}.${signature.toString('base64url')}`;
}
#decodeIdToken(idToken: string): OIDCUserInfo | null {
try {
const parts = idToken.split('.');
if (parts.length !== 3) return null;
const payload = JSON.parse(
Buffer.from(parts[1], 'base64url').toString(),
);
return {
sub: payload.sub,
email: payload.email,
email_verified: payload.email_verified,
};
} catch {
return null;
}
}
+1 -1
View File
@@ -17,7 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { EventClient } from '../../clients/EventClient.js';
import type { EventClient } from '../../clients/event/EventClient.js';
import type { FSEntryStore } from '../../stores/fs/FSEntryStore.js';
import type {
FsRemoveNodeEventPayload,
+30
View File
@@ -37,6 +37,36 @@ import { SocketService } from './socket/SocketService';
import { SubdomainPermissionService } from './subdomain/SubdomainPermissionService';
import type { IPuterServiceRegistry } from './types';
/**
* Populate `IPuterServiceInstances` (declared in `./types`) with the concrete
* types of built-in services. Done via declaration merging instead of
* `LayerInstances<typeof puterServices>` because every concrete service
* extends `PuterService`, whose `protected services` field references this
* type — a direct `typeof puterServices` lookup would self-cycle.
*/
declare module './types' {
interface IPuterServiceInstances {
metering: MeteringService;
permission: PermissionService;
acl: ACLService;
token: TokenService;
auth: AuthService;
fs: FSService;
appPermission: AppPermissionService;
subdomainPermission: SubdomainPermissionService;
recommendedApps: RecommendedAppsService;
suggestedApps: SuggestedAppsService;
socket: SocketService;
notification: NotificationService;
broadcast: BroadcastService;
oidc: OIDCService;
appIcon: AppIconService;
defaultUser: DefaultUserService;
homepage: PuterHomepageService;
health: ServerHealthService;
}
}
// Ordering matters: services declared later see earlier ones as peers.
// ACLService depends on PermissionService (for scan + grant/revoke), so
// PermissionService must be constructed first.
@@ -0,0 +1,754 @@
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import type { Actor } from '../../core/actor.ts';
import { SYSTEM_ACTOR } from '../../core/actor.ts';
import { PuterServer } from '../../server.ts';
import { setupTestServer } from '../../testUtil.ts';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
GLOBAL_APP_KEY,
METRICS_PREFIX,
PERIOD_ESCAPE,
POLICY_PREFIX,
} from './consts.ts';
import type { MeteringService } from './MeteringService.ts';
import { toMicroCents } from './utils.ts';
const escape = (usageType: string) => usageType.replace(/\./g, PERIOD_ESCAPE);
describe('MeteringService', () => {
let server: PuterServer;
let target: MeteringService;
let originalShardCount: number;
// Resolvers and extra policies are stored on private fields of the service
// and there's no public reset. Tests that register hooks pollute later
// tests, so we snapshot the originals once and restore after each test.
type Internals = {
subscriptionResolvers: unknown[];
defaultSubscriptionResolvers: unknown[];
extraPolicies: unknown[];
};
let internals: Internals;
let snapshot: {
subs: unknown[];
defs: unknown[];
pols: unknown[];
};
beforeAll(async () => {
server = await setupTestServer();
target = server.services.metering;
// Smaller shard count makes getGlobalUsage cheap in tests; the
// production value (10000) means ~100 batchGet round-trips per call.
originalShardCount = (target.constructor as typeof MeteringService)
.GLOBAL_SHARD_COUNT;
(target.constructor as typeof MeteringService).GLOBAL_SHARD_COUNT = 4;
(target.constructor as typeof MeteringService).APP_SHARD_COUNT = 4;
internals = target as unknown as Internals;
snapshot = {
subs: [...internals.subscriptionResolvers],
defs: [...internals.defaultSubscriptionResolvers],
pols: [...internals.extraPolicies],
};
});
afterEach(() => {
internals.subscriptionResolvers.length = 0;
internals.subscriptionResolvers.push(...snapshot.subs);
internals.defaultSubscriptionResolvers.length = 0;
internals.defaultSubscriptionResolvers.push(...snapshot.defs);
internals.extraPolicies.length = 0;
internals.extraPolicies.push(...snapshot.pols);
});
afterAll(async () => {
(target.constructor as typeof MeteringService).GLOBAL_SHARD_COUNT =
originalShardCount;
(target.constructor as typeof MeteringService).APP_SHARD_COUNT =
originalShardCount;
await server.shutdown();
});
// Each test uses a fresh user so KV state from one test never leaks into
// the next. Email present → registered-user policy; absent → temp.
let actor: Actor;
const makeUser = (
overrides: Partial<Actor['user']> = {},
): Actor['user'] => ({
uuid: `meter-user-${Math.random().toString(36).slice(2)}`,
username: 'meter-user',
email: '[email protected]',
...overrides,
});
const makeActor = (overrides: Partial<Actor> = {}): Actor => ({
user: makeUser(),
...overrides,
});
beforeEach(() => {
actor = makeActor();
});
// Aux KV writes inside increment paths are fire-and-forget; this helper
// polls until the assertion passes so tests stay deterministic without
// arbitrary sleeps.
const waitFor = (fn: () => unknown | Promise<unknown>) =>
vi.waitFor(fn, { timeout: 2000, interval: 10 });
// ── Subscriptions ────────────────────────────────────────────────
describe('getActorSubscription', () => {
it('returns the registered-user free policy for a user with email', async () => {
const policy = await target.getActorSubscription(actor);
expect(policy.id).toBe(DEFAULT_FREE_SUBSCRIPTION);
expect(policy.monthUsageAllowance).toBeGreaterThan(0);
});
it('returns the temp policy for a user without email', async () => {
const tempActor: Actor = {
user: makeUser({ email: null }),
};
const policy = await target.getActorSubscription(tempActor);
expect(policy.id).toBe(DEFAULT_TEMP_SUBSCRIPTION);
});
it('uses the first non-empty subscription resolver', async () => {
const customPolicy = {
id: 'custom-paid',
monthUsageAllowance: toMicroCents(10),
monthlyStorageAllowance: 1024 * 1024 * 1024,
};
target.registerPolicy(customPolicy);
const stub = vi.fn(async () => 'custom-paid');
target.registerSubscriptionResolver(stub);
const policy = await target.getActorSubscription(actor);
expect(policy.id).toBe('custom-paid');
expect(stub).toHaveBeenCalledWith(actor);
});
it('falls through to the default resolver when the primary returns nothing', async () => {
const customDefault = {
id: 'custom-default',
monthUsageAllowance: toMicroCents(2),
monthlyStorageAllowance: 1024 * 1024 * 1024,
};
target.registerPolicy(customDefault);
target.registerSubscriptionResolver(async () => null);
target.registerDefaultSubscriptionResolver(
async () => 'custom-default',
);
const policy = await target.getActorSubscription(actor);
expect(policy.id).toBe('custom-default');
});
it('rejects an actor with no user uuid', async () => {
await expect(
target.getActorSubscription({
user: { uuid: '' },
}),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── Addons ───────────────────────────────────────────────────────
describe('getActorAddons / updateAddonCredit', () => {
it('returns an empty addon map for a fresh user', async () => {
const addons = await target.getActorAddons(actor);
expect(addons).toEqual({});
});
it('updateAddonCredit increments purchasedCredits', async () => {
await target.updateAddonCredit(actor.user.uuid, 1000);
const addons = await target.getActorAddons(actor);
expect(addons.purchasedCredits).toBe(1000);
await target.updateAddonCredit(actor.user.uuid, 500);
const updated = await target.getActorAddons(actor);
expect(updated.purchasedCredits).toBe(1500);
});
it('updateAddonCredit throws without a userId', async () => {
await expect(target.updateAddonCredit('', 100)).rejects.toThrow();
});
it('rejects getActorAddons for an actor with no user uuid', async () => {
await expect(
target.getActorAddons({ user: { uuid: '' } }),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── incrementUsage ───────────────────────────────────────────────
describe('incrementUsage', () => {
it('records cost, units, and count for a single usage type', async () => {
const cost = 250;
const result = await target.incrementUsage(
actor,
'kv:read',
4,
cost,
);
expect(result.total).toBe(cost);
const record = result['kv:read'];
expect(record).toMatchObject({ cost, units: 4, count: 1 });
});
it('escapes dots in usage type names so KV nested paths do not collide', async () => {
await target.incrementUsage(actor, 'driver.foo.bar', 2, 100);
const { usage } =
await target.getActorCurrentMonthUsageDetails(actor);
// Returned shape uses the escaped key (raw KV layout).
const record = (usage as Record<string, unknown>)[
escape('driver.foo.bar')
];
expect(record).toMatchObject({ cost: 100, units: 2, count: 1 });
});
it('accumulates across calls', async () => {
await target.incrementUsage(actor, 'kv:read', 1, 10);
const second = await target.incrementUsage(actor, 'kv:read', 3, 20);
expect(second.total).toBe(30);
expect(second['kv:read']).toMatchObject({
cost: 30,
units: 4,
count: 2,
});
});
it('returns a zero result for a system actor and writes nothing', async () => {
const result = await target.incrementUsage(
SYSTEM_ACTOR,
'kv:read',
1,
100,
);
expect(result).toEqual({ total: 0 });
});
it.each([
['zero amount', 'kv:read', 0],
['empty usage type', '', 1],
])('skips when %s', async (_label, type, amount) => {
const result = await target.incrementUsage(actor, type, amount, 5);
expect(result).toEqual({ total: 0 });
const { usage } =
await target.getActorCurrentMonthUsageDetails(actor);
expect(usage.total ?? 0).toBe(0);
});
it('normalizes a negative usageAmount to 1', async () => {
const result = await target.incrementUsage(
actor,
'kv:read',
-5,
10,
);
expect(result['kv:read']).toMatchObject({ units: 1 });
});
it('normalizes a negative costOverride to 1 and raises an alarm', async () => {
const alarmSpy = vi.spyOn(server.clients.alarm, 'create');
const result = await target.incrementUsage(
actor,
'kv:read',
1,
-42,
);
expect(result['kv:read']).toMatchObject({ cost: 1, units: 1 });
expect(alarmSpy).toHaveBeenCalledWith(
expect.stringContaining('negative cost'),
expect.any(String),
expect.objectContaining({ usageType: 'kv:read' }),
);
alarmSpy.mockRestore();
});
it('treats a missing costOverride as zero cost', async () => {
const result = await target.incrementUsage(actor, 'kv:read', 2);
expect(result.total).toBe(0);
expect(result['kv:read']).toMatchObject({
cost: 0,
units: 2,
count: 1,
});
});
it('writes the per-actor / per-app aux record', async () => {
const appActor: Actor = {
user: makeUser(),
app: { uid: 'my-app', id: 1 },
};
await target.incrementUsage(appActor, 'kv:read', 1, 100);
await waitFor(async () => {
const u = await target.getActorAppUsage(appActor, 'my-app');
expect(u.total).toBe(100);
});
});
it('consumes purchased credits once monthly allowance is exceeded', async () => {
const overActor: Actor = { user: makeUser() };
const sub = await target.getActorSubscription(overActor);
await target.updateAddonCredit(overActor.user.uuid, 5_000_000);
// Spend the entire monthly allowance — no overage yet.
await target.incrementUsage(
overActor,
'kv:read',
1,
sub.monthUsageAllowance,
);
// First overage of 1_000_000 micro-cents should pull from credits.
await target.incrementUsage(overActor, 'kv:read', 1, 1_000_000);
await waitFor(async () => {
const addons = await target.getActorAddons(overActor);
expect(addons.consumedPurchaseCredits).toBe(1_000_000);
});
});
});
// ── batchIncrementUsages ─────────────────────────────────────────
describe('batchIncrementUsages', () => {
it('aggregates multiple usages into a single actor record', async () => {
const result = await target.batchIncrementUsages(actor, [
{ usageType: 'kv:read', usageAmount: 2, costOverride: 100 },
{ usageType: 'kv:write', usageAmount: 1, costOverride: 50 },
{ usageType: 'kv:read', usageAmount: 3, costOverride: 30 },
]);
expect(result.total).toBe(180);
expect(result['kv:read']).toMatchObject({
cost: 130,
units: 5,
count: 2,
});
expect(result['kv:write']).toMatchObject({
cost: 50,
units: 1,
count: 1,
});
});
it('returns zero for an empty list', async () => {
const result = await target.batchIncrementUsages(actor, []);
expect(result).toEqual({ total: 0 });
});
it('returns zero for a system actor and writes nothing', async () => {
const result = await target.batchIncrementUsages(SYSTEM_ACTOR, [
{ usageType: 'kv:read', usageAmount: 1, costOverride: 100 },
]);
expect(result).toEqual({ total: 0 });
});
it('skips items with missing fields but still writes the rest', async () => {
const result = await target.batchIncrementUsages(actor, [
{ usageType: 'kv:read', usageAmount: 1, costOverride: 10 },
{ usageType: '', usageAmount: 1, costOverride: 999 },
{ usageType: 'kv:write', usageAmount: 0, costOverride: 999 },
{ usageType: 'kv:write', usageAmount: 2, costOverride: 20 },
]);
expect(result.total).toBe(30);
expect(result['kv:read']).toMatchObject({ count: 1, units: 1 });
expect(result['kv:write']).toMatchObject({ count: 1, units: 2 });
});
it('raises an alarm for any negative costOverride in the batch', async () => {
const alarmSpy = vi.spyOn(server.clients.alarm, 'create');
await target.batchIncrementUsages(actor, [
{ usageType: 'kv:read', usageAmount: 1, costOverride: -7 },
]);
expect(alarmSpy).toHaveBeenCalledWith(
expect.stringContaining('negative cost'),
expect.any(String),
expect.objectContaining({ usageType: 'kv:read' }),
);
alarmSpy.mockRestore();
});
});
// ── utilRecordUsageObject ────────────────────────────────────────
describe('utilRecordUsageObject', () => {
it('prefixes each usage kind with the modelPrefix and applies overrides', async () => {
const result = await target.utilRecordUsageObject(
{ prompt_tokens: 100, completion_tokens: 50 },
actor,
'gpt-4',
{ prompt_tokens: 1000 },
);
expect(result['gpt-4:prompt_tokens']).toMatchObject({
cost: 1000,
units: 100,
count: 1,
});
// No override → cost defaults to 0
expect(result['gpt-4:completion_tokens']).toMatchObject({
cost: 0,
units: 50,
count: 1,
});
expect(result.total).toBe(1000);
});
it('ignores non-numeric override values', async () => {
const result = await target.utilRecordUsageObject(
{ prompt_tokens: 1 },
actor,
'm',
{ prompt_tokens: Number.NaN },
);
expect(result['m:prompt_tokens']).toMatchObject({ cost: 0 });
});
});
// ── getActorCurrentMonthUsageDetails ─────────────────────────────
describe('getActorCurrentMonthUsageDetails', () => {
it('returns an empty envelope for a fresh user', async () => {
const result = await target.getActorCurrentMonthUsageDetails(actor);
expect(result.usage).toEqual({ total: 0 });
expect(result.appTotals).toEqual({});
});
it('returns the recorded usage and app totals after increments', async () => {
const userId = actor.user.uuid;
const appA: Actor = {
user: { uuid: userId },
app: { uid: 'A', id: 1 },
};
const appB: Actor = {
user: { uuid: userId },
app: { uid: 'B', id: 2 },
};
await target.incrementUsage(appA, 'kv:read', 1, 100);
await target.incrementUsage(appB, 'kv:read', 1, 50);
await waitFor(async () => {
const r = await target.getActorCurrentMonthUsageDetails({
user: { uuid: userId },
});
expect(r.appTotals.A?.total).toBe(100);
expect(r.appTotals.B?.total).toBe(50);
});
const result = await target.getActorCurrentMonthUsageDetails({
user: { uuid: userId },
});
expect(result.usage.total).toBe(150);
});
it('filters appTotals by actor.app.uid and rolls others into "others"', async () => {
const userId = actor.user.uuid;
const appA: Actor = {
user: { uuid: userId },
app: { uid: 'A', id: 1 },
};
const appB: Actor = {
user: { uuid: userId },
app: { uid: 'B', id: 2 },
};
await target.incrementUsage(appA, 'kv:read', 1, 100);
await target.incrementUsage(appB, 'kv:read', 1, 50);
await waitFor(async () => {
const r = await target.getActorCurrentMonthUsageDetails(appA);
expect(r.appTotals.A?.total).toBe(100);
expect(r.appTotals.others?.total).toBe(50);
expect(r.appTotals).not.toHaveProperty('B');
});
});
it('rejects an actor with no user uuid', async () => {
await expect(
target.getActorCurrentMonthUsageDetails({
user: { uuid: '' },
}),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── getActorCurrentMonthAppUsageDetails ──────────────────────────
describe('getActorCurrentMonthAppUsageDetails', () => {
it('returns the per-app record for an explicit appId', async () => {
const appActor: Actor = {
user: makeUser(),
app: { uid: 'my-app', id: 1 },
};
await target.incrementUsage(appActor, 'kv:read', 1, 250);
await waitFor(async () => {
const r = await target.getActorCurrentMonthAppUsageDetails(
appActor,
'my-app',
);
expect(r.total).toBe(250);
});
});
it('defaults to the actor app id when none is supplied', async () => {
const appActor: Actor = {
user: makeUser(),
app: { uid: 'my-app', id: 1 },
};
await target.incrementUsage(appActor, 'kv:read', 1, 75);
await waitFor(async () => {
const r =
await target.getActorCurrentMonthAppUsageDetails(appActor);
expect(r.total).toBe(75);
});
});
it('allows an app actor to query the global namespace', async () => {
const userOnly: Actor = { user: makeUser() };
await target.incrementUsage(userOnly, 'kv:read', 1, 60);
const appActor: Actor = {
user: userOnly.user,
app: { uid: 'my-app', id: 1 },
};
await waitFor(async () => {
const r = await target.getActorCurrentMonthAppUsageDetails(
appActor,
GLOBAL_APP_KEY,
);
expect(r.total).toBe(60);
});
});
it('forbids an app actor from querying another app', async () => {
const appActor: Actor = {
user: makeUser(),
app: { uid: 'mine', id: 1 },
};
await expect(
target.getActorCurrentMonthAppUsageDetails(
appActor,
'someone-else',
),
).rejects.toMatchObject({ statusCode: 403 });
});
it('rejects an actor with no user uuid', async () => {
await expect(
target.getActorCurrentMonthAppUsageDetails({
user: { uuid: '' },
}),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── setActorCurrentMonthUsageTotal ───────────────────────────────
describe('setActorCurrentMonthUsageTotal', () => {
it('sets the total via a manual_adjustment delta when no usage exists', async () => {
const result = await target.setActorCurrentMonthUsageTotal(
actor,
500,
);
expect(result.total).toBe(500);
const adj = (result as Record<string, unknown>)
.manual_adjustment as
| { cost: number; units: number; count: number }
| undefined;
expect(adj).toMatchObject({ cost: 500, units: 500, count: 1 });
});
it('applies a delta against an existing total', async () => {
await target.incrementUsage(actor, 'kv:read', 1, 100);
const result = await target.setActorCurrentMonthUsageTotal(
actor,
300,
);
expect(result.total).toBe(300);
});
it('is a no-op when delta is zero', async () => {
await target.incrementUsage(actor, 'kv:read', 1, 100);
const result = await target.setActorCurrentMonthUsageTotal(
actor,
100,
);
expect(result.total).toBe(100);
});
it('rejects a negative total', async () => {
await expect(
target.setActorCurrentMonthUsageTotal(actor, -1),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a non-finite total', async () => {
await expect(
target.setActorCurrentMonthUsageTotal(actor, Number.NaN),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects an actor with no user uuid', async () => {
await expect(
target.setActorCurrentMonthUsageTotal(
{ user: { uuid: '' } },
100,
),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── getActorAppUsage ─────────────────────────────────────────────
describe('getActorAppUsage', () => {
it('returns zero for an app the user has no usage in', async () => {
const result = await target.getActorAppUsage(actor, 'untouched');
expect(result.total).toBe(0);
});
it('forbids an app actor from reading another app', async () => {
const appActor: Actor = {
user: makeUser(),
app: { uid: 'mine', id: 1 },
};
await expect(
target.getActorAppUsage(appActor, 'theirs'),
).rejects.toMatchObject({ statusCode: 403 });
});
it('rejects an actor with no user uuid', async () => {
await expect(
target.getActorAppUsage({ user: { uuid: '' } }, 'app'),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── allowance / credits ──────────────────────────────────────────
describe('getRemainingUsage / getAllowedUsage / hasAnyUsage / hasEnoughCredits', () => {
it('a fresh user has the full subscription allowance remaining', async () => {
const allowed = await target.getAllowedUsage(actor);
expect(allowed.remaining).toBe(allowed.monthUsageAllowance);
expect(allowed.monthUsageAllowance).toBeGreaterThan(0);
expect(allowed.addons).toEqual({});
});
it('subtracts spent usage from remaining', async () => {
await target.incrementUsage(actor, 'kv:read', 1, 1_000);
const allowed = await target.getAllowedUsage(actor);
expect(allowed.remaining).toBe(allowed.monthUsageAllowance - 1_000);
});
it('adds purchased credits to remaining', async () => {
await target.updateAddonCredit(actor.user.uuid, 5_000);
const allowed = await target.getAllowedUsage(actor);
expect(allowed.remaining).toBe(allowed.monthUsageAllowance + 5_000);
});
it('clamps remaining at zero when over allowance with no credits', async () => {
const sub = await target.getActorSubscription(actor);
await target.incrementUsage(
actor,
'kv:read',
1,
sub.monthUsageAllowance + 5_000,
);
const remaining = await target.getRemainingUsage(actor);
expect(remaining).toBe(0);
});
it('hasAnyUsage tracks remaining', async () => {
const sub = await target.getActorSubscription(actor);
expect(await target.hasAnyUsage(actor)).toBe(true);
await target.incrementUsage(
actor,
'kv:read',
1,
sub.monthUsageAllowance,
);
expect(await target.hasAnyUsage(actor)).toBe(false);
});
it('hasEnoughCredits compares remaining against the requested amount', async () => {
await target.updateAddonCredit(actor.user.uuid, 1_000);
expect(await target.hasEnoughCredits(actor, 100)).toBe(true);
expect(
await target.hasEnoughCredits(actor, Number.MAX_SAFE_INTEGER),
).toBe(false);
});
});
// ── getGlobalUsage ───────────────────────────────────────────────
describe('getGlobalUsage', () => {
it('aggregates increments across actors into the same global view', async () => {
const before = await target.getGlobalUsage();
const user1: Actor = { user: makeUser() };
const user2: Actor = { user: makeUser() };
await target.incrementUsage(user1, 'kv:read', 1, 100);
await target.incrementUsage(user2, 'kv:read', 1, 200);
await waitFor(async () => {
const now = await target.getGlobalUsage();
expect(now.total - before.total).toBe(300);
const beforeRead = (before['kv:read']?.cost ?? 0) as number;
const nowRead = (now['kv:read']?.cost ?? 0) as number;
expect(nowRead - beforeRead).toBe(300);
});
});
});
// ── KV layout sanity check ───────────────────────────────────────
describe('KV layout', () => {
it('writes the actor monthly record at the expected key shape', async () => {
await target.incrementUsage(actor, 'kv:read', 1, 100);
const month = `${new Date().getUTCFullYear()}-${String(
new Date().getUTCMonth() + 1,
).padStart(2, '0')}`;
const key = `${METRICS_PREFIX}:actor:${actor.user.uuid}:${month}`;
const { res } = await server.stores.kv.get({ key });
expect(res).toMatchObject({ total: 100 });
});
it('persists addons under the policy prefix', async () => {
await target.updateAddonCredit(actor.user.uuid, 250);
const key = `${POLICY_PREFIX}:actor:${actor.user.uuid}:addons`;
const { res } = await server.stores.kv.get({ key });
expect(res).toMatchObject({ purchasedCredits: 250 });
});
it('persists lastUpdated after an increment', async () => {
const userId = actor.user.uuid;
await target.incrementUsage(actor, 'kv:read', 1, 50);
await waitFor(async () => {
const { res } = await server.stores.kv.get({
key: `${METRICS_PREFIX}:actor:${userId}:lastUpdated`,
});
expect(typeof res).toBe('number');
expect(res).toBeGreaterThan(0);
});
});
});
// ── Resolver registration ────────────────────────────────────────
describe('resolver registration', () => {
it('a default resolver that throws does not break subscription resolution', async () => {
target.registerDefaultSubscriptionResolver(async () => {
throw new Error('boom');
});
const policy = await target.getActorSubscription(actor);
expect(policy.id).toBe(DEFAULT_FREE_SUBSCRIPTION);
});
});
});
@@ -18,10 +18,10 @@
*/
import murmurhash from 'murmurhash';
import { PuterService } from '../types';
import type { Actor } from '../../core/actor';
import { isSystemActor } from '../../core/actor';
import { HttpError } from '../../core/http/HttpError.js';
import { PuterService } from '../types';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
@@ -697,7 +697,7 @@ export class MeteringService extends PuterService {
...this.extraPolicies,
...SUB_POLICIES,
...(this.config.unlimitedMetering ? [UNLIMITED_SUBSCRIPTION] : []),
];
] as SubscriptionPolicy[];
return (
availablePolicies.find((p) => p.id === resolvedUser) ??
availablePolicies.find((p) => p.id === resolvedDefault)!
@@ -43,6 +43,7 @@ const ADMIN_STORAGE_BYTES = 10 * 1024 * 1024 * 1024;
*/
export class DefaultUserService extends PuterService {
override async onServerStart(): Promise<void> {
if (this.config.no_default_user) return;
let user = await this.stores.user.getByUsername(USERNAME);
let tmpPassword: string;
+60 -10
View File
@@ -18,27 +18,77 @@
*/
import type { puterClients } from '../clients';
import type { puterStores } from '../stores';
import type { IExtensionClientInstances } from '../clients/types';
import type {
IExtensionStoreInstances,
IPuterStoreInstances,
} from '../stores/types';
import type { IConfig, LayerInstances, WithLifecycle } from '../types';
/**
* Services may depend on clients, stores, and *prior* services (those declared
* earlier in the registry). The `services` argument is the accumulating
* registry — it only contains peers constructed before this one.
* Built-in service instance registry. Forward-declared here and populated
* via declaration merging from `services/index.ts` to avoid the circular
* `typeof puterServices` reference (services extend `PuterService`, whose
* `protected services` field references this type).
*
* Consumers see the merged `IPuterServiceInstances & IExtensionServiceInstances`
* type — built-in keys + extension-augmented keys.
*/
// eslint-disable-next-line @typescript-eslint/no-empty-object-type
export interface IPuterServiceInstances {}
/**
* Extension-augmentable service registry. Extensions add their own service
* instance types via TypeScript declaration merging:
*
* declare module '@heyputer/backend/services/types' {
* interface IExtensionServiceInstances {
* myService: MyService;
* }
* }
*
* Augmentations flow into `this.services` (PuterController, PuterDriver) and
* into the `extension.import('service')` proxy. NOT applied to `PuterService`'s
* own `services` constructor argument — that view is the partial registry of
* peers declared earlier than this service.
*/
export interface IExtensionServiceInstances {
/**
* Open index signature so reads of extension-only service keys return
* `unknown` instead of a type error. Concrete declaration-merged keys
* override this for that name.
*/
[key: string]: unknown;
}
/**
* Services may depend on clients, stores, and *prior* services (those
* declared earlier in the registry).
*
* Type contract caveat: `services` is typed as the FULLY-populated registry,
* even though at construction time only prior services exist. This is a
* deliberate trade-off — almost every `this.services.X` access happens in
* handler/lifecycle methods (which run after all services are wired up), so
* the convenience of typed access in those sites outweighs the construction-
* time inaccuracy. Don't read `this.services.X` from a service constructor
* unless you've verified `X` is registered earlier in the registry.
*/
export type IPuterService<T extends WithLifecycle = WithLifecycle> = new (
config: IConfig,
clients: LayerInstances<typeof puterClients>,
stores: LayerInstances<typeof puterStores>,
services: Partial<Record<string, WithLifecycle>>,
clients: LayerInstances<typeof puterClients> & IExtensionClientInstances,
stores: IPuterStoreInstances & IExtensionStoreInstances,
services: IPuterServiceInstances & IExtensionServiceInstances,
) => T;
export const PuterService = class PuterService implements WithLifecycle {
constructor(
protected config: IConfig,
protected clients: LayerInstances<typeof puterClients>,
protected stores: LayerInstances<typeof puterStores>,
protected services: Partial<Record<string, WithLifecycle>> = {},
protected clients: LayerInstances<typeof puterClients> &
IExtensionClientInstances,
protected stores: IPuterStoreInstances & IExtensionStoreInstances,
protected services: IPuterServiceInstances &
IExtensionServiceInstances = {} as IPuterServiceInstances &
IExtensionServiceInstances,
) {}
public onServerStart() {
return;
+111
View File
@@ -38,6 +38,10 @@ const LIST_CACHE_TTL_SECONDS = 15 * 60;
const FILETYPE_CACHE_KEY_PREFIX = 'apps:by-filetype';
const FILETYPE_CACHE_TTL_SECONDS = 60;
const APP_ID_PROPERTIES = ['id', 'uid', 'name'];
// Old-name redirect window. After this many months an entry in
// `old_app_names` is considered stale and is deleted on the next read
// against that name (lazy GC — no background sweep needed).
const OLD_APP_NAME_TTL_MONTHS = 3;
// Cap on placeholders per `IN (?, ?, …)` query. SQLite's default parameter
// limit is 999; staying well under that keeps `getByIds` portable across
// backends without splitting the cap by driver.
@@ -341,6 +345,9 @@ export class AppStore extends PuterStore {
throw new Error('Failed to create app — no insertId returned');
const fresh = await this.getById(insertId);
if (fresh?.name) {
await this.#clearOldAppNamesForName(fresh.name);
}
await this.#invalidateListCachesForApps([fresh]);
return fresh;
}
@@ -398,14 +405,75 @@ export class AppStore extends PuterStore {
[...values, appId],
);
const renamed =
before &&
typeof allowed.name === 'string' &&
allowed.name !== before.name;
if (renamed) {
await this.#recordOldAppName(before.uid, before.name);
// The new name now belongs to `appId`, so any redirect that
// used to point to a different app for that name is no
// longer accurate — drop it. Also covers the reclaim case
// where this same app's earlier (uid, newName) row would
// otherwise linger as a self-redirect.
await this.#clearOldAppNamesForName(allowed.name);
}
const fresh = await this.#readFromDb('id', appId);
await this.#invalidateListCachesForApps([before, fresh]);
if (fresh) {
await this.#refreshCache({ ...fresh, ...allowed });
}
if (renamed) {
// Drop any stale `apps:name:<old>` cache so the next
// getByName(old) goes through the redirect path with fresh
// data instead of returning a copy of the app under its old
// name.
await this.publishCacheKeys({
keys: [this.#cacheKey('name', before.name)],
broadcast: true,
});
}
return fresh;
}
/**
* Record `oldName` as a redirect to the app identified by `appUid`.
* The (app_uid, name) tuple is unique in `old_app_names`, so the
* upsert refreshes the timestamp when the same app re-records the
* same previous name (which happens whenever a user renames back
* and forth) — that resets the TTL clock so the redirect stays live
* for another {@link OLD_APP_NAME_TTL_MONTHS} months.
*/
async #recordOldAppName(appUid, oldName) {
if (!appUid || typeof oldName !== 'string' || oldName.length === 0) {
return;
}
const upsertClause = this.clients.db.case({
mysql: 'ON DUPLICATE KEY UPDATE `timestamp` = CURRENT_TIMESTAMP',
otherwise:
'ON CONFLICT(`app_uid`, `name`) DO UPDATE SET `timestamp` = CURRENT_TIMESTAMP',
});
await this.clients.db.write(
`INSERT INTO \`old_app_names\` (\`app_uid\`, \`name\`) VALUES (?, ?) ${upsertClause}`,
[appUid, oldName],
);
}
/**
* Drop every `old_app_names` row for `name`. Called whenever a name
* is freshly claimed in `apps` (create or rename-into) so the
* authoritative live row isn't shadowed by a stale redirect that
* pointed `name` at some prior owner.
*/
async #clearOldAppNamesForName(name) {
if (typeof name !== 'string' || name.length === 0) return;
await this.clients.db.write(
'DELETE FROM `old_app_names` WHERE `name` = ?',
[name],
);
}
/**
* Delete an app row. Also deletes its filetype associations.
* Invalidates cache.
@@ -595,6 +663,20 @@ export class AppStore extends PuterStore {
}
async #readFromDb(prop, value) {
if (prop === 'name') {
// Direct match on the live `apps.name` always wins — a current
// owner of the name takes precedence over any historical
// redirect still lingering in `old_app_names`.
const directRows = await this.clients.db.read(
'SELECT * FROM `apps` WHERE `name` = ? LIMIT 1',
[value],
);
if (directRows.length > 0) {
return this.#normalizeRow(directRows[0]);
}
return this.#resolveByOldName(value);
}
const rows = await this.clients.db.read(
`SELECT * FROM \`apps\` WHERE \`${prop}\` = ? LIMIT 1`,
[value],
@@ -603,6 +685,35 @@ export class AppStore extends PuterStore {
return this.#normalizeRow(rows[0]);
}
/**
* Resolve an app by a previously-used name via `old_app_names`.
* Lazy-GCs entries older than {@link OLD_APP_NAME_TTL_MONTHS}: a
* cutoff DELETE runs before the JOIN, so an expired redirect is
* removed and the lookup returns null on the very same call.
*/
async #resolveByOldName(name) {
const cutoffClause = this.clients.db.case({
sqlite: `datetime('now', '-${OLD_APP_NAME_TTL_MONTHS} months')`,
otherwise: `(NOW() - INTERVAL ${OLD_APP_NAME_TTL_MONTHS} MONTH)`,
});
await this.clients.db.write(
`DELETE FROM \`old_app_names\` WHERE \`name\` = ? AND \`timestamp\` < ${cutoffClause}`,
[name],
);
const rows = await this.clients.db.read(
`SELECT a.* FROM \`apps\` AS a
INNER JOIN \`old_app_names\` AS o ON o.\`app_uid\` = a.\`uid\`
WHERE o.\`name\` = ?
ORDER BY o.\`timestamp\` DESC
LIMIT 1`,
[name],
);
if (rows.length === 0) return null;
return this.#normalizeRow(rows[0]);
}
#cacheKey(prop, value) {
return `${CACHE_KEY_PREFIX}:${prop}:${value}`;
}
+24
View File
@@ -31,6 +31,30 @@ import { SystemKVStore } from './systemKv/SystemKVStore.js';
import { UserStore } from './user/UserStore.js';
import type { IPuterStoreRegistry } from './types.js';
/**
* Populate `IPuterStoreInstances` (declared in `./types`) with the concrete
* types of built-in stores. Done via declaration merging instead of
* `LayerInstances<typeof puterStores>` because every concrete store extends
* `PuterStore`, whose `protected stores` field references this type — a
* direct `typeof puterStores` lookup would self-cycle.
*/
declare module './types.js' {
interface IPuterStoreInstances {
kv: SystemKVStore;
user: UserStore;
app: AppStore;
fsEntry: FSEntryStore;
s3Object: S3ObjectStore;
subdomain: SubdomainStore;
notification: NotificationStore;
share: ShareStore;
group: GroupStore;
permission: PermissionStore;
session: SessionStore;
oidc: OIDCStore;
}
}
// Ordering matters: stores declared later see earlier ones as peers.
// PermissionStore depends on `kv`, so `kv` must come first.
// UserStore / AppStore are leaves (db + redis only); sit early so other
@@ -0,0 +1,559 @@
import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest';
import { setupTestServer } from '../../testUtil.ts';
import type { SystemKVStore } from './SystemKVStore.ts';
import { PuterServer } from '../../server.ts';
import type { Actor } from '../../core/actor.ts';
describe('SystemKVStore', () => {
let server: PuterServer;
let target: SystemKVStore;
beforeAll(async () => {
server = await setupTestServer();
target = server.stores.kv;
});
afterAll(async () => {
await server.shutdown();
});
// Each test runs against a fresh actor namespace so state from one test
// never leaks into another. Actors are cheap; creating a unique uuid per
// test gives full isolation without flush() teardown ceremony.
let actor: Actor;
let opts: { actor: Actor };
beforeEach(() => {
actor = {
user: { uuid: `test-user-${Math.random().toString(36).slice(2)}` },
};
opts = { actor };
});
describe('set / get', () => {
it('round-trips a value through the system namespace', async () => {
await target.set({ key: 'systemKey', value: 'systemValue' });
const value = await target.get({ key: 'systemKey' });
expect(value.res).toBe('systemValue');
});
it('returns null for a missing key', async () => {
const result = await target.get({ key: 'doesNotExist' }, opts);
expect(result.res).toBeNull();
});
it('overwrites a previously-set value', async () => {
await target.set({ key: 'k', value: 'first' }, opts);
await target.set({ key: 'k', value: 'second' }, opts);
const result = await target.get({ key: 'k' }, opts);
expect(result.res).toBe('second');
});
it('stores complex object values', async () => {
const value = { nested: { count: 1 }, items: [1, 2, 3] };
await target.set({ key: 'obj', value }, opts);
const result = await target.get({ key: 'obj' }, opts);
expect(result.res).toEqual(value);
});
it('rejects an empty key', async () => {
await expect(
target.set({ key: '', value: 'x' }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a key over 1024 bytes', async () => {
const oversized = 'a'.repeat(1025);
await expect(
target.set({ key: oversized, value: 'x' }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a value over the size limit', async () => {
const huge = 'a'.repeat(400 * 1024);
await expect(
target.set({ key: 'big', value: huge }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('treats a value with an already-elapsed TTL as missing on read', async () => {
const past = Math.floor(Date.now() / 1000) - 10;
await target.set(
{ key: 'expired', value: 'gone', expireAt: past },
opts,
);
const result = await target.get({ key: 'expired' }, opts);
expect(result.res).toBeNull();
});
it('isolates values by actor namespace', async () => {
const otherActor: Actor = { user: { uuid: 'other-user-uuid' } };
await target.set({ key: 'shared', value: 'mine' }, opts);
const otherResult = await target.get(
{ key: 'shared' },
{ actor: otherActor },
);
expect(otherResult.res).toBeNull();
});
it('returns an array of values when called with an array of keys', async () => {
await target.set({ key: 'a', value: 1 }, opts);
await target.set({ key: 'b', value: 2 }, opts);
const result = await target.get(
{ key: ['a', 'b', 'missing'] },
opts,
);
expect(result.res).toEqual([1, 2, null]);
});
});
describe('batchPut', () => {
it('writes multiple items and they read back', async () => {
await target.batchPut(
{
items: [
{ key: 'bp1', value: 'v1' },
{ key: 'bp2', value: 'v2' },
{ key: 'bp3', value: { nested: true } },
],
},
opts,
);
const result = await target.get(
{ key: ['bp1', 'bp2', 'bp3'] },
opts,
);
expect(result.res).toEqual(['v1', 'v2', { nested: true }]);
});
it('is a no-op for an empty items array', async () => {
const result = await target.batchPut({ items: [] }, opts);
expect(result.res).toBe(true);
});
it('deduplicates by key, keeping the last value for repeated keys', async () => {
await target.batchPut(
{
items: [
{ key: 'dup', value: 'first' },
{ key: 'dup', value: 'last' },
],
},
opts,
);
const result = await target.get({ key: 'dup' }, opts);
expect(result.res).toBe('last');
});
it('rejects when any item has an oversized key', async () => {
await expect(
target.batchPut(
{
items: [
{ key: 'ok', value: 1 },
{ key: 'a'.repeat(1025), value: 2 },
],
},
opts,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('del', () => {
it('removes a previously-set key', async () => {
await target.set({ key: 'gone', value: 'bye' }, opts);
await target.del({ key: 'gone' }, opts);
const result = await target.get({ key: 'gone' }, opts);
expect(result.res).toBeNull();
});
it('is idempotent when deleting a missing key', async () => {
const result = await target.del({ key: 'never-existed' }, opts);
expect(result.res).toBe(true);
});
});
describe('list', () => {
beforeEach(async () => {
await target.batchPut(
{
items: [
{ key: 'fruit:apple', value: 'red' },
{ key: 'fruit:banana', value: 'yellow' },
{ key: 'veg:carrot', value: 'orange' },
],
},
opts,
);
});
it('returns key/value entries by default', async () => {
const result = await target.list({}, opts);
expect(Array.isArray(result.res)).toBe(true);
expect(result.res).toEqual(
expect.arrayContaining([
{ key: 'fruit:apple', value: 'red' },
{ key: 'fruit:banana', value: 'yellow' },
{ key: 'veg:carrot', value: 'orange' },
]),
);
});
it('returns just keys when as=keys', async () => {
const result = await target.list({ as: 'keys' }, opts);
expect(result.res).toEqual(
expect.arrayContaining([
'fruit:apple',
'fruit:banana',
'veg:carrot',
]),
);
});
it('returns just values when as=values', async () => {
const result = await target.list({ as: 'values' }, opts);
expect(result.res).toEqual(
expect.arrayContaining(['red', 'yellow', 'orange']),
);
});
it('rejects an unsupported as= value', async () => {
await expect(
// @ts-expect-error intentionally bad input
target.list({ as: 'bogus' }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('filters by a wildcard prefix pattern', async () => {
const result = await target.list(
{ as: 'keys', pattern: 'fruit:*' },
opts,
);
expect(result.res).toEqual(
expect.arrayContaining(['fruit:apple', 'fruit:banana']),
);
expect(result.res as string[]).not.toContain('veg:carrot');
});
it('returns a paginated envelope when limit is supplied', async () => {
const result = await target.list({ limit: 1 }, opts);
const envelope = result.res as {
items: unknown[];
cursor?: string;
};
expect(envelope.items.length).toBe(1);
// With three items and limit 1 there should be a continuation cursor
expect(typeof envelope.cursor).toBe('string');
});
it('rejects a non-positive limit', async () => {
await expect(
target.list({ limit: 0 }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a malformed cursor', async () => {
await expect(
target.list({ cursor: 'not-base64-or-json' }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('omits TTL-expired entries', async () => {
await target.set(
{
key: 'short-lived',
value: 'old',
expireAt: Math.floor(Date.now() / 1000) - 10,
},
opts,
);
const result = await target.list({ as: 'keys' }, opts);
expect(result.res as string[]).not.toContain('short-lived');
});
});
describe('flush', () => {
it('removes every key in the actor namespace', async () => {
await target.batchPut(
{
items: [
{ key: 'f1', value: 1 },
{ key: 'f2', value: 2 },
],
},
opts,
);
await target.flush(opts);
const result = await target.list({ as: 'keys' }, opts);
expect(result.res).toEqual([]);
});
it('only flushes the calling actor namespace', async () => {
const otherActor: Actor = { user: { uuid: 'flush-other-user' } };
await target.set({ key: 'mine', value: 1 }, opts);
await target.set(
{ key: 'theirs', value: 2 },
{ actor: otherActor },
);
await target.flush(opts);
const mine = await target.get({ key: 'mine' }, opts);
const theirs = await target.get(
{ key: 'theirs' },
{ actor: otherActor },
);
expect(mine.res).toBeNull();
expect(theirs.res).toBe(2);
});
});
describe('expireAt / expire', () => {
it('expireAt makes a key invisible once the timestamp passes', async () => {
await target.set({ key: 'fade', value: 'soon' }, opts);
await target.expireAt(
{ key: 'fade', timestamp: Math.floor(Date.now() / 1000) - 5 },
opts,
);
const result = await target.get({ key: 'fade' }, opts);
expect(result.res).toBeNull();
});
it('expire computes the TTL relative to now', async () => {
await target.set({ key: 'fade2', value: 'soon' }, opts);
// negative TTL is effectively expired
await target.expire({ key: 'fade2', ttl: -10 }, opts);
const result = await target.get({ key: 'fade2' }, opts);
expect(result.res).toBeNull();
});
it('rejects an empty key', async () => {
await expect(
target.expireAt({ key: '', timestamp: 0 }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('incr / decr', () => {
it('increments a top-level numeric counter from zero', async () => {
const result = await target.incr(
{ key: 'counter', pathAndAmountMap: { hits: 1 } },
opts,
);
expect(result.res).toMatchObject({ hits: 1 });
});
it('accumulates across calls', async () => {
await target.incr(
{ key: 'counter2', pathAndAmountMap: { hits: 2 } },
opts,
);
const result = await target.incr(
{ key: 'counter2', pathAndAmountMap: { hits: 3 } },
opts,
);
expect(result.res).toMatchObject({ hits: 5 });
});
it('increments nested paths and creates intermediate maps', async () => {
const result = await target.incr(
{
key: 'metrics',
pathAndAmountMap: { 'page.views': 4 },
},
opts,
);
expect(result.res).toMatchObject({ page: { views: 4 } });
});
it('decr subtracts via the same machinery', async () => {
await target.incr(
{ key: 'counter3', pathAndAmountMap: { hits: 10 } },
opts,
);
const result = await target.decr(
{ key: 'counter3', pathAndAmountMap: { hits: 3 } },
opts,
);
expect(result.res).toMatchObject({ hits: 7 });
});
it('rejects when pathAndAmountMap is missing', async () => {
await expect(
target.incr(
{
key: 'k',
// @ts-expect-error intentionally bad input
pathAndAmountMap: undefined,
},
opts,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects when any value in pathAndAmountMap is not a number', async () => {
await expect(
target.incr(
{
key: 'k',
// @ts-expect-error intentionally bad input
pathAndAmountMap: { x: 'nope' },
},
opts,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('add', () => {
it('appends a single element to an empty path, creating a new list', async () => {
const result = await target.add(
{ key: 'list1', pathAndValueMap: { items: 'a' } },
opts,
);
expect(result.res).toMatchObject({ items: ['a'] });
});
it('appends an array to an existing list', async () => {
await target.add(
{ key: 'list2', pathAndValueMap: { items: ['a'] } },
opts,
);
const result = await target.add(
{ key: 'list2', pathAndValueMap: { items: ['b', 'c'] } },
opts,
);
expect(result.res).toMatchObject({ items: ['a', 'b', 'c'] });
});
it('rejects when pathAndValueMap is empty', async () => {
await expect(
target.add({ key: 'k', pathAndValueMap: {} }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('update', () => {
it('sets a top-level path on a fresh key', async () => {
const result = await target.update(
{
key: 'doc',
pathAndValueMap: { name: 'puter' },
},
opts,
);
expect(result.res).toMatchObject({ name: 'puter' });
});
it('writes nested paths and creates intermediate maps', async () => {
const result = await target.update(
{
key: 'doc2',
pathAndValueMap: { 'profile.email': '[email protected]' },
},
opts,
);
expect(result.res).toMatchObject({
profile: { email: '[email protected]' },
});
});
it('preserves untouched fields when updating a single path', async () => {
await target.update(
{
key: 'doc3',
pathAndValueMap: { name: 'first', age: 1 },
},
opts,
);
const result = await target.update(
{ key: 'doc3', pathAndValueMap: { age: 2 } },
opts,
);
expect(result.res).toMatchObject({ name: 'first', age: 2 });
});
it('applies a TTL when ttl is supplied', async () => {
await target.update(
{
key: 'doc4',
pathAndValueMap: { name: 'temp' },
ttl: -10,
},
opts,
);
const result = await target.get({ key: 'doc4' }, opts);
expect(result.res).toBeNull();
});
it('rejects an empty pathAndValueMap', async () => {
await expect(
target.update({ key: 'k', pathAndValueMap: {} }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a non-numeric ttl', async () => {
await expect(
target.update(
{
key: 'k',
pathAndValueMap: { x: 1 },
ttl: Number.NaN,
},
opts,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('remove', () => {
it('removes a path that exists', async () => {
await target.update(
{
key: 'doc-rm',
pathAndValueMap: { keep: 1, drop: 2 },
},
opts,
);
const result = await target.remove(
{ key: 'doc-rm', paths: ['drop'] },
opts,
);
expect(result.res).toMatchObject({ keep: 1 });
expect(result.res).not.toHaveProperty('drop');
});
it('treats a missing path as a no-op and returns current value', async () => {
await target.update(
{ key: 'doc-rm2', pathAndValueMap: { keep: 1 } },
opts,
);
const result = await target.remove(
{ key: 'doc-rm2', paths: ['never.was.here'] },
opts,
);
expect(result.res).toMatchObject({ keep: 1 });
});
it('rejects when paths is empty', async () => {
await expect(
target.remove({ key: 'k', paths: [] }, opts),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('usage accounting', () => {
it('reports write usage on set and read usage on get', async () => {
const setRes = await target.set(
{ key: 'usage-k', value: 'v' },
opts,
);
expect(setRes.usage.write).toBeGreaterThanOrEqual(0);
expect(setRes.usage.read).toBe(0);
const getRes = await target.get({ key: 'usage-k' }, opts);
expect(getRes.usage.read).toBeGreaterThanOrEqual(0);
expect(getRes.usage.write).toBe(0);
});
});
});
+52 -7
View File
@@ -18,17 +18,59 @@
*/
import type { puterClients } from '../clients';
import type { IExtensionClientInstances } from '../clients/types';
import type { IConfig, LayerInstances, WithLifecycle } from '../types';
/**
* Stores may depend on clients and on *prior* stores (those declared earlier
* in the registry). The `stores` argument is the accumulating registry — it
* only contains peers constructed before this one.
* Built-in store instance registry. Forward-declared here and populated via
* declaration merging from `stores/index.ts` to avoid the circular
* `typeof puterStores` reference (stores extend `PuterStore`, whose
* `protected stores` field references this type).
*
* Consumers see the merged `IPuterStoreInstances & IExtensionStoreInstances`
* type — built-in keys + extension-augmented keys.
*/
// eslint-disable-next-line @typescript-eslint/no-empty-object-type
export interface IPuterStoreInstances {}
/**
* Extension-augmentable store registry. Extensions add their own store
* instance types via TypeScript declaration merging:
*
* declare module '@heyputer/backend/stores/types' {
* interface IExtensionStoreInstances {
* myStore: MyStore;
* }
* }
*
* Augmentations flow into `this.stores` (PuterStore, PuterService,
* PuterController, PuterDriver) and into the `extension.import('store')`
* proxy.
*/
export interface IExtensionStoreInstances {
/**
* Open index signature so reads of extension-only store keys return
* `unknown` instead of a type error. Concrete declaration-merged keys
* override this for that name.
*/
[key: string]: unknown;
}
/**
* Stores may depend on clients and on *prior* stores (those declared
* earlier in the registry).
*
* Type contract caveat: `stores` is typed as the FULLY-populated registry,
* even though at construction time only prior stores exist. Same trade-off
* as `PuterService.services` — handler/lifecycle methods (the dominant
* read site) run after all stores are wired, so typed access wins. Don't
* read `this.stores.X` from a store constructor unless `X` is registered
* earlier.
*/
export type IPuterStore<T extends WithLifecycle = WithLifecycle> = new (
config: IConfig,
clients: LayerInstances<typeof puterClients>,
stores: Partial<Record<string, WithLifecycle>>,
clients: LayerInstances<typeof puterClients> & IExtensionClientInstances,
stores: IPuterStoreInstances & IExtensionStoreInstances,
) => T;
const DEFAULT_BROADCAST_REFRESH_TTL_SECONDS = 15 * 60;
@@ -36,8 +78,11 @@ const DEFAULT_BROADCAST_REFRESH_TTL_SECONDS = 15 * 60;
export const PuterStore = class PuterStore implements WithLifecycle {
constructor(
protected config: IConfig,
protected clients: LayerInstances<typeof puterClients>,
protected stores: Partial<Record<string, WithLifecycle>> = {},
protected clients: LayerInstances<typeof puterClients> &
IExtensionClientInstances,
protected stores: IPuterStoreInstances &
IExtensionStoreInstances = {} as IPuterStoreInstances &
IExtensionStoreInstances,
) {}
public onServerStart() {
return;
+29
View File
@@ -0,0 +1,29 @@
import { deepMerge } from '../../tools/lib/configMigration.mjs';
import { PuterServer } from './server';
import { IConfig } from './types';
export const setupTestServer = async (configOverrides?: IConfig) => {
// read default config json
const defaultConfig = await import('../../config.default.json', {
with: {
type: 'json',
},
});
// merge default config with overrides and test defaults
const config = deepMerge(
deepMerge(defaultConfig, {
extensions: [],
port: 0,
database: { engine: 'sqlite', inMemory: true },
dynamo: { inMemory: true, bootstrapTables: true },
redis: { useMock: true },
s3: { localConfig: { inMemory: true } },
no_default_user: true,
no_devwatch: true,
}),
configOverrides ?? {},
);
const server = new PuterServer(config);
await server.start(true);
return server;
};
+69 -1
View File
@@ -28,7 +28,19 @@ export interface IAWSCredentials {
export interface IDynamoConfig {
aws?: IAWSCredentials;
endpoint?: string;
/**
* Filesystem path for the local dynalite store. Defaults to
* `./volatile/runtime/puter-ddb`. Pass `':memory:'` (or set
* `inMemory: true`) to run dynalite without persistence — the
* recommended setup for unit/integration tests.
*/
path?: string;
/**
* Run dynalite in-memory with no on-disk state. Equivalent to
* `path: ':memory:'`. Intended for tests so each suite gets a
* pristine in-process DynamoDB.
*/
inMemory?: boolean;
/**
* Create required tables on startup if they don't exist. Off by
* default because real-AWS deployments provision tables externally
@@ -48,6 +60,12 @@ export interface IRedisConfig {
* ElastiCache). Set `false` for self-host plain-TCP Valkey/Redis.
*/
tls?: boolean;
/**
* Use ioredis-mock instead of a real Redis cluster — fully
* in-process, no network. Defaults to `true` when `startupNodes`
* is empty (so tests with no redis config get a mock for free).
* Intended for unit/integration tests.
*/
useMock?: boolean;
}
@@ -149,6 +167,12 @@ export interface IOIDCProviderConfig {
userinfo_endpoint?: string;
/** Space-separated OAuth scopes. Default depends on provider. */
scopes?: string;
/** Apple Developer Team ID (apple provider only). */
team_id?: string;
/** Key ID for the Sign in with Apple private key (apple provider only). */
key_id?: string;
/** PKCS#8 PEM private key content from Apple (apple provider only). */
private_key?: string;
[key: string]: unknown;
}
@@ -239,6 +263,11 @@ export interface IServerHealthConfig {
}
export interface IS3LocalConfig {
/**
* Run fauxqs entirely in-memory: random port on `127.0.0.1`, no
* `dataDir` / `s3StorageDir`. Intended for tests so each suite
* gets a pristine in-process S3.
*/
inMemory?: boolean;
host?: string;
port?: number;
@@ -277,7 +306,18 @@ export interface IS3Config {
export interface IDatabaseConfig {
engine: 'sqlite' | 'mysql';
// sqlite
/**
* SQLite database file path. Defaults to `':memory:'` (the
* better-sqlite3 in-memory mode), which is also what tests should
* use. `inMemory: true` is an explicit alias for the same.
*/
path?: string;
/**
* Force in-memory SQLite (ignores `path`). Equivalent to
* `path: ':memory:'`. Intended for tests so each suite gets a
* pristine in-process database.
*/
inMemory?: boolean;
targetVersion?: number;
// mysql
host?: string;
@@ -426,6 +466,11 @@ interface IConfigOptional {
no_browser_launch: boolean;
/** Disable dev-time frontend webpack watchers. */
no_devwatch: boolean;
/**
* Skip first-boot bootstrap of the `admin` user and the credentials
* banner that DefaultUserService prints. Intended for tests.
*/
no_default_user: boolean;
/** Optional dev-time frontend watcher overrides. */
devwatch: IDevWatcherConfig;
@@ -583,10 +628,33 @@ interface IConfigOptional {
unlimitedMetering?: boolean;
}
/**
* Extension-augmentable config surface. Extensions add their own config keys
* via TypeScript declaration merging:
*
* declare module '@heyputer/backend/types' {
* interface IExtensionConfig {
* myExtension?: { foo: string };
* }
* }
*
* Augmentations flow into `IConfig`, which is what `this.config` /
* `extension.config` are typed as everywhere.
*/
export interface IExtensionConfig {
/**
* Open index signature so config reads of extension-only keys return
* `unknown` (not a type error). Extensions that declare-merge concrete
* keys (`myExt?: { foo: string }`) override this for the named key —
* the concrete property type wins over the index signature.
*/
[key: string]: unknown;
}
export type IConfig = Partial<IConfigOptional> & {
extensions: string[];
port: number;
};
} & IExtensionConfig;
// eslint-disable-next-line @typescript-eslint/no-wrapper-object-types
export interface WithLifecycle extends Object {
+9 -4
View File
@@ -17,7 +17,12 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
export const nativeImport = new Function(
'specifier',
'return import(specifier)',
) as <TModule = unknown>(specifier: string) => Promise<TModule>;
// `new Function('return import(s)')` would defeat any static bundler
// analysis, but the resulting function inherits a vm context with no
// HostImportModuleDynamically callback under vitest, which throws
// ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING. A direct `import()` with
// `@vite-ignore` accomplishes the same thing — vite skips analysis,
// node performs the import — and works in tests.
export const nativeImport = <TModule = unknown>(
specifier: string,
): Promise<TModule> => import(/* @vite-ignore */ specifier) as Promise<TModule>;
+1 -1
View File
@@ -17,7 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { EventClient } from '../clients/EventClient';
import type { EventClient } from '../clients/event/EventClient';
/**
* Emits `app.privateAccess.resolveLaunch` so the marketplace extension can
+60 -16
View File
@@ -18,12 +18,58 @@
*/
// vite.config.ts - Vite configuration for Puter API tests (TypeScript)
import path from 'node:path';
import { transform } from 'esbuild';
import { loadEnv } from 'vite';
import { defineConfig } from 'vitest/config';
const isCi = process.env.CI === 'true';
const backendDir = __dirname;
const repoRoot = path.resolve(backendDir, '../..');
// Vite 8's oxc transform leaves TC39 stage-3 decorators in place
// (used by `@Controller`/`@Post`), so they reach Node verbatim and
// crash with "SyntaxError: Invalid or unexpected token". Pre-transform
// `.ts`/`.mts` source through esbuild — which DOES lower stage-3
// decorators — locked to `es2024` to match `tsconfig.json`'s target.
const lowerDecoratorsPlugin = {
name: 'puter:lower-decorators',
enforce: 'pre' as const,
async transform(code: string, id: string) {
if (id.includes('/node_modules/')) return null;
if (!/\.(m?ts)$/.test(id)) return null;
if (!code.includes('@')) return null;
const result = await transform(code, {
loader: 'ts',
target: 'es2024',
sourcefile: id,
sourcemap: 'inline',
});
return { code: result.code, map: null };
},
};
export default defineConfig(({ mode }) => ({
plugins: [lowerDecoratorsPlugin],
resolve: {
// Mirror the `@heyputer/backend` path aliases declared in
// tsconfig.json so backend code under test can use the same
// imports it does in production.
alias: [
{
find: /^@heyputer\/backend\/src\/(.*)$/,
replacement: path.join(backendDir, '$1'),
},
{
find: /^@heyputer\/backend\/(.*)$/,
replacement: path.join(backendDir, '$1'),
},
{
find: /^@heyputer\/backend$/,
replacement: path.join(backendDir, 'exports.ts'),
},
],
},
test: {
globals: true,
coverage: {
@@ -32,24 +78,22 @@ export default defineConfig(({ mode }) => ({
? ['json', 'json-summary', 'lcov']
: ['text', 'json', 'json-summary', 'html', 'lcov'],
excludeAfterRemap: true,
// Keep coverage focused on executed files to avoid high-memory
// uncovered-file remapping in CI.
exclude: [
'src/**/types/**',
'src/**/constants/**',
'src/**/*.d.ts',
'src/**/*.d.mts',
'src/**/*.d.cts',
'src/**/dist/**',
'src/**/*.min.*',
'src/**/*.bench.{js,mjs,ts,mts}',
'src/**/*.{test,spec}.{js,mjs,ts,mts}',
'src/public/**',
'src/services/worker/template/**',
// Listing both trees explicitly ensures untested files show
// as 0% instead of being silently dropped from the report.
include: [
'src/backend/**/*.{js,ts}',
'extensions/**/*.{js,ts}',
],
reportsDirectory: path.join(backendDir, 'coverage'),
},
env: loadEnv(mode, '', 'PUTER_'),
include: ['**/*.{test,spec}.{ts,js}'],
root: __dirname, // Ensures paths are relative to backend/
include: [
'src/backend/**/*.test.{js,ts}',
'extensions/**/*.test.{js,ts}',
],
// Root is the repo root so that the file transformer (which
// applies `lowerDecoratorsPlugin`) sees both src/backend and
// extensions/ — vitest skips transform for files outside root.
root: repoRoot,
},
}));
+28 -15
View File
@@ -95,24 +95,37 @@ For more details, see the [Together AI API reference](https://docs.together.ai/r
Available when `provider: 'replicate-image-generation'` or inferred from model:
##### Common options
| Option | Type | Description |
|--------|------|-------------|
| `model` | `String` | Image model to use. |
| `ratio` | `Object` | Aspect ratio as `{ w, h }` (e.g., `{ w: 16, h: 9 }`) |
| `seed` | `Number` | Random seed for reproducible generation |
| `steps` | `Number` | Number of inference steps |
| `guidance` | `Number` | Guidance scale for generation |
| `go_fast` | `Boolean` | Use optimized fast mode. Defaults to `true` for `flux-2-dev`. Affects pricing on supported models |
| `output_quality` | `Number` | Output quality (0-100). |
| `output_megapixels` | `String` | Approximate output megapixels (`'0.25'`, `'0.5'`, `'1'`, `'2'`, `'4'`) |
| `input_image` | `String` | URL of an input image for image-to-image generation |
| `input_images` | `Array<String>` | Array of input image URLs for multi-image generation |
| `negative_prompt` | `String` | Text to guide what to avoid in the image |
| `prompt_strength` | `Number` | How strongly the prompt influences the output |
| `disable_safety_checker` | `Boolean` | If `true`, disables the safety checker |
| `response_format` | `String` | Output format: `'webp'`, `'jpg'`, `'png'` |
| `model` | `String` | Model id (e.g. `'black-forest-labs/flux-schnell'`, `'leonardoai/lucid-origin'`). |
| `ratio` | `Object` | Aspect ratio as `{ w, h }` (e.g., `{ w: 16, h: 9 }`). |
| `input_image` | `String` | URL of an input image for image-to-image generation. |
| `input_images` | `Array<String>` | Array of input image URLs for multi-image generation. |
For more details, see the [Replicate API reference](https://replicate.com/docs).
##### Per-model options
These keys are only forwarded for models that whitelist them (see per-model `allowed_params`):
| Option | Type | Models | Description |
|--------|------|--------|-------------|
| `seed` | `Number` | most models | Random seed for reproducible generation. |
| `steps` | `Number` | `flux-schnell` | Number of inference steps. |
| `guidance` | `Number` | `flux-2-klein-9b-base` | Guidance scale. |
| `go_fast` | `Boolean` | `flux-2-dev` | Use optimized fast mode. Defaults to `true` for `flux-2-dev`; affects pricing. |
| `output_quality` | `Number` | flux family | Output quality (0–100). |
| `output_megapixels` | `String` | flux family | Approximate output megapixels (e.g. `'0.25'`, `'0.5'`, `'1'`, `'2'`). |
| `disable_safety_checker` | `Boolean` | flux-2-dev / klein / flux-schnell | If `true`, disables the safety checker. |
| `safety_tolerance` | `Number` | `flux-2-pro`, `flux-1.1-pro` | Safety tolerance level. |
| `prompt_upsampling` | `Boolean` | `flux-1.1-pro` | Enable prompt upsampling. |
| `response_format` | `String` | most models | Output format (e.g. `'webp'`, `'jpg'`, `'png'`). |
| `generation_mode` | `String` | Leonardo (`lucid-origin`, `phoenix-1.0`) | Generation tier — affects pricing. e.g. `'standard'`/`'ultra'` (lucid-origin), `'fast'`/`'quality'`/`'ultra'` (phoenix-1.0). |
| `style` | `String` | Leonardo | Stylistic preset. |
| `contrast` | `String` | Leonardo | Contrast preset. |
| `prompt_enhance` | `Boolean` | Leonardo | Server-side prompt enhancement. |
For more details, see the [Replicate API reference](https://replicate.com/docs) and each model's schema page on Replicate.
Any properties not set fall back to provider defaults.
Loaded 100 of 115 files, more files were not shown because too many files have changed in this diff. Show more