mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b144f1d11c | ||
|
|
2ceeaacd31 | ||
|
|
0c984ca75b | ||
|
|
16916a903d | ||
|
|
eda4015dc7 | ||
|
|
429b7e2f90 | ||
|
|
b41ed5dfd6 | ||
|
|
b3dd98d24e | ||
|
|
1522ed3cdd | ||
|
|
e936038f80 | ||
|
|
a9f5e818db |
No files matched your search
+1
-1
@@ -40,7 +40,7 @@ The following have already been reviewed and determined **not to be vulnerabilit
|
||||
- **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
|
||||
- **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
|
||||
- **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
|
||||
- **App metadata or app user-count "leaks".** This information is currently public by design. This covers **private apps** too: private apps are an unreleased, admin-only beta for paid listings in the Puter App Center. Their `index_url` and other metadata are not secret, the flag is not user-settable, and no production app relies on it. Reports that a private app's metadata is reachable through a sibling route (`/get-launch-apps`, `index_url` drift on the hosting gate, etc.) are normalization work, not vulnerabilities.
|
||||
- **App metadata or app user-count "leaks".** This information is currently public by design. This covers **private apps** too: private apps are an unreleased, admin-only beta for paid listings in the Puter App Center. Their `index_url` and other metadata are not secret, the flag is not user-settable, and no production app relies on it. Reports that a private app's metadata is reachable through a sibling route (taskbar items, `/get-launch-apps`, `index_url` drift on the hosting gate, etc.) are normalization work, not vulnerabilities.
|
||||
- **Worker hot-reload driven by a share recipient.** Sharing a worker's source file with write access is how you let a collaborator change that worker's code and have it deploy live. The redeploy running as the file owner, with the owner's worker credentials, is the intended behavior of that grant — the owner chose to hand out write access to executable code. If you do not want that, do not share the source file with write access.
|
||||
- **Re-claiming a site or worker name its owner deleted.** A hosted site or worker name that its owner deletes is treated as abandoned by that owner. If one of their apps still points at it, whoever claims the name next serves that app's `index_url` and receives what the app is launched with, and a new app on the same origin may share the old one's identity. Guarding app identities against being recycled this way is ongoing hardening, not a security boundary, so new ways to re-claim an abandoned name, or to inherit an app's tokens or data through one, are not eligible.
|
||||
- **Share-mode downgrades that do not cascade.** Demoting a delegate does not retroactively revoke the grants that delegate already issued downstream. Shares are independent grants once issued, not a live-evaluated chain. Revoking the downstream grant is a separate, supported action.
|
||||
|
||||
@@ -322,36 +322,6 @@ app reads from the desktop's answer. A call made while the frame shows settles
|
||||
with it. A frame that is refused, never ready, or `unavailable` resolves
|
||||
`{ response: false }`, since there's no popup to fall back to.
|
||||
|
||||
## Custom domains
|
||||
|
||||
`GET /embed/attach-custom-domain` is the custom domain dialog of
|
||||
`puter.hosting.attach()`, the popup at `/action/attach-custom-domain`, served
|
||||
like the upgrade: the GUI shell with the same framing rules, redirects and
|
||||
`no-store`, run only when framed by an allowed page. puter.js frames it on a
|
||||
signed-in first-party page (the same pages as the pickers).
|
||||
|
||||
The frame covers the page with no box or close button, over a transparent GUI
|
||||
page, so the dialog floats over the host with its own scrim as it does on the
|
||||
desktop. The frame allows `clipboard-write`, for the dialog's copy buttons, as
|
||||
well as `otp-credentials`. Escape on the page is left to the dialog, which
|
||||
doesn't close while a request is in flight. Its fragment names `user` (the
|
||||
page's signed-in user), `subdomain` (the site to attach a domain to) and
|
||||
`msg_id`.
|
||||
|
||||
The GUI reports `unavailable` unless its session belongs to `user` and the
|
||||
account has no verification or password change waiting. It then runs the
|
||||
popup's action unchanged, with the framing page as the requester the dialog
|
||||
names: it posts `ready` once the dialog is on the page, answers
|
||||
`{ msg: 'customDomainAttached', domain, original_msg_id }` once the dialog has
|
||||
closed and its exit has played, and then posts `closed`. `domain` is the domain the user chose, or
|
||||
null. `attach()` resolves it. A frame that closes without an answer, or a
|
||||
loading scrim the user dismisses, resolves null.
|
||||
|
||||
A call made while the frame shows resolves null without replacing it, as a
|
||||
second call does on the desktop. A frame that is refused, never ready, or
|
||||
`unavailable` falls back to the popup, which can sign in, through a consent
|
||||
dialog when the click has expired.
|
||||
|
||||
## Checks
|
||||
|
||||
Run the GUI component/browser tests (with the SDK bundle built and Playwright
|
||||
@@ -359,7 +329,7 @@ Chromium installed), and the backend route tests:
|
||||
|
||||
```sh
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser src/feedbackEmbed
|
||||
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/modules/UI.showTopUp.test.js src/puter-js/src/modules/UI.attachCustomDomainFrame.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
|
||||
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/modules/UI.showTopUp.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/helpers/popupFrame.test.js
|
||||
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
|
||||
```
|
||||
+10
-40
@@ -257,41 +257,8 @@ describe('whoami extension — handleWhoami', () => {
|
||||
expect(body.created_ts).toBeGreaterThan(0);
|
||||
// `directories` is only sent to user actors — confirm it’s present.
|
||||
expect(body.directories).toBeDefined();
|
||||
});
|
||||
|
||||
it('leaves out taskbar items saved on the account', async () => {
|
||||
const user = await seedUser();
|
||||
await server.stores.user.update(user.id as number, {
|
||||
taskbar_items: JSON.stringify([{ type: 'app', name: 'editor' }]),
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
|
||||
await runWithContext(
|
||||
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
|
||||
() => handleWhoami(makeReq(), res),
|
||||
);
|
||||
|
||||
expect(captured.body).not.toHaveProperty('taskbar_items');
|
||||
});
|
||||
|
||||
it('leaves out a desktop background saved on the account', async () => {
|
||||
const user = await seedUser();
|
||||
await server.stores.user.update(user.id as number, {
|
||||
desktop_bg_url: 'https://cdn.test/wallpaper.png',
|
||||
desktop_bg_color: '#000000',
|
||||
desktop_bg_fit: 'cover',
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
|
||||
await runWithContext(
|
||||
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
|
||||
() => handleWhoami(makeReq(), res),
|
||||
);
|
||||
|
||||
const body = captured.body as Record<string, unknown>;
|
||||
expect(body).not.toHaveProperty('desktop_bg_url');
|
||||
expect(body).not.toHaveProperty('desktop_bg_color');
|
||||
expect(body).not.toHaveProperty('desktop_bg_fit');
|
||||
// taskbar_items is only sent to user actors.
|
||||
expect(body).toHaveProperty('taskbar_items');
|
||||
});
|
||||
|
||||
it('reports the SMS-to-card fallback only once a send has opened it', async () => {
|
||||
@@ -355,7 +322,7 @@ describe('whoami extension — handleWhoami', () => {
|
||||
expect(flags.payment_bypass).toBeUndefined();
|
||||
});
|
||||
|
||||
it('strips created_ts and human_readable_age fields for app actors', async () => {
|
||||
it('strips desktop_bg_*, created_ts and human_readable_age fields for app actors', async () => {
|
||||
const user = await seedUser();
|
||||
const { res, captured } = makeRes();
|
||||
|
||||
@@ -373,6 +340,9 @@ describe('whoami extension — handleWhoami', () => {
|
||||
expect(body.app_name).toBe('app-test-actor');
|
||||
// is_user_token is stripped for app actors.
|
||||
expect(body.is_user_token).toBeUndefined();
|
||||
expect(body.desktop_bg_url).toBeUndefined();
|
||||
expect(body.desktop_bg_color).toBeUndefined();
|
||||
expect(body.desktop_bg_fit).toBeUndefined();
|
||||
expect(body.human_readable_age).toBeUndefined();
|
||||
// Account age, in either form, is not exposed to apps.
|
||||
expect(body.created_ts).toBeUndefined();
|
||||
@@ -462,7 +432,7 @@ describe('whoami extension — handleWhoami', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('cannot add directories for an app actor, which core omits', async () => {
|
||||
it('cannot add taskbar_items for an app actor, which core omits', async () => {
|
||||
const user = await seedUser();
|
||||
const body = await whoamiWithListener(
|
||||
makeActor({
|
||||
@@ -470,11 +440,11 @@ describe('whoami extension — handleWhoami', () => {
|
||||
app: { uid: 'app-test-actor' },
|
||||
}),
|
||||
(_key, event) => {
|
||||
event.details.directories = { '/injected': 'uuid' };
|
||||
event.details.taskbar_items = [{ name: 'injected' }];
|
||||
},
|
||||
);
|
||||
|
||||
expect(body).not.toHaveProperty('directories');
|
||||
expect(body).not.toHaveProperty('taskbar_items');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -555,7 +525,7 @@ describe('whoami extension — handleWhoami', () => {
|
||||
expect(body.otp_secret).toBeUndefined();
|
||||
expect(body.signup_ip).toBeUndefined();
|
||||
// Not just absent as a key — the values must not appear anywhere
|
||||
// in the payload (nested under metadata, …).
|
||||
// in the payload (nested under metadata, taskbar items, …).
|
||||
const serialized = JSON.stringify(body);
|
||||
expect(serialized).not.toContain('5551234567');
|
||||
expect(serialized).not.toContain('fp_ABC123');
|
||||
|
||||
+16
-2
@@ -1,5 +1,6 @@
|
||||
import { Context } from '@heyputer/backend/src/core';
|
||||
import { extension } from '@heyputer/backend/src/extensions';
|
||||
import { APP_ICON_SIZES } from '@heyputer/backend/src/util/appIcon.js';
|
||||
import {
|
||||
buildUserDetails,
|
||||
scrubSensitive,
|
||||
@@ -53,10 +54,19 @@ export const handleWhoami = async (
|
||||
return;
|
||||
}
|
||||
|
||||
const rawIconSize =
|
||||
typeof req.query?.icon_size === 'string'
|
||||
? Number(req.query.icon_size)
|
||||
: undefined;
|
||||
const iconSize =
|
||||
rawIconSize !== undefined && APP_ICON_SIZES.includes(rawIconSize)
|
||||
? rawIconSize
|
||||
: undefined;
|
||||
|
||||
const details = await buildUserDetails(
|
||||
user,
|
||||
{ config: extension.config, clients, stores, services },
|
||||
{ isUser },
|
||||
{ isUser, iconSize, noIcons: !iconSize },
|
||||
);
|
||||
details.human_readable_age = user.timestamp
|
||||
? timeago.format(new Date(user.timestamp as string))
|
||||
@@ -71,6 +81,9 @@ export const handleWhoami = async (
|
||||
delete details.email;
|
||||
delete details.unconfirmed_email;
|
||||
}
|
||||
delete details.desktop_bg_url;
|
||||
delete details.desktop_bg_color;
|
||||
delete details.desktop_bg_fit;
|
||||
delete details.human_readable_age;
|
||||
delete details.created_ts;
|
||||
delete details.is_user_token;
|
||||
@@ -108,7 +121,8 @@ export const handleWhoami = async (
|
||||
}
|
||||
|
||||
const subscription = details.subscription as
|
||||
{ offering?: Record<string, unknown> } | undefined;
|
||||
| { offering?: Record<string, unknown> }
|
||||
| undefined;
|
||||
if (subscription?.offering) {
|
||||
delete subscription.offering.group;
|
||||
delete subscription.offering.benefits;
|
||||
|
||||
@@ -190,13 +190,6 @@ S3_BUCKET=puter-local
|
||||
providers = [ordered]@{
|
||||
ollama = [ordered]@{ enabled = $false }
|
||||
}
|
||||
# Off because a self-hosted install has no paid plans: every account
|
||||
# is on the free policy, so the plan gates would 402 the surfaces that
|
||||
# declare one (the OpenAI/Anthropic-compatible AI endpoints) for
|
||||
# everybody. Same as install.sh.
|
||||
meteringEnforcement = [ordered]@{
|
||||
subscriptions = $false
|
||||
}
|
||||
trust_proxy = 1
|
||||
}
|
||||
$configJson = $config | ConvertTo-Json -Depth 10
|
||||
|
||||
@@ -125,6 +125,7 @@ describe('core without a teams registration', () => {
|
||||
actor: { user: row },
|
||||
} as never,
|
||||
{ title: 'Puter' } as never,
|
||||
{} as never,
|
||||
);
|
||||
const params = JSON.parse(
|
||||
/gui\((\{.*?\})\);/s.exec(html)![1].replaceAll('\\u003c', '<'),
|
||||
|
||||
@@ -736,8 +736,6 @@ export type EventMap = {
|
||||
'outer.permission.flatInvalidated': {
|
||||
entries: Array<{ holderUserId: number; permission: string }>;
|
||||
};
|
||||
/** Links on these nodes were removed; read them from the primary. */
|
||||
'outer.share.linksRevoked': { fsentryIds: number[]; uuids: string[] };
|
||||
/**
|
||||
* A user's subscription set changed, so every process must drop its cached
|
||||
* "does this user have any subscriptions" answer. Carries the counter the
|
||||
@@ -876,11 +874,8 @@ export type EventMap = {
|
||||
session_uids: string[];
|
||||
};
|
||||
|
||||
/**
|
||||
* One access token was revoked; only its own connections should drop.
|
||||
* `outer.*` for the same reason as `outer.auth.sessions.revoked`.
|
||||
*/
|
||||
'outer.auth.access-token.revoked': { token_uid: string };
|
||||
/** One access token was revoked; only its own connections should drop. */
|
||||
'auth.access-token.revoked': { token_uid: string };
|
||||
|
||||
/** Same-cluster eviction for raw realtime WebSockets. */
|
||||
'pubsub.realtime.auth-revoked': {
|
||||
@@ -1088,10 +1083,11 @@ export type EventKey = keyof EventMap & string;
|
||||
// Generates a wildcard for every non-final dot-separated prefix of K.
|
||||
export type WildcardPrefixes<K extends string> =
|
||||
K extends `${infer Head}.${infer Tail}`
|
||||
? | `${Head}.*`
|
||||
| (Tail extends `${string}.${string}`
|
||||
? `${Head}.${WildcardPrefixes<Tail>}`
|
||||
: never)
|
||||
?
|
||||
| `${Head}.*`
|
||||
| (Tail extends `${string}.${string}`
|
||||
? `${Head}.${WildcardPrefixes<Tail>}`
|
||||
: never)
|
||||
: never;
|
||||
|
||||
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
|
||||
@@ -1102,7 +1098,10 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
|
||||
|
||||
/** Acked, not acked, no socket anywhere, or sockets only in a peer region. */
|
||||
export type NotifDeliveryOutcome =
|
||||
'received' | 'unreceived' | 'no-socket' | 'remote-only';
|
||||
| 'received'
|
||||
| 'unreceived'
|
||||
| 'no-socket'
|
||||
| 'remote-only';
|
||||
|
||||
/** One socket's receipt for a live notification. */
|
||||
export interface NotifDeliveryResponder {
|
||||
|
||||
@@ -52,8 +52,8 @@ const APP_READ_LIMIT = {
|
||||
/**
|
||||
* Unauthenticated icon serving; no actor to key on, so the bucket is the
|
||||
* address — and an address is a NAT, a campus or a carrier gateway that can
|
||||
* hold hundreds of desktops. Each dashboard load pulls its app icons at once,
|
||||
* so a single burst from one network is already thousands of requests.
|
||||
* hold hundreds of desktops. Each desktop boot pulls the taskbar's icons at
|
||||
* once, so a single burst from one network is already thousands of requests.
|
||||
* Responses are publicly cacheable and usually a redirect, so the ceiling is
|
||||
* not protecting bandwidth; it is there so a client looping on a broken icon
|
||||
* can't spin unbounded.
|
||||
@@ -523,7 +523,7 @@ export class AppController extends PuterController {
|
||||
};
|
||||
|
||||
// Icons are <img src> targets from the GUI (root) AND resolved via
|
||||
// api_base_url in app summaries. Register on both so either origin
|
||||
// api_base_url in taskbar payloads. Register on both so either origin
|
||||
// works without a cross-subdomain redirect.
|
||||
router.get(
|
||||
'/app-icon/:app_uid',
|
||||
|
||||
@@ -1271,7 +1271,7 @@ describe('AppController app-icon rate limit', () => {
|
||||
it('sizes the icon bucket for a whole network rather than one desktop', () => {
|
||||
// Icons are unauthenticated `<img src>` targets, so the only key is
|
||||
// the address — which one NAT shares across every desktop behind it,
|
||||
// and each dashboard load pulls its app icons in a burst.
|
||||
// and each boot pulls the taskbar's icons in a burst.
|
||||
for (const path of ['/app-icon/:app_uid', '/app-icon/:app_uid/:size']) {
|
||||
const route = router.routes.find(
|
||||
(r) => r.method === 'get' && r.path === path,
|
||||
|
||||
@@ -23,7 +23,6 @@ import { makeActor } from '../../core/actor.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { createSecret as otpCreateSecret } from '../../services/auth/OTPUtil.js';
|
||||
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
|
||||
import { tokenClaimsDigest } from '../../../gui/src/helpers/guiToken.js';
|
||||
import {
|
||||
createTestUser,
|
||||
setupPuterTestEnv,
|
||||
@@ -715,67 +714,4 @@ describe('session token helpers over HTTP', () => {
|
||||
'puter_auth_token=',
|
||||
);
|
||||
});
|
||||
|
||||
describe("the shell's guiTokenDigest", () => {
|
||||
/** The session cookie `/session/sync-cookie` sets for `token`. */
|
||||
const sessionCookie = async (token: string) => {
|
||||
const sync = await get('/session/sync-cookie', token);
|
||||
expect(sync.status).toBe(204);
|
||||
const value = /puter_auth_token=([^;]+)/.exec(
|
||||
sync.headers.get('set-cookie') ?? '',
|
||||
)?.[1];
|
||||
expect(value).toBeTruthy();
|
||||
return `puter_auth_token=${value}`;
|
||||
};
|
||||
|
||||
/** What the shell at `path` renders for `guiTokenDigest`. */
|
||||
const shellDigest = async (path: string, cookie?: string) => {
|
||||
const res = await fetch(new URL(path, env.origin), {
|
||||
headers: cookie ? { cookie } : {},
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
const html = await res.text();
|
||||
const match = /"guiTokenDigest":(null|"[0-9a-f]{32}")/.exec(html);
|
||||
if (!match) {
|
||||
expect(html).not.toContain('guiTokenDigest');
|
||||
return undefined;
|
||||
}
|
||||
return JSON.parse(match[1]) as string | null;
|
||||
};
|
||||
|
||||
it("names the token /get-gui-token hands over for the session cookie", async () => {
|
||||
const { token } = await signupTemp();
|
||||
const cookie = await sessionCookie(token);
|
||||
|
||||
const gui = await fetch(new URL('/get-gui-token', env.origin), {
|
||||
headers: { cookie },
|
||||
});
|
||||
expect(gui.status).toBe(200);
|
||||
const { token: guiToken } = (await gui.json()) as { token: string };
|
||||
|
||||
const digest = await shellDigest('/', cookie);
|
||||
expect(digest).toBe(await tokenClaimsDigest(guiToken));
|
||||
// The token the browser stored at sign-up is for the same session.
|
||||
expect(await tokenClaimsDigest(token)).toBe(digest);
|
||||
});
|
||||
|
||||
it('is null without a session cookie, even with a token in the URL', async () => {
|
||||
const { token } = await signupTemp();
|
||||
expect(await shellDigest('/')).toBe(null);
|
||||
expect(
|
||||
await shellDigest(`/?auth_token=${encodeURIComponent(token)}`),
|
||||
).toBe(null);
|
||||
});
|
||||
|
||||
it('is left out when the session cookie does not resolve to a session', async () => {
|
||||
const { token } = await signupTemp();
|
||||
const cookie = await sessionCookie(token);
|
||||
await env.server.services.auth.revokeSession(claims(token).session_uid);
|
||||
|
||||
expect(await shellDigest('/', cookie)).toBe(undefined);
|
||||
expect(await shellDigest('/', 'puter_auth_token=garbage')).toBe(
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1416,7 +1416,7 @@ describe('AuthController.handleLogin', () => {
|
||||
is_user_token: true,
|
||||
});
|
||||
expect(body.user.directories).toBeTypeOf('object');
|
||||
expect(body.user).not.toHaveProperty('taskbar_items');
|
||||
expect(Array.isArray(body.user.taskbar_items)).toBe(true);
|
||||
});
|
||||
|
||||
// A deleted row still answering from cache reaches the session INSERT.
|
||||
@@ -8293,10 +8293,11 @@ describe('AuthController session endpoints', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('revoke-session: 404 when revoking someone else’s session', async () => {
|
||||
it('revoke-session: 403 when revoking someone else’s session', async () => {
|
||||
const { user: u1 } = await makeUserAndActor();
|
||||
const { actor: a2 } = await makeUserAndActor();
|
||||
// A real session of u1, revoked as a2: answers as a missing one.
|
||||
// Create a real session for u1 so the lookup succeeds, then attempt
|
||||
// to revoke it as a2 — must 403.
|
||||
const sessionRes = await server.services.auth.createSessionToken(
|
||||
u1,
|
||||
{},
|
||||
@@ -8309,7 +8310,7 @@ describe('AuthController session endpoints', () => {
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
).rejects.toMatchObject({ statusCode: 403 });
|
||||
});
|
||||
|
||||
it('rename-session: 400 when uuid param is missing', async () => {
|
||||
@@ -10081,77 +10082,6 @@ describe('AuthController.handleRevokeSession additional branches', () => {
|
||||
expect(Array.isArray(body.sessions)).toBe(true);
|
||||
});
|
||||
|
||||
const expiredSessionOf = async (user: { id: number }) => {
|
||||
const row = await server.stores.session.create(user.id, {
|
||||
expires_at: Math.floor(Date.now() / 1000) - 1,
|
||||
});
|
||||
return row.uuid as string;
|
||||
};
|
||||
|
||||
it('revokes an expired session the caller owns', async () => {
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const uuid = await expiredSessionOf(user);
|
||||
const res = makeRes();
|
||||
await controller.handleRevokeSession(makeReq({ uuid }, { actor }), res);
|
||||
|
||||
const body = res.body as { sessions: Array<{ uuid: string }> };
|
||||
expect(body.sessions.map((s) => s.uuid)).not.toContain(uuid);
|
||||
const rows = await server.clients.db.read(
|
||||
'SELECT `revoked_at` FROM `sessions` WHERE `uuid` = ?',
|
||||
[uuid],
|
||||
);
|
||||
expect(rows[0].revoked_at).not.toBeNull();
|
||||
});
|
||||
|
||||
it('revoking an expired session ends its live child token', async () => {
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const parent = await expiredSessionOf(user);
|
||||
const child = await server.stores.session.getOrCreateGodmodeToken(
|
||||
user.id,
|
||||
{
|
||||
parentSessionUid: parent,
|
||||
appUid: `app-${uuidv4()}`,
|
||||
expiresAt: Math.floor(Date.now() / 1000) + 3600,
|
||||
maxAgeSeconds: 3600,
|
||||
label: 'Godmode app',
|
||||
auth_id: user.uuid,
|
||||
},
|
||||
);
|
||||
const childUuid = child!.row.uuid as string;
|
||||
expect(await server.stores.session.getByUuid(childUuid)).toBeTruthy();
|
||||
|
||||
await controller.handleRevokeSession(
|
||||
makeReq({ uuid: parent }, { actor }),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
expect(await server.stores.session.getByUuid(childUuid)).toBeNull();
|
||||
});
|
||||
|
||||
it('answers 404 for an expired session of another user', async () => {
|
||||
const { user: owner } = await makeUserAndActor();
|
||||
const { actor } = await makeUserAndActor();
|
||||
const uuid = await expiredSessionOf(owner);
|
||||
await expect(
|
||||
controller.handleRevokeSession(
|
||||
makeReq({ uuid }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
});
|
||||
|
||||
it('answers 404 for a session that is already revoked', async () => {
|
||||
const { user, actor } = await makeUserAndActor();
|
||||
const row = await server.stores.session.create(user.id);
|
||||
await server.stores.session.removeByUuid(row.uuid);
|
||||
await expect(
|
||||
controller.handleRevokeSession(
|
||||
makeReq({ uuid: row.uuid }, { actor }),
|
||||
makeRes(),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
});
|
||||
|
||||
it('refuses to revoke the caller’s OWN current session row (400)', async () => {
|
||||
// Invariant: a self-revoke leaves the client in an ambiguous
|
||||
// identity state because the response can't write fresh auth
|
||||
|
||||
@@ -3585,18 +3585,21 @@ export class AuthController extends PuterController {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
// Missing, already revoked and other users' rows all answer 404; an
|
||||
// expired row is still revoked so its children go with it.
|
||||
const session = await this.stores.session.getByUuidAny(uuid);
|
||||
if (
|
||||
!session ||
|
||||
session.revoked_at != null ||
|
||||
session.user_id !== req.actor!.user.id
|
||||
) {
|
||||
// `getByUuid` returns null when the row is missing, already
|
||||
// soft-revoked, or past `expires_at` — surface as 404 so a stale
|
||||
// manage-sessions UI doesn't 500 when it clicks revoke on a row
|
||||
// that already went away.
|
||||
const session = await this.stores.session.getByUuid(uuid);
|
||||
if (!session) {
|
||||
throw new HttpError(404, 'Session not found', {
|
||||
legacyCode: 'not_found',
|
||||
});
|
||||
}
|
||||
if (session.user_id !== req.actor!.user.id) {
|
||||
throw new HttpError(403, 'Can only revoke your own sessions', {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
}
|
||||
await this.services.auth.revokeSession(uuid);
|
||||
const sessions = await this.services.auth.listSessions(req.actor!);
|
||||
res.json({ sessions });
|
||||
|
||||
@@ -25,13 +25,18 @@ const ALLOWED_SORT_BY = ['name', 'size', 'modified', 'type'];
|
||||
const ALLOWED_SORT_ORDER = ['asc', 'desc'];
|
||||
|
||||
/**
|
||||
* Folder view preference routes: layout and sort_by/sort_order, stored on the
|
||||
* folder's fsentries row.
|
||||
* Desktop/UI preference routes.
|
||||
*
|
||||
* Two categories:
|
||||
*
|
||||
* - User-level: desktop background, taskbar items (UserStore)
|
||||
* - Folder-level: layout, sort_by/sort_order (fsentries table)
|
||||
*/
|
||||
/**
|
||||
* Desktop preference writes — layout, sort order. Each persists on every call,
|
||||
* and the GUI fires them on direct user action, so a per-minute ceiling well
|
||||
* above human speed is enough to catch a stuck client.
|
||||
* Desktop preference writes — background, taskbar, layout, sort order. All four
|
||||
* persist to the user row on every call, and the GUI fires them on direct user
|
||||
* action, so a per-minute ceiling well above human speed is enough to catch a
|
||||
* stuck client.
|
||||
*/
|
||||
const PREFERENCE_WRITE_LIMIT = {
|
||||
scope: 'desktop-preference',
|
||||
@@ -42,6 +47,89 @@ const PREFERENCE_WRITE_LIMIT = {
|
||||
|
||||
export class DesktopController extends PuterController {
|
||||
registerRoutes(router) {
|
||||
// -- Desktop background --------------------------------------
|
||||
|
||||
router.post(
|
||||
'/set-desktop-bg',
|
||||
{
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
allowFullAccessToken: true,
|
||||
rateLimit: PREFERENCE_WRITE_LIMIT,
|
||||
},
|
||||
async (req, res) => {
|
||||
const { url, color, fit } = req.body ?? {};
|
||||
|
||||
const patch = {};
|
||||
if (url !== undefined) {
|
||||
if (url !== null && typeof url !== 'string') {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'`url` must be a string or null',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
patch.desktop_bg_url = url;
|
||||
}
|
||||
if (color !== undefined) {
|
||||
if (color !== null && typeof color !== 'string') {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'`color` must be a string or null',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
patch.desktop_bg_color = color;
|
||||
}
|
||||
if (fit !== undefined) {
|
||||
if (fit !== null && typeof fit !== 'string') {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'`fit` must be a string or null',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
patch.desktop_bg_fit = fit;
|
||||
}
|
||||
|
||||
if (Object.keys(patch).length === 0) {
|
||||
throw new HttpError(400, 'No fields provided', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
|
||||
await this.stores.user.update(req.actor.user.id, patch);
|
||||
res.json({});
|
||||
},
|
||||
);
|
||||
|
||||
// -- Taskbar items -------------------------------------------
|
||||
|
||||
router.post(
|
||||
'/update-taskbar-items',
|
||||
{
|
||||
subdomain: 'api',
|
||||
requireUserActor: true,
|
||||
allowFullAccessToken: true,
|
||||
rateLimit: PREFERENCE_WRITE_LIMIT,
|
||||
},
|
||||
async (req, res) => {
|
||||
const { items } = req.body ?? {};
|
||||
if (!Array.isArray(items)) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'Missing or invalid `items` array',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
|
||||
await this.stores.user.update(req.actor.user.id, {
|
||||
taskbar_items: JSON.stringify(items),
|
||||
});
|
||||
res.json({});
|
||||
},
|
||||
);
|
||||
|
||||
// -- Folder layout -------------------------------------------
|
||||
|
||||
router.post(
|
||||
|
||||
@@ -130,17 +130,147 @@ const callRoute = async (
|
||||
});
|
||||
};
|
||||
|
||||
describe('DesktopController routes', () => {
|
||||
it('has no desktop background route', () => {
|
||||
expect(
|
||||
router.routes.some((r) => r.path === '/set-desktop-bg'),
|
||||
).toBe(false);
|
||||
// ── /set-desktop-bg ─────────────────────────────────────────────────
|
||||
|
||||
describe('DesktopController POST /set-desktop-bg', () => {
|
||||
it('persists url/color/fit on the user row', async () => {
|
||||
const { actor, userId } = await makeUser();
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'post',
|
||||
'/set-desktop-bg',
|
||||
makeReq({
|
||||
body: {
|
||||
url: 'https://cdn.test/wallpaper.png',
|
||||
color: '#000000',
|
||||
fit: 'cover',
|
||||
},
|
||||
actor,
|
||||
}),
|
||||
res,
|
||||
);
|
||||
expect(captured.body).toEqual({});
|
||||
|
||||
const refreshed = await server.stores.user.getById(userId);
|
||||
expect(refreshed?.desktop_bg_url).toBe(
|
||||
'https://cdn.test/wallpaper.png',
|
||||
);
|
||||
expect(refreshed?.desktop_bg_color).toBe('#000000');
|
||||
expect(refreshed?.desktop_bg_fit).toBe('cover');
|
||||
});
|
||||
|
||||
it('has no taskbar items route', () => {
|
||||
expect(
|
||||
router.routes.some((r) => r.path === '/update-taskbar-items'),
|
||||
).toBe(false);
|
||||
it('persists only the supplied fields (partial update)', async () => {
|
||||
const { actor, userId } = await makeUser();
|
||||
const { res } = makeRes();
|
||||
await callRoute(
|
||||
'post',
|
||||
'/set-desktop-bg',
|
||||
makeReq({ body: { color: '#ffffff' }, actor }),
|
||||
res,
|
||||
);
|
||||
const refreshed = await server.stores.user.getById(userId);
|
||||
expect(refreshed?.desktop_bg_color).toBe('#ffffff');
|
||||
// Untouched fields remain at their column default.
|
||||
expect(refreshed?.desktop_bg_url).toBeFalsy();
|
||||
expect(refreshed?.desktop_bg_fit).toBeFalsy();
|
||||
});
|
||||
|
||||
it('passes through `null` to clear a field', async () => {
|
||||
const { actor, userId } = await makeUser();
|
||||
// First populate, then clear.
|
||||
const populate = makeRes();
|
||||
await callRoute(
|
||||
'post',
|
||||
'/set-desktop-bg',
|
||||
makeReq({
|
||||
body: { url: 'https://cdn.test/x.png' },
|
||||
actor,
|
||||
}),
|
||||
populate.res,
|
||||
);
|
||||
const before = await server.stores.user.getById(userId);
|
||||
expect(before?.desktop_bg_url).toBe('https://cdn.test/x.png');
|
||||
|
||||
const clear = makeRes();
|
||||
await callRoute(
|
||||
'post',
|
||||
'/set-desktop-bg',
|
||||
makeReq({ body: { url: null }, actor }),
|
||||
clear.res,
|
||||
);
|
||||
const after = await server.stores.user.getById(userId);
|
||||
expect(after?.desktop_bg_url).toBeNull();
|
||||
});
|
||||
|
||||
it('throws 400 when url is not a string or null', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/set-desktop-bg',
|
||||
makeReq({ body: { url: 123 }, actor }),
|
||||
res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('throws 400 when no fields are provided', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/set-desktop-bg',
|
||||
makeReq({ body: {}, actor }),
|
||||
res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
});
|
||||
|
||||
// ── /update-taskbar-items ───────────────────────────────────────────
|
||||
|
||||
describe('DesktopController POST /update-taskbar-items', () => {
|
||||
it('persists items as a JSON-encoded string', async () => {
|
||||
const { actor, userId } = await makeUser();
|
||||
const items = [{ name: 'editor' }, { name: 'browser' }];
|
||||
const { res } = makeRes();
|
||||
await callRoute(
|
||||
'post',
|
||||
'/update-taskbar-items',
|
||||
makeReq({ body: { items }, actor }),
|
||||
res,
|
||||
);
|
||||
|
||||
const refreshed = await server.stores.user.getById(userId);
|
||||
expect(refreshed?.taskbar_items).toBe(JSON.stringify(items));
|
||||
});
|
||||
|
||||
it('throws 400 when items is missing', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/update-taskbar-items',
|
||||
makeReq({ body: {}, actor }),
|
||||
res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('throws 400 when items is not an array', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const { res } = makeRes();
|
||||
await expect(
|
||||
callRoute(
|
||||
'post',
|
||||
'/update-taskbar-items',
|
||||
makeReq({ body: { items: 'oops' }, actor }),
|
||||
res,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -501,29 +501,6 @@ describe('DriverController.#handleCall (via captured router)', () => {
|
||||
expect(body.service.name).toBe('puter-kvstore');
|
||||
});
|
||||
|
||||
it('treats null `args` as no arguments', async () => {
|
||||
const actor = await makeUserActor();
|
||||
await server.stores.permission.setFlatUserPerm(
|
||||
actor.user!.id!,
|
||||
'service:puter-subdomains:ii:puter-subdomains',
|
||||
{
|
||||
permission: 'service:puter-subdomains:ii:puter-subdomains',
|
||||
deleted: false,
|
||||
issuer_user_id: actor.user!.id!,
|
||||
} as never,
|
||||
);
|
||||
|
||||
const res = makeRes();
|
||||
const req = makeReq(
|
||||
{ interface: 'puter-subdomains', method: 'select', args: null },
|
||||
actor,
|
||||
);
|
||||
await runWithContext({ actor }, () =>
|
||||
routes['POST /call'](req, res as unknown as Response, () => {}),
|
||||
);
|
||||
expect((res.body as { success: boolean }).success).toBe(true);
|
||||
});
|
||||
|
||||
it('pipes a stream-shaped result instead of JSON', async () => {
|
||||
// Use an ephemeral driver registered onto the controller's bag.
|
||||
const streamPayload = Buffer.from('hello-world');
|
||||
|
||||
@@ -276,10 +276,8 @@ export class DriverController extends PuterController {
|
||||
interface: ifaceName,
|
||||
method,
|
||||
driver: driverName,
|
||||
args: rawArgs,
|
||||
args = {},
|
||||
} = (req.body ?? {}) as Record<string, unknown>;
|
||||
// Drivers read fields off `args`, so a null one counts as absent.
|
||||
const args = rawArgs ?? {};
|
||||
|
||||
if (!ifaceName || typeof ifaceName !== 'string') {
|
||||
throw new HttpError(400, 'Missing or invalid `interface`', {
|
||||
|
||||
@@ -154,26 +154,6 @@ describe('GET /fs/readdir over HTTP', () => {
|
||||
).toEqual(['a', 'a/b']);
|
||||
});
|
||||
|
||||
it('answers 404 for a directory whose parent chain is broken', async () => {
|
||||
const { username, token } = env.users.user;
|
||||
const path = `/${username}/Documents/http-orphan-${Date.now()}`;
|
||||
const created = await fetch(new URL('/fs/mkdir', env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ path, auth_token: token }),
|
||||
});
|
||||
expect(created.status).toBe(200);
|
||||
const { uid } = (await created.json()) as { uid: string };
|
||||
await env.server.clients.db.write(
|
||||
'UPDATE fsentries SET path = NULL, parent_uid = ? WHERE uuid = ?',
|
||||
['does-not-exist', uid],
|
||||
);
|
||||
await env.server.stores.fsEntry.invalidateEntryCacheByUuid(uid);
|
||||
|
||||
const response = await fetch(readdirUrl({ uid, auth_token: token }));
|
||||
expect(response.status).toBe(404);
|
||||
});
|
||||
|
||||
it('mkdir does not return internal ids, storage columns or tokens', async () => {
|
||||
const { username, token } = env.users.user;
|
||||
const response = await fetch(new URL('/fs/mkdir', env.apiOrigin), {
|
||||
|
||||
@@ -5111,8 +5111,9 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
|
||||
// ── GET /get-launch-apps ────────────────────────────────────────────
|
||||
//
|
||||
// `recent` is a launch-metadata producer: it returns `index_url` for
|
||||
// apps the user has opened before. The field is served whoever reads it,
|
||||
// so AppDriver's hosted-backing guard is applied here too.
|
||||
// apps the user has opened before. The taskbar happens to launch these
|
||||
// by name (so AppDriver's hosted-backing guard applies), but the field
|
||||
// is served regardless, so the guard is applied here too.
|
||||
|
||||
describe('LegacyFSController GET /get-launch-apps', () => {
|
||||
const hostedUrl = (sub: string) => `https://${sub}.site.puter.localhost/`;
|
||||
|
||||
@@ -401,9 +401,11 @@ export class LegacyFSController extends PuterController {
|
||||
.filter(Boolean);
|
||||
|
||||
// Don't hand out an index_url whose puter-hosted backing is
|
||||
// gone or reclaimed. This list is a launch-metadata producer
|
||||
// like any other — a consumer reading index_url straight off
|
||||
// it shouldn't inherit a stale origin.
|
||||
// gone or reclaimed. The taskbar launches recents by name (so
|
||||
// AppDriver's guard applies), but this list is a
|
||||
// launch-metadata producer like any other — a future consumer
|
||||
// reading index_url straight off it shouldn't inherit a stale
|
||||
// origin.
|
||||
let backingGoneFlags: boolean[];
|
||||
try {
|
||||
backingGoneFlags =
|
||||
@@ -948,7 +950,9 @@ export class LegacyFSController extends PuterController {
|
||||
// Trash, and `null`/`{}` when restoring. See
|
||||
// `src/gui/src/helpers.js` → `window.move_items`.
|
||||
newMetadata: (body.new_metadata ?? undefined) as
|
||||
Record<string, unknown> | null | undefined,
|
||||
| Record<string, unknown>
|
||||
| null
|
||||
| undefined,
|
||||
});
|
||||
const overwrittenEntry = replacedByMove(overwriteTarget, moved, name);
|
||||
const oldPath = source.path;
|
||||
@@ -1422,7 +1426,8 @@ export class LegacyFSController extends PuterController {
|
||||
}
|
||||
|
||||
type SignedOrEmpty =
|
||||
(SignedFile & { path?: string }) | Record<string, never>;
|
||||
| (SignedFile & { path?: string })
|
||||
| Record<string, never>;
|
||||
const result: { signatures: SignedOrEmpty[]; token?: string } = {
|
||||
signatures: [],
|
||||
};
|
||||
@@ -2049,7 +2054,10 @@ export class LegacyFSController extends PuterController {
|
||||
const subjectRef = body.subject;
|
||||
const appRef = body.app;
|
||||
const mode = (getString(body, 'mode') ?? 'read') as
|
||||
'see' | 'list' | 'read' | 'write';
|
||||
| 'see'
|
||||
| 'list'
|
||||
| 'read'
|
||||
| 'write';
|
||||
if (!subjectRef || !appRef)
|
||||
throw new HttpError(400, '`subject` and `app` are required', {
|
||||
legacyCode: 'bad_request',
|
||||
|
||||
@@ -79,7 +79,6 @@ interface CapturedResponse {
|
||||
statusCode: number;
|
||||
body: unknown;
|
||||
contentType?: string;
|
||||
location?: string;
|
||||
}
|
||||
|
||||
const makeReq = (init: {
|
||||
@@ -88,7 +87,6 @@ const makeReq = (init: {
|
||||
actor?: Actor;
|
||||
hostname?: string;
|
||||
protocol?: string;
|
||||
originalUrl?: string;
|
||||
}): Request => {
|
||||
return {
|
||||
body: {},
|
||||
@@ -96,7 +94,6 @@ const makeReq = (init: {
|
||||
headers: {},
|
||||
params: init.params ?? {},
|
||||
path: init.path ?? '/',
|
||||
originalUrl: init.originalUrl ?? init.path ?? '/',
|
||||
hostname: init.hostname ?? 'test.local',
|
||||
protocol: init.protocol ?? 'http',
|
||||
actor: init.actor,
|
||||
@@ -134,11 +131,6 @@ const makeRes = () => {
|
||||
captured.contentType = value;
|
||||
return res;
|
||||
}),
|
||||
redirect: vi.fn((location: string) => {
|
||||
captured.statusCode = 302;
|
||||
captured.location = location;
|
||||
return res;
|
||||
}),
|
||||
};
|
||||
return { res: res as unknown as Response, captured };
|
||||
};
|
||||
@@ -222,72 +214,23 @@ describe('HomepageController shell routes', () => {
|
||||
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
|
||||
});
|
||||
|
||||
it('leaves guiTokenDigest out for a session cookie that is not a user session', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const appActor = {
|
||||
...actor,
|
||||
app: { id: 1, uid: 'app-test' },
|
||||
session: { uid: uuidv4() },
|
||||
} as unknown as Actor;
|
||||
const req = Object.assign(makeReq({ path: '/', actor: appActor }), {
|
||||
tokenSource: 'cookie',
|
||||
cookies: { puter_token_v2: 'legacy' },
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute('get', '/', req, res);
|
||||
expect(String(captured.body)).not.toContain('guiTokenDigest');
|
||||
});
|
||||
|
||||
it('serves the shell on /dashboard and /action', async () => {
|
||||
for (const path of ['/dashboard', '/dashboard/', '/action/*splat']) {
|
||||
it('serves the shell on /settings, /dashboard, /desktop, /action, /@:username', async () => {
|
||||
for (const path of [
|
||||
'/settings',
|
||||
'/settings/*splat',
|
||||
'/dashboard',
|
||||
'/dashboard/',
|
||||
'/desktop',
|
||||
'/desktop/',
|
||||
'/action/*splat',
|
||||
'/@:username',
|
||||
]) {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute('get', path, makeReq({ path }), res);
|
||||
expect(typeof captured.body).toBe('string');
|
||||
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
|
||||
}
|
||||
});
|
||||
|
||||
it('redirects /desktop to the root, keeping the query', async () => {
|
||||
for (const path of ['/desktop', '/desktop/']) {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
path,
|
||||
makeReq({ path, originalUrl: `${path}?ref=abc&x=1` }),
|
||||
res,
|
||||
);
|
||||
expect(captured.statusCode).toBe(302);
|
||||
expect(captured.location).toBe('/?ref=abc&x=1');
|
||||
expect(captured.body).toBeUndefined();
|
||||
}
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute('get', '/desktop', makeReq({ path: '/desktop' }), res);
|
||||
expect(captured.location).toBe('/');
|
||||
});
|
||||
|
||||
it('redirects /settings to the dashboard Home tab', async () => {
|
||||
for (const path of ['/settings', '/settings/*splat']) {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute('get', path, makeReq({ path }), res);
|
||||
expect(captured.statusCode).toBe(302);
|
||||
expect(captured.location).toBe('/dashboard#home');
|
||||
expect(captured.body).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('redirects /@:username to the dashboard', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/@:username',
|
||||
makeReq({ path: '/@alice', params: { username: 'alice' } }),
|
||||
res,
|
||||
);
|
||||
expect(captured.statusCode).toBe(302);
|
||||
expect(captured.location).toBe('/');
|
||||
expect(captured.body).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
// ── /app/:name ──────────────────────────────────────────────────────
|
||||
@@ -417,21 +360,47 @@ describe('HomepageController GET /app/:name', () => {
|
||||
expect(html).not.toContain('owner_user_id');
|
||||
});
|
||||
|
||||
it('redirects /desktop/app/:name to /app/:name, keeping the query', async () => {
|
||||
it('serves the same app shell under /desktop/app/:name', async () => {
|
||||
const { userId } = await makeUser();
|
||||
const name = `desk-${Math.random().toString(36).slice(2, 10)}`;
|
||||
await server.stores.app.create(
|
||||
{
|
||||
name,
|
||||
title: 'Desktop App',
|
||||
description: 'opens on the desktop instead of the dashboard',
|
||||
index_url: `https://example.com/${name}/`,
|
||||
approved_for_listing: 1,
|
||||
},
|
||||
{ ownerUserId: userId },
|
||||
);
|
||||
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/desktop/app/:name',
|
||||
makeReq({ params: { name }, path: `/desktop/app/${name}` }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(captured.statusCode).toBe(200);
|
||||
const html = String(captured.body);
|
||||
expect(html).toMatch(/<!DOCTYPE html>/i);
|
||||
expect(html).toContain('Desktop App');
|
||||
});
|
||||
|
||||
it('returns 404 under /desktop/app/:name when the app is unknown', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/desktop/app/:name',
|
||||
makeReq({
|
||||
params: { name: 'editor' },
|
||||
path: '/desktop/app/editor',
|
||||
originalUrl: '/desktop/app/editor?file=%2Falice%2Fa.txt&x=1',
|
||||
params: { name: 'no-such-app' },
|
||||
path: '/desktop/app/no-such-app',
|
||||
}),
|
||||
res,
|
||||
);
|
||||
expect(captured.statusCode).toBe(302);
|
||||
expect(captured.location).toBe('/app/editor?file=%2Falice%2Fa.txt&x=1');
|
||||
expect(captured.body).toBeUndefined();
|
||||
expect(captured.statusCode).toBe(404);
|
||||
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
|
||||
});
|
||||
|
||||
it('omits index_url even for a public app — the shell is not the launch authority', async () => {
|
||||
@@ -465,39 +434,40 @@ describe('HomepageController GET /app/:name', () => {
|
||||
// -- app landing rate limit --
|
||||
|
||||
describe('HomepageController app landing rate limit', () => {
|
||||
it('rate-limits /app/:name per network', () => {
|
||||
const route = router.routes.find(
|
||||
(r) => r.method === 'get' && r.path === '/app/:name',
|
||||
);
|
||||
if (!route) throw new Error('No GET /app/:name route');
|
||||
expect(route.options.rateLimit).toEqual({
|
||||
scope: 'app-landing',
|
||||
limit: 600,
|
||||
window: 60_000,
|
||||
key: 'ip',
|
||||
});
|
||||
it('rate-limits /app/:name and /desktop/app/:name per network', () => {
|
||||
for (const path of ['/app/:name', '/desktop/app/:name']) {
|
||||
const route = router.routes.find(
|
||||
(r) => r.method === 'get' && r.path === path,
|
||||
);
|
||||
if (!route) throw new Error(`No GET ${path} route`);
|
||||
expect(route.options.rateLimit).toEqual({
|
||||
scope: 'app-landing',
|
||||
limit: 600,
|
||||
window: 60_000,
|
||||
key: 'ip',
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── /show/* ─────────────────────────────────────────────────────────
|
||||
|
||||
describe('HomepageController GET /show/*splat', () => {
|
||||
it('redirects to the path in the dashboard Files tab, keeping the query', async () => {
|
||||
it('emits a launch_app explorer call with the post-/show path', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/show/*splat',
|
||||
makeReq({
|
||||
path: '/show/alice/My%20Documents',
|
||||
originalUrl: '/show/alice/My%20Documents?ref=abc',
|
||||
}),
|
||||
makeReq({ path: '/show/alice/Documents' }),
|
||||
res,
|
||||
);
|
||||
expect(captured.statusCode).toBe(302);
|
||||
expect(captured.location).toBe(
|
||||
'/dashboard?ref=abc#files/alice/My%20Documents',
|
||||
);
|
||||
expect(captured.body).toBeUndefined();
|
||||
// The launch payload is JSON-serialized into the rendered HTML.
|
||||
// We assert the rendered shell included the explorer launch hint
|
||||
// pointing at the expected (slashed) path.
|
||||
const html = String(captured.body);
|
||||
expect(html).toContain('launch_app');
|
||||
expect(html).toContain('explorer');
|
||||
expect(html).toContain('/alice/Documents');
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
import express from 'express';
|
||||
import path from 'node:path';
|
||||
import { PuterController } from '../types.js';
|
||||
import { isPlainUserActor } from '../../core/actor.js';
|
||||
import { toAppShellView } from '../../util/appShellView.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter';
|
||||
import {
|
||||
@@ -28,12 +27,11 @@ import {
|
||||
subdomainLabelOf,
|
||||
} from '../../core/http/middleware/hostRedirects.js';
|
||||
import type { IConfig } from '../../types';
|
||||
import type { UserRow } from '../../stores/user/UserStore';
|
||||
import type {
|
||||
PuterHomepageService,
|
||||
PageMeta,
|
||||
LaunchOptions,
|
||||
PopupFrame,
|
||||
ShellContext,
|
||||
} from '../../services/homepage/PuterHomepageService';
|
||||
|
||||
/**
|
||||
@@ -50,7 +48,6 @@ export const FRAMED_POPUP_ACTIONS = [
|
||||
'upgrade',
|
||||
'top-up',
|
||||
'request-email-confirmation',
|
||||
'attach-custom-domain',
|
||||
] as const;
|
||||
|
||||
/** Longest description/og:description/twitter:description on `/app/<name>`. */
|
||||
@@ -172,6 +169,7 @@ export class HomepageController extends PuterController {
|
||||
req: express.Request,
|
||||
res: express.Response,
|
||||
metaOverrides: Partial<PageMeta> = {},
|
||||
launch: LaunchOptions = {},
|
||||
popupFrame?: PopupFrame,
|
||||
) => {
|
||||
const meta = { ...defaultMeta(req), ...metaOverrides };
|
||||
@@ -181,16 +179,7 @@ export class HomepageController extends PuterController {
|
||||
actor?: Parameters<typeof homepage.send>[0]['actor'];
|
||||
}
|
||||
).actor ?? null;
|
||||
await homepage.send(
|
||||
{
|
||||
req,
|
||||
res,
|
||||
actor,
|
||||
guiTokenDigest: this.#guiTokenDigest(req, actor),
|
||||
},
|
||||
meta,
|
||||
popupFrame,
|
||||
);
|
||||
await homepage.send({ req, res, actor }, meta, launch, popupFrame);
|
||||
};
|
||||
|
||||
// -- Root + path-aliased shell routes ------------------------
|
||||
@@ -262,6 +251,7 @@ export class HomepageController extends PuterController {
|
||||
req,
|
||||
res,
|
||||
{},
|
||||
{},
|
||||
{
|
||||
hostOrigin: embedHostOrigin(
|
||||
this.config,
|
||||
@@ -272,42 +262,23 @@ export class HomepageController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
// Old settings links land on the dashboard's Home tab.
|
||||
router.get('/settings', {}, (_req, res) =>
|
||||
res.redirect('/dashboard#home'),
|
||||
);
|
||||
router.get('/settings/*splat', {}, (_req, res) =>
|
||||
res.redirect('/dashboard#home'),
|
||||
);
|
||||
router.get('/settings', {}, (req, res) => sendShell(req, res));
|
||||
router.get('/settings/*splat', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
router.get('/dashboard', {}, (req, res) => sendShell(req, res));
|
||||
router.get('/dashboard/', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
const queryOf = (req: express.Request) => {
|
||||
const separator = req.originalUrl.indexOf('?');
|
||||
return separator === -1 ? '' : req.originalUrl.slice(separator);
|
||||
};
|
||||
|
||||
// Old desktop links land on the dashboard, query intact.
|
||||
router.get('/desktop', {}, (req, res) =>
|
||||
res.redirect(`/${queryOf(req)}`),
|
||||
);
|
||||
router.get('/desktop/', {}, (req, res) =>
|
||||
res.redirect(`/${queryOf(req)}`),
|
||||
);
|
||||
router.get('/desktop', {}, (req, res) => sendShell(req, res));
|
||||
router.get('/desktop/', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
router.get('/action/*splat', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
// Old links to a user's public folder land on the dashboard.
|
||||
router.get('/@:username', {}, (_req, res) => res.redirect('/'));
|
||||
|
||||
// Old desktop app links open the app in the dashboard, query intact.
|
||||
router.get('/desktop/app/:name', {}, (req, res) => {
|
||||
const name = encodeURIComponent(String(req.params.name ?? ''));
|
||||
res.redirect(`/app/${name}${queryOf(req)}`);
|
||||
});
|
||||
router.get('/@:username', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
// -- /app/:name - app metadata baked into the shell ----------
|
||||
// `/desktop/app/:name` is the same landing booted on the desktop
|
||||
// instead of the dashboard; the GUI strips the prefix and treats the
|
||||
// rest of the path identically, so both render the same shell.
|
||||
|
||||
const sendAppShell = async (
|
||||
req: express.Request,
|
||||
@@ -362,11 +333,26 @@ export class HomepageController extends PuterController {
|
||||
{ rateLimit: APP_LANDING_LIMIT },
|
||||
sendAppShell,
|
||||
);
|
||||
router.get(
|
||||
'/desktop/app/:name',
|
||||
{ rateLimit: APP_LANDING_LIMIT },
|
||||
sendAppShell,
|
||||
);
|
||||
|
||||
// -- /show/* - launch explorer with the requested file path --
|
||||
|
||||
// Old /show links open the path in the dashboard's Files tab, query intact.
|
||||
router.get('/show/*splat', {}, (req, res) => {
|
||||
const filePath = req.path.slice('/show'.length);
|
||||
res.redirect(`/dashboard${queryOf(req)}#files${filePath}`);
|
||||
const launch: LaunchOptions = {
|
||||
on_initialized: [
|
||||
{
|
||||
$: 'window-call',
|
||||
fn_name: 'launch_app',
|
||||
args: [{ name: 'explorer', path: filePath }],
|
||||
},
|
||||
],
|
||||
};
|
||||
return sendShell(req, res, {}, launch);
|
||||
});
|
||||
|
||||
// -- /sw.js - the service worker, scoped to the whole origin --
|
||||
@@ -395,38 +381,6 @@ export class HomepageController extends PuterController {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Lets the GUI skip `/get-gui-token`, which reads the session cookie: the
|
||||
* digest of the token it would hand over, null when there is no session
|
||||
* cookie (either name the auth probe reads), and undefined (the GUI asks)
|
||||
* when a cookie didn't resolve to a session.
|
||||
*/
|
||||
#guiTokenDigest(
|
||||
req: express.Request,
|
||||
actor: ShellContext['actor'],
|
||||
): string | null | undefined {
|
||||
if (req.tokenSource === 'cookie') {
|
||||
if (!actor || !isPlainUserActor(actor) || !actor.session?.uid) {
|
||||
return undefined;
|
||||
}
|
||||
return this.services.auth.guiTokenDigest(
|
||||
actor.user as UserRow,
|
||||
actor.session.uid,
|
||||
);
|
||||
}
|
||||
const cookies = (req.cookies ?? {}) as Record<string, unknown>;
|
||||
const hasSessionCookie = [
|
||||
this.config.cookie_name,
|
||||
'puter_token_v2',
|
||||
].some(
|
||||
(name) =>
|
||||
!!name &&
|
||||
typeof cookies[name] === 'string' &&
|
||||
cookies[name] !== '',
|
||||
);
|
||||
return hasSessionCookie ? undefined : null;
|
||||
}
|
||||
}
|
||||
|
||||
const safeJsonParse = (s: string): Record<string, unknown> | null => {
|
||||
|
||||
@@ -372,6 +372,26 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('bakes a whitelisted desktop app landing (/desktop/app/<name>) into the state redirect_uri', async () => {
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/:provider/start',
|
||||
makeReq({
|
||||
params: { provider: 'custom' },
|
||||
query: { return_to: '/desktop/app/my-App_2' },
|
||||
}),
|
||||
res,
|
||||
);
|
||||
const state = new URL(captured.redirectUrl ?? '').searchParams.get(
|
||||
'state',
|
||||
);
|
||||
const decoded = oidc().verifyState(state!);
|
||||
expect(String(decoded?.redirect_uri)).toBe(
|
||||
`${TEST_ORIGIN}/desktop/app/my-App_2`,
|
||||
);
|
||||
});
|
||||
|
||||
it('ignores a non-whitelisted return_to', async () => {
|
||||
const bad_values = [
|
||||
'/app/evil/extra',
|
||||
@@ -380,8 +400,6 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
|
||||
'//evil.test',
|
||||
'/settings',
|
||||
`/app/${'a'.repeat(101)}`,
|
||||
'/desktop',
|
||||
'/desktop/app/my-App_2',
|
||||
'/desktop/app/evil/extra',
|
||||
'/desktop/app/',
|
||||
'/dashboard/app/name',
|
||||
@@ -458,11 +476,11 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
|
||||
sharedPath('b.txt'),
|
||||
sharedPath('a.txt'),
|
||||
],
|
||||
'/dashboard',
|
||||
'/desktop',
|
||||
),
|
||||
);
|
||||
const url = new URL(uri);
|
||||
expect(url.origin + url.pathname).toBe(`${TEST_ORIGIN}/dashboard`);
|
||||
expect(url.origin + url.pathname).toBe(`${TEST_ORIGIN}/desktop`);
|
||||
expect(url.searchParams.getAll('shared')).toEqual([
|
||||
sharedPath('a.txt'),
|
||||
sharedPath('b.txt'),
|
||||
|
||||
@@ -102,14 +102,16 @@ function signupOptionsFromState(
|
||||
}
|
||||
|
||||
// GUI pages an OIDC flow may return to: the root (where a share email lands),
|
||||
// /dashboard, and direct app landings (/app/<name>, mirroring
|
||||
// APP_NAME_REGEX in AppDriver). Strict whitelist — never a client-supplied URL
|
||||
// (no open redirect).
|
||||
// /desktop, /dashboard, and direct app landings (/app/<name> and its
|
||||
// desktop-booted twin /desktop/app/<name>, mirroring APP_NAME_REGEX in
|
||||
// AppDriver). Strict whitelist — never a client-supplied URL (no open
|
||||
// redirect).
|
||||
function isWhitelistedReturnPath(path: string): boolean {
|
||||
return (
|
||||
path === '/' ||
|
||||
path === '/desktop' ||
|
||||
path === '/dashboard' ||
|
||||
/^\/app\/[a-zA-Z0-9_-]{1,100}$/.test(path)
|
||||
/^(\/desktop)?\/app\/[a-zA-Z0-9_-]{1,100}$/.test(path)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -436,8 +438,9 @@ export class OIDCController extends PuterController {
|
||||
|
||||
let appRedirectUri = flowRedirects[flow] ?? (origin || '/');
|
||||
|
||||
// Optional GUI return path so login started from /dashboard,
|
||||
// an /app/<name> landing, or a share link lands back there.
|
||||
// Optional GUI return path so login started from /desktop,
|
||||
// /dashboard, an /app/<name> landing, or a share link lands
|
||||
// back there.
|
||||
const rawReturnTo = Array.isArray(req.query.return_to)
|
||||
? req.query.return_to[0]
|
||||
: req.query.return_to;
|
||||
|
||||
@@ -314,8 +314,8 @@ describe('OIDC account second-factor login', () => {
|
||||
|
||||
it('preserves ordinary OIDC login for accounts without 2FA', async () => {
|
||||
await account(false);
|
||||
const response = await callback('login', { return_to: '/dashboard' });
|
||||
expect(response.headers.get('location')).toBe(`${env.origin}/dashboard`);
|
||||
const response = await callback('login', { return_to: '/desktop' });
|
||||
expect(response.headers.get('location')).toBe(`${env.origin}/desktop`);
|
||||
expect(
|
||||
response.headers.get('set-cookie')?.includes('puter_auth_token='),
|
||||
).toBe(true);
|
||||
|
||||
@@ -24,7 +24,11 @@ import { computeNetworkFingerprint } from '../../core/http/middleware/rateLimit.
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { PuterController } from '../types.js';
|
||||
import { PEER_COSTS } from './costs.js';
|
||||
import { signGuestGrant, verifyGuestGrant } from './guestGrant.js';
|
||||
import {
|
||||
readClaimedGrantIdentifier,
|
||||
signGuestGrant,
|
||||
verifyGuestGrant,
|
||||
} from './guestGrant.js';
|
||||
import {
|
||||
DEFAULT_FREE_SUBSCRIPTION,
|
||||
DEFAULT_TEMP_SUBSCRIPTION,
|
||||
@@ -187,30 +191,19 @@ export class PeerController extends PuterController {
|
||||
// guests share one bucket and no host can be relayed for by
|
||||
// more guests per minute than this — the only ceiling on guest
|
||||
// spend we can apply before the bytes are already spent.
|
||||
// The limiter counts a request when it admits it, so only a
|
||||
// grant that would redeem names a host; otherwise anyone
|
||||
// holding the host's identifier could drain the bucket and
|
||||
// lock out every real guest. Anything else - forged, expired,
|
||||
// malformed, or no secret configured - is charged to the
|
||||
// caller's own network.
|
||||
// A grant that doesn't parse can't name a bucket, so those
|
||||
// requests fall back to the caller's own network.
|
||||
rateLimit: {
|
||||
scope: 'peer-guest-turn',
|
||||
limit: 60,
|
||||
window: 60_000,
|
||||
key: (req: Request) => {
|
||||
const secret =
|
||||
this.config.peers?.guest_turn?.grant_secret;
|
||||
const verified = secret
|
||||
? verifyGuestGrant({
|
||||
grant: (
|
||||
req.body as
|
||||
{ grant?: unknown } | undefined
|
||||
)?.grant,
|
||||
secret,
|
||||
})
|
||||
: null;
|
||||
return verified?.status === 'ok'
|
||||
? `host:${verified.customIdentifier}`
|
||||
const claimed = readClaimedGrantIdentifier(
|
||||
(req.body as { grant?: unknown } | undefined)
|
||||
?.grant,
|
||||
);
|
||||
return claimed
|
||||
? `host:${claimed}`
|
||||
: `net:${computeNetworkFingerprint(req)}`;
|
||||
},
|
||||
},
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
|
||||
import { toMicroCents } from '../../services/metering/utils';
|
||||
|
||||
// Microcents per byte of TURN egress ($0.10/GB).
|
||||
// Microcents per byte of TURN egress ($0.05/GB).
|
||||
export const PEER_COSTS = {
|
||||
'turn:egress-bytes': toMicroCents(0.1 / 1000 ** 3),
|
||||
} as const;
|
||||
@@ -1,6 +1,7 @@
|
||||
import { createHmac } from 'node:crypto';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
readClaimedGrantIdentifier,
|
||||
signGuestGrant,
|
||||
verifyGuestGrant,
|
||||
} from './guestGrant.js';
|
||||
@@ -242,3 +243,59 @@ describe('signGuestGrant / verifyGuestGrant', () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('readClaimedGrantIdentifier', () => {
|
||||
it('reads the identifier from a valid grant', () => {
|
||||
const { grant } = signGuestGrant({
|
||||
customIdentifier: USER_ID,
|
||||
ttlSeconds: 60,
|
||||
secret: SECRET,
|
||||
});
|
||||
|
||||
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
|
||||
});
|
||||
|
||||
it('reads the claimed identifier without checking the signature', () => {
|
||||
// Bucketing runs before verification, so this is expected: a forged
|
||||
// grant still names a bucket, and the handler still rejects it.
|
||||
const grant = forgeGrant(
|
||||
{ id: USER_ID, exp: 9_999_999_999, n: 'x' },
|
||||
{ secret: 'wrong-secret' },
|
||||
);
|
||||
|
||||
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
|
||||
});
|
||||
|
||||
it('reads an expired grant, which the handler then rejects', () => {
|
||||
const { grant } = signGuestGrant({
|
||||
customIdentifier: USER_ID,
|
||||
ttlSeconds: 60,
|
||||
secret: SECRET,
|
||||
now: 1_000_000_000_000,
|
||||
});
|
||||
|
||||
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
|
||||
expect(verifyGuestGrant({ grant, secret: SECRET })).toEqual({
|
||||
status: 'expired',
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
['a non-string', 42],
|
||||
['undefined', undefined],
|
||||
['a wrong-shaped string', 'not-a-grant'],
|
||||
['an unknown version', 'pg2.abc.def'],
|
||||
['an over-long string', `pg1.${'a'.repeat(600)}.sig`],
|
||||
[
|
||||
'a non-JSON payload',
|
||||
`pg1.${Buffer.from('x').toString('base64url')}.sig`,
|
||||
],
|
||||
])('returns null for %s', (_label, grant) => {
|
||||
expect(readClaimedGrantIdentifier(grant)).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null for an identifier of the wrong shape', () => {
|
||||
const grant = forgeGrant({ id: 'nope', exp: 9_999_999_999 });
|
||||
expect(readClaimedGrantIdentifier(grant)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -171,3 +171,37 @@ export const verifyGuestGrant = ({
|
||||
expiresAt: payload.exp,
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Read the claimed identifier out of a grant _without_ verifying it, for
|
||||
* rate-limit bucketing only.
|
||||
*
|
||||
* Bucketing has to happen before the handler runs, and a forged grant is
|
||||
* rejected there without reaching the upstream API — so an unverified read is
|
||||
* enough to put a host's guests in one bucket, and the worst a forger achieves
|
||||
* is choosing which bucket their own rejections land in. Never use this to
|
||||
* decide attribution.
|
||||
*
|
||||
* @returns The claimed identifier, or null if the grant doesn't parse.
|
||||
*/
|
||||
export const readClaimedGrantIdentifier = (grant: unknown): string | null => {
|
||||
if (typeof grant !== 'string' || grant.length > MAX_GRANT_LENGTH) {
|
||||
return null;
|
||||
}
|
||||
const parts = grant.split('.');
|
||||
if (parts.length !== 3 || parts[0] !== GRANT_VERSION) return null;
|
||||
try {
|
||||
const payload = JSON.parse(
|
||||
Buffer.from(parts[1]!, 'base64url').toString('utf8'),
|
||||
) as { id?: unknown };
|
||||
if (
|
||||
typeof payload?.id !== 'string' ||
|
||||
!IDENTIFIER_RE.test(payload.id)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return payload.id;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
@@ -1,89 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import {
|
||||
setupPuterTestEnv,
|
||||
TEST_USER_CREDENTIALS,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
|
||||
/** A PUT body is the file, whatever content type the client labels it with. */
|
||||
describe('PUT stores the body as sent', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
const dav = (path: string, init: RequestInit) => {
|
||||
const port = new URL(env.apiOrigin).port;
|
||||
const auth = Buffer.from(
|
||||
`${TEST_USER_CREDENTIALS.username}:${TEST_USER_CREDENTIALS.password}`,
|
||||
).toString('base64');
|
||||
return fetch(`http://dav.puter.localhost:${port}${encodeURI(path)}`, {
|
||||
...init,
|
||||
headers: { Authorization: `Basic ${auth}`, ...init.headers },
|
||||
});
|
||||
};
|
||||
|
||||
it.each([
|
||||
['application/json', '{"name":"report","rows":[1,2,3]}'],
|
||||
// Not JSON at all: the label alone must not get the body rejected.
|
||||
['application/json', 'not json'],
|
||||
['text/plain;actually=json', '{"a":1}'],
|
||||
['application/x-www-form-urlencoded', 'a=1&b=two'],
|
||||
['application/octet-stream', 'plain bytes'],
|
||||
])(
|
||||
'with Content-Type %s',
|
||||
async (contentType, content) => {
|
||||
const path = `/${env.users.user.username}/Documents/put-${randomUUID().slice(0, 8)}.bin`;
|
||||
const put = await dav(path, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': contentType },
|
||||
body: content,
|
||||
});
|
||||
expect(put.status).toBe(201);
|
||||
|
||||
const get = await dav(path, { method: 'GET' });
|
||||
expect(get.status).toBe(200);
|
||||
expect(await get.text()).toBe(content);
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
it('with a form body over the form parser limit', async () => {
|
||||
const path = `/${env.users.user.username}/Documents/put-${randomUUID().slice(0, 8)}.bin`;
|
||||
const content = `a=${'x'.repeat(200 * 1024)}`;
|
||||
const put = await dav(path, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
||||
body: content,
|
||||
});
|
||||
expect(put.status).toBe(201);
|
||||
|
||||
const get = await dav(path, { method: 'GET' });
|
||||
expect(await get.text()).toBe(content);
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -243,33 +243,6 @@ describe('WispController', () => {
|
||||
expect(await verifyStatus(token)).toBe(403);
|
||||
});
|
||||
|
||||
it('binds a token minted by an access token to that token while the replica lags', async () => {
|
||||
const { sessionActor, apiTokenActor, apiToken } = await freshUser();
|
||||
const db = server.clients.db;
|
||||
const original = db.read.bind(db);
|
||||
const spy = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql: string, params?: unknown[]) =>
|
||||
sql.includes("FROM `sessions` WHERE `kind` = 'access_token'")
|
||||
? []
|
||||
: original(sql, params),
|
||||
);
|
||||
let token: string;
|
||||
try {
|
||||
token = await relayTokenFor(apiTokenActor);
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
}
|
||||
expect(await verifyStatus(token)).toBe(200);
|
||||
|
||||
await server.services.auth.revokeAccessToken(
|
||||
sessionActor,
|
||||
apiToken,
|
||||
);
|
||||
|
||||
expect(await verifyStatus(token)).toBe(403);
|
||||
});
|
||||
|
||||
it('applies the account checks to a token with no session claim', async () => {
|
||||
const { row } = await freshUser();
|
||||
const token = server.services.token.sign(
|
||||
|
||||
@@ -151,12 +151,11 @@ export class WispController extends PuterController {
|
||||
|
||||
/**
|
||||
* The session a relay token lives no longer than. An access token's is its
|
||||
* own session row.
|
||||
* own session row; the actor doesn't carry it.
|
||||
*/
|
||||
async #sessionUidFor(actor: Actor): Promise<string | undefined> {
|
||||
if (actor.session?.uid) return actor.session.uid;
|
||||
if (!actor.accessToken) return undefined;
|
||||
if (actor.accessToken.sessionUid) return actor.accessToken.sessionUid;
|
||||
const row = await this.stores.session.findActiveByAccessTokenUid(
|
||||
actor.accessToken.uid,
|
||||
);
|
||||
|
||||
@@ -47,11 +47,6 @@ export interface ActorAccessToken {
|
||||
* reach while revokes, audits and gate hooks can still name the app.
|
||||
*/
|
||||
godmodeApp?: ActorApp | null;
|
||||
/**
|
||||
* The token's own session row, checked live when it authenticated. Unset
|
||||
* when the token carries no `session_uid` naming its row.
|
||||
*/
|
||||
sessionUid?: string;
|
||||
}
|
||||
|
||||
export interface Actor {
|
||||
|
||||
@@ -50,13 +50,6 @@ declare global {
|
||||
|
||||
tokenAuthFailed?: boolean;
|
||||
|
||||
/**
|
||||
* What `authenticate` threw while checking the token (a database
|
||||
* failure, say). The token may be fine; gates rethrow this instead
|
||||
* of answering 401.
|
||||
*/
|
||||
tokenAuthError?: unknown;
|
||||
|
||||
/**
|
||||
* Set when a token authenticated but its app is on the origin
|
||||
* blocklist. The auth probe leaves `actor` unset; gates translate
|
||||
|
||||
@@ -1,110 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../../testUtil.js';
|
||||
|
||||
const loadShed = () =>
|
||||
Object.assign(new Error('Database operation failed'), {
|
||||
code: 'dbBatchFailed',
|
||||
});
|
||||
|
||||
/**
|
||||
* A token whose lookup fails is not a dead token: clients discard a token on
|
||||
* 401 `token_auth_failed`, so that answer is kept for tokens that checked out
|
||||
* invalid.
|
||||
*/
|
||||
describe('auth probe when the token lookup fails', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
}, 120_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
const get = (path: string, token: string) =>
|
||||
fetch(new URL(path, env.apiOrigin), {
|
||||
headers: { authorization: `Bearer ${token}` },
|
||||
});
|
||||
|
||||
/** Run `fn` while the next user lookup rejects with `error`. */
|
||||
const withFailedUserLookup = async <T>(
|
||||
error: Error,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<T> => {
|
||||
const spy = vi
|
||||
.spyOn(env.server.stores.user, 'getByUuid')
|
||||
.mockRejectedValueOnce(error);
|
||||
try {
|
||||
return await fn();
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
}
|
||||
};
|
||||
|
||||
it.each([
|
||||
['a session token', () => env.users.user.token],
|
||||
['an access token', () => env.users.user.apiToken],
|
||||
])(
|
||||
'answers %s with 503 db_unavailable during a database outage',
|
||||
async (_kind, tokenOf) => {
|
||||
const token = tokenOf();
|
||||
const res = await withFailedUserLookup(loadShed(), () =>
|
||||
get('/whoami', token),
|
||||
);
|
||||
expect(res.status).toBe(503);
|
||||
expect(res.headers.get('retry-after')).toBe('5');
|
||||
expect(await res.json()).toEqual({
|
||||
error: 'Service temporarily unavailable',
|
||||
message: 'Service temporarily unavailable',
|
||||
code: 'db_unavailable',
|
||||
});
|
||||
|
||||
expect((await get('/whoami', token)).status).toBe(200);
|
||||
},
|
||||
);
|
||||
|
||||
it('answers an unexpected lookup failure with a generic 500, not a 401', async () => {
|
||||
const res = await withFailedUserLookup(new Error('boom'), () =>
|
||||
get('/whoami', env.users.user.token),
|
||||
);
|
||||
expect(res.status).toBe(500);
|
||||
expect((await res.json()).code).toBe('internal_error');
|
||||
});
|
||||
|
||||
it('serves a route that does not require auth anonymously', async () => {
|
||||
const res = await withFailedUserLookup(loadShed(), () =>
|
||||
get('/version', env.users.user.token),
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
});
|
||||
|
||||
it('still answers an invalid token with 401 token_auth_failed', async () => {
|
||||
const res = await get('/whoami', 'not-a-real-jwt');
|
||||
expect(res.status).toBe(401);
|
||||
expect(await res.json()).toEqual({
|
||||
error: 'Authentication failed',
|
||||
message: 'Authentication failed',
|
||||
code: 'token_auth_failed',
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -555,18 +555,17 @@ describe('createAuthProbe — actor attachment + failure tracking', () => {
|
||||
expect(req.tokenAuthFailed).toBe(true);
|
||||
});
|
||||
|
||||
it('never rejects — records the error, not tokenAuthFailed, when AuthService throws', async () => {
|
||||
it('never rejects — sets tokenAuthFailed even when AuthService throws', async () => {
|
||||
const stub = makeStubAuth();
|
||||
stub.setNext('throw');
|
||||
const probe = createAuthProbe({ authService: stub.service });
|
||||
const { req, next } = await runProbe(
|
||||
probe,
|
||||
makeReq({ headers: { authorization: 'Bearer some-tok' } }),
|
||||
makeReq({ headers: { authorization: 'Bearer bad-tok' } }),
|
||||
);
|
||||
// Critical invariant: the probe NEVER rejects, no matter what.
|
||||
expect(next).toHaveBeenCalledWith();
|
||||
expect(req.tokenAuthFailed).toBeUndefined();
|
||||
expect(req.tokenAuthError).toBeInstanceOf(Error);
|
||||
expect(req.tokenAuthFailed).toBe(true);
|
||||
expect(req.actor).toBeUndefined();
|
||||
});
|
||||
|
||||
@@ -881,27 +880,4 @@ describe('createAuthProbe (integration) — real session token → real actor',
|
||||
expect(req.actor).toBeUndefined();
|
||||
expect(req.tokenAuthFailed).toBe(true);
|
||||
});
|
||||
|
||||
it('records a failed user lookup on a valid token without marking it invalid', async () => {
|
||||
const user = await makeUser();
|
||||
const { token } = await authService.createSessionToken(user);
|
||||
const lookupError = Object.assign(new Error('db down'), {
|
||||
code: 'dbBatchFailed',
|
||||
});
|
||||
const spy = vi
|
||||
.spyOn(server.stores.user, 'getByUuid')
|
||||
.mockRejectedValueOnce(lookupError);
|
||||
try {
|
||||
const probe = createAuthProbe({ authService });
|
||||
const { req } = await runProbe(
|
||||
probe,
|
||||
makeReq({ headers: { authorization: `Bearer ${token}` } }),
|
||||
);
|
||||
expect(req.actor).toBeUndefined();
|
||||
expect(req.tokenAuthFailed).toBeUndefined();
|
||||
expect(req.tokenAuthError).toBe(lookupError);
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -45,9 +45,9 @@ interface AuthProbeOptions {
|
||||
* and valid — attaches an `Actor` to `req.actor`.
|
||||
*
|
||||
* Key property: this middleware **never rejects**. Missing tokens, malformed
|
||||
* tokens, expired tokens, tokens pointing at deleted users, and lookups that
|
||||
* fail outright all result in `req.actor` being left undefined. Per-route gates
|
||||
* decide whether absence is acceptable.
|
||||
* tokens, expired tokens, tokens pointing at deleted users — all result in
|
||||
* `req.actor` being left undefined. Per-route gates decide whether absence is
|
||||
* acceptable.
|
||||
*
|
||||
* Token lookup order:
|
||||
*
|
||||
@@ -153,10 +153,8 @@ export const createAuthProbe = (opts: AuthProbeOptions): RequestHandler => {
|
||||
} else if (result.invalid) {
|
||||
req.tokenAuthFailed = true;
|
||||
}
|
||||
} catch (err) {
|
||||
// Not a verdict on the token: routes that require auth answer
|
||||
// with this error, routes that don't serve the request anonymously.
|
||||
req.tokenAuthError = err;
|
||||
} catch {
|
||||
req.tokenAuthFailed = true;
|
||||
}
|
||||
next();
|
||||
};
|
||||
@@ -226,7 +224,7 @@ const extractToken = (
|
||||
}
|
||||
}
|
||||
|
||||
// 5. Query string (used by e.g. asset URLs).
|
||||
// 5. Query string (used by e.g. QR login, asset URLs).
|
||||
const queryToken = (req.query as { auth_token?: unknown } | undefined)
|
||||
?.auth_token;
|
||||
if (typeof queryToken === 'string' && queryToken.length > 0) {
|
||||
|
||||
@@ -288,22 +288,6 @@ describe('requireAuthGate', () => {
|
||||
});
|
||||
expectHttpError(got, 401, 'reauth_required');
|
||||
});
|
||||
|
||||
it('passes on the error when the token could not be checked', () => {
|
||||
const lookupError = Object.assign(new Error('down'), {
|
||||
code: 'dbBatchFailed',
|
||||
});
|
||||
const got = runGate(requireAuthGate(), { tokenAuthError: lookupError });
|
||||
expect(got).toBe(lookupError);
|
||||
});
|
||||
|
||||
it('still answers token_auth_failed when a check found the token invalid', () => {
|
||||
const got = runGate(requireAuthGate(), {
|
||||
tokenAuthFailed: true,
|
||||
tokenAuthError: new Error('down'),
|
||||
});
|
||||
expectHttpError(got, 401, 'token_auth_failed');
|
||||
});
|
||||
});
|
||||
|
||||
// ── requireUserActorGate ────────────────────────────────────────────
|
||||
@@ -346,14 +330,6 @@ describe('requireUserActorGate', () => {
|
||||
expectHttpError(got, 401, 'token_missing');
|
||||
});
|
||||
|
||||
it('passes on the error when the token could not be checked', () => {
|
||||
const lookupError = new Error('down');
|
||||
const got = runGate(requireUserActorGate(), {
|
||||
tokenAuthError: lookupError,
|
||||
});
|
||||
expect(got).toBe(lookupError);
|
||||
});
|
||||
|
||||
it('rejects a FULL-ACCESS access token too — account routes stay closed', () => {
|
||||
// The account wall is actor-type based: even a full-access PAT (which
|
||||
// the resource wall lets through) is rejected here, so it can never
|
||||
|
||||
@@ -50,7 +50,7 @@ import { assertVerifiedEmail } from '../verifiedEmail';
|
||||
// Make sure the `Express.Request.actor` augmentation is in scope.
|
||||
import '../expressAugmentation';
|
||||
|
||||
const rejectAuth = (req: Request): unknown => {
|
||||
const rejectAuth = (req: Request): HttpError => {
|
||||
if (req.requiresReauth) {
|
||||
return new HttpError(401, 'Re-authentication required', {
|
||||
legacyCode: 'reauth_required',
|
||||
@@ -71,9 +71,6 @@ const rejectAuth = (req: Request): unknown => {
|
||||
legacyCode: 'token_auth_failed',
|
||||
});
|
||||
}
|
||||
// The token couldn't be checked, so a 401 would wrongly tell the client
|
||||
// to discard it; the error handler answers (503 for a database outage).
|
||||
if (req.tokenAuthError !== undefined) return req.tokenAuthError;
|
||||
return new HttpError(401, 'Missing authentication token', {
|
||||
legacyCode: 'token_missing',
|
||||
});
|
||||
|
||||
@@ -1,263 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Server } from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import { brotliCompressSync, deflateSync, gzipSync } from 'node:zlib';
|
||||
import express, {
|
||||
type ErrorRequestHandler,
|
||||
type RequestHandler,
|
||||
} from 'express';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { SLICE_BYTES } from '../../../util/slicedJson';
|
||||
import { createJsonBodyParser } from './jsonBody';
|
||||
|
||||
const LIMIT = 4 * 1024 * 1024;
|
||||
const TEXT_AS_JSON = 'text/plain;actually=json';
|
||||
|
||||
type Verify = NonNullable<Parameters<typeof express.json>[0]>['verify'];
|
||||
const captureRawBody: Verify = (req, _res, buf) => {
|
||||
(req as { rawBody?: Buffer }).rawBody = buf;
|
||||
};
|
||||
|
||||
/** An app that answers with what its parsers left on the request. */
|
||||
const listen = (parsers: RequestHandler[]): Promise<Server> => {
|
||||
const app = express();
|
||||
app.use(...parsers);
|
||||
app.post('/', (req, res) => {
|
||||
const rawBody = (req as { rawBody?: Buffer }).rawBody;
|
||||
res.json({
|
||||
body: req.body,
|
||||
hasBody: req.body !== undefined,
|
||||
rawBody: rawBody?.toString('base64'),
|
||||
});
|
||||
});
|
||||
const onError: ErrorRequestHandler = (err, _req, res, _next) => {
|
||||
res.status(err.statusCode ?? 500).json({ type: err.type });
|
||||
};
|
||||
app.use(onError);
|
||||
return new Promise((resolve) => {
|
||||
const server = app.listen(0, '127.0.0.1', () => resolve(server));
|
||||
});
|
||||
};
|
||||
|
||||
describe('createJsonBodyParser', () => {
|
||||
let sliced: Server;
|
||||
let framework: Server;
|
||||
|
||||
beforeAll(async () => {
|
||||
sliced = await listen([
|
||||
createJsonBodyParser({ limit: LIMIT, verify: captureRawBody }),
|
||||
]);
|
||||
// What the server ran before: one parser per content type.
|
||||
framework = await listen([
|
||||
express.json({ limit: LIMIT, verify: captureRawBody }),
|
||||
express.json({
|
||||
limit: LIMIT,
|
||||
type: (req) => req.headers['content-type'] === TEXT_AS_JSON,
|
||||
verify: captureRawBody,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
sliced?.close();
|
||||
framework?.close();
|
||||
});
|
||||
|
||||
const post = async (
|
||||
server: Server,
|
||||
body: string | Buffer | undefined,
|
||||
headers: Record<string, string>,
|
||||
) => {
|
||||
const { port } = server.address() as AddressInfo;
|
||||
const response = await fetch(`http://127.0.0.1:${port}/`, {
|
||||
method: 'POST',
|
||||
headers,
|
||||
body,
|
||||
});
|
||||
return { status: response.status, answer: await response.json() };
|
||||
};
|
||||
|
||||
const large = `${JSON.stringify({
|
||||
text: 'é😀 "quoted" \\ line\n'.repeat(SLICE_BYTES / 8),
|
||||
list: Array.from({ length: 20_000 }, (_, i) => ({
|
||||
i,
|
||||
on: i % 2 === 0,
|
||||
})),
|
||||
}).slice(0, -1)},"__proto__":{"nested":true}}`;
|
||||
|
||||
it.each<[string, string | Buffer | undefined, Record<string, string>]>([
|
||||
['an object', '{"a":1}', { 'Content-Type': 'application/json' }],
|
||||
['an array', '[1,2]', { 'Content-Type': 'application/json' }],
|
||||
[
|
||||
'a body sent as text',
|
||||
'{"interface":"x","auth_token":"t"}',
|
||||
{ 'Content-Type': TEXT_AS_JSON },
|
||||
],
|
||||
['a large body', large, { 'Content-Type': 'application/json' }],
|
||||
['a large body sent as text', large, { 'Content-Type': TEXT_AS_JSON }],
|
||||
[
|
||||
'a charset parameter',
|
||||
large,
|
||||
{ 'Content-Type': 'application/json; charset=UTF-8' },
|
||||
],
|
||||
[
|
||||
'a quoted charset parameter',
|
||||
'{"a":"é"}',
|
||||
{ 'Content-Type': 'application/json;charset="utf-8"' },
|
||||
],
|
||||
[
|
||||
'another parameter',
|
||||
'{"a":1}',
|
||||
{ 'Content-Type': 'application/json; profile=x' },
|
||||
],
|
||||
['an empty body', '', { 'Content-Type': 'application/json' }],
|
||||
['no body', undefined, { 'Content-Type': 'application/json' }],
|
||||
[
|
||||
'a byte order mark',
|
||||
`\ufeff${large}`,
|
||||
{ 'Content-Type': 'application/json' },
|
||||
],
|
||||
[
|
||||
'only a byte order mark',
|
||||
'\ufeff',
|
||||
{ 'Content-Type': 'application/json' },
|
||||
],
|
||||
['only whitespace', ' \n', { 'Content-Type': 'application/json' }],
|
||||
['a bare string', '"text"', { 'Content-Type': 'application/json' }],
|
||||
['a bare number', ' 12', { 'Content-Type': 'application/json' }],
|
||||
['null', 'null', { 'Content-Type': 'application/json' }],
|
||||
['malformed JSON', '{"a":', { 'Content-Type': 'application/json' }],
|
||||
[
|
||||
'malformed large JSON',
|
||||
`${large.slice(0, -1)},]`,
|
||||
{ 'Content-Type': 'application/json' },
|
||||
],
|
||||
[
|
||||
'a large bare string',
|
||||
JSON.stringify('x'.repeat(4 * SLICE_BYTES)),
|
||||
{ 'Content-Type': 'application/json' },
|
||||
],
|
||||
[
|
||||
'invalid UTF-8',
|
||||
Buffer.concat([
|
||||
Buffer.from('{"a":"'),
|
||||
Buffer.from([0xff, 0xe2, 0x82]),
|
||||
Buffer.from('"}'),
|
||||
]),
|
||||
{ 'Content-Type': 'application/json' },
|
||||
],
|
||||
[
|
||||
'a body over the limit',
|
||||
`["${'x'.repeat(LIMIT)}"]`,
|
||||
{ 'Content-Type': 'application/json' },
|
||||
],
|
||||
[
|
||||
'a gzipped body',
|
||||
gzipSync(large),
|
||||
{ 'Content-Type': 'application/json', 'Content-Encoding': 'gzip' },
|
||||
],
|
||||
[
|
||||
'a deflated body',
|
||||
deflateSync(large),
|
||||
{
|
||||
'Content-Type': 'application/json',
|
||||
'Content-Encoding': 'deflate',
|
||||
},
|
||||
],
|
||||
[
|
||||
'a brotli body',
|
||||
brotliCompressSync(large),
|
||||
{ 'Content-Type': 'application/json', 'Content-Encoding': 'br' },
|
||||
],
|
||||
[
|
||||
'a body that inflates past the limit',
|
||||
gzipSync(`["${'x'.repeat(LIMIT)}"]`),
|
||||
{ 'Content-Type': 'application/json', 'Content-Encoding': 'gzip' },
|
||||
],
|
||||
[
|
||||
'an unknown content encoding',
|
||||
'{"a":1}',
|
||||
{ 'Content-Type': 'application/json', 'Content-Encoding': 'lzw' },
|
||||
],
|
||||
['plain text', '{"a":1}', { 'Content-Type': 'text/plain' }],
|
||||
[
|
||||
'text with the marker and a charset',
|
||||
'{"a":1}',
|
||||
{ 'Content-Type': `${TEXT_AS_JSON}; charset=utf-8` },
|
||||
],
|
||||
[
|
||||
'a JSON-suffixed type',
|
||||
'{"a":1}',
|
||||
{ 'Content-Type': 'application/vnd.api+json' },
|
||||
],
|
||||
[
|
||||
'form data',
|
||||
'a=1',
|
||||
{ 'Content-Type': 'application/x-www-form-urlencoded' },
|
||||
],
|
||||
['no content type', '{"a":1}', { 'Content-Type': '' }],
|
||||
])('answers %s the way express.json does', async (_name, body, headers) => {
|
||||
const got = await post(sliced, body, headers);
|
||||
const want = await post(framework, body, headers);
|
||||
expect(got).toEqual(want);
|
||||
});
|
||||
|
||||
it('leaves a body in another charset unread', async () => {
|
||||
for (const charset of ['utf-16', 'utf8', 'latin1']) {
|
||||
const { status, answer } = await post(
|
||||
sliced,
|
||||
Buffer.from('{"a":1}', 'utf16le'),
|
||||
{ 'Content-Type': `application/json; charset=${charset}` },
|
||||
);
|
||||
expect(status).toBe(200);
|
||||
expect(answer.hasBody).toBe(false);
|
||||
}
|
||||
// The last charset named is the one that counts.
|
||||
const { answer } = await post(sliced, '{"a":1}', {
|
||||
'Content-Type': 'application/json; charset=utf-8; charset=utf-16',
|
||||
});
|
||||
expect(answer.hasBody).toBe(false);
|
||||
});
|
||||
|
||||
it('parses bodies that arrive together, each to its own result', async () => {
|
||||
const bodies = Array.from({ length: 6 }, (_, n) =>
|
||||
JSON.stringify({ n, pad: 'x'.repeat((n + 1) * SLICE_BYTES) }),
|
||||
);
|
||||
const answers = await Promise.all(
|
||||
bodies.map((body) =>
|
||||
post(sliced, body, { 'Content-Type': 'application/json' }),
|
||||
),
|
||||
);
|
||||
answers.forEach(({ status, answer }, n) => {
|
||||
expect(status).toBe(200);
|
||||
expect(answer.body).toEqual(JSON.parse(bodies[n]!));
|
||||
});
|
||||
// A failed parse hands its turn on like a finished one.
|
||||
const mixed = await Promise.all([
|
||||
post(sliced, `${bodies[5]!},`, {
|
||||
'Content-Type': 'application/json',
|
||||
}),
|
||||
post(sliced, bodies[4], { 'Content-Type': 'application/json' }),
|
||||
post(sliced, bodies[3], { 'Content-Type': 'application/json' }),
|
||||
]);
|
||||
expect(mixed.map(({ status }) => status)).toEqual([400, 200, 200]);
|
||||
});
|
||||
});
|
||||
@@ -1,127 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import express, { type Request, type RequestHandler } from 'express';
|
||||
import { parseJsonInSlices, SLICE_BYTES } from '../../../util/slicedJson';
|
||||
|
||||
/** What puter.js labels a JSON driver call, to keep it a simple CORS request. */
|
||||
const TEXT_AS_JSON = 'text/plain;actually=json';
|
||||
|
||||
const UTF8_CHARSET = /;\s*charset=("?)utf-8\1\s*(?:;|$)/i;
|
||||
|
||||
/** No charset, or exactly UTF-8. Anything else is left to `express.json`. */
|
||||
const declaresUtf8 = (contentType: string): boolean => {
|
||||
const mentions = contentType.match(/charset/gi)?.length ?? 0;
|
||||
return mentions === 0 || (mentions === 1 && UTF8_CHARSET.test(contentType));
|
||||
};
|
||||
|
||||
const isUtf8Json = (req: Request): boolean => {
|
||||
const contentType = req.headers['content-type'];
|
||||
if (contentType === TEXT_AS_JSON) return true;
|
||||
return (
|
||||
typeof contentType === 'string' &&
|
||||
Boolean(req.is('application/json')) &&
|
||||
declaresUtf8(contentType)
|
||||
);
|
||||
};
|
||||
|
||||
/** The error `express.json` raises, which the error handler answers with a 400. */
|
||||
const parseFailed = (message: string): Error =>
|
||||
Object.assign(new SyntaxError(message), {
|
||||
status: 400,
|
||||
statusCode: 400,
|
||||
expose: true,
|
||||
type: 'entity.parse.failed',
|
||||
});
|
||||
|
||||
const waiting: Array<() => void> = [];
|
||||
let parsing = false;
|
||||
|
||||
/**
|
||||
* Bodies too large for one `JSON.parse` call parse one at a time, so a single
|
||||
* tree is under construction however many arrive together.
|
||||
*/
|
||||
const parseInTurn = async (body: Buffer): Promise<unknown> => {
|
||||
if (body.length <= SLICE_BYTES) return parseJsonInSlices(body);
|
||||
if (parsing) await new Promise<void>((resolve) => waiting.push(resolve));
|
||||
parsing = true;
|
||||
try {
|
||||
return await parseJsonInSlices(body);
|
||||
} finally {
|
||||
// Handing over keeps `parsing` set, so a new arrival can't cut in.
|
||||
const next = waiting.shift();
|
||||
if (next) next();
|
||||
else parsing = false;
|
||||
}
|
||||
};
|
||||
|
||||
interface JsonBodyOptions {
|
||||
limit: number;
|
||||
verify?: NonNullable<Parameters<typeof express.raw>[0]>['verify'];
|
||||
}
|
||||
|
||||
/**
|
||||
* `express.json` for UTF-8 bodies, with the parse done in slices so a large
|
||||
* body can't hold the event loop. Reading, limits, decompression and `verify`
|
||||
* are the framework's; the result and the errors match `express.json`.
|
||||
*/
|
||||
export const createJsonBodyParser = ({
|
||||
limit,
|
||||
verify,
|
||||
}: JsonBodyOptions): RequestHandler => {
|
||||
const read = express.raw({
|
||||
limit,
|
||||
verify,
|
||||
type: (req) => isUtf8Json(req as Request),
|
||||
});
|
||||
|
||||
return (req, res, next) => {
|
||||
read(req, res, async (err?: unknown) => {
|
||||
// Not a Buffer: the reader skipped this request.
|
||||
if (err || !Buffer.isBuffer(req.body)) {
|
||||
next(err);
|
||||
return;
|
||||
}
|
||||
let body: Buffer = req.body;
|
||||
req.body = undefined;
|
||||
// The framework's decoder drops a byte order mark.
|
||||
if (body[0] === 0xef && body[1] === 0xbb && body[2] === 0xbf) {
|
||||
body = body.subarray(3);
|
||||
}
|
||||
if (body.length === 0) {
|
||||
req.body = {};
|
||||
next();
|
||||
return;
|
||||
}
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = await parseInTurn(body);
|
||||
} catch (e) {
|
||||
next(parseFailed((e as Error).message));
|
||||
return;
|
||||
}
|
||||
if (typeof parsed !== 'object' || parsed === null) {
|
||||
next(parseFailed('JSON body must be an object or an array'));
|
||||
return;
|
||||
}
|
||||
req.body = parsed;
|
||||
next();
|
||||
});
|
||||
};
|
||||
};
|
||||
@@ -1,184 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { request, type Server } from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import { setImmediate as nextTurn } from 'node:timers/promises';
|
||||
import express, {
|
||||
type ErrorRequestHandler,
|
||||
type RequestHandler,
|
||||
} from 'express';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { createJsonBodyParser } from './jsonBody';
|
||||
import {
|
||||
createLargeJsonBodyParser,
|
||||
DEFAULT_JSON_LIMIT_BYTES,
|
||||
} from './largeJsonBody';
|
||||
|
||||
const MiB = 1024 * 1024;
|
||||
const TOKEN = 'valid-token';
|
||||
const PAD = 'x'.repeat(DEFAULT_JSON_LIMIT_BYTES);
|
||||
|
||||
/** Stands in for the auth probe: one token is valid, wherever it is sent. */
|
||||
const authProbe: RequestHandler = (req, _res, next) => {
|
||||
const token =
|
||||
(req.body as { auth_token?: string } | undefined)?.auth_token ??
|
||||
req.headers.authorization;
|
||||
if (!req.actor && token === TOKEN) {
|
||||
req.actor = { user: { uuid: 'u1' } } as typeof req.actor;
|
||||
}
|
||||
next();
|
||||
};
|
||||
|
||||
describe('a JSON body over the default limit', () => {
|
||||
let server: Server;
|
||||
|
||||
beforeAll(async () => {
|
||||
const app = express();
|
||||
app.post(
|
||||
'/call',
|
||||
createLargeJsonBodyParser({
|
||||
authProbe,
|
||||
limitBytes: 128 * MiB,
|
||||
budgetBytes: 256 * MiB,
|
||||
perUserBytes: 128 * MiB,
|
||||
}),
|
||||
);
|
||||
app.use(createJsonBodyParser({ limit: DEFAULT_JSON_LIMIT_BYTES }));
|
||||
app.use(authProbe);
|
||||
app.post('/call', (req, res) => {
|
||||
const body = req.body as { auth_token?: string; pad: string };
|
||||
res.json({
|
||||
user: req.actor?.user.uuid ?? null,
|
||||
token: body.auth_token ?? null,
|
||||
padLength: body.pad.length,
|
||||
});
|
||||
});
|
||||
const onError: ErrorRequestHandler = (err, _req, res, _next) => {
|
||||
res.status(err.statusCode ?? 500).json({});
|
||||
};
|
||||
app.use(onError);
|
||||
await new Promise<void>((resolve) => {
|
||||
server = app.listen(0, '127.0.0.1', resolve);
|
||||
});
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
server?.close();
|
||||
});
|
||||
|
||||
/** Sends `chunks` one write at a time, so the server sees them apart. */
|
||||
const post = (chunks: string[], headers: Record<string, string> = {}) =>
|
||||
new Promise<{ status: number; answer: Record<string, unknown> }>(
|
||||
(resolve, reject) => {
|
||||
const { port } = server.address() as AddressInfo;
|
||||
const req = request(
|
||||
{
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
path: '/call',
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'text/plain;actually=json',
|
||||
'Content-Length': chunks.reduce(
|
||||
(total, chunk) =>
|
||||
total + Buffer.byteLength(chunk),
|
||||
0,
|
||||
),
|
||||
...headers,
|
||||
},
|
||||
},
|
||||
(res) => {
|
||||
const parts: Buffer[] = [];
|
||||
res.on('data', (part: Buffer) => parts.push(part));
|
||||
res.on('end', () =>
|
||||
resolve({
|
||||
status: res.statusCode!,
|
||||
answer: JSON.parse(
|
||||
Buffer.concat(parts).toString(),
|
||||
),
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
req.on('error', reject);
|
||||
void (async () => {
|
||||
for (const chunk of chunks) {
|
||||
req.write(chunk);
|
||||
await nextTurn();
|
||||
}
|
||||
req.end();
|
||||
})();
|
||||
},
|
||||
);
|
||||
|
||||
it('is accepted when it opens with a valid auth_token', async () => {
|
||||
const { status, answer } = await post([
|
||||
`{"auth_token":"${TOKEN}","pad":"${PAD}"}`,
|
||||
]);
|
||||
expect(status).toBe(200);
|
||||
expect(answer).toEqual({
|
||||
user: 'u1',
|
||||
token: TOKEN,
|
||||
padLength: PAD.length,
|
||||
});
|
||||
}, 60_000);
|
||||
|
||||
it('is accepted when the token arrives in pieces, with spacing', async () => {
|
||||
const opening = ` {\n "auth_token" : "${TOKEN}" ,`;
|
||||
const { status, answer } = await post([...opening, `"pad":"${PAD}"}`]);
|
||||
expect(status).toBe(200);
|
||||
expect(answer.user).toBe('u1');
|
||||
}, 60_000);
|
||||
|
||||
it('is accepted with a header credential, as before', async () => {
|
||||
const { status, answer } = await post([`{"pad":"${PAD}"}`], {
|
||||
Authorization: TOKEN,
|
||||
});
|
||||
expect(status).toBe(200);
|
||||
expect(answer).toEqual({
|
||||
user: 'u1',
|
||||
token: null,
|
||||
padLength: PAD.length,
|
||||
});
|
||||
}, 60_000);
|
||||
|
||||
it.each([
|
||||
[
|
||||
'its token comes after other fields',
|
||||
`{"pad":"${PAD}","auth_token":"${TOKEN}"}`,
|
||||
],
|
||||
['its token is not valid', `{"auth_token":"expired","pad":"${PAD}"}`],
|
||||
['it has no token', `{"pad":"${PAD}"}`],
|
||||
[
|
||||
'a later field replaces the token it opens with',
|
||||
`{"auth_token":"${TOKEN}","pad":"${PAD}","auth_token":"other"}`,
|
||||
],
|
||||
[
|
||||
'the token it opens with is nested',
|
||||
`{"args":{"auth_token":"${TOKEN}"},"pad":"${PAD}"}`,
|
||||
],
|
||||
])(
|
||||
'is refused with 413 when %s',
|
||||
async (_name, body) => {
|
||||
expect((await post([body])).status).toBe(413);
|
||||
},
|
||||
60_000,
|
||||
);
|
||||
});
|
||||
@@ -18,7 +18,6 @@
|
||||
*/
|
||||
|
||||
import { EventEmitter } from 'node:events';
|
||||
import { Readable } from 'node:stream';
|
||||
import type { NextFunction, Request, RequestHandler, Response } from 'express';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { isHttpError } from '../HttpError';
|
||||
@@ -36,7 +35,7 @@ const makeReq = (
|
||||
user: string | null = 'u1',
|
||||
body?: unknown,
|
||||
) =>
|
||||
Object.assign(Readable.from([]), {
|
||||
({
|
||||
complete: true,
|
||||
body,
|
||||
headers: {
|
||||
|
||||
@@ -17,54 +17,13 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, RequestHandler, Response } from 'express';
|
||||
import express, { type Request, type RequestHandler } from 'express';
|
||||
import { actorUid } from '../../actor';
|
||||
import { HttpError } from '../HttpError';
|
||||
import { createJsonBodyParser } from './jsonBody';
|
||||
|
||||
/** The global JSON parser's limit, which applies before auth. */
|
||||
export const DEFAULT_JSON_LIMIT_BYTES = 50 * 1024 * 1024;
|
||||
|
||||
/** How much of a body is read to find a token it opens with. */
|
||||
const LEADING_TOKEN_BYTES = 8 * 1024;
|
||||
const LEADING_TOKEN =
|
||||
/^[ \t\n\r]*\{[ \t\n\r]*"auth_token"[ \t\n\r]*:[ \t\n\r]*"([^"\\]+)"/;
|
||||
|
||||
/**
|
||||
* The first `bytes` of the body, put back for the parser to read. Empty when
|
||||
* the body ends or fails before that many arrive.
|
||||
*/
|
||||
const peekBody = (req: Request, bytes: number): Promise<Buffer> =>
|
||||
new Promise((resolve) => {
|
||||
const settle = (head: Buffer | null) => {
|
||||
req.off('readable', onReadable);
|
||||
req.off('end', onGone);
|
||||
req.off('close', onGone);
|
||||
req.off('error', onGone);
|
||||
if (head) req.unshift(head);
|
||||
resolve(head ?? Buffer.alloc(0));
|
||||
};
|
||||
const onReadable = () => {
|
||||
// Null until `bytes` are buffered.
|
||||
const head = req.read(bytes) as Buffer | null;
|
||||
if (head) settle(head);
|
||||
};
|
||||
const onGone = () => settle(null);
|
||||
req.on('readable', onReadable);
|
||||
req.on('end', onGone);
|
||||
req.on('close', onGone);
|
||||
req.on('error', onGone);
|
||||
});
|
||||
|
||||
/**
|
||||
* The `auth_token` a JSON body opens with, which puter.js sends first so it can
|
||||
* be read ahead of the body.
|
||||
*/
|
||||
const leadingBodyToken = async (req: Request): Promise<string | undefined> => {
|
||||
const head = await peekBody(req, LEADING_TOKEN_BYTES);
|
||||
return LEADING_TOKEN.exec(head.toString('latin1'))?.[1];
|
||||
};
|
||||
|
||||
interface LargeJsonBodyOptions {
|
||||
/** The global auth probe; it skips requests that already carry an actor. */
|
||||
authProbe: RequestHandler;
|
||||
@@ -79,9 +38,9 @@ interface LargeJsonBodyOptions {
|
||||
|
||||
/**
|
||||
* Parses JSON bodies over the default limit, up to `limitBytes`, once the
|
||||
* caller's credential resolves: a header, a cookie, or an `auth_token` the body
|
||||
* opens with. Unauthenticated requests and requests without a Content-Length
|
||||
* fall through to the global parser and its limit.
|
||||
* caller's header or cookie credential resolves. Unauthenticated requests and
|
||||
* requests without a Content-Length fall through to the global parser and its
|
||||
* limit.
|
||||
*
|
||||
* The parsed body lives as long as the request, so a body's bytes count against
|
||||
* both budgets until its response closes. Over the user's budget gets a 429,
|
||||
@@ -94,91 +53,82 @@ export const createLargeJsonBodyParser = ({
|
||||
perUserBytes,
|
||||
accepts,
|
||||
}: LargeJsonBodyOptions): RequestHandler => {
|
||||
const json = createJsonBodyParser({ limit: limitBytes });
|
||||
const json = express.json({ limit: limitBytes });
|
||||
const textAsJson = express.json({
|
||||
limit: limitBytes,
|
||||
type: (req) =>
|
||||
req.headers['content-type'] === 'text/plain;actually=json',
|
||||
});
|
||||
let inFlightBytes = 0;
|
||||
const inFlightByUser = new Map<string, number>();
|
||||
|
||||
const probe = (req: Request, res: Response) =>
|
||||
new Promise<void>((resolve) => {
|
||||
void authProbe(req, res, () => resolve());
|
||||
});
|
||||
|
||||
return async (req, res, next) => {
|
||||
return (req, res, next) => {
|
||||
const declared = Number(req.headers['content-length']);
|
||||
if (!(declared > DEFAULT_JSON_LIMIT_BYTES && declared <= limitBytes)) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
await probe(req, res);
|
||||
let bodyToken: string | undefined;
|
||||
if (!req.actor && !res.closed) {
|
||||
bodyToken = await leadingBodyToken(req);
|
||||
if (bodyToken) {
|
||||
// Where the probe looks for a body token.
|
||||
req.body = { auth_token: bodyToken };
|
||||
await probe(req, res);
|
||||
req.body = undefined;
|
||||
}
|
||||
}
|
||||
// `res.closed`: the client left during the probe, so `close` has
|
||||
// already fired and would never release a charge.
|
||||
if (!req.actor || res.closed) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
const user = req.actor.user.uuid ?? actorUid(req.actor);
|
||||
const userBytes = inFlightByUser.get(user) ?? 0;
|
||||
if (userBytes + declared > perUserBytes) {
|
||||
res.setHeader('Retry-After', '5');
|
||||
next(
|
||||
new HttpError(
|
||||
429,
|
||||
'Too many large requests in progress for this account',
|
||||
{ legacyCode: 'too_many_requests' },
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (inFlightBytes + declared > budgetBytes) {
|
||||
res.setHeader('Retry-After', '5');
|
||||
next(
|
||||
new HttpError(503, 'Too many large requests in progress', {
|
||||
code: 'large_body_capacity',
|
||||
// Load shedding, not a fault: record it, don't page.
|
||||
alarmSeverity: 'info',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
inFlightBytes += declared;
|
||||
inFlightByUser.set(user, userBytes + declared);
|
||||
res.once('close', () => {
|
||||
inFlightBytes -= declared;
|
||||
const left = (inFlightByUser.get(user) ?? 0) - declared;
|
||||
if (left > 0) inFlightByUser.set(user, left);
|
||||
else inFlightByUser.delete(user);
|
||||
});
|
||||
json(req, res, (err?: unknown) => {
|
||||
if (err) {
|
||||
next(err);
|
||||
void authProbe(req, res, () => {
|
||||
// `res.closed`: the client left during the probe, so `close` has
|
||||
// already fired and would never release a charge.
|
||||
if (!req.actor || res.closed) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
// The caller was resolved from the token the body opens with,
|
||||
// so the parsed body has to name the same one.
|
||||
const tokenMoved =
|
||||
bodyToken !== undefined &&
|
||||
(req.body as { auth_token?: unknown } | undefined)
|
||||
?.auth_token !== bodyToken;
|
||||
if (tokenMoved || (accepts && !accepts(req))) {
|
||||
// The same answer the default limit gives.
|
||||
const user = req.actor.user.uuid ?? actorUid(req.actor);
|
||||
const userBytes = inFlightByUser.get(user) ?? 0;
|
||||
if (userBytes + declared > perUserBytes) {
|
||||
res.setHeader('Retry-After', '5');
|
||||
next(
|
||||
new HttpError(413, 'request entity too large', {
|
||||
legacyCode: 'client_bad_request',
|
||||
new HttpError(
|
||||
429,
|
||||
'Too many large requests in progress for this account',
|
||||
{ legacyCode: 'too_many_requests' },
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (inFlightBytes + declared > budgetBytes) {
|
||||
res.setHeader('Retry-After', '5');
|
||||
next(
|
||||
new HttpError(503, 'Too many large requests in progress', {
|
||||
code: 'large_body_capacity',
|
||||
// Load shedding, not a fault: record it, don't page.
|
||||
alarmSeverity: 'info',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
inFlightBytes += declared;
|
||||
inFlightByUser.set(user, userBytes + declared);
|
||||
res.once('close', () => {
|
||||
inFlightBytes -= declared;
|
||||
const left = (inFlightByUser.get(user) ?? 0) - declared;
|
||||
if (left > 0) inFlightByUser.set(user, left);
|
||||
else inFlightByUser.delete(user);
|
||||
});
|
||||
json(req, res, (err?: unknown) => {
|
||||
if (err) {
|
||||
next(err);
|
||||
return;
|
||||
}
|
||||
textAsJson(req, res, (err?: unknown) => {
|
||||
if (err) {
|
||||
next(err);
|
||||
return;
|
||||
}
|
||||
if (accepts && !accepts(req)) {
|
||||
// The same answer the default limit gives.
|
||||
next(
|
||||
new HttpError(413, 'request entity too large', {
|
||||
legacyCode: 'client_bad_request',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
});
|
||||
});
|
||||
});
|
||||
};
|
||||
};
|
||||
@@ -18,7 +18,7 @@
|
||||
*/
|
||||
|
||||
import type { Request } from 'express';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { makeActor } from '../../actor';
|
||||
import {
|
||||
@@ -1132,150 +1132,14 @@ describe('resolvePublicHostedIdentity', () => {
|
||||
const reqOf = (init: {
|
||||
cookies?: Record<string, string>;
|
||||
actor?: unknown;
|
||||
headers?: Record<string, string>;
|
||||
query?: Record<string, string>;
|
||||
}): Request =>
|
||||
({
|
||||
cookies: init.cookies ?? {},
|
||||
actor: init.actor,
|
||||
headers: init.headers ?? {},
|
||||
query: init.query ?? {},
|
||||
headers: {},
|
||||
query: {},
|
||||
}) as unknown as Request;
|
||||
|
||||
describe('with the launch token in the Referer', () => {
|
||||
const host = 'launch.site.puter.localhost';
|
||||
const cookieName = HOSTED_ASSET_COOKIES.public.name;
|
||||
|
||||
const launch = async () => {
|
||||
const user = await makeUser();
|
||||
const sub = `launch-${uuidv4().slice(0, 8)}`;
|
||||
const app = await server.stores.app.create(
|
||||
{ name: sub, title: sub, index_url: `http://${host}/` },
|
||||
{ ownerUserId: user.id },
|
||||
);
|
||||
const token = await authService.getUserAppToken(
|
||||
{ user } as unknown as Parameters<
|
||||
typeof authService.getUserAppToken
|
||||
>[0],
|
||||
app.uid,
|
||||
);
|
||||
const actor = (await authService.authenticateFromToken(token))!;
|
||||
const cookie = await authService.createHostedAssetToken('public', {
|
||||
appUid: app.uid,
|
||||
userUid: user.uuid,
|
||||
sessionUuid: actor.session?.uid,
|
||||
subdomain: 'launch',
|
||||
host,
|
||||
});
|
||||
return { user, appUid: app.uid, token, cookie, actor };
|
||||
};
|
||||
|
||||
const resolve = (
|
||||
appUid: string,
|
||||
init: Parameters<typeof reqOf>[0],
|
||||
expectedHost = host,
|
||||
) =>
|
||||
resolvePublicHostedIdentity({
|
||||
req: reqOf(init),
|
||||
authService,
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
expectedAppUid: appUid,
|
||||
expectedSubdomain: 'launch',
|
||||
expectedHost,
|
||||
});
|
||||
|
||||
const refererFor = (token: string) =>
|
||||
`http://${host}/?puter.auth.token=${encodeURIComponent(token)}`;
|
||||
|
||||
it('takes the cookie when the token names its user and session', async () => {
|
||||
const { user, appUid, token, cookie } = await launch();
|
||||
const authenticate = vi.spyOn(authService, 'authenticate');
|
||||
try {
|
||||
const out = await resolve(appUid, {
|
||||
cookies: { [cookieName]: cookie },
|
||||
headers: { referer: refererFor(token) },
|
||||
});
|
||||
expect(out).toMatchObject({
|
||||
source: 'public-cookie',
|
||||
userUid: user.uuid,
|
||||
hasValidPublicCookie: true,
|
||||
});
|
||||
expect(authenticate).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
authenticate.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('authenticates a query token even when it names the cookie identity', async () => {
|
||||
const { user, appUid, token, cookie } = await launch();
|
||||
const out = await resolve(appUid, {
|
||||
cookies: { [cookieName]: cookie },
|
||||
query: { 'puter.auth.token': token },
|
||||
});
|
||||
expect(out.source).toBe('query');
|
||||
expect(out.userUid).toBe(user.uuid);
|
||||
expect(
|
||||
(out as { hasValidPublicCookie?: boolean })
|
||||
.hasValidPublicCookie,
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('authenticates the token when the cookie was minted for another host', async () => {
|
||||
const { user, appUid, token, cookie } = await launch();
|
||||
const out = await resolve(
|
||||
appUid,
|
||||
{
|
||||
cookies: { [cookieName]: cookie },
|
||||
headers: { referer: refererFor(token) },
|
||||
},
|
||||
'other.site.puter.localhost',
|
||||
);
|
||||
expect(out.source).toBe('referrer');
|
||||
expect(out.userUid).toBe(user.uuid);
|
||||
});
|
||||
|
||||
it('authenticates a token that names another app under the same session', async () => {
|
||||
const { user, appUid, cookie, actor } = await launch();
|
||||
const otherAppToken = server.services.token.sign('auth', {
|
||||
type: 'app-under-user',
|
||||
version: '2',
|
||||
user_uid: user.uuid,
|
||||
app_uid: `app-${uuidv4()}`,
|
||||
session_uid: actor.session!.uid,
|
||||
});
|
||||
const authenticate = vi.spyOn(authService, 'authenticate');
|
||||
try {
|
||||
const out = await resolve(appUid, {
|
||||
cookies: { [cookieName]: cookie },
|
||||
headers: { referer: refererFor(otherAppToken) },
|
||||
});
|
||||
expect(authenticate).toHaveBeenCalledTimes(1);
|
||||
expect(out.source).toBe('public-cookie');
|
||||
} finally {
|
||||
authenticate.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('never resolves a scoped access token, with or without the cookie', async () => {
|
||||
const { user, appUid, cookie } = await launch();
|
||||
const scoped = await authService.createAccessToken(
|
||||
makeActor({
|
||||
user: { id: user.id, uuid: user.uuid, username: user.username },
|
||||
}),
|
||||
[[`user:${user.uuid}:email:read`]],
|
||||
);
|
||||
const withCookie = await resolve(appUid, {
|
||||
cookies: { [cookieName]: cookie },
|
||||
headers: { referer: refererFor(scoped) },
|
||||
});
|
||||
expect(withCookie.source).toBe('public-cookie');
|
||||
const withoutCookie = await resolve(appUid, {
|
||||
headers: { referer: refererFor(scoped) },
|
||||
});
|
||||
expect(withoutCookie.source).toBe('none');
|
||||
});
|
||||
});
|
||||
|
||||
it('returns the cookie identity when present and valid', async () => {
|
||||
const user = await makeUser();
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
|
||||
@@ -86,10 +86,6 @@ export interface PrivateIdentity {
|
||||
hasValidPrivateCookie?: boolean;
|
||||
}
|
||||
|
||||
type PublicCookieClaims = Awaited<
|
||||
ReturnType<AuthService['verifyHostedAssetToken']>
|
||||
>;
|
||||
|
||||
interface SubdomainLike {
|
||||
user_id?: number | null;
|
||||
}
|
||||
@@ -537,63 +533,8 @@ export async function resolvePublicHostedIdentity(opts: {
|
||||
}
|
||||
}
|
||||
|
||||
const { name: publicCookieNameV2, legacyName: publicCookieNameLegacy } =
|
||||
HOSTED_ASSET_COOKIES.public;
|
||||
const publicCookieToken =
|
||||
(typeof cookies?.[publicCookieNameV2] === 'string'
|
||||
? cookies[publicCookieNameV2]
|
||||
: null) ??
|
||||
(typeof cookies?.[publicCookieNameLegacy] === 'string'
|
||||
? cookies[publicCookieNameLegacy]
|
||||
: null);
|
||||
let publicCookieClaims: Promise<PublicCookieClaims | null> | undefined;
|
||||
const verifyPublicCookie = () =>
|
||||
(publicCookieClaims ??= (async () => {
|
||||
if (!publicCookieToken) return null;
|
||||
try {
|
||||
return await authService.verifyHostedAssetToken(
|
||||
'public',
|
||||
publicCookieToken,
|
||||
{
|
||||
appUid: expectedAppUid,
|
||||
subdomain: expectedSubdomain,
|
||||
host: expectedHost,
|
||||
},
|
||||
);
|
||||
} catch {
|
||||
// v1-signed cookies land here and get re-minted.
|
||||
return null;
|
||||
}
|
||||
})());
|
||||
const publicCookieIdentity = (claims: PublicCookieClaims) => ({
|
||||
source: 'public-cookie' as const,
|
||||
userUid: claims.userUid,
|
||||
sessionUuid: claims.sessionUuid,
|
||||
hasValidPublicCookie: true,
|
||||
});
|
||||
|
||||
const bootstrap = getBootstrapToken(req);
|
||||
if (bootstrap) {
|
||||
// A launched app's own assets carry its launch token in the Referer.
|
||||
// When that token names the cookie's user and session, the cookie
|
||||
// already stands for it and nothing needs re-authenticating or minting.
|
||||
if (
|
||||
bootstrap.source === 'referrer' &&
|
||||
publicCookieToken &&
|
||||
expectedAppUid
|
||||
) {
|
||||
const named = authService.signedSessionClaims(bootstrap.token);
|
||||
if (named && (!named.appUid || named.appUid === expectedAppUid)) {
|
||||
const claims = await verifyPublicCookie();
|
||||
if (
|
||||
claims &&
|
||||
claims.userUid === named.userUid &&
|
||||
claims.sessionUuid === named.sessionUuid
|
||||
) {
|
||||
return publicCookieIdentity(claims);
|
||||
}
|
||||
}
|
||||
}
|
||||
try {
|
||||
const actor = await authService.authenticateFromToken(
|
||||
bootstrap.token,
|
||||
@@ -613,8 +554,36 @@ export async function resolvePublicHostedIdentity(opts: {
|
||||
}
|
||||
}
|
||||
|
||||
const claims = await verifyPublicCookie();
|
||||
if (claims) return publicCookieIdentity(claims);
|
||||
const { name: publicCookieNameV2, legacyName: publicCookieNameLegacy } =
|
||||
HOSTED_ASSET_COOKIES.public;
|
||||
const publicCookieToken =
|
||||
(typeof cookies?.[publicCookieNameV2] === 'string'
|
||||
? cookies[publicCookieNameV2]
|
||||
: null) ??
|
||||
(typeof cookies?.[publicCookieNameLegacy] === 'string'
|
||||
? cookies[publicCookieNameLegacy]
|
||||
: null);
|
||||
if (publicCookieToken) {
|
||||
try {
|
||||
const claims = await authService.verifyHostedAssetToken(
|
||||
'public',
|
||||
publicCookieToken,
|
||||
{
|
||||
appUid: expectedAppUid,
|
||||
subdomain: expectedSubdomain,
|
||||
host: expectedHost,
|
||||
},
|
||||
);
|
||||
return {
|
||||
source: 'public-cookie',
|
||||
userUid: claims.userUid,
|
||||
sessionUuid: claims.sessionUuid,
|
||||
hasValidPublicCookie: true,
|
||||
};
|
||||
} catch {
|
||||
/* fall through — v1-signed cookies land here and get re-minted */
|
||||
}
|
||||
}
|
||||
|
||||
return { source: 'none' };
|
||||
}
|
||||
|
||||
@@ -2763,321 +2763,6 @@ describe('public hosted asset billing identity', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('public hosted assets requested with the launch token in the Referer', () => {
|
||||
const cookieName = HOSTED_ASSET_COOKIES.public.name;
|
||||
|
||||
const makeLaunchedApp = async () => {
|
||||
const owner = await makeUserWithHome();
|
||||
const home = (await server.stores.fsEntry.getEntryByPath(
|
||||
`/${owner.username}`,
|
||||
))!;
|
||||
const sub = `ref-${uuidv4().slice(0, 8)}`;
|
||||
const host = `${sub}.site.puter.localhost`;
|
||||
await server.stores.subdomain.create({
|
||||
userId: owner.id,
|
||||
subdomain: sub,
|
||||
rootDirId: home.id,
|
||||
});
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`/${owner.username}/index.html`,
|
||||
Buffer.from('<html></html>'),
|
||||
);
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`/${owner.username}/app.css`,
|
||||
Buffer.from('body{}'),
|
||||
);
|
||||
const app = await server.stores.app.create(
|
||||
{ name: sub, title: sub, index_url: `http://${host}/` },
|
||||
{ ownerUserId: owner.id },
|
||||
);
|
||||
return { owner, sub, host, appUid: app.uid };
|
||||
};
|
||||
|
||||
const cookieFor = async (
|
||||
token: string,
|
||||
site: { sub: string; host: string; appUid: string },
|
||||
) => {
|
||||
const actor = (await server.services.auth.authenticateFromToken(
|
||||
token,
|
||||
))!;
|
||||
return server.services.auth.createHostedAssetToken('public', {
|
||||
appUid: site.appUid,
|
||||
userUid: actor.user.uuid,
|
||||
sessionUuid: actor.session?.uid,
|
||||
subdomain: site.sub,
|
||||
host: site.host,
|
||||
});
|
||||
};
|
||||
|
||||
const refererFor = (host: string, token: string) =>
|
||||
`http://${host}/?puter.app_instance_id=x&puter.auth.token=${encodeURIComponent(token)}`;
|
||||
|
||||
const serve = async (
|
||||
host: string,
|
||||
init: {
|
||||
path?: string;
|
||||
cookies?: Record<string, string>;
|
||||
headers?: Record<string, string>;
|
||||
query?: Record<string, string>;
|
||||
} = {},
|
||||
) => {
|
||||
const req = makeReq({ hostname: host, path: '/app.css', ...init });
|
||||
const { res, out } = makeRes();
|
||||
await buildMiddleware()(req, res, vi.fn());
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
expect(out.statusCode).toBe(200);
|
||||
return { req, out };
|
||||
};
|
||||
|
||||
const claimsOf = (cookie: string, appUid: string, host: string) =>
|
||||
server.services.auth.verifyHostedAssetToken('public', cookie, {
|
||||
appUid,
|
||||
host,
|
||||
});
|
||||
|
||||
const REDIS_METHODS = [
|
||||
'get',
|
||||
'mget',
|
||||
'exists',
|
||||
'hget',
|
||||
'hgetall',
|
||||
'smembers',
|
||||
'set',
|
||||
'del',
|
||||
'expire',
|
||||
'incr',
|
||||
'incrby',
|
||||
'hset',
|
||||
'sadd',
|
||||
'pipeline',
|
||||
'multi',
|
||||
];
|
||||
|
||||
/** Redis calls, JWT signs and full authentications made by `fn`. */
|
||||
const measure = async <T>(fn: () => Promise<T>) => {
|
||||
const redis = server.clients.redis as unknown as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
const redisSpies = REDIS_METHODS.filter(
|
||||
(m) => typeof redis[m] === 'function',
|
||||
).map((m) => [m, vi.spyOn(redis as never, m as never)] as const);
|
||||
const sign = vi.spyOn(server.services.token, 'sign');
|
||||
const authenticate = vi.spyOn(server.services.auth, 'authenticate');
|
||||
try {
|
||||
const result = await fn();
|
||||
const redisCalls: Record<string, number> = {};
|
||||
for (const [m, spy] of redisSpies) {
|
||||
const n = (spy as unknown as { mock: { calls: unknown[] } })
|
||||
.mock.calls.length;
|
||||
if (n) redisCalls[m] = n;
|
||||
}
|
||||
return {
|
||||
result,
|
||||
redisCalls,
|
||||
signs: sign.mock.calls.length,
|
||||
authentications: authenticate.mock.calls.length,
|
||||
};
|
||||
} finally {
|
||||
for (const [, spy] of redisSpies) spy.mockRestore();
|
||||
sign.mockRestore();
|
||||
authenticate.mockRestore();
|
||||
}
|
||||
};
|
||||
|
||||
it('costs the same as the cookie alone when the token names the cookie user and session', async () => {
|
||||
const site = await makeLaunchedApp();
|
||||
const visitor = await makeUser();
|
||||
const token = await server.services.auth.getUserAppToken(
|
||||
actorFor(visitor),
|
||||
site.appUid,
|
||||
);
|
||||
const cookies = { [cookieName]: await cookieFor(token, site) };
|
||||
const headers = { referer: refererFor(site.host, token) };
|
||||
|
||||
// Warm the caches both paths read.
|
||||
await serve(site.host, { cookies });
|
||||
await serve(site.host, { cookies, headers });
|
||||
|
||||
const cookieOnly = await measure(() => serve(site.host, { cookies }));
|
||||
const withReferer = await measure(() =>
|
||||
serve(site.host, { cookies, headers }),
|
||||
);
|
||||
|
||||
expect(withReferer.result.req.egressActor?.user?.uuid).toBe(
|
||||
visitor.uuid,
|
||||
);
|
||||
expect(withReferer.result.out.cookies).toEqual({});
|
||||
expect(withReferer.signs).toBe(0);
|
||||
expect(withReferer.authentications).toBe(0);
|
||||
expect(withReferer.redisCalls).toEqual(cookieOnly.redisCalls);
|
||||
expect(cookieOnly.signs).toBe(0);
|
||||
});
|
||||
|
||||
it('bills the token account when the launch could not replace an older cookie', async () => {
|
||||
const site = await makeLaunchedApp();
|
||||
const previous = await makeUser();
|
||||
const current = await makeUser();
|
||||
const previousToken = await server.services.auth.getUserAppToken(
|
||||
actorFor(previous),
|
||||
site.appUid,
|
||||
);
|
||||
const currentToken = await server.services.auth.getUserAppToken(
|
||||
actorFor(current),
|
||||
site.appUid,
|
||||
);
|
||||
const oldCookies = {
|
||||
[cookieName]: await cookieFor(previousToken, site),
|
||||
};
|
||||
|
||||
const mint = vi
|
||||
.spyOn(server.services.auth, 'createHostedAssetToken')
|
||||
.mockRejectedValueOnce(new Error('mint failed'));
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
let launch;
|
||||
try {
|
||||
launch = await serve(site.host, {
|
||||
path: '/index.html',
|
||||
cookies: oldCookies,
|
||||
query: { 'puter.auth.token': currentToken },
|
||||
});
|
||||
} finally {
|
||||
mint.mockRestore();
|
||||
warn.mockRestore();
|
||||
}
|
||||
expect(launch.out.cookies).toEqual({});
|
||||
expect(launch.req.egressActor?.user?.uuid).toBe(current.uuid);
|
||||
|
||||
const asset = await serve(site.host, {
|
||||
cookies: oldCookies,
|
||||
headers: { referer: refererFor(site.host, currentToken) },
|
||||
});
|
||||
expect(asset.req.egressActor?.user?.uuid).toBe(current.uuid);
|
||||
const replaced = asset.out.cookies[cookieName];
|
||||
expect(replaced).toBeTruthy();
|
||||
expect((await claimsOf(replaced, site.appUid, site.host)).userUid).toBe(
|
||||
current.uuid,
|
||||
);
|
||||
});
|
||||
|
||||
it('bills the token account when a cached page of another account loads assets', async () => {
|
||||
// The document never reaches the server (served from an app's own
|
||||
// cache), so only the assets show which account launched it.
|
||||
const site = await makeLaunchedApp();
|
||||
const previous = await makeUser();
|
||||
const current = await makeUser();
|
||||
const previousToken = await server.services.auth.getUserAppToken(
|
||||
actorFor(previous),
|
||||
site.appUid,
|
||||
);
|
||||
const currentToken = await server.services.auth.getUserAppToken(
|
||||
actorFor(current),
|
||||
site.appUid,
|
||||
);
|
||||
const oldCookies = {
|
||||
[cookieName]: await cookieFor(previousToken, site),
|
||||
};
|
||||
|
||||
for (const path of ['/app.css', '/index.html']) {
|
||||
const asset = await serve(site.host, {
|
||||
path,
|
||||
cookies: oldCookies,
|
||||
headers: { referer: refererFor(site.host, currentToken) },
|
||||
});
|
||||
expect(asset.req.egressActor?.user?.uuid).toBe(current.uuid);
|
||||
const replaced = asset.out.cookies[cookieName];
|
||||
expect(
|
||||
(await claimsOf(replaced, site.appUid, site.host)).userUid,
|
||||
).toBe(current.uuid);
|
||||
}
|
||||
});
|
||||
|
||||
it('re-mints the cookie when the token names the same user under another session', async () => {
|
||||
const site = await makeLaunchedApp();
|
||||
const visitor = await makeUser();
|
||||
const appToken = await server.services.auth.getUserAppToken(
|
||||
actorFor(visitor),
|
||||
site.appUid,
|
||||
);
|
||||
const cookies = { [cookieName]: await cookieFor(appToken, site) };
|
||||
const { session, token: sessionToken } =
|
||||
await server.services.auth.createSessionToken(visitor as never);
|
||||
|
||||
const asset = await serve(site.host, {
|
||||
cookies,
|
||||
headers: { referer: refererFor(site.host, sessionToken) },
|
||||
});
|
||||
expect(asset.req.egressActor?.user?.uuid).toBe(visitor.uuid);
|
||||
const replaced = asset.out.cookies[cookieName];
|
||||
expect(replaced).toBeTruthy();
|
||||
expect(
|
||||
(await claimsOf(replaced, site.appUid, site.host)).sessionUuid,
|
||||
).toBe(session.uuid);
|
||||
});
|
||||
|
||||
it('falls back to the cookie when the Referer token does not verify', async () => {
|
||||
const site = await makeLaunchedApp();
|
||||
const visitor = await makeUser();
|
||||
const token = await server.services.auth.getUserAppToken(
|
||||
actorFor(visitor),
|
||||
site.appUid,
|
||||
);
|
||||
const cookies = { [cookieName]: await cookieFor(token, site) };
|
||||
const tampered = `${token.slice(0, -4)}AAAA`;
|
||||
|
||||
const asset = await serve(site.host, {
|
||||
cookies,
|
||||
headers: { referer: refererFor(site.host, tampered) },
|
||||
});
|
||||
expect(asset.req.egressActor?.user?.uuid).toBe(visitor.uuid);
|
||||
expect(asset.out.cookies).toEqual({});
|
||||
});
|
||||
|
||||
it('bills the site owner once the shared session is revoked', async () => {
|
||||
const site = await makeLaunchedApp();
|
||||
const visitor = await makeUser();
|
||||
const token = await server.services.auth.getUserAppToken(
|
||||
actorFor(visitor),
|
||||
site.appUid,
|
||||
);
|
||||
const cookies = { [cookieName]: await cookieFor(token, site) };
|
||||
await server.services.auth.revokeAllSessionsForUserId(visitor.id);
|
||||
|
||||
const asset = await serve(site.host, {
|
||||
cookies,
|
||||
headers: { referer: refererFor(site.host, token) },
|
||||
});
|
||||
expect(asset.req.egressActor?.user?.uuid).toBe(site.owner.uuid);
|
||||
expect(asset.out.cookies).toEqual({});
|
||||
});
|
||||
|
||||
it('keeps responses that set the cookie out of shared caches', async () => {
|
||||
const site = await makeLaunchedApp();
|
||||
const visitor = await makeUser();
|
||||
const token = await server.services.auth.getUserAppToken(
|
||||
actorFor(visitor),
|
||||
site.appUid,
|
||||
);
|
||||
|
||||
const launch = await serve(site.host, {
|
||||
path: '/index.html',
|
||||
query: { 'puter.auth.token': token },
|
||||
});
|
||||
const cookie = launch.out.cookies[cookieName];
|
||||
expect(cookie).toBeTruthy();
|
||||
expect(launch.out.headers['Cache-Control']).toBe('private');
|
||||
|
||||
const asset = await serve(site.host, {
|
||||
cookies: { [cookieName]: cookie },
|
||||
headers: { referer: refererFor(site.host, token) },
|
||||
});
|
||||
expect(asset.out.cookies).toEqual({});
|
||||
expect(asset.out.headers['Cache-Control']).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('site rooted in the owner tree, root moved to Trash', () => {
|
||||
it('stops serving a trashed folder and resumes when restored', async () => {
|
||||
const owner = await makeUserWithHome();
|
||||
|
||||
@@ -548,8 +548,6 @@ export const createPuterSiteMiddleware = (
|
||||
requestHostname: host,
|
||||
}),
|
||||
);
|
||||
// A shared cache must not hand this visitor's cookie to others.
|
||||
res.set('Cache-Control', 'private');
|
||||
}
|
||||
} catch (e) {
|
||||
// Best-effort — don't block the public file serve.
|
||||
|
||||
@@ -355,17 +355,6 @@ describe('ChatCompletionDriver.complete auth and model resolution', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('rejects a model that is not a string with 400', async () => {
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
driver.complete({
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
model: { id: 'gpt' },
|
||||
} as unknown as ICompleteArguments),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('routes by alias when a provider exposes one (puterId is auto-aliased on the bucket)', async () => {
|
||||
// Inject a model that carries a `puterId` so we can verify the
|
||||
// alias bucket resolves back to the canonical id. Spy on
|
||||
|
||||
@@ -79,7 +79,6 @@ import type {
|
||||
} from './types.js';
|
||||
import { normalize_tools_object } from './utils/FunctionCalling.js';
|
||||
import {
|
||||
requestedServiceSpeed,
|
||||
selectServiceSpeed,
|
||||
lowerServiceSpeed,
|
||||
isServiceSpeedUnavailable,
|
||||
@@ -910,16 +909,18 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
attemptProvider: IChatProvider,
|
||||
baseModel: IChatModel,
|
||||
) => {
|
||||
let requestedSpeed = requestedServiceSpeed(
|
||||
args,
|
||||
baseModel.provider,
|
||||
);
|
||||
const offers =
|
||||
requestedSpeed !== undefined && requestedSpeed !== 'standard'
|
||||
args.serviceSpeed !== undefined &&
|
||||
args.serviceSpeed !== 'standard'
|
||||
? ((await attemptProvider.serviceSpeedModels?.(
|
||||
baseModel,
|
||||
)) ?? {})
|
||||
: {};
|
||||
let requestedSpeed =
|
||||
args.serviceSpeed ??
|
||||
(baseModel.provider === 'claude' && args.speed === 'fast'
|
||||
? 'fast'
|
||||
: undefined);
|
||||
for (;;) {
|
||||
const selection = selectServiceSpeed(
|
||||
baseModel,
|
||||
@@ -2129,9 +2130,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
}
|
||||
|
||||
#resolveModel(modelId: string, provider?: string): IChatModel | null {
|
||||
// `model` comes straight from the caller and may not be a string.
|
||||
if (typeof modelId !== 'string') return null;
|
||||
const models = this.#modelIdMap[normalizeModelKey(modelId)];
|
||||
const models = this.#modelIdMap[normalizeModelKey(modelId ?? '')];
|
||||
if (!models || models.length === 0) return null;
|
||||
// An explicitly requested provider is honoured even if its route is
|
||||
// marked — the caller asked for that one, not for the cheapest hop.
|
||||
|
||||
@@ -44,9 +44,7 @@ import {
|
||||
type MockInstance,
|
||||
} from 'vitest';
|
||||
|
||||
import { SYSTEM_ACTOR, type Actor } from '../../../../core/actor.js';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { ChatCompletionDriver } from '../../ChatCompletionDriver.js';
|
||||
import { SYSTEM_ACTOR } from '../../../../core/actor.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import { PuterServer } from '../../../../server.js';
|
||||
import { setupTestServer } from '../../../../testUtil.js';
|
||||
@@ -1582,117 +1580,3 @@ describe('OpenAiResponsesChatProvider.complete error mapping', () => {
|
||||
expect(recordSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver OpenAI billing', () => {
|
||||
const makeBillingUser = async () => {
|
||||
const user = await server.stores.user.create({
|
||||
username: `billing-${uuidv4().slice(0, 8)}`,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email: null,
|
||||
free_storage: 1000000,
|
||||
requires_email_confirmation: false,
|
||||
});
|
||||
return { user: { ...user, email_confirmed: true } } as Actor;
|
||||
};
|
||||
const makeBillingDriver = async () => {
|
||||
const driver = new ChatCompletionDriver(
|
||||
{
|
||||
providers: {
|
||||
ollama: { enabled: false },
|
||||
'openai-completion': { apiKey: 'test-key' },
|
||||
},
|
||||
} as never,
|
||||
server.clients,
|
||||
server.stores,
|
||||
server.services,
|
||||
);
|
||||
await driver.onServerStart();
|
||||
return driver;
|
||||
};
|
||||
it('meters an empty Responses completion before returning its error', async () => {
|
||||
const actor = await makeBillingUser();
|
||||
const driver = await makeBillingDriver();
|
||||
const before = await server.services.metering.getRemainingUsage(actor);
|
||||
responsesCreateMock.mockResolvedValueOnce({
|
||||
status: 'incomplete',
|
||||
incomplete_details: { reason: 'max_output_tokens' },
|
||||
output: [],
|
||||
output_text: '',
|
||||
usage: {
|
||||
input_tokens: 1200,
|
||||
output_tokens: 2000,
|
||||
output_tokens_details: { reasoning_tokens: 2000 },
|
||||
},
|
||||
});
|
||||
await expect(
|
||||
withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'gpt-5.4-mini',
|
||||
provider: 'openai-responses',
|
||||
messages: [{ role: 'user', content: 'hello' }],
|
||||
max_tokens: 1,
|
||||
fallback: false,
|
||||
}),
|
||||
actor,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(responsesCreateMock).toHaveBeenCalledTimes(1);
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
expect(recordSpy.mock.calls[0][0]).toMatchObject({
|
||||
prompt_tokens: 1200,
|
||||
completion_tokens: 2000,
|
||||
});
|
||||
await Promise.all(recordSpy.mock.results.map((result) => result.value));
|
||||
expect(
|
||||
await server.services.metering.getRemainingUsage(actor),
|
||||
).toBeLessThan(before);
|
||||
});
|
||||
it('prices a raw priority tier with the same rate card as fast service', async () => {
|
||||
const actor = await makeBillingUser();
|
||||
const driver = await makeBillingDriver();
|
||||
const charges: number[] = [];
|
||||
for (const extra of [
|
||||
{},
|
||||
{ service_tier: 'priority' },
|
||||
{ serviceSpeed: 'fast' },
|
||||
]) {
|
||||
responsesCreateMock.mockResolvedValueOnce({
|
||||
output: [],
|
||||
output_text: 'hi',
|
||||
service_tier: Object.keys(extra).length
|
||||
? 'priority'
|
||||
: 'default',
|
||||
usage: { input_tokens: 100, output_tokens: 100 },
|
||||
});
|
||||
const before =
|
||||
await server.services.metering.getRemainingUsage(actor);
|
||||
await withTestActor(
|
||||
() =>
|
||||
driver.complete({
|
||||
model: 'gpt-6.1-terra',
|
||||
provider: 'openai-responses',
|
||||
messages: [{ role: 'user', content: 'hello' }],
|
||||
max_tokens: 100,
|
||||
fallback: false,
|
||||
...extra,
|
||||
} as never),
|
||||
actor,
|
||||
);
|
||||
await Promise.all(
|
||||
recordSpy.mock.results.map((result) => result.value),
|
||||
);
|
||||
charges.push(
|
||||
before -
|
||||
(await server.services.metering.getRemainingUsage(actor)),
|
||||
);
|
||||
}
|
||||
expect(responsesCreateMock.mock.calls[1][0].service_tier).toBe(
|
||||
'priority',
|
||||
);
|
||||
expect(charges[0]).toBeGreaterThan(0);
|
||||
expect(charges[1]).toBe(charges[2]);
|
||||
expect(charges[2]).toBe(charges[0] * 2);
|
||||
});
|
||||
});
|
||||
@@ -1122,6 +1122,16 @@ export const handle_completion_output_responses_api = async ({
|
||||
// Inline-compaction artifact, if the upstream compacted this turn.
|
||||
const compactionItem = output.find((item) => item?.type === 'compaction');
|
||||
|
||||
const is_empty = completion.output_text.trim() === '';
|
||||
if (is_empty && responseToolCalls.length < 1 && !compactionItem) {
|
||||
// GPT refuses to generate an empty response if you ask it to,
|
||||
// so this will probably only happen on an error condition.
|
||||
// A compaction-only output is legitimate, so don't reject it.
|
||||
throw new HttpError(400, 'an empty response was generated', {
|
||||
legacyCode: 'bad_response',
|
||||
});
|
||||
}
|
||||
|
||||
// Reasoning models return `reasoning` output items; their human-readable
|
||||
// text only exists when the caller requested summaries via
|
||||
// `reasoning: { summary: ... }` (raw chain-of-thought is never returned).
|
||||
@@ -1224,14 +1234,6 @@ export const handle_completion_output_responses_api = async ({
|
||||
if (usageDetails) ret.usageDetails = usageDetails;
|
||||
}
|
||||
|
||||
// Empty output can still include paid reasoning and input tokens.
|
||||
const isEmpty = completion.output_text.trim() === '';
|
||||
if (isEmpty && responseToolCalls.length < 1 && !compactionItem) {
|
||||
throw new HttpError(400, 'an empty response was generated', {
|
||||
legacyCode: 'bad_response',
|
||||
});
|
||||
}
|
||||
|
||||
const mod_text = completion.output_text;
|
||||
if (moderate && mod_text !== null) {
|
||||
await applyModeration(moderate, mod_text);
|
||||
|
||||
@@ -26,7 +26,6 @@ import {
|
||||
} from './OpenAIUtil.js';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import {
|
||||
requestedServiceSpeed,
|
||||
selectServiceSpeed,
|
||||
servedServiceSpeed,
|
||||
serviceSpeedBillingModel,
|
||||
@@ -84,31 +83,6 @@ describe('service speed selection', () => {
|
||||
expect(selected.model).toBe(entry);
|
||||
expect(selected.args).toBe(args);
|
||||
});
|
||||
it.each([
|
||||
[{ service_tier: 'priority' }, 'openai-responses', 'fast'],
|
||||
[{ service_tier: 'ultrafast' }, 'openai-responses', 'ultrafast'],
|
||||
[{ service_tier: 'auto' }, 'mistral', 'fast'],
|
||||
[{ speed: 'fast' }, 'claude', 'fast'],
|
||||
[{ service_tier: 'auto' }, 'openai-responses', undefined],
|
||||
[{ service_tier: 'flex' }, 'openai-responses', undefined],
|
||||
[{ speed: 'fast' }, 'openai-responses', undefined],
|
||||
[{ serviceSpeed: 'standard', service_tier: 'priority' }, 'openai-responses', 'standard'],
|
||||
] as const)('reads %j on %s as speed %s', (args, provider, speed) => {
|
||||
expect(requestedServiceSpeed(args, provider)).toBe(speed);
|
||||
});
|
||||
it('prices a native priority tier like fast service', () => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
const args = { ...request(), service_tier: 'priority' as const };
|
||||
const selected = selectServiceSpeed(
|
||||
entry,
|
||||
{ ...args, serviceSpeed: requestedServiceSpeed(args, entry.provider) },
|
||||
noVariant,
|
||||
);
|
||||
const explicit = selectServiceSpeed(entry, request('fast'), noVariant);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(selected.model).toEqual(explicit.model);
|
||||
expect(selected.args.service_tier).toBe('priority');
|
||||
});
|
||||
it.each([
|
||||
'deepseek',
|
||||
'zai',
|
||||
@@ -274,33 +248,6 @@ describe('served speed billing', () => {
|
||||
expect(recorded).toEqual([result.costs]);
|
||||
},
|
||||
);
|
||||
it.each([
|
||||
['fast', 2],
|
||||
['ultrafast', 6],
|
||||
] as const)(
|
||||
'bills a served %s tier at its catalog rate without a selection snapshot',
|
||||
(speed, multiplier) => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
entry.pricingOverrides = [
|
||||
{
|
||||
minPromptTokens: 100,
|
||||
costs: { prompt_tokens: 400, completion_tokens: 1500 },
|
||||
},
|
||||
];
|
||||
const billed = serviceSpeedBillingModel(entry, speed);
|
||||
expect(billed.costs.prompt_tokens).toBe(200 * multiplier);
|
||||
expect(billed.pricingOverrides?.[0]?.costs.prompt_tokens).toBe(
|
||||
400 * multiplier,
|
||||
);
|
||||
expect(serviceSpeedBillingModel(entry, 'standard')).toBe(entry);
|
||||
},
|
||||
);
|
||||
it('bills a served premium tier as standard when the catalog has no rate for it', () => {
|
||||
const entry = model('groq', 'some-model');
|
||||
expect(serviceSpeedBillingModel(entry, 'fast').costs).toEqual(
|
||||
entry.costs,
|
||||
);
|
||||
});
|
||||
it('restores standard scheduled and long-context rates after a downgrade', () => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
entry.pricingOverrides = [
|
||||
|
||||
@@ -59,26 +59,6 @@ export interface ServiceSpeedSelection {
|
||||
speed: ServiceSpeed;
|
||||
}
|
||||
|
||||
/**
|
||||
* The speed a request asks for. Native premium tier parameters count as the
|
||||
* matching `serviceSpeed`, so they are priced like it; `serviceSpeed` wins.
|
||||
*/
|
||||
export const requestedServiceSpeed = (
|
||||
args: Pick<ICompleteArguments, 'serviceSpeed' | 'service_tier' | 'speed'>,
|
||||
provider: string | undefined,
|
||||
): ServiceSpeed | undefined => {
|
||||
if (args.serviceSpeed !== undefined) return args.serviceSpeed;
|
||||
if (args.service_tier === 'ultrafast') return 'ultrafast';
|
||||
if (
|
||||
args.service_tier === 'priority' ||
|
||||
(provider === 'claude' && args.speed === 'fast') ||
|
||||
// Mistral's `auto` tier is its priority tier.
|
||||
(provider === 'mistral' && args.service_tier === 'auto')
|
||||
)
|
||||
return 'fast';
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** Select only an equivalent model from the same provider's loaded catalog. */
|
||||
export const selectServiceSpeed = (
|
||||
model: IChatModel,
|
||||
@@ -221,27 +201,16 @@ export const servedServiceSpeed = (
|
||||
: 'standard';
|
||||
};
|
||||
|
||||
/**
|
||||
* The model priced at the speed the provider served. Without a selection
|
||||
* snapshot the catalog's rates for that speed apply, so a premium tier reached
|
||||
* some other way is never billed as standard.
|
||||
*/
|
||||
export const serviceSpeedBillingModel = (
|
||||
model: IChatModel,
|
||||
speed: ServiceSpeed,
|
||||
): IChatModel => {
|
||||
const pricing = model.serviceSpeedPricing ?? {
|
||||
speed: 'standard',
|
||||
standardCosts: model.costs,
|
||||
standardOverrides: model.pricingOverrides,
|
||||
fastCosts: serviceSpeedCosts(model, 'fast'),
|
||||
ultrafastCosts: serviceSpeedCosts(model, 'ultrafast'),
|
||||
};
|
||||
if (pricing.speed === speed) return model;
|
||||
const pricing = model.serviceSpeedPricing;
|
||||
if (!pricing || pricing.speed === speed) return model;
|
||||
const costs =
|
||||
(speed === 'ultrafast' ? pricing.ultrafastCosts : undefined) ??
|
||||
(speed !== 'standard' ? pricing.fastCosts : undefined) ??
|
||||
pricing.standardCosts;
|
||||
speed === 'fast'
|
||||
? (pricing.fastCosts ?? pricing.standardCosts)
|
||||
: pricing.standardCosts;
|
||||
return {
|
||||
...model,
|
||||
costs,
|
||||
|
||||
@@ -190,39 +190,6 @@ describe('VoiceChangerDriver.convert argument validation', () => {
|
||||
withActor(actor, () => driver.convert({ audio: undefined })),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it.each([
|
||||
['an unpriced', 'eleven_multilingual_sts_v1'],
|
||||
['a malformed', 'eleven_english_sts_v2&x=1'],
|
||||
])(
|
||||
'rejects %s model before loading audio or calling ElevenLabs',
|
||||
async (_label, model) => {
|
||||
const { actor } = await makeUser();
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
driver.convert({
|
||||
audio: 'https://audio.example/sample.wav',
|
||||
model,
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'field_invalid',
|
||||
fields: {
|
||||
key: 'model',
|
||||
expected: [
|
||||
'eleven_multilingual_sts_v2',
|
||||
'eleven_english_sts_v2',
|
||||
],
|
||||
got: model,
|
||||
},
|
||||
});
|
||||
expect(webFetchMock).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
// ── Credit gate ─────────────────────────────────────────────────────
|
||||
|
||||
@@ -51,10 +51,6 @@ const MAX_AUDIO_FILE_SIZE = 25 * 1024 * 1024;
|
||||
// Covers the upload as well as the conversion.
|
||||
const CONVERT_TIMEOUT_MS = 10 * 60 * 1000;
|
||||
|
||||
const SUPPORTED_MODELS = Object.keys(VOICE_CHANGER_COSTS).map(
|
||||
(key) => key.split(':')[1]!,
|
||||
);
|
||||
|
||||
const VOICE_CHANGER_CATALOG: ProviderCatalog = {
|
||||
label: 'Speech-to-speech',
|
||||
ids: ['elevenlabs'],
|
||||
@@ -169,6 +165,14 @@ export class VoiceChangerDriver extends PuterDriver {
|
||||
});
|
||||
}
|
||||
|
||||
const loaded = await loadFileInput(
|
||||
this.stores,
|
||||
this.services.fs,
|
||||
actor,
|
||||
args.audio,
|
||||
{ maxBytes: MAX_AUDIO_FILE_SIZE, acceptWebInput: true },
|
||||
);
|
||||
|
||||
const modelId =
|
||||
args.model_id || args.model || elevenlabs.defaultModelId;
|
||||
const voiceId =
|
||||
@@ -180,41 +184,21 @@ export class VoiceChangerDriver extends PuterDriver {
|
||||
throw new HttpError(400, '`voice` is required', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
// `voiceId` lands in the request URL path, which carries our API key,
|
||||
// so anything outside ElevenLabs' `[A-Za-z0-9_-]+` id shape could
|
||||
// steer the request at a different endpoint.
|
||||
if (!/^[A-Za-z0-9_-]+$/.test(voiceId))
|
||||
// `voiceId` lands in the request URL path; `modelId` lands in a
|
||||
// multipart field. Both are forwarded to ElevenLabs with our
|
||||
// long-lived API key, so anything other than a strict alphanumeric
|
||||
// shape lets a caller steer the request at a different endpoint or
|
||||
// inject parameters. ElevenLabs voice/model IDs are always
|
||||
// `[A-Za-z0-9_-]+` in practice.
|
||||
const ID_REGEX = /^[A-Za-z0-9_-]+$/;
|
||||
if (!ID_REGEX.test(voiceId))
|
||||
throw new HttpError(400, '`voice` must be alphanumeric', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
// Gate on the cost table: an id we can't price is one we can't bill
|
||||
// for, and ElevenLabs bills us for it either way. This also keeps
|
||||
// anything but a known id out of the multipart `model_id` field.
|
||||
const usageKey = `elevenlabs:${modelId}:second`;
|
||||
if (!Object.hasOwn(VOICE_CHANGER_COSTS, usageKey)) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
`Invalid model: ${modelId}. Expected: ${SUPPORTED_MODELS.join(', ')}`,
|
||||
{
|
||||
legacyCode: 'field_invalid',
|
||||
fields: {
|
||||
key: 'model',
|
||||
expected: SUPPORTED_MODELS,
|
||||
got: modelId,
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
const ucentsPerSecond = VOICE_CHANGER_COSTS[usageKey]!;
|
||||
|
||||
const loaded = await loadFileInput(
|
||||
this.stores,
|
||||
this.services.fs,
|
||||
actor,
|
||||
args.audio,
|
||||
{ maxBytes: MAX_AUDIO_FILE_SIZE, acceptWebInput: true },
|
||||
);
|
||||
if (!ID_REGEX.test(modelId))
|
||||
throw new HttpError(400, '`model` must be alphanumeric', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
// The hold estimates seconds from the input at 128 kbps: compact audio
|
||||
// is under-held rather than refused. Usage is metered from the length
|
||||
@@ -223,6 +207,8 @@ export class VoiceChangerDriver extends PuterDriver {
|
||||
1,
|
||||
Math.ceil(loaded.buffer.byteLength / 16000),
|
||||
);
|
||||
const usageKey = `elevenlabs:${modelId}:second`;
|
||||
const ucentsPerSecond = VOICE_CHANGER_COSTS[usageKey] ?? 0;
|
||||
const estimatedCost = ucentsPerSecond * estimatedSeconds;
|
||||
|
||||
const formData = new FormData();
|
||||
|
||||
@@ -37,7 +37,7 @@ import {
|
||||
postGeminiJson,
|
||||
geminiTokenCount as tokenCount,
|
||||
} from '../../../util/geminiRest.js';
|
||||
import { decodeStrictBase64InSlices } from '../../../../util/mediaSniff.js';
|
||||
import { decodeStrictBase64 } from '../../../../util/mediaSniff.js';
|
||||
import { isUpstreamTimeoutError } from '../../../util/upstreamErrors.js';
|
||||
import { SAMPLE_AUDIO_URL } from '../../../util/testMode.js';
|
||||
import type {
|
||||
@@ -456,8 +456,7 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
},
|
||||
);
|
||||
}
|
||||
const { body, contentType } =
|
||||
await this.#decodeAudio(audio);
|
||||
const { body, contentType } = this.#decodeAudio(audio);
|
||||
|
||||
outputTokens = usageTokens(
|
||||
usage?.output_tokens_by_modality,
|
||||
@@ -475,7 +474,7 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
dataType: 'stream',
|
||||
content_type: contentType,
|
||||
chunked: true,
|
||||
stream: Readable.from(body, { objectMode: false }),
|
||||
stream: Readable.from(body),
|
||||
};
|
||||
} finally {
|
||||
// Every outcome pays once for the counts it reports.
|
||||
@@ -520,15 +519,13 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
}
|
||||
|
||||
/** WAV passes through byte for byte; raw PCM is wrapped in a WAV header. */
|
||||
async #decodeAudio(audio: InteractionContent): Promise<{
|
||||
body: Buffer[];
|
||||
#decodeAudio(audio: InteractionContent): {
|
||||
body: Buffer;
|
||||
contentType: string;
|
||||
}> {
|
||||
} {
|
||||
const { data, mime_type: mimeType } = audio;
|
||||
const bytes =
|
||||
typeof data === 'string'
|
||||
? await decodeStrictBase64InSlices(data)
|
||||
: null;
|
||||
typeof data === 'string' ? decodeStrictBase64(data) : null;
|
||||
if (!bytes || typeof mimeType !== 'string') {
|
||||
throw upstreamFailure('Gemini TTS returned invalid audio data');
|
||||
}
|
||||
@@ -544,7 +541,7 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
) {
|
||||
throw upstreamFailure('Gemini TTS returned invalid audio data');
|
||||
}
|
||||
return { body: [bytes], contentType: 'audio/wav' };
|
||||
return { body: bytes, contentType: 'audio/wav' };
|
||||
}
|
||||
if (PCM_MIME_TYPES.has(essence) && bytes.length > 0) {
|
||||
const rate = Number(
|
||||
@@ -554,9 +551,8 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
Number.isSafeInteger(rate) && rate > 0
|
||||
? rate
|
||||
: DEFAULT_PCM_SAMPLE_RATE;
|
||||
// Sent ahead of the samples, which are never copied behind it.
|
||||
return {
|
||||
body: [this.#wavHeader(bytes.length, sampleRate, 1, 16), bytes],
|
||||
body: this.#wrapPcmInWav(bytes, sampleRate, 1, 16),
|
||||
contentType: 'audio/wav',
|
||||
};
|
||||
}
|
||||
@@ -564,15 +560,17 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
}
|
||||
|
||||
/** Wrap raw PCM samples in a WAV container so browsers can play it. */
|
||||
#wavHeader(
|
||||
dataSize: number,
|
||||
#wrapPcmInWav(
|
||||
pcm: Buffer,
|
||||
sampleRate: number,
|
||||
channels: number,
|
||||
bitsPerSample: number,
|
||||
): Buffer {
|
||||
const byteRate = (sampleRate * channels * bitsPerSample) / 8;
|
||||
const blockAlign = (channels * bitsPerSample) / 8;
|
||||
const buffer = Buffer.alloc(44);
|
||||
const dataSize = pcm.length;
|
||||
const headerSize = 44;
|
||||
const buffer = Buffer.alloc(headerSize + dataSize);
|
||||
|
||||
// RIFF header
|
||||
buffer.write('RIFF', 0);
|
||||
@@ -592,6 +590,7 @@ export class GeminiTTSProvider implements ITTSProvider {
|
||||
// data sub-chunk
|
||||
buffer.write('data', 36);
|
||||
buffer.writeUInt32LE(dataSize, 40);
|
||||
pcm.copy(buffer, headerSize);
|
||||
|
||||
return buffer;
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ import { insufficientCreditsError } from '../../../../services/metering/enforcem
|
||||
import type { UsageInput } from '../../../../services/metering/types.js';
|
||||
import { normalizeRawBase64ImageString } from '../../../../util/appIcon.js';
|
||||
import {
|
||||
decodeStrictBase64InSlices,
|
||||
decodeStrictBase64,
|
||||
sniffImageMime,
|
||||
sniffVideoMime,
|
||||
} from '../../../../util/mediaSniff.js';
|
||||
@@ -405,7 +405,7 @@ export class GeminiVideoProvider extends VideoProvider {
|
||||
'Gemini video generation did not complete the request',
|
||||
);
|
||||
}
|
||||
const video = await this.#readVideo(videoBlock);
|
||||
const video = this.#readVideo(videoBlock);
|
||||
if (!billed) {
|
||||
throw geminiUpstreamFailure(
|
||||
'Gemini video generation returned unusable usage',
|
||||
@@ -722,7 +722,7 @@ export class GeminiVideoProvider extends VideoProvider {
|
||||
}
|
||||
|
||||
/** The base64 MP4 held by the video block. */
|
||||
async #readVideo(video: InteractionContent | undefined): Promise<string> {
|
||||
#readVideo(video: InteractionContent | undefined): string {
|
||||
if (!video) {
|
||||
throw new HttpError(400, 'Gemini did not return a video', {
|
||||
legacyCode: 'upstream_bad_request',
|
||||
@@ -731,7 +731,7 @@ export class GeminiVideoProvider extends VideoProvider {
|
||||
}
|
||||
const bytes =
|
||||
video.mime_type === 'video/mp4' && typeof video.data === 'string'
|
||||
? await decodeStrictBase64InSlices(video.data)
|
||||
? decodeStrictBase64(video.data)
|
||||
: null;
|
||||
if (!bytes || sniffVideoMime(bytes) !== 'video/mp4') {
|
||||
throw geminiUpstreamFailure(
|
||||
|
||||
@@ -18,7 +18,6 @@
|
||||
*/
|
||||
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { parseJsonInSlices } from '../../util/slicedJson.js';
|
||||
import { readBoundedBody } from './fileInput.js';
|
||||
import { upstreamFetch } from './upstreamErrors.js';
|
||||
|
||||
@@ -40,8 +39,8 @@ export const geminiUpstreamFailure = (message: string): HttpError =>
|
||||
/**
|
||||
* POSTs `body` to a Gemini REST `path` and parses the JSON object it answers
|
||||
* with. `timeoutMs` covers the body read, and a reply over `maxBytes` fails as
|
||||
* an upstream error unless `tooLarge` supplies another. `T` is the documented
|
||||
* reply shape; callers check every field before use.
|
||||
* an upstream error unless `tooLarge` supplies another. `T` is the documented reply shape; callers check every
|
||||
* field before use.
|
||||
*/
|
||||
export async function postGeminiJson<T extends object>(
|
||||
apiKey: string,
|
||||
@@ -88,8 +87,7 @@ export async function postGeminiJson<T extends object>(
|
||||
const raw = await readBoundedBody(response, maxBytes, tooLarge);
|
||||
let parsed: unknown = null;
|
||||
try {
|
||||
// Media replies run to tens of megabytes of base64.
|
||||
parsed = await parseJsonInSlices(raw);
|
||||
parsed = JSON.parse(raw.toString('utf8'));
|
||||
} catch {
|
||||
// Handled with the non-object case below.
|
||||
}
|
||||
|
||||
@@ -416,7 +416,6 @@ describe('PuterServer host header validation', () => {
|
||||
'upgrade',
|
||||
'top-up',
|
||||
'request-email-confirmation',
|
||||
'attach-custom-domain',
|
||||
])('lets a deployment subdomain frame the %s popup and names it to the GUI', async (action) => {
|
||||
const parent = `http://spreadsheet.puter.localhost:${port}`;
|
||||
const guiParams = (body: string) =>
|
||||
@@ -794,26 +793,6 @@ describe('PuterServer request parsing', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('parses a chat call above 50 MiB whose body opens with its token', async () => {
|
||||
const fields = `"interface":"puter-chat-completion","method":"no-such-method","pad":"${'x'.repeat(51 * MiB)}"`;
|
||||
const asText = { 'content-type': 'text/plain;actually=json' };
|
||||
const first = await postJson(
|
||||
'/drivers/call',
|
||||
`{"auth_token":"${token}",${fields}}`,
|
||||
asText,
|
||||
);
|
||||
expect(first.status).toBe(404);
|
||||
expect(first.body).toContain('no-such-method');
|
||||
|
||||
// A token after other fields can't be read ahead of the body.
|
||||
const last = await postJson(
|
||||
'/drivers/call',
|
||||
`{${fields},"auth_token":"${token}"}`,
|
||||
asText,
|
||||
);
|
||||
expect(last.status).toBe(413);
|
||||
});
|
||||
|
||||
it('keeps the default JSON limit for other drivers', async () => {
|
||||
const payload = JSON.stringify({
|
||||
interface: 'puter-kvstore',
|
||||
|
||||
+11
-30
@@ -36,7 +36,6 @@ import { createAuthProbe } from './core/http/middleware/authProbe';
|
||||
import { createRequestContextMiddleware } from './core/http/middleware/requestContext';
|
||||
import { createFingerprintMiddleware } from './core/http/middleware/fingerprint';
|
||||
import { createErrorHandler } from './core/http/middleware/errorHandler';
|
||||
import { createJsonBodyParser } from './core/http/middleware/jsonBody';
|
||||
import {
|
||||
createLargeJsonBodyParser,
|
||||
DEFAULT_JSON_LIMIT_BYTES,
|
||||
@@ -145,8 +144,6 @@ const LARGE_JSON_ROUTES = [
|
||||
const LARGE_JSON_LIMIT_BYTES = 128 * 1024 * 1024;
|
||||
/** Two max-size bodies per process. */
|
||||
const LARGE_JSON_BUDGET_BYTES = 2 * LARGE_JSON_LIMIT_BYTES;
|
||||
/** These parse in one go, so they stay small enough for that to be quick. */
|
||||
const NON_UTF8_JSON_LIMIT_BYTES = 1024 * 1024;
|
||||
|
||||
/**
|
||||
* Through `/drivers/call`, only chat takes a large body; other drivers keep the
|
||||
@@ -578,42 +575,26 @@ export class PuterServer {
|
||||
}),
|
||||
);
|
||||
}
|
||||
// A WebDAV body is the file being stored, whatever content type the
|
||||
// client labels it with, so the controller reads it off the stream.
|
||||
const exceptDav =
|
||||
(parser: RequestHandler): RequestHandler =>
|
||||
(req, res, next) => {
|
||||
if (activeSubdomain(req) === 'dav') {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
return parser(req, res, next);
|
||||
};
|
||||
this.#app.use(
|
||||
exceptDav(
|
||||
createJsonBodyParser({
|
||||
limit: DEFAULT_JSON_LIMIT_BYTES,
|
||||
verify: captureRawBody,
|
||||
}),
|
||||
),
|
||||
express.json({
|
||||
limit: DEFAULT_JSON_LIMIT_BYTES,
|
||||
verify: captureRawBody,
|
||||
}),
|
||||
);
|
||||
// JSON in another charset, which the parser above leaves alone.
|
||||
this.#app.use(
|
||||
exceptDav(
|
||||
express.json({
|
||||
limit: NON_UTF8_JSON_LIMIT_BYTES,
|
||||
verify: captureRawBody,
|
||||
}),
|
||||
),
|
||||
express.json({
|
||||
limit: DEFAULT_JSON_LIMIT_BYTES,
|
||||
type: (req) =>
|
||||
req.headers['content-type'] === 'text/plain;actually=json',
|
||||
verify: captureRawBody,
|
||||
}),
|
||||
);
|
||||
// Form-encoded bodies (e.g. `/down` from the GUI's iframe-triggered
|
||||
// download form). Needs to run before the auth probe so
|
||||
// `req.body.auth_token` is populated for urlencoded POSTs the same
|
||||
// way it is for JSON POSTs. Small cap — this parser is only here to
|
||||
// cover the auth_token / anti_csrf field shape, not file uploads.
|
||||
this.#app.use(
|
||||
exceptDav(express.urlencoded({ extended: true, limit: '100kb' })),
|
||||
);
|
||||
this.#app.use(express.urlencoded({ extended: true, limit: '100kb' }));
|
||||
|
||||
// -- Auth probe ----------------------------------------------
|
||||
if (authProbe) {
|
||||
|
||||
@@ -31,13 +31,24 @@ export const RECOMMENDED_APP_NAMES = [
|
||||
'spreadsheet',
|
||||
'presentation',
|
||||
'whiteboard',
|
||||
'notes',
|
||||
'pdf-editor',
|
||||
'contacts',
|
||||
'calendar',
|
||||
'meetings',
|
||||
'teamchat',
|
||||
'email',
|
||||
'dev-center',
|
||||
'cap-table',
|
||||
'invoices',
|
||||
'crm',
|
||||
'signatures',
|
||||
'projects',
|
||||
'point-of-sale',
|
||||
'inventory',
|
||||
'hiring',
|
||||
'time-tracking',
|
||||
'shift-planner',
|
||||
'equipment',
|
||||
'chess',
|
||||
'checkers',
|
||||
'backgammon',
|
||||
|
||||
@@ -186,96 +186,6 @@ describe('AuthService (integration)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('signedSessionClaims', () => {
|
||||
it('names the user and session of a session token', async () => {
|
||||
const user = await makeUser();
|
||||
const { token, session } = await authService.createSessionToken(
|
||||
user,
|
||||
{},
|
||||
);
|
||||
expect(authService.signedSessionClaims(token)).toEqual({
|
||||
userUid: user.uuid,
|
||||
sessionUuid: (session as { uuid: string }).uuid,
|
||||
});
|
||||
});
|
||||
|
||||
it('names the app too for an app token', async () => {
|
||||
const user = await makeUser();
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
const sessionUuid = uuidv4();
|
||||
const token = server.services.token.sign('auth', {
|
||||
type: 'app-under-user',
|
||||
version: '2',
|
||||
user_uid: user.uuid,
|
||||
app_uid: appUid,
|
||||
session_uid: sessionUuid,
|
||||
});
|
||||
expect(authService.signedSessionClaims(token)).toEqual({
|
||||
userUid: user.uuid,
|
||||
sessionUuid,
|
||||
appUid,
|
||||
});
|
||||
});
|
||||
|
||||
it('reads the session from `uuid` on a token without `session_uid`', () => {
|
||||
const [userUid, sessionUuid] = [uuidv4(), uuidv4()];
|
||||
const token = server.services.token.sign('auth', {
|
||||
type: 'gui',
|
||||
version: '0.0.0',
|
||||
uuid: sessionUuid,
|
||||
user_uid: userUid,
|
||||
});
|
||||
expect(authService.signedSessionClaims(token)).toEqual({
|
||||
userUid,
|
||||
sessionUuid,
|
||||
});
|
||||
});
|
||||
|
||||
it('returns null for worker, access, expired, tampered and session-less tokens', async () => {
|
||||
const user = await makeUser();
|
||||
const accessToken = await authService.createAccessToken(
|
||||
{
|
||||
user: {
|
||||
id: user.id,
|
||||
uuid: user.uuid,
|
||||
username: user.username,
|
||||
},
|
||||
} as Actor,
|
||||
[[`user:${user.uuid}:email:read`]],
|
||||
);
|
||||
const sign = (claims: Record<string, unknown>, expiresIn?: number) =>
|
||||
server.services.token.sign(
|
||||
'auth',
|
||||
claims,
|
||||
expiresIn === undefined ? undefined : { expiresIn },
|
||||
);
|
||||
const sessionUuid = uuidv4();
|
||||
const session = {
|
||||
type: 'session',
|
||||
version: '2',
|
||||
uuid: sessionUuid,
|
||||
session_uid: sessionUuid,
|
||||
user_uid: user.uuid,
|
||||
};
|
||||
const valid = sign(session);
|
||||
for (const token of [
|
||||
accessToken,
|
||||
sign({ ...session, worker: true, worker_name: 'w' }),
|
||||
sign({
|
||||
type: 'app-under-user',
|
||||
version: '2',
|
||||
user_uid: user.uuid,
|
||||
app_uid: `app-${uuidv4()}`,
|
||||
}),
|
||||
sign(session, -120),
|
||||
`${valid.slice(0, -4)}AAAA`,
|
||||
'not-a-jwt',
|
||||
]) {
|
||||
expect(authService.signedSessionClaims(token)).toBeNull();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── Rich `authenticate()` result shape ──────────────────────────
|
||||
|
||||
describe('authenticate (reauth signal)', () => {
|
||||
@@ -740,60 +650,6 @@ describe('AuthService (integration)', () => {
|
||||
).toBeFalsy();
|
||||
});
|
||||
|
||||
describe('listSessions and expiry', () => {
|
||||
const nowS = () => Math.floor(Date.now() / 1000);
|
||||
|
||||
const listedUuids = async (user: { id: number }) =>
|
||||
(await authService.listSessions({ user } as Actor)).map(
|
||||
(r) => (r as { uuid: string }).uuid,
|
||||
);
|
||||
|
||||
it.each(['web', 'app', 'access_token'])(
|
||||
'leaves out an expired %s session and lists a live one',
|
||||
async (kind) => {
|
||||
const user = await makeUser();
|
||||
const sessions = server.stores.session;
|
||||
const live = await sessions.create(user.id, {
|
||||
kind,
|
||||
expires_at: nowS() + 3600,
|
||||
});
|
||||
const expired = await sessions.create(user.id, {
|
||||
kind,
|
||||
expires_at: nowS() - 1,
|
||||
});
|
||||
|
||||
const uuids = await listedUuids(user);
|
||||
expect(uuids).toContain(live.uuid);
|
||||
expect(uuids).not.toContain(expired.uuid);
|
||||
},
|
||||
);
|
||||
|
||||
it('leaves out a token whose parent session has expired or is gone', async () => {
|
||||
const user = await makeUser();
|
||||
const sessions = server.stores.session;
|
||||
const token = (parent: string) =>
|
||||
sessions.create(user.id, {
|
||||
kind: 'access_token',
|
||||
parent_session_id: parent,
|
||||
expires_at: nowS() + 3600,
|
||||
});
|
||||
const liveParent = await sessions.create(user.id);
|
||||
const expiredParent = await sessions.create(user.id, {
|
||||
expires_at: nowS() - 1,
|
||||
});
|
||||
const listed = await token(liveParent.uuid);
|
||||
const underExpired = await token(expiredParent.uuid);
|
||||
const underMissing = await token(uuidv4());
|
||||
|
||||
const uuids = await listedUuids(user);
|
||||
expect(uuids).toContain(liveParent.uuid);
|
||||
expect(uuids).toContain(listed.uuid);
|
||||
expect(uuids).not.toContain(expiredParent.uuid);
|
||||
expect(uuids).not.toContain(underExpired.uuid);
|
||||
expect(uuids).not.toContain(underMissing.uuid);
|
||||
});
|
||||
});
|
||||
|
||||
it('listSessions excludes kind="asset" rows', async () => {
|
||||
// Asset rows are per-cookie children of `web` rows, revoked
|
||||
// transitively via the cascade — surfacing them in the
|
||||
@@ -3243,7 +3099,7 @@ describe('AuthService (integration)', () => {
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
const announced = () =>
|
||||
emit.mock.calls.filter(
|
||||
(c) => c[0] === 'outer.auth.access-token.revoked',
|
||||
(c) => c[0] === 'auth.access-token.revoked',
|
||||
).length;
|
||||
|
||||
const scoped = await authService.createAccessToken(actor, [
|
||||
@@ -3261,35 +3117,6 @@ describe('AuthService (integration)', () => {
|
||||
emit.mockRestore();
|
||||
});
|
||||
|
||||
it('announces a desktop app token it replaces past its max age', async () => {
|
||||
const user = await makeUser();
|
||||
const { session } = await authService.createSessionToken(user);
|
||||
const actor = makeActor({
|
||||
user: { id: user.id, uuid: user.uuid, username: user.username },
|
||||
session: { uid: String(session.uuid), kind: 'web' },
|
||||
});
|
||||
const app = { uid: `app-${uuidv4()}`, title: 'Retire probe' };
|
||||
const { token } = await authService.getGodmodeAppToken(actor, app);
|
||||
const { token_uid } = server.services.token.verify<{
|
||||
token_uid: string;
|
||||
}>('auth', token);
|
||||
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
const later = Date.now() + 8 * 24 * 60 * 60 * 1000;
|
||||
vi.spyOn(Date, 'now').mockReturnValue(later);
|
||||
try {
|
||||
await authService.getGodmodeAppToken(actor, app);
|
||||
expect(emit).toHaveBeenCalledWith(
|
||||
'outer.auth.access-token.revoked',
|
||||
{ token_uid },
|
||||
{},
|
||||
);
|
||||
} finally {
|
||||
vi.mocked(Date.now).mockRestore();
|
||||
emit.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('revokeAccessToken rejects with 404 when the token belongs to another user', async () => {
|
||||
const u1 = await makeUser();
|
||||
const u2 = await makeUser();
|
||||
@@ -3898,163 +3725,6 @@ describe('AuthService (integration)', () => {
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
expect(await authService.authenticateFromToken(jwt)).toBeTruthy();
|
||||
});
|
||||
|
||||
const migratedTokenRow = async (userId: number) => {
|
||||
const legacyUid = uuidv4();
|
||||
const row = await server.stores.session.create(userId, {
|
||||
kind: 'access_token',
|
||||
legacy_token_uid: legacyUid,
|
||||
created_via: 'legacy_backfill',
|
||||
});
|
||||
return { legacyUid, rowUuid: row.uuid as string };
|
||||
};
|
||||
|
||||
it('soft-revokes a migrated token row named by its legacy token uid', async () => {
|
||||
const user = await makeUser();
|
||||
const actor = {
|
||||
user: { id: user.id, uuid: user.uuid, username: user.username },
|
||||
} as Actor;
|
||||
const { legacyUid, rowUuid } = await migratedTokenRow(user.id);
|
||||
|
||||
await authService.revokeAccessToken(actor, legacyUid);
|
||||
|
||||
expect(await server.stores.session.getByUuid(rowUuid)).toBeNull();
|
||||
});
|
||||
|
||||
it('404s for a raw uid that names no token row', async () => {
|
||||
const user = await makeUser();
|
||||
const actor = {
|
||||
user: { id: user.id, uuid: user.uuid, username: user.username },
|
||||
} as Actor;
|
||||
|
||||
await expect(
|
||||
authService.revokeAccessToken(actor, uuidv4()),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
});
|
||||
|
||||
it("404s when another user names a migrated token's legacy uid", async () => {
|
||||
const owner = await makeUser();
|
||||
const other = await makeUser();
|
||||
const otherActor = {
|
||||
user: {
|
||||
id: other.id,
|
||||
uuid: other.uuid,
|
||||
username: other.username,
|
||||
},
|
||||
} as Actor;
|
||||
const { legacyUid, rowUuid } = await migratedTokenRow(owner.id);
|
||||
|
||||
await expect(
|
||||
authService.revokeAccessToken(otherActor, legacyUid),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
expect(
|
||||
await server.stores.session.getByUuid(rowUuid),
|
||||
).toBeTruthy();
|
||||
});
|
||||
});
|
||||
|
||||
describe('tokens minted by a full-access token', () => {
|
||||
const fullAccessToken = async () => {
|
||||
const user = await makeUser();
|
||||
const userActor = makeActor({
|
||||
user: { id: user.id, uuid: user.uuid, username: user.username },
|
||||
});
|
||||
const jwt = await authService.createAccessToken(userActor, [
|
||||
[FULL_API_ACCESS],
|
||||
]);
|
||||
const decoded = server.services.token.verify('auth', jwt) as {
|
||||
token_uid: string;
|
||||
session_uid: string;
|
||||
auth_id: string;
|
||||
};
|
||||
return { user, userActor, jwt, decoded };
|
||||
};
|
||||
|
||||
const parentOf = async (token: string) => {
|
||||
const { session_uid } = server.services.token.verify(
|
||||
'auth',
|
||||
token,
|
||||
) as { session_uid: string };
|
||||
const rows = (await server.clients.db.read(
|
||||
'SELECT `parent_session_id` FROM `sessions` WHERE `uuid` = ?',
|
||||
[session_uid],
|
||||
)) as Array<{ parent_session_id: string | null }>;
|
||||
return rows[0]?.parent_session_id;
|
||||
};
|
||||
|
||||
/** A replica that has not seen any access-token row yet. */
|
||||
const lagTokenRows = () => {
|
||||
const db = server.clients.db;
|
||||
const original = db.read.bind(db);
|
||||
const spy = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql: string, params?: unknown[]) =>
|
||||
sql.includes("FROM `sessions` WHERE `kind` = 'access_token'")
|
||||
? []
|
||||
: original(sql, params),
|
||||
);
|
||||
return () => spy.mockRestore();
|
||||
};
|
||||
|
||||
it('parents the child to the minting token row while the replica lags, and revoking it takes the child', async () => {
|
||||
const { user, userActor, jwt, decoded } = await fullAccessToken();
|
||||
const minter = await authService.authenticateFromToken(jwt);
|
||||
|
||||
const restore = lagTokenRows();
|
||||
let child: string;
|
||||
try {
|
||||
child = await authService.createAccessToken(minter!, [
|
||||
[`user:${user.uuid}:email:read`],
|
||||
]);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
|
||||
expect(await parentOf(child)).toBe(decoded.session_uid);
|
||||
expect(await authService.authenticateFromToken(child)).toBeTruthy();
|
||||
await authService.revokeAccessToken(userActor, jwt);
|
||||
expect(await authService.authenticateFromToken(child)).toBeNull();
|
||||
});
|
||||
|
||||
it('carries the row whether auth read it from the cache or the database', async () => {
|
||||
const { jwt, decoded } = await fullAccessToken();
|
||||
const key = `sessions:v2:uuid:${decoded.session_uid}`;
|
||||
|
||||
expect(await server.clients.redis.get(key)).toBeTruthy();
|
||||
const cached = await authService.authenticateFromToken(jwt);
|
||||
await server.clients.redis.del(key);
|
||||
const fromDb = await authService.authenticateFromToken(jwt);
|
||||
|
||||
expect(cached?.accessToken?.sessionUid).toBe(decoded.session_uid);
|
||||
expect(fromDb?.accessToken?.sessionUid).toBe(decoded.session_uid);
|
||||
});
|
||||
|
||||
it('looks the row up by token uid when the claim names no row of its own', async () => {
|
||||
const { user, decoded } = await fullAccessToken();
|
||||
const otherSession = await authService.createSessionToken(user, {});
|
||||
const claims = {
|
||||
type: 'access-token',
|
||||
version: '2',
|
||||
token_uid: decoded.token_uid,
|
||||
user_uid: user.uuid,
|
||||
auth_id: decoded.auth_id,
|
||||
full_access: true,
|
||||
};
|
||||
const unclaimed = server.services.token.sign('auth', claims);
|
||||
const misclaimed = server.services.token.sign('auth', {
|
||||
...claims,
|
||||
session_uid: (otherSession.session as { uuid: string }).uuid,
|
||||
});
|
||||
|
||||
for (const token of [unclaimed, misclaimed]) {
|
||||
const minter = await authService.authenticateFromToken(token);
|
||||
expect(minter?.accessToken?.sessionUid).toBeUndefined();
|
||||
const child = await authService.createAccessToken(minter!, [
|
||||
[`user:${user.uuid}:email:read`],
|
||||
]);
|
||||
expect(await parentOf(child)).toBe(decoded.session_uid);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('revokeSession on access-token rows', () => {
|
||||
@@ -4355,32 +4025,6 @@ describe('AuthService (integration)', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it.each(revokers)(
|
||||
'%s revokes an expired session and its live children',
|
||||
async (_name, revoke) => {
|
||||
const user = await makeUser();
|
||||
const sessions = server.stores.session;
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const web = await sessions.create(user.id, {
|
||||
expires_at: now - 1,
|
||||
});
|
||||
const child = await sessions.create(user.id, {
|
||||
kind: 'access_token',
|
||||
parent_session_id: web.uuid,
|
||||
expires_at: now + 3600,
|
||||
});
|
||||
|
||||
await revoke(user);
|
||||
|
||||
const rows = await server.clients.db.read(
|
||||
'SELECT `uuid`, `revoked_at` FROM `sessions` WHERE `uuid` IN (?, ?)',
|
||||
[web.uuid, child.uuid],
|
||||
);
|
||||
expect(rows).toHaveLength(2);
|
||||
for (const row of rows) expect(row.revoked_at).not.toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
it('a password reset leaves workers and standalone access tokens; a forced sign-out takes them', async () => {
|
||||
const user = await makeUser();
|
||||
const sessions = server.stores.session;
|
||||
|
||||
@@ -216,10 +216,6 @@ const sequentialAuthenticate = async (
|
||||
if (!app || !isGodmodeApp(app)) return { invalid: true };
|
||||
godmodeApp = { uid: app.uid, id: app.id };
|
||||
}
|
||||
const ownRow =
|
||||
session?.kind === 'access_token' &&
|
||||
(session.access_token_uid === d.token_uid ||
|
||||
session.legacy_token_uid === d.token_uid);
|
||||
const actor = makeActor({
|
||||
user: actorUser(user),
|
||||
accessToken: {
|
||||
@@ -228,7 +224,6 @@ const sequentialAuthenticate = async (
|
||||
authorized: null,
|
||||
fullAccess: !d.app_uid && d.full_access === true,
|
||||
...(godmodeApp ? { godmodeApp } : {}),
|
||||
...(ownRow ? { sessionUid: session!.uuid } : {}),
|
||||
},
|
||||
});
|
||||
applyHandlerDepth(actor);
|
||||
|
||||
@@ -106,15 +106,6 @@ const HOSTED_ASSET_LABELS = {
|
||||
|
||||
export type ReauthReason = 'session_revoked' | 'session_expired';
|
||||
|
||||
/** Worker claims a session/gui token can carry. */
|
||||
interface SessionTypeTokenOptions {
|
||||
worker?: boolean;
|
||||
workerName?: string;
|
||||
workerUid?: string;
|
||||
workerSiteUid?: string;
|
||||
workerSite?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Request metadata for `authenticate`. `touch: false` checks the token without
|
||||
* recording activity on its session, for re-checks no user made.
|
||||
@@ -376,22 +367,6 @@ export class AuthService extends PuterService {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Digest of the claims `createGuiToken` signs for this session, which
|
||||
* differ between two of its tokens only in `iat`.
|
||||
*/
|
||||
guiTokenDigest(user: UserRow, sessionUuid: string): string {
|
||||
return this.services.token.claimsDigest(
|
||||
'auth',
|
||||
this.#sessionTypeClaims(
|
||||
'gui',
|
||||
user,
|
||||
sessionUuid,
|
||||
this.#authIdFor(user),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Sign a session token for an existing session (upgrade from GUI token). */
|
||||
createSessionTokenForSession(user: UserRow, sessionUuid: string): string {
|
||||
return this.#signSessionTypeToken(
|
||||
@@ -411,21 +386,14 @@ export class AuthService extends PuterService {
|
||||
user: UserRow,
|
||||
sessionUuid: string,
|
||||
authId: string,
|
||||
opts: SessionTypeTokenOptions = {},
|
||||
opts: {
|
||||
worker?: boolean;
|
||||
workerName?: string;
|
||||
workerUid?: string;
|
||||
workerSiteUid?: string;
|
||||
workerSite?: string;
|
||||
} = {},
|
||||
): string {
|
||||
return this.services.token.sign(
|
||||
'auth',
|
||||
this.#sessionTypeClaims(type, user, sessionUuid, authId, opts),
|
||||
);
|
||||
}
|
||||
|
||||
#sessionTypeClaims(
|
||||
type: 'session' | 'gui',
|
||||
user: UserRow,
|
||||
sessionUuid: string,
|
||||
authId: string,
|
||||
opts: SessionTypeTokenOptions = {},
|
||||
): Record<string, unknown> {
|
||||
const claims: Record<string, unknown> = {
|
||||
type,
|
||||
version: '2',
|
||||
@@ -442,7 +410,7 @@ export class AuthService extends PuterService {
|
||||
if (opts.workerUid) claims.worker_uid = opts.workerUid;
|
||||
if (opts.workerSiteUid) claims.workerSiteUid = opts.workerSiteUid;
|
||||
if (opts.workerSite) claims.workerSite = opts.workerSite;
|
||||
return claims;
|
||||
return this.services.token.sign('auth', claims);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -775,22 +743,9 @@ export class AuthService extends PuterService {
|
||||
|
||||
const rows = (await this.stores.session.getByUserId(
|
||||
actor.user.id,
|
||||
)) as SessionRow[];
|
||||
)) as Array<Record<string, unknown>>;
|
||||
|
||||
// Listed while authentication still accepts it: not expired, and for
|
||||
// a token minted under another session, that session live as well.
|
||||
const live = new Map(
|
||||
rows
|
||||
.filter((row) => !this.#endedSession(row))
|
||||
.map((row) => [row.uuid, row]),
|
||||
);
|
||||
const visible = [...live.values()].filter(
|
||||
(row) =>
|
||||
row.kind !== 'asset' &&
|
||||
(row.kind !== 'access_token' ||
|
||||
!row.parent_session_id ||
|
||||
live.has(row.parent_session_id)),
|
||||
);
|
||||
const visible = rows.filter((row) => row.kind !== 'asset');
|
||||
|
||||
const appUids = [
|
||||
...new Set(
|
||||
@@ -830,7 +785,7 @@ export class AuthService extends PuterService {
|
||||
const enriched = visible.map((row) => {
|
||||
const meta =
|
||||
(typeof row.meta === 'string'
|
||||
? JSON.parse(row.meta)
|
||||
? JSON.parse(row.meta as string)
|
||||
: row.meta) ?? {};
|
||||
const isCurrent = actor.session?.uid === row.uuid;
|
||||
const appUid = typeof row.app_uid === 'string' ? row.app_uid : null;
|
||||
@@ -1934,7 +1889,7 @@ export class AuthService extends PuterService {
|
||||
}
|
||||
if (row.access_token_uid) {
|
||||
this.clients.event?.emit(
|
||||
'outer.auth.access-token.revoked',
|
||||
'auth.access-token.revoked',
|
||||
{ token_uid: row.access_token_uid },
|
||||
{},
|
||||
);
|
||||
@@ -2096,43 +2051,6 @@ export class AuthService extends PuterService {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* User, session and app a session, GUI or app token names, checked against
|
||||
* its signature and expiry only: nothing is read from the stores, so this
|
||||
* says nothing about whether the session or user is still live. Null for
|
||||
* worker and access tokens, and for anything that doesn't verify.
|
||||
*/
|
||||
signedSessionClaims(
|
||||
token: string,
|
||||
): { userUid: string; sessionUuid: string; appUid?: string } | null {
|
||||
let decoded: AnyTokenPayload;
|
||||
try {
|
||||
decoded = this.services.token.verify<AnyTokenPayload>(
|
||||
'auth',
|
||||
token,
|
||||
);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (decoded.type === 'session' || decoded.type === 'gui') {
|
||||
if (decoded.worker) return null;
|
||||
const sessionUuid = decoded.session_uid ?? decoded.uuid;
|
||||
if (!decoded.user_uid || !sessionUuid) return null;
|
||||
return { userUid: decoded.user_uid, sessionUuid };
|
||||
}
|
||||
if (decoded.type === 'app-under-user') {
|
||||
if (decoded.worker) return null;
|
||||
if (!decoded.user_uid || !decoded.session_uid || !decoded.app_uid)
|
||||
return null;
|
||||
return {
|
||||
userUid: decoded.user_uid,
|
||||
sessionUuid: decoded.session_uid,
|
||||
appUid: decoded.app_uid,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
#verifyHostedAssetToken(
|
||||
token: string,
|
||||
expectedKind: 'private' | 'public',
|
||||
@@ -2300,21 +2218,18 @@ export class AuthService extends PuterService {
|
||||
let parent_session_id =
|
||||
issuingApp && actor.session ? actor.session.uid : null;
|
||||
if (mintingToken) {
|
||||
const mintingRowUuid =
|
||||
mintingToken.sessionUid ??
|
||||
(
|
||||
await this.stores.session.findActiveByAccessTokenUid(
|
||||
mintingToken.uid,
|
||||
)
|
||||
)?.uuid;
|
||||
if (!mintingRowUuid) {
|
||||
const mintingRow =
|
||||
await this.stores.session.findActiveByAccessTokenUid(
|
||||
mintingToken.uid,
|
||||
);
|
||||
if (!mintingRow) {
|
||||
throw new HttpError(
|
||||
403,
|
||||
'Access tokens may not create access tokens',
|
||||
{ legacyCode: 'forbidden' },
|
||||
);
|
||||
}
|
||||
parent_session_id = mintingRowUuid;
|
||||
parent_session_id = mintingRow.uuid;
|
||||
}
|
||||
|
||||
const tokenSession = await this.stores.session.create(
|
||||
@@ -2596,7 +2511,7 @@ export class AuthService extends PuterService {
|
||||
// Only a token the handshake admits has a socket to drop.
|
||||
if (mayBeFullAccess) {
|
||||
this.clients.event?.emit(
|
||||
'outer.auth.access-token.revoked',
|
||||
'auth.access-token.revoked',
|
||||
{ token_uid: tokenUid },
|
||||
{},
|
||||
);
|
||||
@@ -2881,15 +2796,6 @@ export class AuthService extends PuterService {
|
||||
|
||||
if (session) void this.#touchSession(session.uuid, user.id, ctx);
|
||||
|
||||
// The row `findActiveByAccessTokenUid` would return, already checked
|
||||
// live above, so tokens this one mints can parent to it without a read.
|
||||
const ownRow =
|
||||
session?.kind === 'access_token' &&
|
||||
(session.access_token_uid === decoded.token_uid ||
|
||||
session.legacy_token_uid === decoded.token_uid)
|
||||
? session
|
||||
: null;
|
||||
|
||||
const actor = makeActor({
|
||||
user: this.#actorUserFromRow(user),
|
||||
accessToken: {
|
||||
@@ -2902,7 +2808,6 @@ export class AuthService extends PuterService {
|
||||
// claim on one is ignored here.
|
||||
fullAccess: !decoded.app_uid && decoded.full_access === true,
|
||||
...(godmodeApp ? { godmodeApp } : {}),
|
||||
...(ownRow ? { sessionUid: ownRow.uuid } : {}),
|
||||
},
|
||||
});
|
||||
this.#applyHandlerDepth(actor, decoded);
|
||||
@@ -2959,7 +2864,7 @@ export class AuthService extends PuterService {
|
||||
|
||||
#actorUserFromRow(user: UserRow) {
|
||||
// Strip the password hash; pass everything else through so callers
|
||||
// can read metadata, otp_enabled, etc. without
|
||||
// can read metadata, desktop_bg_*, otp_enabled, etc. without
|
||||
// re-fetching the user. Mirrors what /whoami exposes off of UserRow.
|
||||
const { password: _password, ...rest } = user;
|
||||
return {
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
import { createHmac } from 'node:crypto';
|
||||
import jwt from 'jsonwebtoken';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { tokenClaimsDigest } from '../../../gui/src/helpers/guiToken.js';
|
||||
import { isTokenForAnotherUser } from '../../../gui/src/helpers/tokenUser.js';
|
||||
import { TokenService } from './TokenService.js';
|
||||
|
||||
@@ -404,35 +403,3 @@ describe('session token read by the desktop', () => {
|
||||
expect(isTokenForAnotherUser(token, { uuid: otherUid })).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('TokenService.claimsDigest', () => {
|
||||
const claims = {
|
||||
type: 'gui',
|
||||
version: '2',
|
||||
uuid: '6f1e2d3c-0000-4000-8000-000000000001',
|
||||
session_uid: '6f1e2d3c-0000-4000-8000-000000000001',
|
||||
user_uid: '5d4adce0-a381-4982-9c02-6e2540026238',
|
||||
auth_id: '5d4adce0-a381-4982-9c02-6e2540026238',
|
||||
};
|
||||
|
||||
it('is what the desktop computes over a token signed with those claims', async () => {
|
||||
const svc = createTokenService();
|
||||
const digest = svc.claimsDigest('auth', claims);
|
||||
expect(digest).toMatch(/^[0-9a-f]{32}$/);
|
||||
expect(await tokenClaimsDigest(svc.sign('auth', claims))).toBe(digest);
|
||||
});
|
||||
|
||||
it('differs for another session or token type', () => {
|
||||
const svc = createTokenService();
|
||||
const digest = svc.claimsDigest('auth', claims);
|
||||
expect(
|
||||
svc.claimsDigest('auth', {
|
||||
...claims,
|
||||
session_uid: 'b0b0b0b0-0000-4000-8000-000000000002',
|
||||
}),
|
||||
).not.toBe(digest);
|
||||
expect(svc.claimsDigest('auth', { ...claims, type: 'session' })).not.toBe(
|
||||
digest,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -17,7 +17,6 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto';
|
||||
import jwt, { type SignOptions } from 'jsonwebtoken';
|
||||
import { PuterService } from '../types';
|
||||
|
||||
@@ -269,24 +268,6 @@ export class TokenService extends PuterService {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Digest of the claims `sign(scope, payload)` writes, minus `iat`: SHA-256
|
||||
* over the claims as key-sorted `[key, value]` pairs in JSON, first 32 hex
|
||||
* chars. The GUI computes the same over a stored token
|
||||
* (`gui/src/helpers/guiToken.js`), so the two must stay in step.
|
||||
*/
|
||||
claimsDigest(scope: string, payload: Record<string, unknown>): string {
|
||||
const claims = this.#compressPayload(COMPRESSION[scope], payload);
|
||||
const pairs = Object.keys(claims)
|
||||
.filter((key) => key !== 'iat')
|
||||
.sort()
|
||||
.map((key) => [key, claims[key]]);
|
||||
return createHash('sha256')
|
||||
.update(JSON.stringify(pairs))
|
||||
.digest('hex')
|
||||
.slice(0, 32);
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify and decompress. Only v2 tokens (`kid: 'v2'`) verify; any other
|
||||
* token — including the retired v1 format — is rejected as invalid, with no
|
||||
|
||||
@@ -43,7 +43,7 @@ export type TokenType = 'session' | 'gui' | 'app-under-user' | 'access-token';
|
||||
|
||||
/**
|
||||
* Browser session token. `'session'` is the HTTP-only-cookie flavor, `'gui'`
|
||||
* the same shape served in a response body (e.g. the login response).
|
||||
* the same shape served in a response body (e.g. QR login).
|
||||
*/
|
||||
export interface SessionTokenPayload extends TokenPayloadBase {
|
||||
type: 'session' | 'gui';
|
||||
@@ -109,7 +109,9 @@ export interface AccessTokenPayload extends TokenPayloadBase {
|
||||
}
|
||||
|
||||
export type AnyTokenPayload =
|
||||
SessionTokenPayload | AppUnderUserTokenPayload | AccessTokenPayload;
|
||||
| SessionTokenPayload
|
||||
| AppUnderUserTokenPayload
|
||||
| AccessTokenPayload;
|
||||
|
||||
// -- Session row (from `sessions` table) ----------------------------
|
||||
|
||||
@@ -126,12 +128,8 @@ export interface SessionRow {
|
||||
expires_at?: number | null;
|
||||
app_uid?: string | null;
|
||||
legacy_token_uid?: string | null;
|
||||
access_token_uid?: string | null;
|
||||
created_via?: string | null;
|
||||
auth_id?: string | null;
|
||||
label?: string | null;
|
||||
last_ip?: string | null;
|
||||
last_user_agent?: string | null;
|
||||
}
|
||||
|
||||
export {};
|
||||
@@ -647,22 +647,6 @@ describe('FSService overwrite and dedupe resolution', () => {
|
||||
expect(await readBack(second)).toBe('second');
|
||||
});
|
||||
|
||||
it('leaves one readable file when two overwrites create the same path at once', async () => {
|
||||
const path = `${user.home}/Documents/raced.txt`;
|
||||
const written = await Promise.all([
|
||||
writeFile(user, path, 'short', { overwrite: true }),
|
||||
writeFile(user, path, 'a much longer body', { overwrite: true }),
|
||||
]);
|
||||
|
||||
// Both started before the row existed, so each uploaded under its own
|
||||
// key; the row has to end up describing bytes that are really there.
|
||||
expect(written[0].uuid).toBe(written[1].uuid);
|
||||
const stored = (await entryAt(user, '/Documents/raced.txt'))!;
|
||||
const body = await readBack(stored);
|
||||
expect(['short', 'a much longer body']).toContain(body);
|
||||
expect(stored.size).toBe(Buffer.byteLength(body));
|
||||
});
|
||||
|
||||
it('dedupes into an unused " (n)" name, skipping names already taken', async () => {
|
||||
await writeFile(user, `${user.home}/Documents/d.txt`, 'a');
|
||||
await writeFile(user, `${user.home}/Documents/d (1).txt`, 'a');
|
||||
|
||||
@@ -2073,7 +2073,10 @@ export class FSService extends PuterService {
|
||||
);
|
||||
});
|
||||
|
||||
const fsEntries = await this.#createUploadedEntries(createInputs);
|
||||
const fsEntries = await this.stores.fsEntry.batchCreateEntries(
|
||||
createInputs,
|
||||
true,
|
||||
);
|
||||
return preparedBatch.items.map((item, index) => {
|
||||
const fsEntry = fsEntries[index];
|
||||
if (!fsEntry) {
|
||||
@@ -2998,97 +3001,30 @@ export class FSService extends PuterService {
|
||||
legacyCode: 'conflict',
|
||||
});
|
||||
}
|
||||
await this.#moveObjectOntoKey(
|
||||
session.objectKey,
|
||||
liveKey,
|
||||
{ bucket, region, size },
|
||||
session.contentType,
|
||||
);
|
||||
}
|
||||
|
||||
async #moveObjectOntoKey(
|
||||
fromKey: string,
|
||||
toKey: string,
|
||||
at: { bucket: string; region: string; size: number },
|
||||
contentType?: string | null,
|
||||
): Promise<void> {
|
||||
await this.stores.s3Object.copyObjectOfAnySize(
|
||||
{
|
||||
sourceBucket: at.bucket,
|
||||
sourceKey: fromKey,
|
||||
destinationBucket: at.bucket,
|
||||
destinationKey: toKey,
|
||||
...(contentType ? { contentType } : {}),
|
||||
sourceBucket: bucket,
|
||||
sourceKey: session.objectKey,
|
||||
destinationBucket: bucket,
|
||||
destinationKey: liveKey,
|
||||
contentType: session.contentType,
|
||||
metadataDirective: 'REPLACE',
|
||||
},
|
||||
at.region,
|
||||
at.size,
|
||||
region,
|
||||
size,
|
||||
);
|
||||
try {
|
||||
await this.stores.s3Object.deleteObject(
|
||||
at.bucket,
|
||||
fromKey,
|
||||
at.region,
|
||||
bucket,
|
||||
session.objectKey,
|
||||
region,
|
||||
);
|
||||
} catch (e) {
|
||||
// The destination is already right; the source is only litter.
|
||||
console.warn('[fs] moved object left behind:', e);
|
||||
// The live object is already right; the staged one is only litter.
|
||||
console.warn('[fs] staged overwrite left behind:', e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the rows for objects already uploaded. A row another writer
|
||||
* created first keeps its uuid, so those bytes move onto that key.
|
||||
*/
|
||||
async #createUploadedEntries(
|
||||
inputs: FSEntryCreateInput[],
|
||||
): Promise<FSEntry[]> {
|
||||
const entries = await this.stores.fsEntry.batchCreateEntries(
|
||||
inputs,
|
||||
true,
|
||||
);
|
||||
const raced = inputs.flatMap((input, index) => {
|
||||
const liveKey = entries[index]?.uuid;
|
||||
return liveKey && liveKey !== input.uuid
|
||||
? [{ input, index, liveKey }]
|
||||
: [];
|
||||
});
|
||||
if (raced.length === 0) return entries;
|
||||
|
||||
await Promise.all(
|
||||
raced.map(({ input, liveKey }) =>
|
||||
this.#moveObjectOntoKey(
|
||||
input.uuid,
|
||||
liveKey,
|
||||
{
|
||||
bucket: this.stores.s3Object.resolveBucket(
|
||||
input.bucket,
|
||||
),
|
||||
region: this.stores.s3Object.resolveRegion(
|
||||
input.bucketRegion,
|
||||
),
|
||||
size: input.size,
|
||||
},
|
||||
input.contentType,
|
||||
),
|
||||
),
|
||||
);
|
||||
const adopted = await this.stores.fsEntry.batchCreateEntries(
|
||||
raced.map(({ input, liveKey }) => ({ ...input, uuid: liveKey })),
|
||||
true,
|
||||
);
|
||||
raced.forEach(({ index, liveKey }, i) => {
|
||||
const entry = adopted[i];
|
||||
if (entry?.uuid !== liveKey) {
|
||||
throw new HttpError(409, 'Write target changed mid-write', {
|
||||
legacyCode: 'conflict',
|
||||
});
|
||||
}
|
||||
entries[index] = entry;
|
||||
});
|
||||
return entries;
|
||||
}
|
||||
|
||||
async completeUrlWrite(
|
||||
userId: number,
|
||||
completeWriteRequest: CompleteWriteRequest,
|
||||
@@ -3645,10 +3581,10 @@ export class FSService extends PuterService {
|
||||
: uploadBody.contentHashSha256;
|
||||
|
||||
const createInput = this.#toCreateInput(normalizedInput, objectKey);
|
||||
const [fsEntry] = await this.#createUploadedEntries([createInput]);
|
||||
if (!fsEntry) {
|
||||
throw new Error('Failed to create entry');
|
||||
}
|
||||
const fsEntry = await this.stores.fsEntry.createEntry(
|
||||
createInput,
|
||||
normalizedInput.createMissingParents,
|
||||
);
|
||||
|
||||
this.#emitFsEvent(
|
||||
existingEntry ? 'fs.write.file' : 'fs.create.file',
|
||||
@@ -3746,18 +3682,17 @@ export class FSService extends PuterService {
|
||||
updates.filter(isValid),
|
||||
);
|
||||
let next = 0;
|
||||
return updates.map(
|
||||
(update): PromiseSettledResult<FSEntry> =>
|
||||
isValid(update)
|
||||
? stored[next++]
|
||||
: {
|
||||
status: 'rejected',
|
||||
reason: new HttpError(
|
||||
400,
|
||||
'Invalid file entry identifier for thumbnail update',
|
||||
{ legacyCode: 'bad_request' },
|
||||
),
|
||||
},
|
||||
return updates.map((update): PromiseSettledResult<FSEntry> =>
|
||||
isValid(update)
|
||||
? stored[next++]
|
||||
: {
|
||||
status: 'rejected',
|
||||
reason: new HttpError(
|
||||
400,
|
||||
'Invalid file entry identifier for thumbnail update',
|
||||
{ legacyCode: 'bad_request' },
|
||||
),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -124,21 +124,6 @@ describe('resolveNode', () => {
|
||||
expect(error.message).toContain('uuid=ghost');
|
||||
});
|
||||
|
||||
it('treats a path-less row found by uid or id as missing', async () => {
|
||||
const orphan = makeEntry({ path: null as unknown as string });
|
||||
const store = makeStore({
|
||||
getEntryByUuid: vi.fn(async () => orphan),
|
||||
getEntryById: vi.fn(async () => orphan),
|
||||
});
|
||||
|
||||
await expect(resolveNode(store, { uid: 'uuid-7' })).resolves.toBeNull();
|
||||
await expectHttpError(
|
||||
() => resolveNode(store, { id: 7 }, { required: true }),
|
||||
404,
|
||||
'subject_does_not_exist',
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to id lookup when uid is blank', async () => {
|
||||
const entry = makeEntry();
|
||||
const store = makeStore({ getEntryById: vi.fn(async () => entry) });
|
||||
|
||||
@@ -62,12 +62,10 @@ export async function resolveNode(
|
||||
): Promise<FSEntry | null> {
|
||||
if (ref.entry) return ref.entry;
|
||||
|
||||
// A uid/id lookup can return a legacy row with no path when its parent
|
||||
// chain is broken. Nothing can authorize or address it, so it's a miss.
|
||||
const uuid = ref.uid ?? ref.uuid;
|
||||
if (isNonEmptyString(uuid)) {
|
||||
const entry = await fsEntryStore.getEntryByUuid(uuid);
|
||||
if (entry?.path) return entry;
|
||||
if (entry) return entry;
|
||||
return notFoundOrNull(
|
||||
options.required,
|
||||
`Entry not found: uuid=${uuid}`,
|
||||
@@ -82,7 +80,7 @@ export async function resolveNode(
|
||||
});
|
||||
}
|
||||
const entry = await fsEntryStore.getEntryById(numericId);
|
||||
if (entry?.path) return entry;
|
||||
if (entry) return entry;
|
||||
return notFoundOrNull(
|
||||
options.required,
|
||||
`Entry not found: id=${numericId}`,
|
||||
|
||||
@@ -137,6 +137,7 @@ const render = async (
|
||||
service: PuterHomepageService,
|
||||
req: Request = makeReq(),
|
||||
meta: Record<string, unknown> = { title: 'Puter' },
|
||||
launchOptions: Record<string, unknown> = {},
|
||||
actor: unknown = null,
|
||||
): Promise<string> => {
|
||||
let sent = '';
|
||||
@@ -148,6 +149,7 @@ const render = async (
|
||||
await service.send(
|
||||
{ req, res, actor } as never,
|
||||
meta as never,
|
||||
launchOptions as never,
|
||||
);
|
||||
return sent;
|
||||
};
|
||||
@@ -290,6 +292,7 @@ describe('PuterHomepageService — gui() parameters', () => {
|
||||
asset_dir: '/dist',
|
||||
title: 'Editor',
|
||||
app: { name: 'editor' },
|
||||
launch_options: {},
|
||||
});
|
||||
// No `config.origin`: fall back to the request's protocol + domain.
|
||||
expect(params.app_origin).toBe('http://puter.test');
|
||||
@@ -335,7 +338,7 @@ describe('PuterHomepageService — gui() parameters', () => {
|
||||
|
||||
const staff = { user: { id: 7, email: '[email protected]' } };
|
||||
expect(
|
||||
guiParamsOf(await render(service, makeReq(), undefined, staff))
|
||||
guiParamsOf(await render(service, makeReq(), undefined, {}, staff))
|
||||
.teams_ui,
|
||||
).toBe(true);
|
||||
expect(teamsAvailableTo).toHaveBeenCalledWith(7, '[email protected]');
|
||||
@@ -346,7 +349,7 @@ describe('PuterHomepageService — gui() parameters', () => {
|
||||
teamsAvailableTo.mockResolvedValueOnce(false);
|
||||
expect(
|
||||
guiParamsOf(
|
||||
await render(service, makeReq(), undefined, {
|
||||
await render(service, makeReq(), undefined, {}, {
|
||||
user: { id: 9, email: '[email protected]' },
|
||||
}),
|
||||
).teams_ui,
|
||||
@@ -417,6 +420,22 @@ describe('PuterHomepageService — gui() parameters', () => {
|
||||
expect(html).toContain('\\u003c/script>');
|
||||
expect(guiParamsOf(html).injected).toBe('</script><img src=x>');
|
||||
});
|
||||
|
||||
it('passes launch options straight through', async () => {
|
||||
const params = guiParamsOf(
|
||||
await render(
|
||||
makeService(),
|
||||
makeReq(),
|
||||
{ title: 'Puter' },
|
||||
{
|
||||
on_initialized: [{ do: 'thing' }],
|
||||
},
|
||||
),
|
||||
);
|
||||
expect(params.launch_options).toEqual({
|
||||
on_initialized: [{ do: 'thing' }],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('PuterHomepageService — extension hooks', () => {
|
||||
|
||||
@@ -44,6 +44,10 @@ export interface PageMeta {
|
||||
app?: { name?: string; [k: string]: unknown } | null;
|
||||
}
|
||||
|
||||
export interface LaunchOptions {
|
||||
on_initialized?: Array<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* A GUI popup a first-party page frames at `/embed/<action>`. `hostOrigin` is
|
||||
* that page, read from the referrer and pinned with frame-ancestors; null when
|
||||
@@ -53,17 +57,6 @@ export interface PopupFrame {
|
||||
hostOrigin: string | null;
|
||||
}
|
||||
|
||||
export interface ShellContext {
|
||||
req: Request;
|
||||
res: Response;
|
||||
actor?: Actor | null;
|
||||
/**
|
||||
* `AuthService.guiTokenDigest` for the request's session cookie, or null
|
||||
* when it carries none. Omitted when the caller doesn't work it out.
|
||||
*/
|
||||
guiTokenDigest?: string | null;
|
||||
}
|
||||
|
||||
interface PuterGuiAddonsEvent {
|
||||
req: Request;
|
||||
path: string;
|
||||
@@ -316,11 +309,12 @@ export class PuterHomepageService extends PuterService {
|
||||
* called on the happy path.
|
||||
*/
|
||||
async send(
|
||||
ctx: ShellContext,
|
||||
ctx: { req: Request; res: Response; actor?: Actor | null },
|
||||
meta: PageMeta,
|
||||
launchOptions: LaunchOptions = {},
|
||||
popupFrame?: PopupFrame,
|
||||
): Promise<void> {
|
||||
const { req, res, actor, guiTokenDigest } = ctx;
|
||||
const { req, res, actor } = ctx;
|
||||
|
||||
// Easter egg: puter-in-puter detection → render error page instead.
|
||||
if (
|
||||
@@ -340,8 +334,9 @@ export class PuterHomepageService extends PuterService {
|
||||
}
|
||||
|
||||
const html = await this.#renderShell(
|
||||
{ req, actor: actor ?? null, guiTokenDigest },
|
||||
{ req, actor: actor ?? null },
|
||||
meta,
|
||||
launchOptions,
|
||||
popupFrame,
|
||||
);
|
||||
res.send(html);
|
||||
@@ -350,13 +345,12 @@ export class PuterHomepageService extends PuterService {
|
||||
// -- Internals ----------------------------------------------------
|
||||
|
||||
async #renderShell(
|
||||
ctx: Pick<ShellContext, 'req' | 'guiTokenDigest'> & {
|
||||
actor: Actor | null;
|
||||
},
|
||||
ctx: { req: Request; actor: Actor | null },
|
||||
meta: PageMeta,
|
||||
launchOptions: LaunchOptions,
|
||||
popupFrame?: PopupFrame,
|
||||
): Promise<string> {
|
||||
const { req, actor, guiTokenDigest } = ctx;
|
||||
const { req, actor } = ctx;
|
||||
const env = this.config.env ?? 'prod';
|
||||
const bundled = env !== 'dev' || this.config.use_bundled_gui === true;
|
||||
const assetDir = env === 'dev' ? '/src' : '/dist';
|
||||
@@ -399,13 +393,10 @@ export class PuterHomepageService extends PuterService {
|
||||
// instead of the socket wire.
|
||||
eventsNotifications: notificationsFoldInEnabled(this.config),
|
||||
...meta,
|
||||
launch_options: launchOptions,
|
||||
// After `meta`, so nothing page-specific can name the opener.
|
||||
...(popupFrame ? { embed_host_origin: popupFrame.hostOrigin } : {}),
|
||||
};
|
||||
// Only ever this request's; without one the GUI asks for its token as
|
||||
// it always has.
|
||||
if (guiTokenDigest === undefined) delete guiParams.guiTokenDigest;
|
||||
else guiParams.guiTokenDigest = guiTokenDigest;
|
||||
|
||||
const event: PuterGuiAddonsEvent = {
|
||||
req,
|
||||
@@ -526,6 +517,8 @@ export class PuterHomepageService extends PuterService {
|
||||
<meta name="theme-color" content="#ffffff">
|
||||
${bundled ? `<link rel="stylesheet" href="${guiCss}">` : ''}
|
||||
|
||||
<link rel="preload" as="image" href="https://puter-assets.b-cdn.net/wallpaper.webp">
|
||||
|
||||
<script>
|
||||
if ( ! window.service_script ) {
|
||||
window.service_script_api_promise = (() => {
|
||||
|
||||
@@ -4579,209 +4579,6 @@ describe('MeteringService', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('settling when a balance read fails', () => {
|
||||
const monthKey = (a: Actor) =>
|
||||
`${METRICS_V2_PREFIX}:actor:${a.user!.uuid}:${currentMonthString()}`;
|
||||
|
||||
/** Rejects the next `times` reads `match` picks; the rest go through. */
|
||||
const failReads = (
|
||||
match: (args: {
|
||||
key: unknown;
|
||||
consistentRead?: boolean;
|
||||
}) => boolean,
|
||||
times = Infinity,
|
||||
) => {
|
||||
const kv = server.stores.kv;
|
||||
const realGet = kv.get.bind(kv);
|
||||
let left = times;
|
||||
return vi
|
||||
.spyOn(kv, 'get')
|
||||
.mockImplementation(
|
||||
async (...args: Parameters<typeof realGet>) => {
|
||||
if (left > 0 && match(args[0] as never)) {
|
||||
left--;
|
||||
throw new Error(
|
||||
'ProvisionedThroughputExceededException',
|
||||
);
|
||||
}
|
||||
return realGet(...args);
|
||||
},
|
||||
);
|
||||
};
|
||||
const isAddonsRead = ({ key }: { key: unknown }) =>
|
||||
typeof key === 'string' && key.endsWith(':addons');
|
||||
|
||||
const pools = async (a: Actor) => {
|
||||
await server.stores.meteringBuffer.flushCycle();
|
||||
const { usage } = await target.getActorCurrentMonthUsageDetails(a);
|
||||
const addons = await target.getActorAddons(a, {
|
||||
consistentRead: true,
|
||||
});
|
||||
return {
|
||||
total: usage.total,
|
||||
allowanceUsed: usage.allowanceUsed ?? 0,
|
||||
consumed: addons.consumedPurchaseCredits ?? 0,
|
||||
};
|
||||
};
|
||||
|
||||
it('charges the allowance when the purchased-credit read fails', async () => {
|
||||
await target.updateAddonCredit(actor.user.uuid!, 1_000_000);
|
||||
await target.incrementUsage(actor, 'ai:chat', 1, 1000);
|
||||
const before = await target.getAllowedUsage(actor);
|
||||
|
||||
const get = failReads(isAddonsRead);
|
||||
const result = await target.incrementUsage(
|
||||
actor,
|
||||
'ai:chat',
|
||||
1,
|
||||
5000,
|
||||
);
|
||||
get.mockRestore();
|
||||
|
||||
expect(result.total).toBe(6000);
|
||||
expect(await pools(actor)).toEqual({
|
||||
total: 6000,
|
||||
allowanceUsed: 6000,
|
||||
consumed: 0,
|
||||
});
|
||||
const after = await target.getAllowedUsage(actor);
|
||||
expect(before.remaining - after.remaining).toBe(5000);
|
||||
});
|
||||
|
||||
it('charges only the allowance share past the allowance, and nothing later charges the rest', async () => {
|
||||
const { monthUsageAllowance: allowance } =
|
||||
await target.getActorSubscription(actor);
|
||||
await target.updateAddonCredit(actor.user.uuid!, 1_000_000);
|
||||
await target.incrementUsage(actor, 'ai:chat', 1, allowance - 100);
|
||||
expect(await target.hasAnyUsageCached(actor)).toBe(true);
|
||||
|
||||
const alarm = vi.spyOn(server.clients.alarm, 'create');
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
const get = failReads(isAddonsRead);
|
||||
await target.incrementUsage(actor, 'ai:chat', 1, 1000);
|
||||
get.mockRestore();
|
||||
|
||||
// 100 fits the allowance; the 900 credit would cover isn't drawn.
|
||||
expect(await pools(actor)).toEqual({
|
||||
total: allowance + 900,
|
||||
allowanceUsed: allowance,
|
||||
consumed: 0,
|
||||
});
|
||||
expect(alarm).toHaveBeenCalledWith(
|
||||
'metering_settle_read_failed',
|
||||
expect.any(String),
|
||||
expect.objectContaining({
|
||||
userId: actor.user.uuid,
|
||||
unchargedCost: 900,
|
||||
}),
|
||||
'info',
|
||||
);
|
||||
expect(
|
||||
alarm.mock.calls.some(([id]) =>
|
||||
String(id).startsWith('metering service error'),
|
||||
),
|
||||
).toBe(false);
|
||||
// The unread credit isn't taken as spent.
|
||||
expect(
|
||||
emit.mock.calls.some(
|
||||
([name]) => name === 'metering.credit-state',
|
||||
),
|
||||
).toBe(false);
|
||||
expect(await target.hasAnyUsageCached(actor)).toBe(true);
|
||||
alarm.mockRestore();
|
||||
emit.mockRestore();
|
||||
|
||||
// The next settle charges its own cost once, and only that.
|
||||
await target.incrementUsage(actor, 'ai:chat', 1, 700);
|
||||
expect(await pools(actor)).toEqual({
|
||||
total: allowance + 1600,
|
||||
allowanceUsed: allowance,
|
||||
consumed: 700,
|
||||
});
|
||||
});
|
||||
|
||||
it('charges the allowance from the buffered total when the exact read fails', async () => {
|
||||
const Service = target.constructor as typeof MeteringService;
|
||||
const interval = Service.EXACT_READ_MIN_INTERVAL_MS;
|
||||
Service.EXACT_READ_MIN_INTERVAL_MS = 0;
|
||||
try {
|
||||
const { monthUsageAllowance: allowance } =
|
||||
await target.getActorSubscription(actor);
|
||||
await target.incrementUsage(
|
||||
actor,
|
||||
'ai:chat',
|
||||
1,
|
||||
allowance - 3000,
|
||||
);
|
||||
await server.stores.meteringBuffer.flushCycle();
|
||||
|
||||
const key = monthKey(actor);
|
||||
const alarm = vi.spyOn(server.clients.alarm, 'create');
|
||||
const get = failReads(
|
||||
(args) => args.key === key && !!args.consistentRead,
|
||||
);
|
||||
const result = await target.incrementUsage(
|
||||
actor,
|
||||
'ai:chat',
|
||||
1,
|
||||
1000,
|
||||
);
|
||||
const failedReads = get.mock.calls.length;
|
||||
get.mockRestore();
|
||||
|
||||
expect(failedReads).toBeGreaterThan(0);
|
||||
expect(result.total).toBe(allowance - 2000);
|
||||
expect(await pools(actor)).toEqual({
|
||||
total: allowance - 2000,
|
||||
allowanceUsed: allowance - 2000,
|
||||
consumed: 0,
|
||||
});
|
||||
expect(alarm).toHaveBeenCalledWith(
|
||||
'metering_settle_read_failed',
|
||||
expect.any(String),
|
||||
expect.objectContaining({ unchargedCost: 0 }),
|
||||
'info',
|
||||
);
|
||||
alarm.mockRestore();
|
||||
} finally {
|
||||
Service.EXACT_READ_MIN_INTERVAL_MS = interval;
|
||||
}
|
||||
});
|
||||
|
||||
it('charges each of two racing calls once when one of their reads fails', async () => {
|
||||
await target.updateAddonCredit(actor.user.uuid!, 1_000_000);
|
||||
await target.incrementUsage(actor, 'ai:chat', 1, 1000);
|
||||
|
||||
const get = failReads(isAddonsRead, 1);
|
||||
await Promise.all([
|
||||
target.incrementUsage(actor, 'ai:chat', 1, 2000),
|
||||
target.incrementUsage(actor, 'ai:chat', 1, 3000),
|
||||
]);
|
||||
get.mockRestore();
|
||||
|
||||
expect(await pools(actor)).toEqual({
|
||||
total: 6000,
|
||||
allowanceUsed: 6000,
|
||||
consumed: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it('charges a failed call and the calls after it exactly once each', async () => {
|
||||
await target.updateAddonCredit(actor.user.uuid!, 1_000_000);
|
||||
const costs = [1000, 2000, 3000, 4000];
|
||||
for (const [i, cost] of costs.entries()) {
|
||||
const get = failReads(isAddonsRead, i % 2 === 0 ? 1 : 0);
|
||||
await target.incrementUsage(actor, 'ai:chat', 1, cost);
|
||||
get.mockRestore();
|
||||
}
|
||||
expect(await pools(actor)).toEqual({
|
||||
total: 10_000,
|
||||
allowanceUsed: 10_000,
|
||||
consumed: 0,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ── Resolver registration ────────────────────────────────────────
|
||||
|
||||
describe('resolver registration', () => {
|
||||
|
||||
@@ -342,9 +342,6 @@ export class MeteringService extends PuterService {
|
||||
/** Bumped on every drop, so a read that straddles one isn't cached. */
|
||||
#spentAddonsDrops = 0;
|
||||
|
||||
/** Running sum reported by `#alarmSettleReadFailed`. */
|
||||
#unchargedSinceStart = 0;
|
||||
|
||||
/**
|
||||
* Uuid → until when a near-allowance read bypasses the exact-read throttle.
|
||||
* Set on a credits change (a purchase, an admin correction): for a short
|
||||
@@ -678,84 +675,49 @@ export class MeteringService extends PuterService {
|
||||
appCallCount: usages.length,
|
||||
});
|
||||
|
||||
// The usage is already being recorded, so a failed balance read
|
||||
// must not skip settling it.
|
||||
let settleReadError: Error | null = null;
|
||||
const [usageResult, subscription, actorAddons] = await Promise.all([
|
||||
usageResultPromise,
|
||||
this.#actorSubscriptionWithStatus(actor),
|
||||
(async () => {
|
||||
try {
|
||||
return await this.#addonsForSettle(actor);
|
||||
} catch (e) {
|
||||
settleReadError = e as Error;
|
||||
return null;
|
||||
}
|
||||
})(),
|
||||
this.#addonsForSettle(actor),
|
||||
]);
|
||||
const { policy: actorSubscription, provisionalUntil } =
|
||||
subscription;
|
||||
|
||||
let actorUsages: UsageByType;
|
||||
try {
|
||||
actorUsages = await this.exactUsageNearAllowance(
|
||||
userId,
|
||||
actorUsageKey,
|
||||
usageResult,
|
||||
actorSubscription.monthUsageAllowance,
|
||||
);
|
||||
} catch (e) {
|
||||
settleReadError ??= e as Error;
|
||||
actorUsages = usageResult.res as unknown as UsageByType;
|
||||
}
|
||||
const actorUsages = await this.exactUsageNearAllowance(
|
||||
userId,
|
||||
actorUsageKey,
|
||||
usageResult,
|
||||
actorSubscription.monthUsageAllowance,
|
||||
);
|
||||
|
||||
// After a failed read only the allowance share is charged:
|
||||
// purchased credit is never drawn against an unread balance.
|
||||
const readAddons = settleReadError ? null : actorAddons;
|
||||
const settledAllowanceUsed = await this.settleIncrementCharges(
|
||||
userId,
|
||||
actorUsageKey,
|
||||
actorUsages,
|
||||
actorSubscription,
|
||||
readAddons ?? ({} as UsageAddons),
|
||||
actorAddons,
|
||||
totalBatchCost,
|
||||
);
|
||||
// Before any hold this usage stands for is released.
|
||||
await this.stores.creditHold.bump(userId);
|
||||
|
||||
if (!readAddons) {
|
||||
const usedBefore = MeteringService.allowanceUsedBefore(
|
||||
actorUsages,
|
||||
actorSubscription.monthUsageAllowance,
|
||||
totalBatchCost,
|
||||
);
|
||||
this.#alarmSettleReadFailed(
|
||||
actor,
|
||||
settleReadError,
|
||||
Math.max(
|
||||
0,
|
||||
totalBatchCost - (settledAllowanceUsed - usedBefore),
|
||||
),
|
||||
);
|
||||
} else {
|
||||
this.maybeAlertOveruse({
|
||||
actor,
|
||||
userId,
|
||||
actorUsages,
|
||||
actorSubscription,
|
||||
actorAddons,
|
||||
incrementCost: totalBatchCost,
|
||||
batchUsages: usages,
|
||||
});
|
||||
this.maybeAlertOveruse({
|
||||
actor,
|
||||
userId,
|
||||
actorUsages,
|
||||
actorSubscription,
|
||||
actorAddons,
|
||||
incrementCost: totalBatchCost,
|
||||
batchUsages: usages,
|
||||
});
|
||||
|
||||
this.rememberRemainingCredits(
|
||||
userId,
|
||||
settledAllowanceUsed,
|
||||
actorSubscription,
|
||||
actorAddons,
|
||||
provisionalUntil,
|
||||
);
|
||||
}
|
||||
this.rememberRemainingCredits(
|
||||
userId,
|
||||
settledAllowanceUsed,
|
||||
actorSubscription,
|
||||
actorAddons,
|
||||
provisionalUntil,
|
||||
);
|
||||
|
||||
const own = this.#withDetail(actorUsages, usageResult.detail);
|
||||
const charged = await this.applyMonthlyCharges(
|
||||
@@ -2995,31 +2957,6 @@ export class MeteringService extends PuterService {
|
||||
return usedBefore + allowanceCharge;
|
||||
}
|
||||
|
||||
/**
|
||||
* One alarm id for every actor, so a store brownout is a single alarm with
|
||||
* a count. `unchargedCost` is what credit would have covered, had any been
|
||||
* left; nothing charges it later.
|
||||
*/
|
||||
#alarmSettleReadFailed(
|
||||
actor: Actor,
|
||||
error: Error | null,
|
||||
unchargedCost: number,
|
||||
): void {
|
||||
this.#unchargedSinceStart += unchargedCost;
|
||||
this.clients.alarm.create(
|
||||
'metering_settle_read_failed',
|
||||
'Metering settled usage to the allowance only: a balance read failed',
|
||||
{
|
||||
userId: actor.user?.uuid,
|
||||
appId: actor.effectiveApp?.uid,
|
||||
unchargedCost,
|
||||
unchargedSinceStart: this.#unchargedSinceStart,
|
||||
error: error ?? undefined,
|
||||
},
|
||||
'info',
|
||||
);
|
||||
}
|
||||
|
||||
private maybeAlertOveruse(ctx: {
|
||||
actor: Actor;
|
||||
userId: string;
|
||||
|
||||
@@ -55,7 +55,10 @@ import type { IConfig } from '../../types.js';
|
||||
*/
|
||||
|
||||
export type NotifWire =
|
||||
'notif.message' | 'notif.unreads' | 'notif.ack' | 'notif.removed';
|
||||
| 'notif.message'
|
||||
| 'notif.unreads'
|
||||
| 'notif.ack'
|
||||
| 'notif.removed';
|
||||
|
||||
export const NOTIF_DELIVERY_EVENT = 'outer.notif.delivery';
|
||||
|
||||
@@ -197,7 +200,7 @@ export class NotificationSocketAdapter {
|
||||
});
|
||||
}
|
||||
|
||||
/** One notification, in the shape the GUI's socket listeners receive. */
|
||||
/** One notification, in the shape `UIDesktop` has always received. */
|
||||
message(userId: number, uid: string, notification: unknown): void {
|
||||
const response = { uid, notification };
|
||||
if (this.#dispatched(userId, 'notif.message', response)) return;
|
||||
|
||||
@@ -17,7 +17,6 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import MockRedis from 'ioredis-mock';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { makeActor, type Actor } from '../../core/actor.js';
|
||||
@@ -28,16 +27,6 @@ import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import { kv } from '../../util/kvSingleton.js';
|
||||
import { PermissionService } from './PermissionService.js';
|
||||
|
||||
// The servers here share one redis mock; start each on an empty one.
|
||||
const setupFreshServer = async (
|
||||
...args: Parameters<typeof setupTestServer>
|
||||
): Promise<PuterServer> => {
|
||||
const redis = new MockRedis();
|
||||
await redis.flushall();
|
||||
redis.disconnect();
|
||||
return setupTestServer(...args);
|
||||
};
|
||||
|
||||
/** `default_temp_group` from config.default.json. */
|
||||
const DEFAULT_TEMP_GROUP_UID = 'b7220104-7905-4985-b996-649fdcdb3c8f';
|
||||
|
||||
@@ -201,7 +190,7 @@ describe('PermissionService (integration)', () => {
|
||||
let permService: PermissionService;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupFreshServer();
|
||||
server = await setupTestServer();
|
||||
permService = server.services
|
||||
.permission as unknown as PermissionService;
|
||||
});
|
||||
@@ -1652,7 +1641,7 @@ describe('PermissionService — scan paths', () => {
|
||||
let permService: PermissionService;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupFreshServer();
|
||||
server = await setupTestServer();
|
||||
permService = server.services
|
||||
.permission as unknown as PermissionService;
|
||||
}, 60_000);
|
||||
@@ -1909,77 +1898,6 @@ describe('PermissionService — scan paths', () => {
|
||||
expect(await permService.check(originActor, permission)).toBe(true);
|
||||
expect(await permService.check(sharerActor, permission)).toBe(true);
|
||||
}, 20_000);
|
||||
|
||||
it('does not read a revoked group grant back from a lagging replica', async () => {
|
||||
const { row: issuer } = await makeGroupedUser();
|
||||
const { row: member, actor: memberActor } = await makeGroupedUser();
|
||||
const groupUid = uuidv4();
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `group` (`uid`, `owner_user_id`, `extra`, `metadata`) ' +
|
||||
'VALUES (?, ?, ?, ?)',
|
||||
[groupUid, issuer.id, '{}', '{}'],
|
||||
);
|
||||
const [group] = await server.clients.db.read(
|
||||
'SELECT `id` FROM `group` WHERE `uid` = ?',
|
||||
[groupUid],
|
||||
);
|
||||
await server.stores.group.addUsers(groupUid, [member.username]);
|
||||
|
||||
const permission = `zztest:lag-${uuidv4()}:ii:read`;
|
||||
await server.stores.permission.setFlatUserPerm(
|
||||
issuer.id,
|
||||
permission,
|
||||
{
|
||||
permission,
|
||||
deleted: false,
|
||||
issuer_user_id: issuer.id,
|
||||
} as never,
|
||||
);
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `user_to_group_permissions` ' +
|
||||
'(`user_id`, `group_id`, `permission`, `extra`) VALUES (?, ?, ?, ?)',
|
||||
[issuer.id, group.id, permission, '{}'],
|
||||
);
|
||||
expect(await permService.check(memberActor, permission)).toBe(true);
|
||||
|
||||
await server.stores.permission.deleteUserGroupPerm(
|
||||
Number(group.id),
|
||||
issuer.id,
|
||||
permission,
|
||||
);
|
||||
await permService.bumpPermissionCacheForUsernames([
|
||||
member.username,
|
||||
]);
|
||||
|
||||
// The replica still returns the row; the primary has deleted it.
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi
|
||||
.spyOn(db, 'pread')
|
||||
.mockImplementation((sql, params) => primaryRead(sql, params));
|
||||
const read = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql, params) =>
|
||||
String(sql).includes('user_to_group_permissions')
|
||||
? [
|
||||
{
|
||||
permission,
|
||||
user_id: issuer.id,
|
||||
group_id: group.id,
|
||||
extra: '{}',
|
||||
},
|
||||
]
|
||||
: primaryRead(sql, params),
|
||||
);
|
||||
try {
|
||||
expect(await permService.check(memberActor, permission)).toBe(
|
||||
false,
|
||||
);
|
||||
} finally {
|
||||
read.mockRestore();
|
||||
pread.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('app-under-user grants', () => {
|
||||
@@ -2688,7 +2606,7 @@ describe('PermissionService — default user permissions vs. group config', () =
|
||||
// Self-hosted deployments may run without default groups. The floor
|
||||
// is not group-derived, so it applies regardless — a deployment that
|
||||
// clears both no longer has every driver call fail closed.
|
||||
const bare = await setupFreshServer({
|
||||
const bare = await setupTestServer({
|
||||
default_user_group: '',
|
||||
default_temp_group: '',
|
||||
} as never);
|
||||
|
||||
@@ -1155,13 +1155,9 @@ export class PermissionService extends PuterService {
|
||||
}
|
||||
|
||||
/** Batched: one event for the whole group, not one per member. */
|
||||
async #bumpGroupCacheGeneration(
|
||||
groupId: number,
|
||||
memberUuids?: string[],
|
||||
): Promise<void> {
|
||||
async #bumpGroupCacheGeneration(groupId: number): Promise<void> {
|
||||
const uuids =
|
||||
memberUuids ??
|
||||
(await this.stores.permission.listGroupMemberUuids(groupId));
|
||||
await this.stores.permission.listGroupMemberUuids(groupId);
|
||||
if (uuids.length === 0) return;
|
||||
await this.stores.permission.bumpCacheGenerations(
|
||||
uuids.map((uuid) => `user:${uuid}`),
|
||||
@@ -1242,12 +1238,11 @@ export class PermissionService extends PuterService {
|
||||
|
||||
await this.#assertCanManage(actor, permission);
|
||||
|
||||
const { removed: revoked, members } =
|
||||
await this.stores.permission.deleteUserGroupPerm(
|
||||
groupId,
|
||||
issuerId,
|
||||
permission,
|
||||
);
|
||||
const revoked = await this.stores.permission.deleteUserGroupPerm(
|
||||
groupId,
|
||||
issuerId,
|
||||
permission,
|
||||
);
|
||||
|
||||
this.#audit(
|
||||
this.stores.permission.auditUserGroupPerm({
|
||||
@@ -1262,18 +1257,15 @@ export class PermissionService extends PuterService {
|
||||
);
|
||||
|
||||
// Bumped even when nothing matched: a cached allow must not survive.
|
||||
await this.#bumpGroupCacheGeneration(
|
||||
groupId,
|
||||
members
|
||||
.map((member) => member.uuid)
|
||||
.filter((uuid): uuid is string => Boolean(uuid)),
|
||||
);
|
||||
await this.#bumpGroupCacheGeneration(groupId);
|
||||
|
||||
// Nothing but the membership names the holders, so without this their
|
||||
// watches outlive the revoke.
|
||||
if (revoked) {
|
||||
for (const member of members) {
|
||||
this.#announceRevoked(member.id, null, permission);
|
||||
for (const memberId of await this.stores.permission.listGroupMemberIds(
|
||||
groupId,
|
||||
)) {
|
||||
this.#announceRevoked(memberId, null, permission);
|
||||
}
|
||||
}
|
||||
return revoked;
|
||||
|
||||
@@ -18,7 +18,6 @@
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
import v8 from 'node:v8';
|
||||
import type { Request, Response } from 'express';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
@@ -230,37 +229,6 @@ describe('ProfileService', () => {
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
// V8 stops optimizing regexps once a process holds enough compiled code,
|
||||
// as long-running servers do, and an unoptimized one overflows on this.
|
||||
it('answers a picture of several megabytes with 413', () => {
|
||||
const picture = `data:image/png;base64,${'A'.repeat(6 * 1024 * 1024)}`;
|
||||
v8.setFlagsFromString('--no-regexp-optimization');
|
||||
try {
|
||||
expect(() => service.validatePatch({ picture })).toThrow(
|
||||
expect.objectContaining({
|
||||
statusCode: 413,
|
||||
code: 'profile_picture_too_large',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
v8.setFlagsFromString('--regexp-optimization');
|
||||
}
|
||||
});
|
||||
|
||||
it('accepts a picture of the largest size, with or without padding', () => {
|
||||
const prefix = 'data:image/png;base64,';
|
||||
const full = `${prefix}${'A'.repeat(PROFILE_PICTURE_MAX_BYTES - prefix.length)}`;
|
||||
expect(service.validatePatch({ picture: full })).toEqual({
|
||||
picture: full,
|
||||
});
|
||||
expect(service.validatePatch({ picture: `${prefix}QQ` })).toEqual({
|
||||
picture: `${prefix}QQ`,
|
||||
});
|
||||
expect(() => service.validatePatch({ picture: `${full}A` })).toThrow(
|
||||
expect.objectContaining({ code: 'profile_picture_too_large' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps what validates from a legacy profile and names the rest', () => {
|
||||
expect(
|
||||
service.normalizeLegacyProfile({
|
||||
|
||||
@@ -83,14 +83,6 @@ const validateField = (field: ProfileField, value: unknown): string | null => {
|
||||
);
|
||||
}
|
||||
if (field === 'picture') {
|
||||
// Sized before the pattern runs, which fails outright on megabytes of
|
||||
// text. Only ASCII passes the pattern, so the length is the byte count.
|
||||
if (value.length > PROFILE_PICTURE_MAX_BYTES) {
|
||||
throw new HttpError(413, 'picture is too large', {
|
||||
code: 'profile_picture_too_large',
|
||||
fields: { field, maxBytes: PROFILE_PICTURE_MAX_BYTES },
|
||||
});
|
||||
}
|
||||
if (!PICTURE_DATA_URL.test(value)) {
|
||||
throw invalid(
|
||||
field,
|
||||
@@ -98,6 +90,12 @@ const validateField = (field: ProfileField, value: unknown): string | null => {
|
||||
'profile_picture_invalid',
|
||||
);
|
||||
}
|
||||
if (Buffer.byteLength(value) > PROFILE_PICTURE_MAX_BYTES) {
|
||||
throw new HttpError(413, 'picture is too large', {
|
||||
code: 'profile_picture_too_large',
|
||||
fields: { field, maxBytes: PROFILE_PICTURE_MAX_BYTES },
|
||||
});
|
||||
}
|
||||
return value;
|
||||
}
|
||||
const maxLength =
|
||||
|
||||
@@ -2354,35 +2354,6 @@ describe('OpenAIRealtimeRelayService relay', () => {
|
||||
b.ws.close();
|
||||
});
|
||||
|
||||
it('evicts an access token revoked on another node', async () => {
|
||||
const row = await server.stores.user.getByUsername(user.username);
|
||||
const actor = makeActor({ user: row! });
|
||||
const token = await server.services.auth.createAccessToken(actor, [
|
||||
[FULL_API_ACCESS],
|
||||
]);
|
||||
const { token_uid } = server.services.token.verify<{
|
||||
token_uid: string;
|
||||
}>('auth', token);
|
||||
const c = open(`${relayUrl}?auth_token=${token}`);
|
||||
await c.waitFor('session.created');
|
||||
|
||||
// As a peer cluster delivers it, after a malformed payload it ignores.
|
||||
server.clients.event.emit(
|
||||
'outer.auth.access-token.revoked',
|
||||
{ token_uid: 42 } as never,
|
||||
{ from_outside: true },
|
||||
);
|
||||
server.clients.event.emit(
|
||||
'outer.auth.access-token.revoked',
|
||||
{ token_uid },
|
||||
{ from_outside: true },
|
||||
);
|
||||
expect(await c.waitFor('error')).toMatchObject({
|
||||
error: { code: 'invalid_api_key' },
|
||||
});
|
||||
await c.closed;
|
||||
});
|
||||
|
||||
it('closes with a retryable error after a sustained verification outage', async () => {
|
||||
const c = open(`${relayUrl}?auth_token=${user.token}`);
|
||||
await c.waitFor('session.created');
|
||||
|
||||
@@ -838,19 +838,13 @@ export class OpenAIRealtimeRelayService extends PuterService {
|
||||
{},
|
||||
);
|
||||
});
|
||||
this.clients.event.on(
|
||||
'outer.auth.access-token.revoked',
|
||||
(_key, data) => {
|
||||
const tokenUid = (data as { token_uid?: unknown } | null)
|
||||
?.token_uid;
|
||||
if (typeof tokenUid !== 'string' || !tokenUid) return;
|
||||
this.clients.event.emit(
|
||||
'pubsub.realtime.auth-revoked',
|
||||
{ tokenUid },
|
||||
{},
|
||||
);
|
||||
},
|
||||
);
|
||||
this.clients.event.on('auth.access-token.revoked', (_key, data) => {
|
||||
this.clients.event.emit(
|
||||
'pubsub.realtime.auth-revoked',
|
||||
{ tokenUid: data.token_uid },
|
||||
{},
|
||||
);
|
||||
});
|
||||
// Raw WebSockets do not participate in socket.io's cluster adapter.
|
||||
this.clients.event.on('pubsub.realtime.auth-revoked', (_key, data) => {
|
||||
const revoked = data.sessionUids
|
||||
|
||||
@@ -1270,13 +1270,9 @@ describe('ShareService', () => {
|
||||
expect(listByFsentry).not.toHaveBeenCalled();
|
||||
expect(getEntriesByIds).not.toHaveBeenCalled();
|
||||
const prefixes = [`fs:${file.uuid}`, `manage:fs:${file.uuid}`];
|
||||
expect(revokeUserGrants).toHaveBeenCalledWith(prefixes, {
|
||||
primary: true,
|
||||
});
|
||||
expect(revokeGroupGrants).toHaveBeenCalledWith(prefixes, {
|
||||
primary: true,
|
||||
});
|
||||
expect(deleteRows).toHaveBeenCalledWith([file.id], [file.uuid]);
|
||||
expect(revokeUserGrants).toHaveBeenCalledWith(prefixes);
|
||||
expect(revokeGroupGrants).toHaveBeenCalledWith(prefixes);
|
||||
expect(deleteRows).toHaveBeenCalledWith([file.id]);
|
||||
// Grants go before the rows that record them.
|
||||
expect(
|
||||
revokeUserGrants.mock.invocationCallOrder[0]!,
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
|
||||
import { dirname as pathDirname } from 'node:path/posix';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
@@ -240,161 +240,6 @@ describe('ShareService: deleting a shared item', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('reads no grants or shares from the primary for an owner with nothing shared', async () => {
|
||||
const owner = await makeUser();
|
||||
const { dir } = await makeDirWithFile(owner.user);
|
||||
const trash = await dirAt(`/${owner.user.username}/Trash`);
|
||||
|
||||
const pread = vi.spyOn(server.clients.db, 'pread');
|
||||
try {
|
||||
await move(owner, dir, trash);
|
||||
const retireReads = pread.mock.calls
|
||||
.map(([sql]) => String(sql))
|
||||
.filter((sql) =>
|
||||
/`user_to_user_permissions`|`user_to_group_permissions`|FROM `share`/u.test(
|
||||
sql,
|
||||
),
|
||||
);
|
||||
expect(retireReads).toEqual([]);
|
||||
} finally {
|
||||
pread.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('withdraws a share made just now that the replica has not seen', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
const { dir, file } = await makeDirWithFile(owner.user);
|
||||
await share(owner.actor, {
|
||||
uid: dir.uuid,
|
||||
recipient: { username: recipient.user.username },
|
||||
mode: 'read',
|
||||
});
|
||||
expect(await canRead(recipient.actor, file.path)).toBe(true);
|
||||
const trash = await dirAt(`/${owner.user.username}/Trash`);
|
||||
|
||||
// The replica has neither the grant nor the share row yet.
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi.spyOn(db, 'pread').mockImplementation(primaryRead);
|
||||
const read = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql, params) =>
|
||||
/`user_to_user_permissions`|FROM `share`/u.test(String(sql))
|
||||
? []
|
||||
: primaryRead(sql, params),
|
||||
);
|
||||
let moved;
|
||||
try {
|
||||
moved = await move(owner, dir, trash);
|
||||
} finally {
|
||||
read.mockRestore();
|
||||
pread.mockRestore();
|
||||
}
|
||||
|
||||
expect(
|
||||
await canRead(recipient.actor, `${moved.path}/${file.name}`),
|
||||
).toBe(false);
|
||||
expect(
|
||||
await server.stores.share.listAllByFsentrySubtree(dir.id),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("does not honour a lagging replica's copy of a link the delete removed", async () => {
|
||||
const owner = await makeUser();
|
||||
const stranger = await makeUser();
|
||||
const { dir, file } = await makeDirWithFile(owner.user);
|
||||
// The owner's plan covers link sharing.
|
||||
const plan = vi
|
||||
.spyOn(server.services.metering, 'getActorSubscription')
|
||||
.mockResolvedValue({ id: 'business' } as never);
|
||||
await server.stores.share.upsertAnyone({
|
||||
issuerUserId: owner.user.id,
|
||||
fsentryId: dir.id,
|
||||
mode: 'read',
|
||||
});
|
||||
expect(await canRead(stranger.actor, file.path)).toBe(true);
|
||||
|
||||
const trash = await dirAt(`/${owner.user.username}/Trash`);
|
||||
const moved = await move(owner, dir, trash);
|
||||
|
||||
// The replica still has the link row the primary just deleted.
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi.spyOn(db, 'pread').mockImplementation(primaryRead);
|
||||
const read = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql, params) =>
|
||||
String(sql).includes('`share`.`anyone` = 1')
|
||||
? [
|
||||
{
|
||||
mode: 'read',
|
||||
entry_uuid: dir.uuid,
|
||||
owner_user_id: owner.user.id,
|
||||
},
|
||||
]
|
||||
: primaryRead(sql, params),
|
||||
);
|
||||
try {
|
||||
expect(
|
||||
await canRead(stranger.actor, `${moved.path}/${file.name}`),
|
||||
).toBe(false);
|
||||
} finally {
|
||||
read.mockRestore();
|
||||
pread.mockRestore();
|
||||
plan.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("drops a team member's cached access when the replica lacks them", async () => {
|
||||
const owner = await makeUser();
|
||||
const member = await makeUser();
|
||||
const file = await makeFile(owner.user);
|
||||
const groupUid = uuidv4();
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `group` (`uid`, `owner_user_id`, `kind`, `name`, ' +
|
||||
'`extra`, `metadata`) VALUES (?, ?, ?, ?, ?, ?)',
|
||||
[groupUid, owner.user.id, 'team', 'Team', '{}', '{}'],
|
||||
);
|
||||
const [group] = (await server.clients.db.read(
|
||||
'SELECT id FROM `group` WHERE uid = ?',
|
||||
[groupUid],
|
||||
)) as { id: number }[];
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[member.user.id, group.id],
|
||||
);
|
||||
await server.stores.permission.upsertUserGroupPerm(
|
||||
Number(group.id),
|
||||
owner.user.id,
|
||||
`fs:${file.uuid}:read`,
|
||||
{},
|
||||
);
|
||||
|
||||
// The replica has not seen the member join yet.
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi.spyOn(db, 'pread').mockImplementation(primaryRead);
|
||||
const read = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql, params) =>
|
||||
String(sql).includes('jct_user_group')
|
||||
? []
|
||||
: primaryRead(sql, params),
|
||||
);
|
||||
const bump = vi.spyOn(server.stores.permission, 'bumpCacheGenerations');
|
||||
try {
|
||||
await server.services.share.onEntryDeleted(file.uuid);
|
||||
expect(bump.mock.calls.flat(2)).toContain(
|
||||
`user:${member.user.uuid}`,
|
||||
);
|
||||
} finally {
|
||||
bump.mockRestore();
|
||||
read.mockRestore();
|
||||
pread.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('does not answer whether a name is already in the owner\'s Trash', async () => {
|
||||
const owner = await makeUser();
|
||||
const recipient = await makeUser();
|
||||
|
||||
@@ -292,9 +292,6 @@ export const modeFromEntryPermission = (permission: string): string | null => {
|
||||
*/
|
||||
const RETIRE_CHUNK_SIZE = 100;
|
||||
|
||||
/** How long after an owner shares the retire reads stay on the primary. */
|
||||
const RECENT_SHARE_WINDOW_MS = 30_000;
|
||||
|
||||
/** Shares one user may create per UTC day, absent a config override. */
|
||||
export const DEFAULT_DAILY_SHARE_LIMIT = 200;
|
||||
|
||||
@@ -400,11 +397,6 @@ export class ShareService extends PuterService {
|
||||
* fleet-wide; its TTL covers a process the bump missed.
|
||||
*/
|
||||
readonly #ownerShares = new SubscriptionCache();
|
||||
/** Owners who shared just now: a replica may not have the row yet. */
|
||||
readonly #recentSharers = new SubscriptionCache(
|
||||
undefined,
|
||||
RECENT_SHARE_WINDOW_MS,
|
||||
);
|
||||
readonly #ownerLookups = new Map<string, Promise<boolean>>();
|
||||
|
||||
/**
|
||||
@@ -627,11 +619,6 @@ export class ShareService extends PuterService {
|
||||
const chunk = entries.slice(i, i + RETIRE_CHUNK_SIZE);
|
||||
const removed = await this.onEntryDeleted(
|
||||
chunk.map((entry) => entry.uuid),
|
||||
{
|
||||
primary: this.#sharedRecently(
|
||||
chunk.map((entry) => entry.userId),
|
||||
),
|
||||
},
|
||||
);
|
||||
|
||||
const holdersByEntry = new Map<string, Set<number>>();
|
||||
@@ -945,16 +932,6 @@ export class ShareService extends PuterService {
|
||||
this.#ownerShares.generationOf(ownerUserId),
|
||||
true,
|
||||
);
|
||||
this.#recentSharers.write(
|
||||
ownerUserId,
|
||||
this.#recentSharers.generationOf(ownerUserId),
|
||||
true,
|
||||
);
|
||||
}
|
||||
|
||||
/** Whether this process heard any of these owners share within the window. */
|
||||
#sharedRecently(ownerUserIds: number[]): boolean {
|
||||
return ownerUserIds.some((id) => this.#recentSharers.read(id) === true);
|
||||
}
|
||||
|
||||
async #emitGui(
|
||||
@@ -1578,9 +1555,7 @@ export class ShareService extends PuterService {
|
||||
// Every row, not just the applied ones: a node whose only share is a
|
||||
// pending invite or a team grant is exactly what the group and invite
|
||||
// sweeps ask about.
|
||||
const rows = await this.stores.share.listAllByFsentrySubtree(entry.id, {
|
||||
primary: true,
|
||||
});
|
||||
const rows = await this.stores.share.listAllByFsentrySubtree(entry.id);
|
||||
const nodes = await this.stores.fsEntry.getEntriesByIds(
|
||||
rows.map((row: { fsentry_id: number }) => Number(row.fsentry_id)),
|
||||
);
|
||||
@@ -1804,13 +1779,9 @@ export class ShareService extends PuterService {
|
||||
/**
|
||||
* Retire the grants pointing at a node that no longer exists. Returns the
|
||||
* rows removed, which is the only record of who had access — the index rows
|
||||
* cascade away with the fsentry. Reads a replica unless `primary` says a
|
||||
* grant may be too new for one.
|
||||
* cascade away with the fsentry.
|
||||
*/
|
||||
async onEntryDeleted(
|
||||
entryUids: string | string[],
|
||||
{ primary = false }: { primary?: boolean } = {},
|
||||
): Promise<
|
||||
async onEntryDeleted(entryUids: string | string[]): Promise<
|
||||
Array<{
|
||||
holder_user_id: number;
|
||||
issuer_user_id: number;
|
||||
@@ -1825,7 +1796,6 @@ export class ShareService extends PuterService {
|
||||
const removed =
|
||||
await this.stores.permission.deleteUserUserPermsByPermissionPrefixes(
|
||||
uids.flatMap((uid) => [`fs:${uid}`, `manage:fs:${uid}`]),
|
||||
{ primary },
|
||||
);
|
||||
|
||||
// Retiring the rows is not enough: a holder's cached scan still answers
|
||||
@@ -1838,14 +1808,19 @@ export class ShareService extends PuterService {
|
||||
|
||||
// Group grants on the same node, and their members' caches: without
|
||||
// this they outlive the file and keep answering "allowed".
|
||||
const { members } =
|
||||
const removedGroups =
|
||||
await this.stores.permission.deleteUserGroupPermsByPermissionPrefixes(
|
||||
uids.flatMap((uid) => [`fs:${uid}`, `manage:fs:${uid}`]),
|
||||
{ primary },
|
||||
);
|
||||
const memberUuids = members
|
||||
.map((member) => member.uuid)
|
||||
.filter((uuid): uuid is string => Boolean(uuid));
|
||||
const memberUuids = (
|
||||
await Promise.all(
|
||||
[...new Set(removedGroups.map((row) => Number(row.group_id)))]
|
||||
.filter((id) => Number.isFinite(id))
|
||||
.map((id) =>
|
||||
this.stores.permission.listGroupMemberUuids(id),
|
||||
),
|
||||
)
|
||||
).flat();
|
||||
|
||||
await this.stores.permission.bumpCacheGenerations([
|
||||
...new Set([
|
||||
@@ -1902,23 +1877,15 @@ export class ShareService extends PuterService {
|
||||
* the subtree stands on its own, and one left behind keeps granting the
|
||||
* issuer's recipients access under an owner who never agreed to them.
|
||||
*/
|
||||
async onEntryOwnerChanged(
|
||||
entry: FSEntry,
|
||||
{ primary = true }: { primary?: boolean } = {},
|
||||
): Promise<void> {
|
||||
async onEntryOwnerChanged(entry: FSEntry): Promise<void> {
|
||||
// A file's share rows can only name the file itself, so there is
|
||||
// nothing to list or hydrate.
|
||||
if (!entry.isDir) {
|
||||
await this.onEntryDeleted([entry.uuid], { primary });
|
||||
await this.stores.share.deleteByFsentryIds(
|
||||
[entry.id],
|
||||
[entry.uuid],
|
||||
);
|
||||
await this.onEntryDeleted([entry.uuid]);
|
||||
await this.stores.share.deleteByFsentryIds([entry.id]);
|
||||
return;
|
||||
}
|
||||
const rows = await this.stores.share.listAllByFsentrySubtree(entry.id, {
|
||||
primary,
|
||||
});
|
||||
const rows = await this.stores.share.listAllByFsentrySubtree(entry.id);
|
||||
const fsentryIds = [
|
||||
...new Set([
|
||||
entry.id,
|
||||
@@ -1930,8 +1897,8 @@ export class ShareService extends PuterService {
|
||||
|
||||
const nodes = await this.stores.fsEntry.getEntriesByIds(fsentryIds);
|
||||
const uuids = [...nodes.values()].map((node) => node.uuid);
|
||||
if (uuids.length > 0) await this.onEntryDeleted(uuids, { primary });
|
||||
await this.stores.share.deleteByFsentryIds(fsentryIds, uuids);
|
||||
if (uuids.length > 0) await this.onEntryDeleted(uuids);
|
||||
await this.stores.share.deleteByFsentryIds(fsentryIds);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1946,11 +1913,7 @@ export class ShareService extends PuterService {
|
||||
async onEntryTrashed(entry: FSEntry): Promise<void> {
|
||||
// The same sweep as a change of owner: every kind of row in the
|
||||
// subtree, since a link share deeper down grants access on its own.
|
||||
// Nearly every trashed entry has no shares, so the reads stay off the
|
||||
// primary unless its owner shared just now.
|
||||
await this.onEntryOwnerChanged(entry, {
|
||||
primary: this.#sharedRecently([entry.userId]),
|
||||
});
|
||||
await this.onEntryOwnerChanged(entry);
|
||||
}
|
||||
|
||||
// -- Reads --------------------------------------------------------
|
||||
@@ -3052,7 +3015,6 @@ export class ShareService extends PuterService {
|
||||
mode,
|
||||
issuerAppUid: this.#actingAppUid(actor),
|
||||
});
|
||||
this.#ownerShared(entry.userId);
|
||||
return {
|
||||
...this.#resolve(row, entry, actor, { username: null }),
|
||||
anyone: true,
|
||||
@@ -3352,7 +3314,6 @@ export class ShareService extends PuterService {
|
||||
mode,
|
||||
issuerAppUid: this.#actingAppUid(actor),
|
||||
});
|
||||
this.#ownerShared(entry.userId);
|
||||
return {
|
||||
...this.#resolve(row, entry, actor, { username: null }),
|
||||
pending: true,
|
||||
|
||||
@@ -428,43 +428,6 @@ describe('SocketService (live socket.io)', () => {
|
||||
sessionSocket.disconnect();
|
||||
});
|
||||
|
||||
it('drops a token`s socket on a revocation that reached it from elsewhere', async () => {
|
||||
const row = await server.stores.user.getByUsername(user.username);
|
||||
const actor = makeActor({ user: row as never });
|
||||
const doomed = await runWithContext({ actor }, () =>
|
||||
server.services.auth.createAccessToken(actor, [[FULL_API_ACCESS]], {
|
||||
label: 'revoke-fanout-probe',
|
||||
}),
|
||||
);
|
||||
const { token_uid } = server.services.token.verify<{
|
||||
token_uid: string;
|
||||
}>('auth', doomed);
|
||||
const tokenSocket = await connect({ auth_token: doomed });
|
||||
expect(tokenSocket.connected).toBe(true);
|
||||
|
||||
// A malformed peer payload is ignored.
|
||||
for (const data of [null, {}, { token_uid: 42 }]) {
|
||||
server.clients.event.emit(
|
||||
'outer.auth.access-token.revoked',
|
||||
data as never,
|
||||
{ from_outside: true },
|
||||
);
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 200));
|
||||
expect(tokenSocket.connected).toBe(true);
|
||||
|
||||
// As a peer cluster delivers it: the token was revoked on another node.
|
||||
server.clients.event.emit(
|
||||
'outer.auth.access-token.revoked',
|
||||
{ token_uid },
|
||||
{ from_outside: true },
|
||||
);
|
||||
|
||||
await vi.waitFor(() => expect(tokenSocket.connected).toBe(false), {
|
||||
timeout: 5000,
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects an app-under-user token while events are off', async () => {
|
||||
const row = await server.stores.user.getByUsername(user.username);
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
@@ -963,31 +926,6 @@ describe('SocketService (live socket.io)', () => {
|
||||
socket.disconnect();
|
||||
});
|
||||
|
||||
it('keeps a connection up when its re-check lookup fails', async () => {
|
||||
const user = await createTestUser(server, {
|
||||
username: 'sock-lookup-down',
|
||||
password: 'sock-lookup-down-password',
|
||||
});
|
||||
const socket = await connect({ auth_token: `Bearer ${user.token}` });
|
||||
const authenticate = vi
|
||||
.spyOn(server.services.auth, 'authenticate')
|
||||
.mockRejectedValue(
|
||||
Object.assign(new Error('Database operation failed'), {
|
||||
code: 'dbBatchFailed',
|
||||
}),
|
||||
);
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
try {
|
||||
await socketService.reauthenticateSockets();
|
||||
await new Promise((r) => setTimeout(r, 200));
|
||||
expect(socket.connected).toBe(true);
|
||||
} finally {
|
||||
authenticate.mockRestore();
|
||||
warn.mockRestore();
|
||||
socket.disconnect();
|
||||
}
|
||||
});
|
||||
|
||||
it('re-checks without recording session activity', async () => {
|
||||
const idle = await createTestUser(server, {
|
||||
username: 'sock-idle',
|
||||
|
||||
@@ -982,22 +982,15 @@ export class SocketService extends PuterService {
|
||||
},
|
||||
);
|
||||
const decisions = new Map<string, SocketAuthDecision>();
|
||||
let unchecked = 0;
|
||||
checked.forEach((token, i) => {
|
||||
const outcome = outcomes[i];
|
||||
// A lookup that failed says nothing about the credential, so its
|
||||
// sockets stay up until a later sweep gets an answer.
|
||||
if (outcome?.status === 'fulfilled') {
|
||||
decisions.set(token, outcome.value);
|
||||
} else {
|
||||
unchecked++;
|
||||
}
|
||||
});
|
||||
if (unchecked > 0) {
|
||||
console.warn(
|
||||
`[socket] reauth sweep could not check ${unchecked} credential(s)`,
|
||||
decisions.set(
|
||||
token,
|
||||
outcome?.status === 'fulfilled'
|
||||
? outcome.value
|
||||
: { reject: new Error('socket reauth failed') },
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
for (const socket of sockets) {
|
||||
const decision = socket.authToken
|
||||
@@ -1182,12 +1175,10 @@ export class SocketService extends PuterService {
|
||||
);
|
||||
|
||||
this.clients.event.on(
|
||||
'outer.auth.access-token.revoked',
|
||||
'auth.access-token.revoked',
|
||||
(_key: string, data: unknown) => {
|
||||
const tokenUid = (data as { token_uid?: unknown } | null)
|
||||
?.token_uid;
|
||||
if (typeof tokenUid !== 'string') return;
|
||||
this.#evictAccessTokenSockets(tokenUid).catch(
|
||||
const { token_uid } = data as { token_uid: string };
|
||||
this.#evictAccessTokenSockets(token_uid).catch(
|
||||
(err: unknown) => {
|
||||
console.error('[socket] token eviction failed', err);
|
||||
},
|
||||
|
||||
@@ -26,7 +26,6 @@ import {
|
||||
PuterStore,
|
||||
} from '../types';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { runWithConcurrencyLimit } from '../../util/concurrency.js';
|
||||
import { isUniqueViolation } from '../../util/dbError.js';
|
||||
import {
|
||||
validateUrl,
|
||||
@@ -90,8 +89,6 @@ const STATS_CACHE_TTL_SECONDS = 30 * 60;
|
||||
// `app-` + a UUID. ClickHouse's `app_opens.app_uid` is FixedString(40) and
|
||||
// fails the whole query on a longer value.
|
||||
const APP_UID_MAX_BYTES = 40;
|
||||
// ClickHouse caps the size of one query parameter, and the uid list is one.
|
||||
const CLICKHOUSE_UIDS_PER_QUERY = 1000;
|
||||
|
||||
// Period helpers for detailed/grouped stats. Ported from v1
|
||||
// AppInformationService — queries go straight to ClickHouse/MySQL on demand
|
||||
@@ -1730,30 +1727,20 @@ export class AppStore extends PuterStore {
|
||||
|
||||
const clickhouse = this.clients.clickhouse;
|
||||
if (clickhouse) {
|
||||
const chunks = [];
|
||||
for (let i = 0; i < uids.length; i += CLICKHOUSE_UIDS_PER_QUERY) {
|
||||
chunks.push(uids.slice(i, i + CLICKHOUSE_UIDS_PER_QUERY));
|
||||
}
|
||||
const results = await runWithConcurrencyLimit(
|
||||
chunks,
|
||||
4,
|
||||
async (chunk) => {
|
||||
const res = await clickhouse.query({
|
||||
query: `
|
||||
SELECT app_uid,
|
||||
uniqExact((user_id, intDiv(ts, ${APP_OPEN_COUNT_PERIOD_SECONDS}))) AS open_count,
|
||||
count(DISTINCT user_id) AS user_count
|
||||
FROM app_opens
|
||||
WHERE app_uid IN {uids:Array(String)}
|
||||
GROUP BY app_uid
|
||||
`,
|
||||
query_params: { uids: chunk.map((u) => String(u)) },
|
||||
format: 'JSONEachRow',
|
||||
});
|
||||
return res.json();
|
||||
},
|
||||
);
|
||||
for (const row of results.flat()) {
|
||||
const res = await clickhouse.query({
|
||||
query: `
|
||||
SELECT app_uid,
|
||||
uniqExact((user_id, intDiv(ts, ${APP_OPEN_COUNT_PERIOD_SECONDS}))) AS open_count,
|
||||
count(DISTINCT user_id) AS user_count
|
||||
FROM app_opens
|
||||
WHERE app_uid IN {uids:Array(String)}
|
||||
GROUP BY app_uid
|
||||
`,
|
||||
query_params: { uids: uids.map((u) => String(u)) },
|
||||
format: 'JSONEachRow',
|
||||
});
|
||||
const rows = await res.json();
|
||||
for (const row of rows) {
|
||||
out.set(row.app_uid, {
|
||||
open_count: parseInt(row.open_count, 10) || 0,
|
||||
user_count: parseInt(row.user_count, 10) || 0,
|
||||
|
||||
@@ -200,37 +200,6 @@ describe('AppStore app stats (cache-on-read)', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('splits a long uid list across ClickHouse queries', async () => {
|
||||
const uids = Array.from({ length: 2500 }, () => freshUid());
|
||||
const queried = [];
|
||||
|
||||
server.clients.clickhouse = {
|
||||
query: async ({ query_params }) => {
|
||||
queried.push(query_params.uids.length);
|
||||
return {
|
||||
json: async () =>
|
||||
query_params.uids.map((u) => ({
|
||||
app_uid: u,
|
||||
open_count: '2',
|
||||
user_count: '1',
|
||||
})),
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
try {
|
||||
const stats = await appStore.getAppsStats(uids);
|
||||
expect(queried.sort((a, b) => b - a)).toEqual([1000, 1000, 500]);
|
||||
expect(stats.size).toBe(2500);
|
||||
expect(stats.get(uids[2499])).toMatchObject({
|
||||
open_count: 2,
|
||||
user_count: 1,
|
||||
});
|
||||
} finally {
|
||||
delete server.clients.clickhouse;
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps ids longer than an app uid out of the ClickHouse query', async () => {
|
||||
const uid = freshUid();
|
||||
const longId = `iap-${'a'.repeat(36)}-extra`;
|
||||
|
||||
@@ -444,7 +444,7 @@ describe('FSEntryStore cache invalidation', () => {
|
||||
store.batchCreateEntries([
|
||||
{
|
||||
userId: user.userId,
|
||||
uuid: file.uuid,
|
||||
uuid: uuidv4(),
|
||||
path: file.path,
|
||||
size: 7,
|
||||
overwrite: true,
|
||||
@@ -1185,7 +1185,7 @@ describe('FSEntryStore entry creation', () => {
|
||||
const [updated] = await store.batchCreateEntries([
|
||||
{
|
||||
userId: user.userId,
|
||||
uuid: original.uuid,
|
||||
uuid: uuidv4(),
|
||||
path: `${user.home}/Documents/ow.txt`,
|
||||
size: 42,
|
||||
overwrite: true,
|
||||
@@ -1204,33 +1204,6 @@ describe('FSEntryStore entry creation', () => {
|
||||
expect(updated?.thumbnail).toBe('data:image/png;base64,AA');
|
||||
});
|
||||
|
||||
it('leaves the row alone when an overwrite brings a different object key', async () => {
|
||||
const user = await makeUser();
|
||||
const original = await createFile(
|
||||
user,
|
||||
`${user.home}/Documents/otherkey.txt`,
|
||||
{ size: 1 },
|
||||
);
|
||||
|
||||
// The bytes for this write sit under a key the row doesn't name, so
|
||||
// repointing bucket or size would describe an object that isn't there.
|
||||
const returned = await createFile(user, original.path, {
|
||||
size: 42,
|
||||
overwrite: true,
|
||||
bucket: 'elsewhere',
|
||||
bucketRegion: 'eu-central-1',
|
||||
});
|
||||
|
||||
expect(returned.uuid).toBe(original.uuid);
|
||||
await expect(
|
||||
store.getEntryByPath(original.path),
|
||||
).resolves.toMatchObject({
|
||||
uuid: original.uuid,
|
||||
size: 1,
|
||||
bucket: 'puter-local',
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses to clobber an existing entry without overwrite, or a directory with it', async () => {
|
||||
const user = await makeUser();
|
||||
await createFile(user, `${user.home}/Documents/keep.txt`);
|
||||
@@ -1254,10 +1227,7 @@ describe('FSEntryStore entry creation', () => {
|
||||
|
||||
it('propagates the underlying failure when an overwrite update fails', async () => {
|
||||
const user = await makeUser();
|
||||
const existing = await createFile(
|
||||
user,
|
||||
`${user.home}/Documents/upfail.txt`,
|
||||
);
|
||||
await createFile(user, `${user.home}/Documents/upfail.txt`);
|
||||
|
||||
const db = server.clients.db;
|
||||
const originalWrite = (Object.getPrototypeOf(db) as typeof db).write;
|
||||
@@ -1273,7 +1243,6 @@ describe('FSEntryStore entry creation', () => {
|
||||
await expect(
|
||||
createFile(user, `${user.home}/Documents/upfail.txt`, {
|
||||
overwrite: true,
|
||||
uuid: existing.uuid,
|
||||
}),
|
||||
).rejects.toThrow('update rejected');
|
||||
|
||||
@@ -3832,7 +3801,7 @@ describe('FSEntryStore lost-insert recovery', () => {
|
||||
spy.mockRestore();
|
||||
});
|
||||
|
||||
it('hands back the race winner untouched when overwrite was requested', async () => {
|
||||
it('updates the race winner in place when overwrite was requested', async () => {
|
||||
const user = await makeUser();
|
||||
const uuid = uuidv4();
|
||||
const spy = loseInsertRace(uuid, () =>
|
||||
@@ -3858,14 +3827,13 @@ describe('FSEntryStore lost-insert recovery', () => {
|
||||
]);
|
||||
spy.mockRestore();
|
||||
|
||||
// Our object is under `uuid`, not the winner's key, so the winner's
|
||||
// row is returned as it stands for the caller to move the bytes onto.
|
||||
// The winner's row survived; our payload was written onto it.
|
||||
expect(entry?.uuid).not.toBe(uuid);
|
||||
expect(entry?.size).not.toBe(77);
|
||||
expect(entry?.thumbnail).not.toBe('thumb');
|
||||
expect(entry?.size).toBe(77);
|
||||
expect(entry?.thumbnail).toBe('thumb');
|
||||
await expect(
|
||||
store.getEntryByPath(`${user.home}/Documents/recovered.txt`),
|
||||
).resolves.toMatchObject({ size: entry?.size, uuid: entry?.uuid });
|
||||
).resolves.toMatchObject({ size: 77, uuid: entry?.uuid });
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -1941,11 +1941,6 @@ export class FSEntryStore extends PuterStore {
|
||||
return entry;
|
||||
}
|
||||
|
||||
/**
|
||||
* An overwrite whose `uuid` is not the existing row's leaves that row
|
||||
* untouched and returns it: the caller's object is under another key, so
|
||||
* the row can't describe it until the caller moves the bytes.
|
||||
*/
|
||||
async batchCreateEntries(
|
||||
entries: FSEntryCreateInput[],
|
||||
createPaths = true,
|
||||
@@ -2079,10 +2074,6 @@ export class FSEntryStore extends PuterStore {
|
||||
{ legacyCode: 'cannot_overwrite_a_directory' },
|
||||
);
|
||||
}
|
||||
if (existingEntry.uuid !== entry.input.uuid) {
|
||||
updatedResultsByIndex.set(entry.index, existingEntry);
|
||||
continue;
|
||||
}
|
||||
|
||||
const updatedEntry = {
|
||||
...existingEntry,
|
||||
@@ -2407,10 +2398,6 @@ export class FSEntryStore extends PuterStore {
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (existing.uuid !== entry.input.uuid) {
|
||||
updatedResultsByIndex.set(entry.index, existing);
|
||||
continue;
|
||||
}
|
||||
|
||||
const updatedEntry: FSEntry = {
|
||||
...existing,
|
||||
|
||||
@@ -64,29 +64,6 @@ describe('PermissionStore', () => {
|
||||
);
|
||||
};
|
||||
|
||||
/** Replica reads of `table` see `stale`; the primary stays real. */
|
||||
const lagReplica = (
|
||||
table: string,
|
||||
stale: (sql: string) => Record<string, unknown>[],
|
||||
) => {
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi
|
||||
.spyOn(db, 'pread')
|
||||
.mockImplementation((sql, params) => primaryRead(sql, params));
|
||||
const read = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (sql, params) =>
|
||||
String(sql).includes(table)
|
||||
? stale(String(sql))
|
||||
: primaryRead(sql, params),
|
||||
);
|
||||
return () => {
|
||||
read.mockRestore();
|
||||
pread.mockRestore();
|
||||
};
|
||||
};
|
||||
|
||||
// -- Flat KV view --------------------------------------------------
|
||||
|
||||
describe('flat user permissions (KV view)', () => {
|
||||
@@ -934,341 +911,6 @@ describe('PermissionStore', () => {
|
||||
expect(await store.readUserGroupPerms(user.id, [])).toEqual([]);
|
||||
});
|
||||
|
||||
it('does not read a revoked grant back from a lagging replica', async () => {
|
||||
const issuer = await makeUser();
|
||||
const member = await makeUser();
|
||||
const groupId = await seedGroup(issuer.id);
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[member.id, groupId],
|
||||
);
|
||||
await seedGroupPerm(issuer.id, groupId, 'fs:lag:read');
|
||||
const [stale] = await store.readUserGroupPerms(member.id, [
|
||||
'fs:lag:read',
|
||||
]);
|
||||
const [staleForHolder] = await store.readUserGroupPermsForHolders(
|
||||
[member.id],
|
||||
['fs:lag:read'],
|
||||
);
|
||||
|
||||
await store.deleteUserGroupPerm(groupId, issuer.id, 'fs:lag:read');
|
||||
|
||||
// The replica still has the row the primary just deleted.
|
||||
const restore = lagReplica('user_to_group_permissions', (sql) =>
|
||||
sql.includes('holder_user_id')
|
||||
? [staleForHolder]
|
||||
: [{ ...stale, extra: '{}' }],
|
||||
);
|
||||
try {
|
||||
expect(
|
||||
await store.readUserGroupPerms(member.id, ['fs:lag:read']),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
await store.readUserGroupPermsForHolders(
|
||||
[member.id],
|
||||
['fs:lag:read'],
|
||||
),
|
||||
).toEqual([]);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
it('does not read grants removed with their node back from a lagging replica', async () => {
|
||||
const issuer = await makeUser();
|
||||
const member = await makeUser();
|
||||
const groupId = await seedGroup(issuer.id);
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[member.id, groupId],
|
||||
);
|
||||
const node = uuidv4();
|
||||
await seedGroupPerm(issuer.id, groupId, `fs:${node}:read`);
|
||||
const [stale] = await store.readUserGroupPerms(member.id, [
|
||||
`fs:${node}:read`,
|
||||
]);
|
||||
|
||||
await store.deleteUserGroupPermsByPermissionPrefixes([
|
||||
`fs:${node}`,
|
||||
]);
|
||||
|
||||
const restore = lagReplica('user_to_group_permissions', () => [
|
||||
{ ...stale, extra: '{}' },
|
||||
]);
|
||||
try {
|
||||
expect(
|
||||
await store.readUserGroupPerms(member.id, [
|
||||
`fs:${node}:read`,
|
||||
]),
|
||||
).toEqual([]);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
it('removes a grant the replica has not seen yet with its node, when asked to read the primary', async () => {
|
||||
const issuer = await makeUser();
|
||||
const groupId = await seedGroup(issuer.id);
|
||||
const node = uuidv4();
|
||||
await seedGroupPerm(issuer.id, groupId, `fs:${node}:read`);
|
||||
|
||||
const restore = lagReplica('user_to_group_permissions', () => []);
|
||||
let unasked;
|
||||
let removed;
|
||||
try {
|
||||
unasked = await store.deleteUserGroupPermsByPermissionPrefixes([
|
||||
`fs:${node}`,
|
||||
]);
|
||||
removed = await store.deleteUserGroupPermsByPermissionPrefixes(
|
||||
[`fs:${node}`],
|
||||
{ primary: true },
|
||||
);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
// Unasked, the replica's answer stands and the primary is left alone.
|
||||
expect(unasked?.rows).toEqual([]);
|
||||
expect(removed?.rows).toEqual([
|
||||
{ group_id: groupId, permission: `fs:${node}:read` },
|
||||
]);
|
||||
expect(
|
||||
await store.queryIssuerGroupPermsByPrefix(
|
||||
issuer.id,
|
||||
groupId,
|
||||
`fs:${node}`,
|
||||
),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it('sends a former member to the primary once membership changes', async () => {
|
||||
const issuer = await makeUser();
|
||||
const member = await makeUser();
|
||||
const groupId = await seedGroup(issuer.id);
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[member.id, groupId],
|
||||
);
|
||||
await seedGroupPerm(issuer.id, groupId, 'fs:left:read');
|
||||
const [stale] = await store.readUserGroupPerms(member.id, [
|
||||
'fs:left:read',
|
||||
]);
|
||||
await server.clients.db.write(
|
||||
'DELETE FROM `jct_user_group` WHERE `user_id` = ? AND `group_id` = ?',
|
||||
[member.id, groupId],
|
||||
);
|
||||
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
try {
|
||||
await store.markGroupMembershipChanged([member.id]);
|
||||
// Only tombstones go out; there is no cached value to drop.
|
||||
const sent = emit.mock.calls
|
||||
.filter(([name]) => name === 'outer.cacheUpdate')
|
||||
.flatMap(
|
||||
([, data]) => (data as { cacheKey: string[] }).cacheKey,
|
||||
);
|
||||
expect(sent).toEqual([
|
||||
`perms:u2g:${member.id}${CACHE_TOMBSTONE_SUFFIX}`,
|
||||
]);
|
||||
} finally {
|
||||
emit.mockRestore();
|
||||
}
|
||||
|
||||
const restore = lagReplica('user_to_group_permissions', () => [
|
||||
{ ...stale, extra: '{}' },
|
||||
]);
|
||||
try {
|
||||
expect(
|
||||
await store.readUserGroupPerms(member.id, ['fs:left:read']),
|
||||
).toEqual([]);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
const seedMember = async (issuerId: number, permission: string) => {
|
||||
const member = await makeUser();
|
||||
const groupId = await seedGroup(issuerId);
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[member.id, groupId],
|
||||
);
|
||||
await seedGroupPerm(issuerId, groupId, permission);
|
||||
return { member, groupId };
|
||||
};
|
||||
|
||||
const tombstoneSet = async (userId: number) =>
|
||||
(await server.clients.redis.exists(
|
||||
`perms:u2g:${userId}${CACHE_TOMBSTONE_SUFFIX}`,
|
||||
)) > 0;
|
||||
|
||||
it('sends member reads to the primary before the grant is deleted', async () => {
|
||||
const issuer = await makeUser();
|
||||
const node = uuidv4();
|
||||
const one = await seedMember(issuer.id, 'fs:order:read');
|
||||
const many = await seedMember(issuer.id, `fs:${node}:read`);
|
||||
|
||||
const db = server.clients.db;
|
||||
const primaryWrite = db.write.bind(db);
|
||||
const seen: boolean[] = [];
|
||||
const write = vi
|
||||
.spyOn(db, 'write')
|
||||
.mockImplementation(async (sql, params) => {
|
||||
if (String(sql).startsWith('DELETE FROM')) {
|
||||
seen.push(
|
||||
(await tombstoneSet(one.member.id)) ||
|
||||
(await tombstoneSet(many.member.id)),
|
||||
);
|
||||
}
|
||||
return primaryWrite(sql, params);
|
||||
});
|
||||
try {
|
||||
await store.deleteUserGroupPerm(
|
||||
one.groupId,
|
||||
issuer.id,
|
||||
'fs:order:read',
|
||||
);
|
||||
await store.deleteUserGroupPermsByPermissionPrefixes([
|
||||
`fs:${node}`,
|
||||
]);
|
||||
} finally {
|
||||
write.mockRestore();
|
||||
}
|
||||
expect(seen).toEqual([true, true]);
|
||||
});
|
||||
|
||||
it('leaves the grant in place when the member read fails', async () => {
|
||||
const issuer = await makeUser();
|
||||
const node = uuidv4();
|
||||
const one = await seedMember(issuer.id, 'fs:retry:read');
|
||||
const many = await seedMember(issuer.id, `fs:${node}:read`);
|
||||
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi
|
||||
.spyOn(db, 'pread')
|
||||
.mockImplementation(async (sql, params) => {
|
||||
if (String(sql).includes('`jct_user_group` ug')) {
|
||||
throw new Error('primary unavailable');
|
||||
}
|
||||
return primaryRead(sql, params);
|
||||
});
|
||||
try {
|
||||
await expect(
|
||||
store.deleteUserGroupPerm(
|
||||
one.groupId,
|
||||
issuer.id,
|
||||
'fs:retry:read',
|
||||
),
|
||||
).rejects.toThrow('primary unavailable');
|
||||
await expect(
|
||||
store.deleteUserGroupPermsByPermissionPrefixes([
|
||||
`fs:${node}`,
|
||||
]),
|
||||
).rejects.toThrow('primary unavailable');
|
||||
} finally {
|
||||
pread.mockRestore();
|
||||
}
|
||||
|
||||
// The retry still finds the grants and whose they were.
|
||||
expect(
|
||||
await store.deleteUserGroupPerm(
|
||||
one.groupId,
|
||||
issuer.id,
|
||||
'fs:retry:read',
|
||||
),
|
||||
).toEqual({
|
||||
removed: true,
|
||||
members: [{ id: one.member.id, uuid: expect.any(String) }],
|
||||
});
|
||||
expect(
|
||||
await store.deleteUserGroupPermsByPermissionPrefixes([
|
||||
`fs:${node}`,
|
||||
]),
|
||||
).toEqual({
|
||||
rows: [
|
||||
{ group_id: many.groupId, permission: `fs:${node}:read` },
|
||||
],
|
||||
members: [{ id: many.member.id, uuid: expect.any(String) }],
|
||||
});
|
||||
});
|
||||
|
||||
it('includes someone who joined while the grant was deleted', async () => {
|
||||
const issuer = await makeUser();
|
||||
const joiner = await makeUser();
|
||||
const { member, groupId } = await seedMember(
|
||||
issuer.id,
|
||||
'fs:joined:read',
|
||||
);
|
||||
|
||||
const db = server.clients.db;
|
||||
const primaryWrite = db.write.bind(db);
|
||||
const write = vi
|
||||
.spyOn(db, 'write')
|
||||
.mockImplementation(async (sql, params) => {
|
||||
if (String(sql).startsWith('DELETE FROM')) {
|
||||
await primaryWrite(
|
||||
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
|
||||
[joiner.id, groupId],
|
||||
);
|
||||
}
|
||||
return primaryWrite(sql, params);
|
||||
});
|
||||
let result;
|
||||
try {
|
||||
result = await store.deleteUserGroupPerm(
|
||||
groupId,
|
||||
issuer.id,
|
||||
'fs:joined:read',
|
||||
);
|
||||
} finally {
|
||||
write.mockRestore();
|
||||
}
|
||||
expect(result.removed).toBe(true);
|
||||
expect(result.members.map((m) => m.id).sort()).toEqual(
|
||||
[member.id, joiner.id].sort(),
|
||||
);
|
||||
expect(await tombstoneSet(joiner.id)).toBe(true);
|
||||
});
|
||||
|
||||
it('still reports the removal when the second member read fails', async () => {
|
||||
const issuer = await makeUser();
|
||||
const { member, groupId } = await seedMember(
|
||||
issuer.id,
|
||||
'fs:reread:read',
|
||||
);
|
||||
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
let memberReads = 0;
|
||||
const pread = vi
|
||||
.spyOn(db, 'pread')
|
||||
.mockImplementation(async (sql, params) => {
|
||||
if (
|
||||
String(sql).includes('`jct_user_group` ug') &&
|
||||
++memberReads > 1
|
||||
) {
|
||||
throw new Error('primary unavailable');
|
||||
}
|
||||
return primaryRead(sql, params);
|
||||
});
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
try {
|
||||
expect(
|
||||
await store.deleteUserGroupPerm(
|
||||
groupId,
|
||||
issuer.id,
|
||||
'fs:reread:read',
|
||||
),
|
||||
).toEqual({
|
||||
removed: true,
|
||||
members: [{ id: member.id, uuid: expect.any(String) }],
|
||||
});
|
||||
} finally {
|
||||
pread.mockRestore();
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('reads many issuer/group pairs under many prefixes at once', async () => {
|
||||
const issuer = await makeUser();
|
||||
const rival = await makeUser();
|
||||
@@ -1831,41 +1473,6 @@ describe('PermissionStore', () => {
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('removes a grant the replica has not seen yet with its node, when asked to read the primary', async () => {
|
||||
const issuer = await makeUser();
|
||||
const holder = await makeUser();
|
||||
const node = uuidv4();
|
||||
await store.upsertUserUserPerm(
|
||||
holder.id,
|
||||
issuer.id,
|
||||
`fs:${node}:read`,
|
||||
{},
|
||||
);
|
||||
|
||||
const restore = lagReplica('user_to_user_permissions', () => []);
|
||||
let unasked;
|
||||
let removed;
|
||||
try {
|
||||
unasked = await store.deleteUserUserPermsByPermissionPrefixes([
|
||||
`fs:${node}`,
|
||||
]);
|
||||
removed = await store.deleteUserUserPermsByPermissionPrefixes(
|
||||
[`fs:${node}`],
|
||||
{ primary: true },
|
||||
);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
// Unasked, the replica's answer stands and the primary is left alone.
|
||||
expect(unasked).toEqual([]);
|
||||
expect(removed).toHaveLength(1);
|
||||
expect(
|
||||
await store.readLinkedUserUserPermsFromPrimary(holder.id, [
|
||||
`fs:${node}:read`,
|
||||
]),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it('does not let a wildcard in the prefix widen the match', async () => {
|
||||
const issuer = await makeUser();
|
||||
const holder = await makeUser();
|
||||
|
||||
@@ -17,11 +17,7 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import {
|
||||
CACHE_TOMBSTONE_SUFFIX,
|
||||
CACHE_TOMBSTONE_TTL_SECONDS,
|
||||
PuterStore,
|
||||
} from '../types';
|
||||
import { PuterStore } from '../types';
|
||||
import type { LayerInstances } from '../../types';
|
||||
import type { puterStores } from '../index';
|
||||
import { PermissionUtil } from '../../services/permission/permissionUtil';
|
||||
@@ -102,11 +98,6 @@ type DbRead = (
|
||||
params: unknown[],
|
||||
) => Promise<Record<string, unknown>[]>;
|
||||
|
||||
export interface GroupMember {
|
||||
id: number;
|
||||
uuid: string | null;
|
||||
}
|
||||
|
||||
export interface LinkedUserUserPermRow {
|
||||
holder_user_id: number;
|
||||
issuer_user_id: number;
|
||||
@@ -654,40 +645,25 @@ export class PermissionStore extends PuterStore {
|
||||
}));
|
||||
}
|
||||
|
||||
/** `primary` when a grant may be too new for a replica to have it. */
|
||||
#grantRead(primary: boolean): DbRead {
|
||||
return primary
|
||||
? this.clients.db.pread.bind(this.clients.db)
|
||||
: this.clients.db.read.bind(this.clients.db);
|
||||
}
|
||||
|
||||
/** The group analogue: a deleted node's group grants must go with it. */
|
||||
async deleteUserGroupPermsByPermissionPrefixes(
|
||||
permissions: string[],
|
||||
{ primary = false }: { primary?: boolean } = {},
|
||||
): Promise<{
|
||||
rows: Array<{ group_id: number; permission: string }>;
|
||||
members: GroupMember[];
|
||||
}> {
|
||||
if (permissions.length === 0) return { rows: [], members: [] };
|
||||
): Promise<Array<{ group_id: number; permission: string }>> {
|
||||
if (permissions.length === 0) return [];
|
||||
const { where, params } = subtreeClause(permissions);
|
||||
|
||||
const rows = (await this.#grantRead(primary)(
|
||||
const rows = (await this.clients.db.read(
|
||||
'SELECT `group_id`, `permission` FROM `user_to_group_permissions` ' +
|
||||
`WHERE ${where}`,
|
||||
params,
|
||||
)) as Array<{ group_id: number; permission: string }>;
|
||||
if (rows.length === 0) return { rows, members: [] };
|
||||
if (rows.length === 0) return [];
|
||||
|
||||
const { members } = await this.#deleteGroupGrants(
|
||||
rows.map((row) => row.group_id),
|
||||
() =>
|
||||
this.clients.db.write(
|
||||
`DELETE FROM \`user_to_group_permissions\` WHERE ${where}`,
|
||||
params,
|
||||
),
|
||||
await this.clients.db.write(
|
||||
`DELETE FROM \`user_to_group_permissions\` WHERE ${where}`,
|
||||
params,
|
||||
);
|
||||
return { rows, members };
|
||||
return rows;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -702,7 +678,6 @@ export class PermissionStore extends PuterStore {
|
||||
*/
|
||||
async deleteUserUserPermsByPermissionPrefixes(
|
||||
permissions: string[],
|
||||
{ primary = false }: { primary?: boolean } = {},
|
||||
): Promise<
|
||||
Array<{
|
||||
holder_user_id: number;
|
||||
@@ -713,7 +688,7 @@ export class PermissionStore extends PuterStore {
|
||||
if (permissions.length === 0) return [];
|
||||
const { where, params } = subtreeClause(permissions);
|
||||
|
||||
const rows = (await this.#grantRead(primary)(
|
||||
const rows = (await this.clients.db.read(
|
||||
'SELECT `holder_user_id`, `issuer_user_id`, `permission` FROM `user_to_user_permissions` ' +
|
||||
`WHERE ${where}`,
|
||||
params,
|
||||
@@ -1357,6 +1332,9 @@ export class PermissionStore extends PuterStore {
|
||||
}
|
||||
|
||||
// -- SQL: user-to-group permissions ------------------------------
|
||||
//
|
||||
// Read-only. Group grants are seeded by migration (the admin group's
|
||||
// unrestricted `driver` access); nothing writes them at runtime.
|
||||
|
||||
/**
|
||||
* Reads group permissions granted to groups the user is a member of, for a
|
||||
@@ -1373,15 +1351,9 @@ export class PermissionStore extends PuterStore {
|
||||
if (permissions.length === 0) return [];
|
||||
let permClause = permissions.map(() => 'p.permission = ?').join(' OR ');
|
||||
if (permissions.length > 1) permClause = `(${permClause})`;
|
||||
const tombstoned = await this.isCacheKeyTombstoned([
|
||||
this.#u2gCacheKey(userId),
|
||||
]);
|
||||
const read: DbRead = tombstoned
|
||||
? this.clients.db.pread.bind(this.clients.db)
|
||||
: this.clients.db.read.bind(this.clients.db);
|
||||
// Deletion leaves memberships and grants, so a deleted team would
|
||||
// otherwise keep resolving access nothing can withdraw.
|
||||
const rows = await read(
|
||||
const rows = await this.clients.db.read(
|
||||
'SELECT p.permission, p.user_id, p.group_id, p.extra FROM `user_to_group_permissions` p ' +
|
||||
'JOIN `jct_user_group` ug ON p.group_id = ug.group_id ' +
|
||||
'JOIN `group` g ON g.`id` = ug.group_id ' +
|
||||
@@ -1415,44 +1387,13 @@ export class PermissionStore extends PuterStore {
|
||||
.filter((uuid): uuid is string => Boolean(uuid));
|
||||
}
|
||||
|
||||
async #listGroupMembers(groupIds: number[]): Promise<GroupMember[]> {
|
||||
const ids = [...new Set(groupIds.map(Number))].filter((id) =>
|
||||
Number.isFinite(id),
|
||||
);
|
||||
if (ids.length === 0) return [];
|
||||
const rows = (await this.clients.db.pread(
|
||||
'SELECT DISTINCT u.`id`, u.`uuid` FROM `jct_user_group` ug ' +
|
||||
'JOIN `user` u ON u.`id` = ug.`user_id` ' +
|
||||
`WHERE ug.\`group_id\` IN (${ids.map(() => '?').join(', ')})`,
|
||||
ids,
|
||||
)) as { id: number; uuid: string | null }[];
|
||||
return rows.map((row) => ({ id: Number(row.id), uuid: row.uuid }));
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs a delete of these groups' grants with their members' group reads
|
||||
* already on the primary. Returns the members, later joiners included.
|
||||
*/
|
||||
async #deleteGroupGrants<T>(
|
||||
groupIds: number[],
|
||||
deleteGrants: () => Promise<T>,
|
||||
): Promise<{ result: T; members: GroupMember[] }> {
|
||||
// Before the delete: a failed member read leaves the grants to retry.
|
||||
const members = await this.#listGroupMembers(groupIds);
|
||||
await this.markGroupMembershipChanged(members.map((m) => m.id));
|
||||
const result = await deleteGrants();
|
||||
try {
|
||||
const known = new Set(members.map((m) => m.id));
|
||||
const joined = (await this.#listGroupMembers(groupIds)).filter(
|
||||
(m) => !known.has(m.id),
|
||||
);
|
||||
await this.markGroupMembershipChanged(joined.map((m) => m.id));
|
||||
members.push(...joined);
|
||||
} catch (e) {
|
||||
// The grants are gone; only someone who joined meanwhile is missed.
|
||||
console.warn('[permission] group members not re-read:', e);
|
||||
}
|
||||
return { result, members };
|
||||
/** Whose standing access a grant to this group settles. */
|
||||
async listGroupMemberIds(groupId: number): Promise<number[]> {
|
||||
const rows = (await this.clients.db.read(
|
||||
'SELECT `user_id` FROM `jct_user_group` WHERE `group_id` = ?',
|
||||
[groupId],
|
||||
)) as { user_id: number }[];
|
||||
return rows.map((r) => Number(r.user_id));
|
||||
}
|
||||
|
||||
/** Batched `readUserGroupPerms`, tagged with the user each row reached. */
|
||||
@@ -1465,14 +1406,7 @@ export class PermissionStore extends PuterStore {
|
||||
const holders = [...new Set(userIds)];
|
||||
const perms = [...new Set(permissions)];
|
||||
if (holders.length === 0 || perms.length === 0) return [];
|
||||
const changed = await this.tombstonedCacheKeys(
|
||||
holders.map((holder) => this.#u2gCacheKey(holder)),
|
||||
);
|
||||
const read: DbRead =
|
||||
changed.size > 0
|
||||
? this.clients.db.pread.bind(this.clients.db)
|
||||
: this.clients.db.read.bind(this.clients.db);
|
||||
const rows = await read(
|
||||
const rows = await this.clients.db.read(
|
||||
'SELECT ug.`user_id` AS `holder_user_id`, p.`permission`, p.`user_id` ' +
|
||||
'FROM `user_to_group_permissions` p ' +
|
||||
'JOIN `jct_user_group` ug ON p.`group_id` = ug.`group_id` ' +
|
||||
@@ -1558,34 +1492,13 @@ export class PermissionStore extends PuterStore {
|
||||
groupId: number,
|
||||
issuerUserId: number,
|
||||
permission: string,
|
||||
): Promise<{ removed: boolean; members: GroupMember[] }> {
|
||||
const { result, members } = await this.#deleteGroupGrants(
|
||||
[groupId],
|
||||
() =>
|
||||
this.clients.db.write(
|
||||
'DELETE FROM `user_to_group_permissions` ' +
|
||||
'WHERE `group_id` = ? AND `user_id` = ? AND `permission` = ?',
|
||||
[groupId, issuerUserId, permission],
|
||||
),
|
||||
): Promise<boolean> {
|
||||
const result = await this.clients.db.write(
|
||||
'DELETE FROM `user_to_group_permissions` ' +
|
||||
'WHERE `group_id` = ? AND `user_id` = ? AND `permission` = ?',
|
||||
[groupId, issuerUserId, permission],
|
||||
);
|
||||
return { removed: result.anyRowsAffected, members };
|
||||
}
|
||||
|
||||
/** Group reads of these users go to the primary; for leaves and deletes. */
|
||||
async markGroupMembershipChanged(userIds: number[]): Promise<void> {
|
||||
const ids = [...new Set(userIds.map(Number))].filter((id) =>
|
||||
Number.isFinite(id),
|
||||
);
|
||||
if (ids.length === 0) return;
|
||||
// Only the tombstones: nothing is cached under the base key.
|
||||
await this.publishCacheKeys({
|
||||
keys: ids.map(
|
||||
(id) => this.#u2gCacheKey(id) + CACHE_TOMBSTONE_SUFFIX,
|
||||
),
|
||||
serializedData: '1',
|
||||
ttlSeconds: CACHE_TOMBSTONE_TTL_SECONDS,
|
||||
broadcast: true,
|
||||
});
|
||||
return result.anyRowsAffected;
|
||||
}
|
||||
|
||||
async auditUserGroupPerm(
|
||||
@@ -1975,11 +1888,6 @@ export class PermissionStore extends PuterStore {
|
||||
return `perms:u2u:holder:${holderUserId}`;
|
||||
}
|
||||
|
||||
/** Holds only a tombstone: group reads aren't cached here. */
|
||||
#u2gCacheKey(userId: number): string {
|
||||
return `perms:u2g:${userId}`;
|
||||
}
|
||||
|
||||
#u2aCacheKey(userId: number, appId: number): string {
|
||||
return `perms:u2a:${userId}:${appId}`;
|
||||
}
|
||||
|
||||
@@ -658,26 +658,12 @@ export class SessionStore extends PuterStore {
|
||||
*/
|
||||
async findActiveByAccessTokenUid(tokenUid) {
|
||||
if (!tokenUid) return null;
|
||||
// One indexed lookup per column: an OR across both can't use either
|
||||
// index on MySQL. MySQL indexes the legacy uid only through its
|
||||
// active-row generated column, equal to it while `revoked_at` is NULL.
|
||||
const legacyColumn = this.clients.db.case({
|
||||
mysql: 'legacy_token_unique_key',
|
||||
otherwise: 'legacy_token_uid',
|
||||
});
|
||||
const findBy = async (column) => {
|
||||
const rows = await this.clients.db.read(
|
||||
`SELECT * FROM \`sessions\` WHERE \`kind\` = 'access_token' AND \`${column}\` = ? AND \`revoked_at\` IS NULL AND (\`expires_at\` IS NULL OR \`expires_at\` > ?) ORDER BY \`id\` DESC LIMIT 1`,
|
||||
[tokenUid, nowSeconds()],
|
||||
);
|
||||
return this.#normalizeRow(rows[0]);
|
||||
};
|
||||
// Sent together, so a miss on the first costs no second round trip.
|
||||
const [own, legacy] = await Promise.all([
|
||||
findBy('access_token_uid'),
|
||||
findBy(legacyColumn),
|
||||
]);
|
||||
return own ?? legacy;
|
||||
const now = nowSeconds();
|
||||
const rows = await this.clients.db.read(
|
||||
"SELECT * FROM `sessions` WHERE `kind` = 'access_token' AND (`access_token_uid` = ? OR `legacy_token_uid` = ?) AND `revoked_at` IS NULL AND (`expires_at` IS NULL OR `expires_at` > ?) ORDER BY `id` DESC LIMIT 1",
|
||||
[tokenUid, tokenUid, now],
|
||||
);
|
||||
return this.#normalizeRow(rows[0]);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -771,77 +771,6 @@ describe('SessionStore', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('findActiveByAccessTokenUid', () => {
|
||||
const tokenRow = (userId: number, fields: Record<string, unknown>) =>
|
||||
target.create(userId, { kind: 'access_token', ...fields });
|
||||
|
||||
it('finds a token row by its access_token_uid', async () => {
|
||||
const user = await makeUser();
|
||||
const tokenUid = uuidv4();
|
||||
const row = await tokenRow(user.id, { access_token_uid: tokenUid });
|
||||
|
||||
const found = await target.findActiveByAccessTokenUid(tokenUid);
|
||||
expect(found?.uuid).toBe(row.uuid);
|
||||
});
|
||||
|
||||
it('finds a migrated row by its legacy token uid', async () => {
|
||||
const user = await makeUser();
|
||||
const legacyUid = uuidv4();
|
||||
const row = await tokenRow(user.id, {
|
||||
legacy_token_uid: legacyUid,
|
||||
created_via: 'legacy_backfill',
|
||||
});
|
||||
|
||||
const found = await target.findActiveByAccessTokenUid(legacyUid);
|
||||
expect(found?.uuid).toBe(row.uuid);
|
||||
});
|
||||
|
||||
it('returns null for a uid no token row carries', async () => {
|
||||
const user = await makeUser();
|
||||
await tokenRow(user.id, { access_token_uid: uuidv4() });
|
||||
|
||||
expect(
|
||||
await target.findActiveByAccessTokenUid(uuidv4()),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('skips revoked and expired rows under either uid', async () => {
|
||||
const user = await makeUser();
|
||||
const revokedUid = uuidv4();
|
||||
const revokedLegacyUid = uuidv4();
|
||||
const expiredUid = uuidv4();
|
||||
const revoked = await tokenRow(user.id, {
|
||||
access_token_uid: revokedUid,
|
||||
});
|
||||
const revokedLegacy = await tokenRow(user.id, {
|
||||
legacy_token_uid: revokedLegacyUid,
|
||||
});
|
||||
await tokenRow(user.id, {
|
||||
access_token_uid: expiredUid,
|
||||
expires_at: Math.floor(Date.now() / 1000) - 60,
|
||||
});
|
||||
await target.removeByUuid(revoked.uuid);
|
||||
await target.removeByUuid(revokedLegacy.uuid);
|
||||
|
||||
for (const uid of [revokedUid, revokedLegacyUid, expiredUid]) {
|
||||
expect(await target.findActiveByAccessTokenUid(uid)).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
it('ignores a non-token row carrying the uid', async () => {
|
||||
const user = await makeUser();
|
||||
const tokenUid = uuidv4();
|
||||
await target.create(user.id, {
|
||||
kind: 'web',
|
||||
access_token_uid: tokenUid,
|
||||
});
|
||||
|
||||
expect(
|
||||
await target.findActiveByAccessTokenUid(tokenUid),
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('getOrCreateGodmodeToken', () => {
|
||||
const nowS = () => Math.floor(Date.now() / 1000);
|
||||
const mint = (
|
||||
|
||||
@@ -48,33 +48,6 @@ const SHARED_IDS_CHUNK_SIZE = 1000;
|
||||
* and no permission behind it, and is itself what the ACL reads.
|
||||
*/
|
||||
export class ShareStore extends PuterStore {
|
||||
onServerStart() {
|
||||
// A peer region's link revokes; our own emit already applied here.
|
||||
this.clients.event.on(
|
||||
'outer.share.linksRevoked',
|
||||
async (_key, data, meta) => {
|
||||
if (!meta?.from_outside) return;
|
||||
const fsentryIds = Array.isArray(data?.fsentryIds)
|
||||
? data.fsentryIds.filter((id) => Number.isInteger(id))
|
||||
: [];
|
||||
const uuids = Array.isArray(data?.uuids)
|
||||
? data.uuids.filter(
|
||||
(uuid) => typeof uuid === 'string' && uuid !== '',
|
||||
)
|
||||
: [];
|
||||
if (fsentryIds.length === 0 && uuids.length === 0) return;
|
||||
try {
|
||||
await this.#applyLinksRevoked(fsentryIds, uuids);
|
||||
} catch (e) {
|
||||
console.warn(
|
||||
'[share] remote link-revoke marker not set:',
|
||||
e,
|
||||
);
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// -- Reads --------------------------------------------------------
|
||||
|
||||
/** Pass `primary` to read back a row just written; a replica may lag. */
|
||||
@@ -365,17 +338,12 @@ export class ShareStore extends PuterStore {
|
||||
* Every share row on a node and everything beneath it, whatever kind —
|
||||
* user, group and link shares plus unclaimed invites.
|
||||
* `listByFsentrySubtree` answers the narrower question; this one is for
|
||||
* retiring the lot. Pass `primary` when a share may be too new for a
|
||||
* replica.
|
||||
* retiring the lot.
|
||||
*
|
||||
* @param {number} fsentryId
|
||||
* @param {{ primary?: boolean }} [opts]
|
||||
*/
|
||||
async listAllByFsentrySubtree(fsentryId, { primary = false } = {}) {
|
||||
const read = primary
|
||||
? this.clients.db.pread.bind(this.clients.db)
|
||||
: this.clients.db.read.bind(this.clients.db);
|
||||
const rows = await read(
|
||||
async listAllByFsentrySubtree(fsentryId) {
|
||||
const rows = await this.clients.db.read(
|
||||
this.#subtreeCte() +
|
||||
'SELECT `share`.* FROM `share` ' +
|
||||
'JOIN `subtree` ON `share`.`fsentry_id` = `subtree`.`id` ' +
|
||||
@@ -996,19 +964,14 @@ export class ShareStore extends PuterStore {
|
||||
* its shares never fires.
|
||||
*
|
||||
* @param {number[]} fsentryIds
|
||||
* @param {string[]} [entryUuids] The nodes' uuids; looked up when omitted.
|
||||
*/
|
||||
async deleteByFsentryIds(fsentryIds, entryUuids) {
|
||||
async deleteByFsentryIds(fsentryIds) {
|
||||
if (fsentryIds.length === 0) return 0;
|
||||
const placeholders = fsentryIds.map(() => '?').join(', ');
|
||||
const result = await this.clients.db.write(
|
||||
`DELETE FROM \`share\` WHERE \`fsentry_id\` IN (${placeholders})`,
|
||||
fsentryIds,
|
||||
);
|
||||
// Which of these nodes held a link is no longer known, so mark them all.
|
||||
if (result.anyRowsAffected) {
|
||||
await this.#markLinksRevoked(fsentryIds, entryUuids);
|
||||
}
|
||||
return result.affectedRows;
|
||||
}
|
||||
|
||||
@@ -1253,64 +1216,36 @@ export class ShareStore extends PuterStore {
|
||||
[fsentryId],
|
||||
);
|
||||
const removed = result.anyRowsAffected;
|
||||
if (removed) {
|
||||
await this.#markLinksRevoked(
|
||||
[fsentryId],
|
||||
entryUuid ? [entryUuid] : undefined,
|
||||
);
|
||||
}
|
||||
if (removed) await this.#markLinkRevoked(fsentryId, entryUuid);
|
||||
return removed;
|
||||
}
|
||||
|
||||
/** Reads of these nodes' links go to the primary, here and in peers. */
|
||||
async #markLinksRevoked(fsentryIds, entryUuids) {
|
||||
const ids = [...new Set(fsentryIds.map(Number))];
|
||||
let uuids = entryUuids?.length >= ids.length ? entryUuids : null;
|
||||
if (!uuids) {
|
||||
try {
|
||||
const rows = await this.clients.db.pread(
|
||||
'SELECT `uuid` FROM `fsentries` WHERE `id` IN ' +
|
||||
`(${ids.map(() => '?').join(', ')})`,
|
||||
ids,
|
||||
);
|
||||
uuids = rows.map((row) => String(row.uuid));
|
||||
} catch (e) {
|
||||
console.warn('[share] link-revoke uuids not found:', e);
|
||||
uuids = entryUuids ?? [];
|
||||
}
|
||||
}
|
||||
/** Open the window in which reads of this node's link go to the primary. */
|
||||
async #markLinkRevoked(fsentryId, entryUuid) {
|
||||
try {
|
||||
await this.#applyLinksRevoked(ids, uuids);
|
||||
let uuid = entryUuid;
|
||||
if (!uuid) {
|
||||
const rows = await this.clients.db.pread(
|
||||
'SELECT `uuid` FROM `fsentries` WHERE `id` = ? LIMIT 1',
|
||||
[fsentryId],
|
||||
);
|
||||
uuid = rows[0]?.uuid;
|
||||
}
|
||||
const keys = [ShareStore.revokedLinkKey('id', fsentryId)];
|
||||
if (uuid) keys.push(ShareStore.revokedLinkKey('uuid', uuid));
|
||||
await Promise.all(
|
||||
keys.map((key) =>
|
||||
this.clients.redis.set(
|
||||
key,
|
||||
'1',
|
||||
'EX',
|
||||
ShareStore.REVOKED_LINK_PRIMARY_WINDOW_SECONDS,
|
||||
),
|
||||
),
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[share] link-revoke marker not set:', e);
|
||||
}
|
||||
// Peers still get the ids when the local half failed.
|
||||
try {
|
||||
this.clients.event.emit(
|
||||
'outer.share.linksRevoked',
|
||||
{ fsentryIds: ids, uuids },
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[share] link-revoke not sent to peers:', e);
|
||||
}
|
||||
}
|
||||
|
||||
/** Local half of {@link #markLinksRevoked}; never emits. */
|
||||
async #applyLinksRevoked(fsentryIds, uuids) {
|
||||
const pipeline = this.clients.redis.pipeline();
|
||||
for (const key of [
|
||||
...fsentryIds.map((id) => ShareStore.revokedLinkKey('id', id)),
|
||||
...uuids.map((uuid) => ShareStore.revokedLinkKey('uuid', uuid)),
|
||||
]) {
|
||||
pipeline.set(
|
||||
key,
|
||||
'1',
|
||||
'EX',
|
||||
ShareStore.REVOKED_LINK_PRIMARY_WINDOW_SECONDS,
|
||||
);
|
||||
}
|
||||
await pipeline.exec();
|
||||
}
|
||||
|
||||
// -- Daily quota --------------------------------------------------
|
||||
|
||||
@@ -337,176 +337,6 @@ describe('ShareStore', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('does not read back a link removed with its node from a lagging replica', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
const stale = await store.upsertAnyone({
|
||||
issuerUserId: issuer.id,
|
||||
fsentryId: entry.id,
|
||||
mode: 'read',
|
||||
});
|
||||
await store.deleteByFsentryIds([entry.id]);
|
||||
|
||||
// The replica still has the link the primary just deleted.
|
||||
const db = server.clients.db;
|
||||
const primaryRead = db.read.bind(db);
|
||||
const pread = vi.spyOn(db, 'pread').mockImplementation(primaryRead);
|
||||
const read = vi
|
||||
.spyOn(db, 'read')
|
||||
.mockImplementation(async (q, p) =>
|
||||
/FROM `share`/u.test(q)
|
||||
? [
|
||||
{
|
||||
...stale,
|
||||
data: '{}',
|
||||
entry_uuid: entry.uuid,
|
||||
owner_user_id: issuer.id,
|
||||
},
|
||||
]
|
||||
: primaryRead(q, p),
|
||||
);
|
||||
try {
|
||||
expect(await store.getAnyone(entry.id)).toBeNull();
|
||||
expect(await store.listAnyoneOnFsentries([entry.id])).toEqual(
|
||||
[],
|
||||
);
|
||||
expect(await store.listAnyoneReaching([entry.uuid])).toEqual(
|
||||
[],
|
||||
);
|
||||
} finally {
|
||||
read.mockRestore();
|
||||
pread.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('tells peer regions when a link is removed', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
await store.upsertAnyone({
|
||||
issuerUserId: issuer.id,
|
||||
fsentryId: entry.id,
|
||||
mode: 'read',
|
||||
});
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
try {
|
||||
await store.deleteByFsentryIds([entry.id]);
|
||||
expect(emit).toHaveBeenCalledWith(
|
||||
'outer.share.linksRevoked',
|
||||
{ fsentryIds: [entry.id], uuids: [entry.uuid] },
|
||||
{},
|
||||
);
|
||||
} finally {
|
||||
emit.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('applies a link removal broadcast by a peer region', async () => {
|
||||
const fsentryId = 900000 + Math.floor(Math.random() * 99999);
|
||||
const uuid = uuidv4();
|
||||
server.clients.event.emit(
|
||||
'outer.share.linksRevoked',
|
||||
{ fsentryIds: [fsentryId], uuids: [uuid] },
|
||||
{ from_outside: true },
|
||||
);
|
||||
await vi.waitFor(async () => {
|
||||
expect(
|
||||
await server.clients.redis.get(
|
||||
ShareStore.revokedLinkKey('uuid', uuid),
|
||||
),
|
||||
).toBe('1');
|
||||
});
|
||||
expect(
|
||||
await server.clients.redis.ttl(
|
||||
ShareStore.revokedLinkKey('id', fsentryId),
|
||||
),
|
||||
).toBeGreaterThan(25);
|
||||
});
|
||||
|
||||
it('still tells peer regions when the local marker cannot be set', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
await store.upsertAnyone({
|
||||
issuerUserId: issuer.id,
|
||||
fsentryId: entry.id,
|
||||
mode: 'read',
|
||||
});
|
||||
const pipeline = vi
|
||||
.spyOn(server.clients.redis, 'pipeline')
|
||||
.mockImplementation(() => {
|
||||
throw new Error('redis unavailable');
|
||||
});
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
try {
|
||||
await store.deleteByFsentryIds([entry.id], [entry.uuid]);
|
||||
expect(emit).toHaveBeenCalledWith(
|
||||
'outer.share.linksRevoked',
|
||||
{ fsentryIds: [entry.id], uuids: [entry.uuid] },
|
||||
{},
|
||||
);
|
||||
} finally {
|
||||
emit.mockRestore();
|
||||
pipeline.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('neither marks nor tells peers when nothing was removed', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
const emit = vi.spyOn(server.clients.event, 'emit');
|
||||
try {
|
||||
expect(await store.deleteByFsentryIds([entry.id])).toBe(0);
|
||||
expect(emit).not.toHaveBeenCalledWith(
|
||||
'outer.share.linksRevoked',
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
);
|
||||
} finally {
|
||||
emit.mockRestore();
|
||||
}
|
||||
expect(
|
||||
await server.clients.redis.get(
|
||||
ShareStore.revokedLinkKey('uuid', entry.uuid),
|
||||
),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("ignores its own region's event and malformed payloads", async () => {
|
||||
const local = uuidv4();
|
||||
const sentinel = uuidv4();
|
||||
const bogus = 900000 + Math.floor(Math.random() * 99999);
|
||||
const emit = (data, meta) =>
|
||||
server.clients.event.emit(
|
||||
'outer.share.linksRevoked',
|
||||
data,
|
||||
meta,
|
||||
);
|
||||
emit({ fsentryIds: [], uuids: [local] }, {});
|
||||
for (const data of [
|
||||
null,
|
||||
{ fsentryIds: 'x', uuids: 'y' },
|
||||
{
|
||||
fsentryIds: [String(bogus), bogus + 0.5, null],
|
||||
uuids: [bogus, '', null],
|
||||
},
|
||||
]) {
|
||||
emit(data, { from_outside: true });
|
||||
}
|
||||
// Handled in order, so once this lands the ones above have too.
|
||||
emit({ fsentryIds: [], uuids: [sentinel] }, { from_outside: true });
|
||||
await vi.waitFor(async () => {
|
||||
expect(
|
||||
await server.clients.redis.get(
|
||||
ShareStore.revokedLinkKey('uuid', sentinel),
|
||||
),
|
||||
).toBe('1');
|
||||
});
|
||||
|
||||
for (const key of [
|
||||
ShareStore.revokedLinkKey('uuid', local),
|
||||
ShareStore.revokedLinkKey('uuid', String(bogus)),
|
||||
ShareStore.revokedLinkKey('id', bogus),
|
||||
ShareStore.revokedLinkKey('id', bogus + 0.5),
|
||||
]) {
|
||||
expect(await server.clients.redis.get(key)).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
it('reads a link past the replica once one has been withdrawn', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
await store.upsertAnyone({
|
||||
@@ -1325,27 +1155,6 @@ describe('ShareStore', () => {
|
||||
};
|
||||
};
|
||||
|
||||
it('lists a share the replica lacks in a subtree when asked to read the primary', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
const link = await store.upsertAnyone({
|
||||
issuerUserId: issuer.id,
|
||||
fsentryId: entry.id,
|
||||
mode: 'read',
|
||||
});
|
||||
const restore = lagShares([]);
|
||||
try {
|
||||
expect(
|
||||
await store.listAllByFsentrySubtree(entry.id),
|
||||
).toEqual([]);
|
||||
const rows = await store.listAllByFsentrySubtree(entry.id, {
|
||||
primary: true,
|
||||
});
|
||||
expect(rows.map((row) => row.uid)).toEqual([link.uid]);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
it('returns the rows it inserts before the replica has them', async () => {
|
||||
const entry = await makeEntry(issuer);
|
||||
// A group row written directly: the share only needs its id,
|
||||
|
||||
@@ -159,8 +159,8 @@ export interface AppSummaryOptions {
|
||||
|
||||
/**
|
||||
* The summary the desktop launches from directly, without re-reading the app
|
||||
* (suggested, recommended and recent apps), so every launch-relevant flag
|
||||
* belongs here.
|
||||
* (suggested, recommended, recent and taskbar apps), so every launch-relevant
|
||||
* flag belongs here.
|
||||
*/
|
||||
export function toAppSummary(
|
||||
app: AppRow,
|
||||
|
||||
@@ -18,10 +18,7 @@
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
decodeStrictBase64,
|
||||
decodeStrictBase64InSlices,
|
||||
} from './mediaSniff.ts';
|
||||
import { decodeStrictBase64 } from './mediaSniff.ts';
|
||||
|
||||
describe('decodeStrictBase64', () => {
|
||||
it.each([
|
||||
@@ -53,52 +50,3 @@ describe('decodeStrictBase64', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('decodeStrictBase64InSlices', () => {
|
||||
// Several slices, with a last one that is partial and padded.
|
||||
const bytes = Buffer.alloc(5 * 1024 * 1024 + 2);
|
||||
for (let i = 0; i < bytes.length; i += 4099) bytes[i] = i % 251;
|
||||
const valid = bytes.toString('base64');
|
||||
|
||||
it('decodes a multi-MB payload to the same bytes', async () => {
|
||||
expect(valid.endsWith('==')).toBe(true);
|
||||
expect((await decodeStrictBase64InSlices(valid))?.equals(bytes)).toBe(
|
||||
true,
|
||||
);
|
||||
const short = Buffer.from('abcd').toString('base64');
|
||||
expect(await decodeStrictBase64InSlices(short)).toEqual(
|
||||
Buffer.from('abcd'),
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['empty input', ''],
|
||||
['unpadded input', valid.slice(0, -2)],
|
||||
[
|
||||
'a character outside the alphabet in the first slice',
|
||||
`*${valid.slice(1)}`,
|
||||
],
|
||||
[
|
||||
'a character outside the alphabet in the last slice',
|
||||
`${valid.slice(0, -5)}*${valid.slice(-4)}`,
|
||||
],
|
||||
[
|
||||
'the URL-safe alphabet',
|
||||
`${valid.slice(0, 3_000_000)}-${valid.slice(3_000_001)}`,
|
||||
],
|
||||
[
|
||||
'a non-ASCII character',
|
||||
`${valid.slice(0, 3_000_000)}ī${valid.slice(3_000_001)}`,
|
||||
],
|
||||
[
|
||||
'padding where a slice ends',
|
||||
`${valid.slice(0, 1024 * 1024 - 1)}=${valid.slice(1024 * 1024)}`,
|
||||
],
|
||||
[
|
||||
'padding inside a slice',
|
||||
`${valid.slice(0, 2_000_002)}==${valid.slice(2_000_004)}`,
|
||||
],
|
||||
])('rejects %s', async (_label, value) => {
|
||||
expect(await decodeStrictBase64InSlices(value)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -17,8 +17,6 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { setImmediate as nextTurn } from 'node:timers/promises';
|
||||
|
||||
// Identify uploaded media by its bytes. A caller-supplied MIME type or file
|
||||
// extension is never trusted.
|
||||
|
||||
@@ -71,33 +69,6 @@ export function decodeStrictBase64(value: string): Buffer | null {
|
||||
return bytes.length === (value.length / 4) * 3 - padding ? bytes : null;
|
||||
}
|
||||
|
||||
/** Base64 characters decoded per turn; a multiple of four. */
|
||||
const BASE64_SLICE_CHARS = 1024 * 1024;
|
||||
|
||||
/**
|
||||
* `decodeStrictBase64` for payloads of many megabytes: decodes a slice per turn
|
||||
* of the event loop, so the whole string never holds it.
|
||||
*/
|
||||
export async function decodeStrictBase64InSlices(
|
||||
value: string,
|
||||
): Promise<Buffer | null> {
|
||||
if (value.length <= BASE64_SLICE_CHARS) return decodeStrictBase64(value);
|
||||
if (value.length % 4 !== 0) return null;
|
||||
const padding = value.endsWith('==') ? 2 : value.endsWith('=') ? 1 : 0;
|
||||
const bytes = Buffer.allocUnsafe((value.length / 4) * 3 - padding);
|
||||
let written = 0;
|
||||
for (let at = 0; at < value.length; at += BASE64_SLICE_CHARS) {
|
||||
const slice = decodeStrictBase64(
|
||||
value.slice(at, at + BASE64_SLICE_CHARS),
|
||||
);
|
||||
// Padding inside the string leaves a slice short, and the total with it.
|
||||
if (!slice || written + slice.length > bytes.length) return null;
|
||||
written += slice.copy(bytes, written);
|
||||
await nextTurn();
|
||||
}
|
||||
return written === bytes.length ? bytes : null;
|
||||
}
|
||||
|
||||
function looksLikeSvg(bytes: Buffer): boolean {
|
||||
let head = bytes.subarray(0, SVG_SNIFF_WINDOW).toString('utf8');
|
||||
if (head.charCodeAt(0) === 0xfeff) head = head.slice(1);
|
||||
|
||||
Loaded 100 of 318 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user