Compare commits

..
Author SHA1 Message Date
Daniel Salazar f3542a6aa3 fix(backend): shutdown hooks don't skip each other, and final flushes wait for in-flight work (#4499)
A throwing prepare/shutdown hook stopped the loop, so shutdown() skipped
closeAllConnections and every later hook, including the final metering
flush; each hook now runs and failures are logged. Broadcast, events
forwarding and share digests joined nothing already in flight and could
drop what was queued behind it; their final flushes now wait for it, ship
the tail, and drain once more at shutdown. Local worker disposal and the
OpenAI image metering call are awaited. settleWithin and
FINAL_FLUSH_TIMEOUT_MS bound each final flush.
2026-11-04 19:41:17 -07:00
Daniel Salazar 5a59db7b6b feat(backend): drain-start hooks run concurrently and can hold the listener open
The listener closes once every onServerDrainStart hook settles, and the
drain window now starts at the signal, so time a hook spends holding the
listener open counts against drainMs instead of extending shutdown. A
failing hook no longer skips the others.
2026-11-04 18:38:33 -07:00
251 changed files with 6772 additions and 20167 deletions

No files matched your search

-27
View File
@@ -161,38 +161,11 @@ An overview page answers "what is this API and is it what I need?", and a functi
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
### Function page layout
A function page opens with one sentence that says what the function does, in its own paragraph, such as "Given a prompt, generate an image using AI." Put any more detail in the paragraphs after it.
After the intro, a function page has these `##` sections, in this order:
1. Syntax
2. Parameters
3. Return value
4. Errors, if the function has any
5. Sections for specific topics, if the page needs them, with `## Model-specific features` last
6. Examples
[chat.md](src/docs/src/AI/chat.md) follows this layout.
- **Options as a table.** Document an options object as a table with Option, Type, and Description columns. Give each option its own row, even when two options go together, such as `width` and `height`. Order the rows by how often developers use them, most common first, without group rows. In chat.md that's the core options, then reasoning, tools, response handling, request control, and output.
- **Common options in the table, model-specific behavior in its own section.** A page whose function works with many models or providers, such as [chat.md](src/docs/src/AI/chat.md) and [txt2img.md](src/docs/src/AI/txt2img.md), splits its options this way:
- **The options table** under Parameters lists the options that work with every model, and describes each one the way it works for every model.
- **`## Model-specific features`**, the last section before Examples, covers everything else. It has a `###` for each model family or provider, named after it, such as "OpenAI models" or "Cloudflare models". Under it go the options only some models accept, and the common options a model handles differently, such as a narrower range, other allowed values, or an option it ignores.
- **Write there only what's different.** A `###` lists an option only when that model or provider adds it or handles it differently. An option that works the same everywhere stays out, even if the provider supports it, and a model or provider with nothing different gets no `###`.
- **Model-specific features is the only place for model-specific details.** Every limit, format, allowed value, default, price, or quirk that applies to some models goes there, in that model's or provider's `###`, and nowhere else. The options table and topic sections (such as Image inputs or Video input) describe what's true for every model and link to Model-specific features for the rest. Don't add a table or sentence there that compares providers or repeats a model's limits. Each copy is another source of truth to keep in sync, and copies drift when a model changes.
- **Link the two.** Directly under the options table, put `Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).` A row whose option behaves differently on some models can also end with `See [Model-specific features](#model-specific-features).`
- **A long topic gets its own `##` section.** When explaining an option or feature takes more than a sentence or two, such as function calling or output dimensions, give it a section between Errors and Examples and link to it from the options table. This keeps the reference sections short.
- **Usage patterns are examples.** A way of using the API, such as stopping a stream, goes in a runnable example that the relevant section links to ("See the [stop a stream example](/playground/ai-chat-stop-stream/)."), not in a section of its own.
- **Each example has a `###` heading** under `## Examples`, so every example shows in the table of contents.
### Writing docs
These apply to reference pages and recipes alike.
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
- **Document Puter as it runs on puter.com.** The docs are for developers building on production Puter. Leave out behavior that only happens on a self-hosted or differently configured server, such as a provider that isn't set up there.
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
-105
View File
@@ -1,105 +0,0 @@
# Embedded app browser
`GET /embed/apps` on the deployment's main GUI origin renders the same app browser
as the dashboard's Apps tab, without the desktop or dashboard navigation.
It supports search, paging, saved folders, and saved ordering. It uses the same
installed-app and recommendation data as the dashboard, including the user's
removed recommendations. Editing, adding, reordering, renaming, and uninstalling
are disabled. Selecting an app opens it in a new browser tab at its own URL, the
same way the dashboard does, and the host page stays open. The app's token isn't
put in the URL: the app's SDK asks the embed for it once the tab loads, so the
embed must still be open then. If the iframe is removed first, the app falls back
to its normal sign-in. Apps that need the desktop (built-in and godmode apps,
private apps the user can't open, or a failed token request) open `/app/<name>`
on the GUI origin, cut off from the embed. Website shortcuts open their external
destination with no opener.
Apps opened from the embed carry `utm_source=puter.com&utm_medium=referral&utm_campaign=app_browser_embed`;
the desktop and dashboard send `utm_campaign=desktop` and `utm_campaign=dashboard`.
## Session and framing
Only show the iframe after the user has signed in. The embed reads the existing
GUI session from local storage on its own origin. It does not create accounts,
show a login dialog, or accept a token from its parent. A missing session or
failed load displays an unavailable/error state. Being signed in on a subdomain
alone does not populate the main GUI origin's storage; the main GUI session must
already exist.
The route's `Content-Security-Policy: frame-ancestors` allows the GUI origin and
the embedding page's origin when it is a subdomain of `config.domain` with the
scheme and port from `config.origin`, and not under one of the user-content
hosting domains (`static_hosting_domain`, `private_app_hosting_domain`, and their
`_alt` variants). For example, `https://puter.com` permits
`https://spreadsheet.puter.com`, and `http://puter.localhost:4100` permits
`http://spreadsheet.puter.localhost:4100` but not
`http://someone.site.puter.localhost:4100`. The embedding page is read from the
iframe request's referrer, so the host must not suppress it (the browser default
`strict-origin-when-cross-origin` is enough). All ancestors in a nested iframe
chain must be allowed. Other GUI routes retain `X-Frame-Options: SAMEORIGIN`.
The embed accepts no query parameters. Requests with a query string redirect to
the same embed path so SDK bootstrap parameters cannot override the session or API
origin. The host needs to allow the GUI origin in its own `frame-src` CSP if it
sets one. If sandboxing the iframe, allow scripts, same-origin storage, popups,
and popups to escape the sandbox.
## Host integration
Create the iframe when “More apps” is opened, and remove it on close so reopening
loads current apps and preferences. The host owns the modal and close control;
there is no parent messaging API and no app list or credential is sent to it.
The app browser fills its iframe and responds to its size.
This example assumes `puterOrigin` comes from the host app's deployment settings
and the caller has already established the GUI session. Translate the button and
frame labels using the host app's localization system.
```html
<button id="more-apps" type="button">More apps</button>
<dialog id="apps-dialog" aria-label="More apps">
<button id="close-apps" type="button">Close</button>
<div id="apps-frame"></div>
</dialog>
<style>
#apps-dialog { padding: 12px; width: min(900px, 90vw); }
#apps-frame iframe { width: 100%; height: min(650px, 75vh); border: 0; }
</style>
<script type="module">
const puterOrigin = 'https://puter.com'; // Use this deployment's configured origin.
const dialog = document.querySelector('#apps-dialog');
const container = document.querySelector('#apps-frame');
const trigger = document.querySelector('#more-apps');
trigger.addEventListener('click', () => {
const frame = document.createElement('iframe');
frame.title = 'Your apps';
frame.src = new URL('/embed/apps', puterOrigin).href;
container.replaceChildren(frame);
dialog.showModal();
});
document.querySelector('#close-apps').addEventListener('click', () => dialog.close());
dialog.addEventListener('close', () => {
container.replaceChildren();
trigger.focus();
});
</script>
```
## Implementation and checks
`createAppBrowser` in `src/gui/src/UI/Dashboard/AppBrowser.js` owns the shared UI
and data behavior. `TabApps.js` injects desktop actions. `appsEmbed.js` supplies
new-tab launching through `launchAppInTab` in `src/gui/src/helpers/appTabLaunch.js`,
which builds the same app URL as `launch_app`, and loads only the embed entry, its
SDK, and shared styles.
The normal GUI webpack build emits `bundle.min.js` and `apps-embed.min.js`.
No puter.js API or app-list endpoint changes are needed.
Run the GUI component/browser tests (with the SDK bundle built and Playwright
Chromium installed), and the backend route tests:
```sh
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
```
+6 -7
View File
@@ -1,6 +1,6 @@
# Browser push client
Puter's desktop shows pushes through `/sw.js`. Core shows Settings → Notifications only when the `pushEnabled` gui param is `true`; core never sets it. The worker is registered only when push is turned on in the browser, and a load removes one left without a subscription. A backend extension that delivers pushes sets it with `services.homepage.setGuiParam('pushEnabled', true)` and can load its own code into the worker with `services.homepage.registerServiceWorkerScript(path)`.
Puter's desktop shows pushes through `/sw.js`. Core registers it, and shows Settings → Notifications, only when the `pushEnabled` gui param is `true`; core never sets it. A backend extension that delivers pushes sets it with `services.homepage.setGuiParam('pushEnabled', true)` and can load its own code into the worker with `services.homepage.registerServiceWorkerScript(path)`.
Core subscribes the browser to a push service only when the server also publishes `pushWebPushKey` (see [Built-in Web Push client](#built-in-web-push-client)), and it has no `/push/*` routes of its own. The device, app, preference and subscription endpoints below come from that extension. On a deployment without one, such as most self-hosted servers, they answer core's plain 404 (`code: 'not_found'`), which `puter.notifications` reports as `push_unavailable` and the Notifications tab shows as unavailable. A device the extension can't find is `device_not_found`.
@@ -22,13 +22,12 @@ When the browser replaces the subscription (`pushsubscriptionchange`), `sw.js` p
Before a notification is pushed, the server asks the open clients whether they received it live. It sends `notif.receipt` with `{ uid }` on the user's sockets for the app that sent it, or on the desktop's sockets for Puter's own notifications. The SDK remembers the uids its page received through `onNotification()`, `onPush()` or the desktop's feed, and answers once its page has that notification, or with `{ received: false }` when it never gets it. The server waits up to 5 seconds (`notificationAckTimeoutMs`).
An answer carries `visibilityState` and `focused` as they were when the page received the notification. Pages never show a browser notification themselves; pushes do.
An answer carries `visibilityState`, `focused` and `showsOwn`, the last being whether the page shows the notification itself. A website page shows it itself when it is hidden or unfocused, has an `onPush()` handler and notification permission, and no other tab of the same site is focused. Inside the desktop, the desktop shows it, and an app's `onPush()` result is never shown as a browser notification.
- An answer from a page that was visible and focused stops the push (`handled` on `notif.delivery.settled`). An older client that doesn't send `focused` counts while visible.
- Otherwise the notification is pushed, to every browser the user subscribed. A website's pushes go only to the browsers that subscribed through its own worker, never to Puter's `/sw.js`. An app inside the desktop pushes through `/sw.js`, which shows the app's name and icon.
- A client that can't answer, such as an older puter.js, doesn't hold the push back.
The ask reaches only the sockets in the asking region. So that other regions hear about a page too, a page that receives a notification live while visible and focused also sends `notif.handled` with `{ uid }` on the socket it came on. Core relays it as the node-local `notif.handled` event with the socket's user and app, at most 60 a minute per user; an extension that delivers pushes decides what to do with it.
- An answer from a page that is visible and focused, or that shows the notification itself, stops the push.
- Otherwise the notification is pushed. A website's pushes go only to the browsers that subscribed through its own worker, never to Puter's `/sw.js`. An app inside the desktop pushes through `/sw.js`, which shows the app's name and icon.
- A hidden desktop tab shows no banner, so answers only from hidden desktop tabs don't stop a push of Puter's own notifications.
- A client that can't answer, such as an older puter.js, holds the push back while it is connected.
## Registering
@@ -318,13 +318,3 @@ describe('AbstractDatabaseClient — SQL generation', () => {
);
});
});
it('builds an insert for batchWrite and shares it with insert', async () => {
const client = new FixtureDatabaseClient([]);
const entry = client.insertEntry('widget', { name: "reader's", enabled: true });
expect(entry.values).toEqual(["reader's", true]);
expect(entry.statement).toContain('VALUES (?, ?)');
await client.insert('widget', { name: "reader's", enabled: true });
expect(client.writes).toEqual([{ query: entry.statement, params: entry.values }]);
});
+7 -16
View File
@@ -110,23 +110,14 @@ export class AbstractDatabaseClient extends PuterClient {
tableName: string,
data: Record<string, unknown>,
): Promise<WriteResult> {
const entry = this.insertEntry(tableName, data);
return this.write(
entry.statement + this.returningIdClause(),
entry.values,
);
}
/** An insert that can join other writes in a transaction. */
insertEntry(tableName: string, data: Record<string, unknown>): BatchEntry {
const cols = Object.keys(data);
return {
statement:
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`,
values: Object.values(data),
};
const values = Object.values(data);
const sql =
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
}
/** Whether `read()` goes to a separate replica rather than the primary. */
+6 -29
View File
@@ -117,7 +117,7 @@ export type EventMap = {
model_used: string;
service_used: string;
intended_service: string;
model_details?: {
model_details: {
id: string;
provider: string;
input_cost_key: string;
@@ -556,14 +556,6 @@ export type EventMap = {
'kv.share.retired': { handles: string[]; userId: number };
// ---- Notifications ----
/** An app message was acknowledged after its state committed. */
'notif.acknowledged': {
userId: number;
uid: string;
type: string;
audience: string;
appUid: string;
};
/**
* A notification row exists. Emitted after the insert commits — nothing
* pushes a uid before there is a row to name — and carries the scope tuple
@@ -597,8 +589,6 @@ export type EventMap = {
value: Record<string, unknown>;
/** An earlier notification rewritten in place. */
updated?: true;
/** Replaced an earlier notification in its group. */
replaced?: true;
/** Names this one send, for a receipt asked about it. */
sendId: string;
/** The same `sendId` may be emitted again, as a delayed send is. */
@@ -636,18 +626,6 @@ export type EventMap = {
/** A peer region holds a socket for the user. */
peerRegions: boolean;
responders: NotifDeliveryResponder[];
/** A page here got it live while visible and focused. */
handled: boolean;
};
/**
* A page reported that it got a notification live while visible and
* focused. Node-local; `appUid` is the reporting socket's app, absent for
* the desktop.
*/
'notif.handled': {
userId: number;
appUid?: string;
uid: string;
};
// ---- Metering ----
@@ -873,12 +851,6 @@ export type EventMap = {
/** One access token was revoked; only its own connections should drop. */
'auth.access-token.revoked': { token_uid: string };
/** Same-cluster eviction for raw realtime WebSockets. */
'pubsub.realtime.auth-revoked': {
sessionUids?: string[];
tokenUid?: string;
};
/**
* A grant was withdrawn, so whatever was standing on it has to be settled
* rather than left to fail its next check.
@@ -1104,6 +1076,11 @@ export interface NotifDeliveryResponder {
visibilityState: 'visible' | 'hidden' | 'unknown';
/** The page's window has focus. Absent from older clients. */
focused?: boolean;
/**
* The page shows its own browser notification while hidden or unfocused.
* Absent from older clients.
*/
showsOwn?: boolean;
}
export type EventMetadata = {
@@ -347,46 +347,6 @@ describe('DispatcherInvokeTransport with every connection busy', () => {
held[INVOKE_CONNECTIONS].answer();
await expect(after).resolves.toMatchObject({ status: 200 });
});
it('rechecks the guard once a connection frees, and drops a superseded call', async () => {
const { transport, held, answerAll } = await saturated();
let current = true;
const guard = vi.fn(async () => current);
const waiting = transport.send({ ...CALL, beforeSend: guard });
current = false;
held[0].answer();
await expect(waiting).resolves.toEqual({
status: null,
error: 'invocation superseded',
});
expect(guard).toHaveBeenCalledTimes(1);
expect(held).toHaveLength(INVOKE_CONNECTIONS);
await answerAll();
});
it('fails a call whose guard throws after the wait, and frees its slot', async () => {
const { transport, held, answerAll } = await saturated();
const waiting = transport.send({
...CALL,
beforeSend: async () => {
throw new Error('lease store down');
},
});
held[0].answer();
await expect(waiting).rejects.toThrow('lease store down');
expect(held).toHaveLength(INVOKE_CONNECTIONS);
await answerAll();
const after = transport.send(CALL);
await vi.waitFor(() =>
expect(held).toHaveLength(INVOKE_CONNECTIONS + 1),
);
held[INVOKE_CONNECTIONS].answer();
await expect(after).resolves.toMatchObject({ status: 200 });
});
});
describe('DispatcherInvokeTransport schedule lane', () => {
@@ -811,138 +771,6 @@ describe('DispatcherInvokeTransport batches', () => {
]);
});
it('leaves a call superseded while gathered out of the batch', async () => {
const { fetchImpl, batches, singles } = streamingFetch();
const transport = transportFor(fetchImpl, { batchWindowMs: 20 });
let current = true;
const sent = [
transport.send(CALL),
transport.send({ ...CALL, beforeSend: async () => current }),
transport.send({ ...CALL, script: 'evw-other' }),
];
current = false;
await vi.waitFor(() => expect(batches).toHaveLength(1));
expect(batches[0].body.calls.map((call) => call.script)).toEqual([
'evw-test',
'evw-other',
]);
await expect(sent[1]).resolves.toEqual({
status: null,
error: 'invocation superseded',
});
batches[0].answer({ i: 0, status: 200, handled: true });
batches[0].answer({ i: 1, status: 200, handled: true });
await expect(Promise.all([sent[0], sent[2]])).resolves.toEqual([
{ status: 200, handled: true },
{ status: 200, handled: true },
]);
expect(singles).toHaveLength(0);
});
it('sends nothing when every gathered call was superseded', async () => {
const fetchImpl = vi.fn(
stubFetch(200, { [EVENTS_HANDLED_HEADER]: '1' }),
);
const transport = transportFor(fetchImpl as unknown as FetchImpl);
const superseded = { ...CALL, beforeSend: async () => false };
await expect(
Promise.all([
transport.send(superseded),
transport.send(superseded),
]),
).resolves.toEqual([
{ status: null, error: 'invocation superseded' },
{ status: null, error: 'invocation superseded' },
]);
expect(fetchImpl).not.toHaveBeenCalled();
});
it('fails only the call whose guard throws at flush', async () => {
const { fetchImpl, batches, singles } = streamingFetch();
const transport = transportFor(fetchImpl);
const failing = transport.send({
...CALL,
beforeSend: async () => {
throw new Error('lease store down');
},
});
const kept = transport.send(CALL);
await expect(failing).rejects.toThrow('lease store down');
// One call left, so it goes alone.
await expect(kept).resolves.toEqual({ status: 200, handled: true });
expect(batches).toHaveLength(0);
expect(singles).toHaveLength(1);
});
it('rechecks the guard before resending a batch call on its own', async () => {
const urls: string[] = [];
let current = true;
const fetchImpl = (async (url: string) => {
urls.push(url);
if (url.endsWith(EVENTS_DISPATCH_BATCH_PATH)) {
// Superseded while the batch was out.
current = false;
return new Response('no-such-route', {
status: 404,
headers: {
[EVENTS_DISPATCH_ERROR_HEADER]: 'no-such-route',
},
});
}
return new Response(null, {
status: 200,
headers: { [EVENTS_HANDLED_HEADER]: '1' },
});
}) as unknown as FetchImpl;
const transport = transportFor(fetchImpl);
await expect(
Promise.all([
transport.send({ ...CALL, beforeSend: async () => current }),
transport.send(CALL),
]),
).resolves.toEqual([
{ status: null, error: 'invocation superseded' },
{ status: 200, handled: true },
]);
expect(urls.map((url) => new URL(url).pathname)).toEqual([
EVENTS_DISPATCH_BATCH_PATH,
EVENTS_DISPATCH_PATH,
]);
});
it('answers a schedule run superseded in the batch as the early check does', async () => {
const { fetchImpl, batches } = streamingFetch();
const client = new EventsWorkerInvokerClient({
events: { invokeTimeoutMs: 5_000 },
} as unknown as IConfig);
client.setTransport(transportFor(fetchImpl, { batchWindowMs: 20 }));
let current = true;
const guard = vi.fn(async () => current);
const run = client.invoke(REQUEST, {
lane: 'schedule',
beforeSend: guard,
});
const other = client.invoke(REQUEST);
const another = client.invoke(REQUEST);
await vi.waitFor(() => expect(guard).toHaveBeenCalledTimes(1));
current = false;
await vi.waitFor(() => expect(batches).toHaveLength(1));
expect(batches[0].body.calls).toHaveLength(2);
await expect(run).resolves.toEqual(
await client.invoke(REQUEST, { beforeSend: async () => false }),
);
await expect(run).resolves.toMatchObject({
outcome: 'unavailable',
error: 'invocation superseded',
});
batches[0].answer({ i: 0, status: 200, handled: true });
batches[0].answer({ i: 1, status: 200, handled: true });
await Promise.all([other, another]);
});
it('builds each body when the batch goes out', async () => {
const { fetchImpl, batches } = streamingFetch();
const transport = transportFor(fetchImpl);
@@ -999,26 +827,6 @@ describe('DispatcherInvokeTransport batches', () => {
});
describe('EventsWorkerInvokerClient invoke options', () => {
it('does not send a superseded invocation', async () => {
const client = makeClient();
const send = vi.fn(async () => ({ status: 200, handled: true }));
client.setTransport({ send });
const result = await client.invoke(REQUEST, { beforeSend: async () => false });
expect(result.outcome).toBe('unavailable');
expect(send).not.toHaveBeenCalled();
});
it('rechecks before resending after a deployment', async () => {
const client = makeClient();
let current = true;
const send = vi.fn(async () => ({ status: null, dispatchReason: 'missing' }));
client.setTransport({ send });
client.setMissHandler(async () => { current = false; return 'deployed'; });
const result = await client.invoke(REQUEST, { beforeSend: async () => current });
expect(result.outcome).toBe('unavailable');
expect(send).toHaveBeenCalledTimes(1);
});
it('stamps the event as each attempt goes out', async () => {
const bodies: string[] = [];
const client = makeClient();
@@ -282,11 +282,6 @@ export interface EventsInvokeCall {
* connection first, so a send time stamped into it is the real one.
*/
render?: () => string;
/**
* Rechecked by a transport that waits before sending; false drops the call
* with {@link supersededAnswer}.
*/
beforeSend?: () => Promise<boolean>;
}
/** What one attempt at delivering an invocation came back with. */
@@ -339,8 +334,6 @@ export interface WorkerInvokeResult {
/** How one invocation is sent. */
export interface WorkerInvokeOptions {
lane?: InvokeLane;
/** Rechecked before each send, including a resend after deployment. */
beforeSend?: () => Promise<boolean>;
/** Applied to the event as each attempt is sent. */
stampEvent?: (event: unknown) => unknown;
}
@@ -348,12 +341,6 @@ export interface WorkerInvokeOptions {
const NO_TRANSPORT =
'no events dispatcher configured (url and internal secret)';
/** What a call whose `beforeSend` said no answers instead of being sent. */
const supersededAnswer = (): EventsInvokeAnswer => ({
status: null,
error: 'invocation superseded',
});
export class EventsWorkerInvokerClient extends PuterClient {
/** Set only where invocations do not leave the process — local development. */
#transport: EventsInvokeTransport | null = null;
@@ -408,13 +395,11 @@ export class EventsWorkerInvokerClient extends PuterClient {
const margin = Math.min(EVENTS_DEADLINE_MARGIN_MS, timeoutMs / 2);
const deadline =
Date.now() + timeoutMs * EVENTS_WORKER_LEASE_FACTOR - margin;
const send = async (
const send = (
deployed: boolean,
sendTimeoutMs = timeoutMs,
): Promise<EventsInvokeAnswer> => {
if (options.beforeSend && !(await options.beforeSend()))
return supersededAnswer();
return transport.send({
): Promise<EventsInvokeAnswer> =>
transport.send({
script: request.script,
appUid: request.appUid,
...(request.holder ? { holder: request.holder } : {}),
@@ -424,11 +409,7 @@ export class EventsWorkerInvokerClient extends PuterClient {
...(deployed ? { deployed: true } : {}),
...(options.lane ? { lane: options.lane } : {}),
...(stampEvent ? { render } : {}),
...(options.beforeSend
? { beforeSend: options.beforeSend }
: {}),
});
};
// A send may wait for a connection for half its timeout before the
// run's own timeout starts, so it is given what is left divided by that.
@@ -553,8 +534,6 @@ export class EventsWorkerInvokerClient extends PuterClient {
interface BatchedCall {
call: EventsInvokeCall;
resolve: (answer: EventsInvokeAnswer) => void;
/** For a `beforeSend` that threw. */
reject: (err: unknown) => void;
/** When its wait for a connection began, for what is left of it. */
waitingSince: number;
}
@@ -625,14 +604,9 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
error: 'no free connection to the events dispatcher',
};
try {
if (this.#batchSize === 1) {
// The wait for a connection may have outlived the call.
if (call.beforeSend && !(await call.beforeSend()))
return supersededAnswer();
return await this.#sendOne(call);
}
return await new Promise<EventsInvokeAnswer>((resolve, reject) => {
this.#batch.push({ call, resolve, reject, waitingSince });
if (this.#batchSize === 1) return await this.#sendOne(call);
return await new Promise<EventsInvokeAnswer>((resolve) => {
this.#batch.push({ call, resolve, waitingSince });
if (this.#batch.length >= this.#batchSize) this.#flush();
else
this.#batchTimer ??= setTimeout(
@@ -652,12 +626,6 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
const batch = this.#batch;
this.#batch = [];
if (batch.length === 0) return;
if (batch.some(({ call }) => call.beforeSend))
void this.#sendStillCurrent(batch);
else this.#sendGathered(batch);
}
#sendGathered(batch: BatchedCall[]): void {
if (batch.length === 1 || Date.now() < this.#batchUnsupportedUntil) {
for (const { call, resolve } of batch)
void this.#sendOne(call).then(resolve);
@@ -666,26 +634,6 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
void this.#sendBatch(batch);
}
/** Drops calls superseded while they were gathered, then sends the rest. */
async #sendStillCurrent(batch: BatchedCall[]): Promise<void> {
const checked = await Promise.all(
batch.map(async (entry) => {
if (!entry.call.beforeSend) return entry;
try {
if (await entry.call.beforeSend()) return entry;
entry.resolve(supersededAnswer());
} catch (err) {
entry.reject(err);
}
return null;
}),
);
const current = checked.filter(
(entry): entry is BatchedCall => entry !== null,
);
if (current.length > 0) this.#sendGathered(current);
}
/** One call on the single route. */
async #sendOne(call: EventsInvokeCall): Promise<EventsInvokeAnswer> {
try {
@@ -740,15 +688,13 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
let open = batch.length;
const answered = batch.map(() => false);
const timers: ReturnType<typeof setTimeout>[] = [];
const finish = (i: number, done: (entry: BatchedCall) => void) => {
const settle = (i: number, answer: EventsInvokeAnswer): void => {
if (answered[i]) return;
answered[i] = true;
clearTimeout(timers[i]);
done(batch[i]);
batch[i].resolve(answer);
if (--open === 0) hangUp.abort();
};
const settle = (i: number, answer: EventsInvokeAnswer): void =>
finish(i, (entry) => entry.resolve(answer));
const settleRest = (answer: EventsInvokeAnswer): void => {
for (let i = 0; i < batch.length; i++) settle(i, answer);
};
@@ -831,22 +777,8 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
this.#batchUnsupportedUntil =
Date.now() + BATCH_UNSUPPORTED_RETRY_MS;
batch.forEach(({ call }, i) => {
if (answered[i]) return;
const { beforeSend } = call;
if (!beforeSend) {
if (!answered[i])
void this.#sendOne(call).then((a) => settle(i, a));
return;
}
// The batch round trip may have outlived the call.
void (async () => {
try {
if (!(await beforeSend()))
return settle(i, supersededAnswer());
} catch (err) {
return finish(i, (entry) => entry.reject(err));
}
settle(i, await this.#sendOne(call));
})();
});
return;
}
@@ -236,17 +236,6 @@ export class EventsController extends PuterController {
});
}
@Post('/schedules/update', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async updateSchedule(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const { scheduleId, changes } = bodyRecord(req);
res.json(await this.#schedules().update(actor, scheduleId, changes));
}
/**
* POST /events/schedules/remove — end one schedule. An id the caller does
* not hold reads as absent.
@@ -75,7 +75,6 @@ const fakeSchedules = () =>
total: 3,
})),
get: vi.fn(async () => view),
update: vi.fn(async () => view),
remove: vi.fn(async () => {}),
receivePeerChanges: vi.fn(async () => ({ applied: 2 })),
prepareSendAt: vi.fn(),
@@ -130,10 +129,6 @@ const routes: Array<[string, (response: Response) => Promise<void>]> = [
(r) =>
events.getSchedule(req({ params: { scheduleId: SCHEDULE_ID } }), r),
],
[
'POST /events/schedules/update',
(r) => events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes: { every: '2h' } } }), r),
],
[
'POST /events/schedules/remove',
(r) =>
@@ -255,16 +250,6 @@ describe('with services.schedules', () => {
expect(sent.body).toEqual(view);
});
it('updates through update and answers its view', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
const changes = { every: '2h' };
await events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes } }), response);
expect(schedules.update).toHaveBeenCalledWith(actor, SCHEDULE_ID, changes);
expect(sent.body).toEqual(view);
});
it('removes through remove and answers {}', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
@@ -1,77 +0,0 @@
import type { Request, Response } from 'express';
import { describe, expect, it } from 'vitest';
import { makeActor } from '../../core/actor.js';
import type {
ScheduleServiceContract,
ScheduleView,
} from '../../core/schedules.js';
import { EventsController } from './EventsController.js';
const actor = makeActor({
user: { id: 1, uuid: 'user-1', username: 'user1' } as never,
});
const request = (body: unknown) => ({ actor, body }) as Request;
const controller = (schedules?: Pick<ScheduleServiceContract, 'update'>) =>
new EventsController(
{} as never,
{} as never,
{} as never,
{ schedules } as never,
{} as never,
);
describe('update schedule route contract', () => {
it('answers events_disabled when schedules are unavailable', async () => {
await expect(
controller().updateSchedule(
request({ scheduleId: 'id', changes: {} }),
{} as Response,
),
).rejects.toMatchObject({
legacyCode: 'events_disabled',
statusCode: 503,
noAlarm: true,
});
});
it('passes the actor, id and changes through and returns the updated view', async () => {
const view = {
scheduleId: 'id',
kind: 'recurring',
state: 'suspended',
} as ScheduleView;
const changes = { cron: '0 9 * * *', tz: 'UTC' };
const calls: unknown[][] = [];
const events = controller({
update: async (...args) => {
calls.push(args);
return view;
},
});
let sent: unknown;
await events.updateSchedule(request({ scheduleId: 'id', changes }), {
json: (value: unknown) => {
sent = value;
},
} as Response);
expect(calls).toEqual([[actor, 'id', changes]]);
expect(sent).toBe(view);
});
it('preserves validation and sending errors from the implementation', async () => {
const error = {
code: 'events_schedule_sending',
message: 'Already sending',
};
await expect(
controller({
update: async () => {
throw error;
},
}).updateSchedule(
request({ scheduleId: 'id', changes: {} }),
{} as Response,
),
).rejects.toBe(error);
});
});
@@ -22,7 +22,6 @@ import path from 'node:path';
import { PuterController } from '../types.js';
import { toAppShellView } from '../../util/appShellView.js';
import type { PuterRouter } from '../../core/http/PuterRouter';
import type { IConfig } from '../../types';
import type {
PuterHomepageService,
PageMeta,
@@ -44,9 +43,9 @@ const APP_LANDING_LIMIT = {
};
/**
* App descriptions can run long; search snippets and social cards want a short
* blurb. Collapses whitespace, then cuts at a word boundary with an ellipsis so
* the result never exceeds `APP_META_DESCRIPTION_MAX`.
* App descriptions can run long; search snippets and social cards want a
* short blurb. Collapses whitespace, then cuts at a word boundary with an
* ellipsis so the result never exceeds `APP_META_DESCRIPTION_MAX`.
*/
export function appMetaDescription(text: string): string {
const clean = text.replace(/\s+/g, ' ').trim();
@@ -57,47 +56,6 @@ export function appMetaDescription(text: string): string {
return `${cut.replace(/[\s,;:.!?…—-]+$/, '')}…`;
}
/**
* `frame-ancestors` for the app browser embed. A wildcard over `*.<domain>`
* would also admit user-hosted sites whenever the hosting domains sit under
* it, so the embedding page is named exactly, from the request's referrer.
*/
export function embedFrameAncestors(
config: Pick<
IConfig,
| 'origin'
| 'domain'
| 'static_hosting_domain'
| 'static_hosting_domain_alt'
| 'private_app_hosting_domain'
| 'private_app_hosting_domain_alt'
>,
referer: string | undefined,
): string {
const gui = new URL(config.origin ?? `https://${config.domain}`);
let parent: URL;
try {
parent = new URL(referer ?? '');
} catch {
return "'self'";
}
const host = parent.hostname.toLowerCase();
const hostingDomains = [
config.static_hosting_domain,
config.static_hosting_domain_alt,
config.private_app_hosting_domain,
config.private_app_hosting_domain_alt,
]
.map((d) => (d ?? '').toLowerCase().split(':')[0])
.filter(Boolean);
const allowed =
parent.protocol === gui.protocol &&
parent.port === gui.port &&
host.endsWith(`.${config.domain.toLowerCase()}`) &&
!hostingDomains.some((d) => host === d || host.endsWith(`.${d}`));
return allowed ? `'self' ${parent.origin}` : "'self'";
}
/**
* Routes that render the Puter GUI shell, plus a catch-all static fallback
* under `<gui_assets_root>/src` for non-dist/src paths (images, fonts, lib
@@ -147,21 +105,6 @@ export class HomepageController extends PuterController {
router.get('/', {}, (req, res) => sendShell(req, res));
router.get('/embed/apps', {}, (req, res) => {
// SDK bootstrap parameters must not change this page's account or API origin.
if (Object.keys(req.query).length) {
res.redirect('/embed/apps');
return;
}
res.removeHeader('X-Frame-Options');
res.setHeader(
'Content-Security-Policy',
`frame-ancestors ${embedFrameAncestors(this.config, req.get('referer'))}`,
);
res.setHeader('Cache-Control', 'no-store');
res.send(homepage.renderAppBrowserEmbed());
});
router.get('/settings', {}, (req, res) => sendShell(req, res));
router.get('/settings/*splat', {}, (req, res) => sendShell(req, res));
@@ -201,7 +201,6 @@ describe('PuterAIController.openaiResponses parameter forwarding', () => {
context_management: [{ type: 'compaction' }],
prompt_cache_key: 'ck',
prompt_cache_retention: '24h',
prompt_cache_options: { ttl: '12h', prewarm: true },
store: true,
truncation: 'auto',
service_tier: 'default',
@@ -227,9 +226,6 @@ describe('PuterAIController.openaiResponses parameter forwarding', () => {
context_management: [{ type: 'compaction' }],
prompt_cache_key: 'ck',
prompt_cache_retention: '24h',
// Forwarded as sent; the provider keeps only the fields it
// accepts.
prompt_cache_options: { ttl: '12h', prewarm: true },
// forced false regardless of the client's request.
store: false,
truncation: 'auto',
@@ -363,18 +359,13 @@ describe('PuterAIController.openaiResponses input normalization', () => {
role: 'user',
content: [
'a bare string part',
{
type: 'input_text',
text: 'typed text',
prompt_cache_breakpoint: { mode: 'explicit', ttl: '12h' },
},
{ type: 'input_text', text: 'typed text' },
{ type: 'output_text', text: 'echoed text' },
{
type: 'input_image',
detail: 'high',
image_url: 'https://img.test/a.png',
file_id: 'file_img',
prompt_cache_breakpoint: { mode: 'explicit' },
},
{ type: 'input_audio', input_audio: { data: 'AAA' } },
{
@@ -395,18 +386,13 @@ describe('PuterAIController.openaiResponses input normalization', () => {
role: 'user',
content: [
{ type: 'text', text: 'a bare string part' },
{
type: 'text',
text: 'typed text',
prompt_cache_breakpoint: { mode: 'explicit', ttl: '12h' },
},
{ type: 'text', text: 'typed text' },
{ type: 'text', text: 'echoed text' },
{
type: 'image_url',
detail: 'high',
image_url: { url: 'https://img.test/a.png' },
file_id: 'file_img',
prompt_cache_breakpoint: { mode: 'explicit' },
},
{ type: 'input_audio', input_audio: { data: 'AAA' } },
{
@@ -653,33 +639,6 @@ describe('PuterAIController.openaiResponses output items', () => {
// -- /openai/v1/chat/completions edges -------------------------------
describe('PuterAIController.openaiChatCompletions translation edges', () => {
it('forwards the prompt-cache fields to the driver', async () => {
const completeSpy = stubChatComplete({
message: { role: 'assistant', content: 'ok' },
});
const { res } = makeRes();
await controller.openaiChatCompletions(
makeReq({
body: {
model: 'gpt-test',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_key: 'ck',
prompt_cache_retention: 'in_memory',
prompt_cache_options: { ttl: '30m' },
},
}),
res,
);
const args = completeSpy.mock.calls[0]![0] as Record<string, unknown>;
expect(args).toMatchObject({
prompt_cache_key: 'ck',
prompt_cache_retention: 'in_memory',
prompt_cache_options: { ttl: '30m' },
});
});
it('derives tool_calls from tool_use content blocks when the message has none', async () => {
stubChatComplete({
message: {
@@ -447,7 +447,6 @@ export class PuterAIController extends PuterController {
body.reasoning_effort as ICompleteArguments['reasoning_effort'],
}
: {}),
...promptCacheArgsFromBody(body),
...openaiCompatProvider(body),
...(body.fallback === false ? { fallback: false } : {}),
};
@@ -843,7 +842,15 @@ export class PuterAIController extends PuterController {
body.compaction as ICompleteArguments['compaction'],
}
: {}),
...promptCacheArgsFromBody(body),
...(body.prompt_cache_key
? { prompt_cache_key: String(body.prompt_cache_key) }
: {}),
...(body.prompt_cache_retention
? {
prompt_cache_retention:
body.prompt_cache_retention as ICompleteArguments['prompt_cache_retention'],
}
: {}),
// forced off regardless of what the caller sent — OpenAI
// defaults `store` to true, which would persist state server-side
// on OpenAI's end that this route gives no way to retrieve.
@@ -1336,33 +1343,6 @@ const openaiCompatProvider = (
return { provider: DEFAULTS.openaiChat };
};
/**
* The prompt-cache fields both OpenAI-shaped routes forward as sent; the
* provider checks the values.
*/
const promptCacheArgsFromBody = (
body: Record<string, unknown>,
): Pick<
ICompleteArguments,
'prompt_cache_key' | 'prompt_cache_retention' | 'prompt_cache_options'
> => ({
...(body.prompt_cache_key
? { prompt_cache_key: String(body.prompt_cache_key) }
: {}),
...(body.prompt_cache_retention
? {
prompt_cache_retention:
body.prompt_cache_retention as ICompleteArguments['prompt_cache_retention'],
}
: {}),
...(body.prompt_cache_options
? {
prompt_cache_options:
body.prompt_cache_options as ICompleteArguments['prompt_cache_options'],
}
: {}),
});
const randomId = (): string => crypto.randomUUID().replace(/-/g, '');
const generateId = (prefix: string): string => `${prefix}_${randomId()}`;
@@ -1610,24 +1590,12 @@ const parseJsonMaybe = (value: unknown): unknown => {
}
};
/** An OpenAI `prompt_cache_breakpoint` marker, kept on the translated part. */
const cacheBreakpointOf = (
p: Record<string, unknown>,
): { prompt_cache_breakpoint?: unknown } =>
p.prompt_cache_breakpoint && typeof p.prompt_cache_breakpoint === 'object'
? { prompt_cache_breakpoint: p.prompt_cache_breakpoint }
: {};
const normalizeContentPart = (part: unknown): Record<string, unknown> => {
if (typeof part === 'string') return { type: 'text', text: part };
if (!part || typeof part !== 'object') return { type: 'text', text: '' };
const p = part as Record<string, unknown>;
if (p.type === 'input_text' || p.type === 'output_text') {
return {
type: 'text',
text: String(p.text ?? ''),
...cacheBreakpointOf(p),
};
return { type: 'text', text: String(p.text ?? '') };
}
if (p.type === 'input_image') {
return {
@@ -1635,7 +1603,6 @@ const normalizeContentPart = (part: unknown): Record<string, unknown> => {
...(p.detail ? { detail: p.detail } : {}),
...(p.image_url ? { image_url: { url: p.image_url } } : {}),
...(p.file_id ? { file_id: p.file_id } : {}),
...cacheBreakpointOf(p),
};
}
if (p.type === 'input_audio')
@@ -1647,7 +1614,6 @@ const normalizeContentPart = (part: unknown): Record<string, unknown> => {
...(p.file_id ? { file_id: p.file_id } : {}),
...(p.file_url ? { file_url: p.file_url } : {}),
...(p.filename ? { filename: p.filename } : {}),
...cacheBreakpointOf(p),
};
}
return p;
+1 -9
View File
@@ -170,16 +170,8 @@ export interface ScheduleServiceContract {
request: ScheduleListRequest,
): Promise<PageResult<ScheduleView>>;
get(actor: Actor, scheduleId: unknown): Promise<ScheduleView>;
update(
actor: Actor,
scheduleId: unknown,
changes: unknown,
): Promise<ScheduleView>;
remove(actor: Actor, scheduleId: unknown): Promise<void>;
/** `results`: each change's index outcome, in order (`null` if dropped). */
receivePeerChanges(
body: unknown,
): Promise<{ applied: number; results?: Array<number | null> }>;
receivePeerChanges(body: unknown): Promise<{ applied: number }>;
/** Checks a caller supplying `sendAt`; `events_disabled` when off. */
prepareSendAt(actor: Actor, sendAt: unknown): Promise<DeferredSendPlan>;
createDeferredAppMessage(
@@ -449,22 +449,6 @@ describe('AzureResponsesProvider.complete request shape', () => {
expect(args.service_tier).toBe('default');
});
it('sends the wire spelling of the in-memory cache retention', async () => {
const provider = makeProvider();
responsesCreateMock.mockResolvedValueOnce(okResponse);
await withTestActor(() =>
provider.complete({
model: 'gpt-5.3-codex',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_retention: 'in-memory',
} as never),
);
const [args] = responsesCreateMock.mock.calls[0]!;
expect(args.prompt_cache_retention).toBe('in_memory');
});
it.each([
['previous_response_id', 'resp_1'],
['conversation', 'conv_1'],
@@ -35,7 +35,6 @@ import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import {
clampReasoningEffort,
openAICompatParams,
promptCacheRetentionToWire,
rejectStatefulResponsesFields,
} from '../../utils/openaiParams.js';
import { buildCostsOverride } from '../../utils/pricing.js';
@@ -210,9 +209,6 @@ export class AzureResponsesProvider implements IChatProvider {
const userIdentifier = upstreamUserIdentifier(actor);
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
const cacheKey = prompt_cache_key ?? userIdentifier;
const cacheRetention = promptCacheRetentionToWire(
prompt_cache_retention,
);
// Resolve any `puter_path` content parts into inline base64 data URLs
// before the Responses API sees them.
@@ -311,8 +307,8 @@ export class AzureResponsesProvider implements IChatProvider {
...(instructions !== undefined ? { instructions } : {}),
...(metadata !== undefined ? { metadata } : {}),
...(cacheKey !== undefined ? { prompt_cache_key: cacheKey } : {}),
...(cacheRetention !== undefined
? { prompt_cache_retention: cacheRetention }
...(prompt_cache_retention !== undefined
? { prompt_cache_retention }
: {}),
...(store !== undefined ? { store } : {}),
...(max_tokens !== undefined
@@ -23,7 +23,6 @@ import { HttpError } from '../../../../core/http/HttpError.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { upstreamUserIdentifier } from '../../../util/upstreamIdentifier.js';
import type { IChatModel, ICompleteArguments } from '../../types.js';
import { promptCacheRetentionToWire } from '../../utils/openaiParams.js';
import { sdkClientOptions } from '../../utils/sdkClient.js';
import {
type ChatProviderConfig,
@@ -108,9 +107,12 @@ export class MetaProvider extends OpenAICompatProvider {
? requestedEffort
: undefined;
const cacheRetention = promptCacheRetentionToWire(
args.prompt_cache_retention,
);
// Puter spells the in-memory retention with a hyphen; Meta's enum
// uses an underscore.
const cacheRetention =
args.prompt_cache_retention === 'in-memory'
? 'in_memory'
: args.prompt_cache_retention;
// The identifier is Puter's abuse attribution, so `custom` can't
// override it. Cache key defaults to it; see upstreamUserIdentifier.
@@ -249,130 +249,6 @@ describe('OpenAiChatProvider.complete argument validation', () => {
expect(result).toEqual({ delegated: true });
});
it('delegates a 12h cache lifetime to the Responses provider, which is the only API that writes it', async () => {
const { provider } = makeProvider();
const sibling = {
complete: vi.fn().mockResolvedValue({ delegated: true }),
};
provider.setResponsesProvider(
sibling as unknown as Parameters<
typeof provider.setResponsesProvider
>[0],
);
const params = {
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_options: { ttl: '12h' as const },
};
const result = await withTestActor(() => provider.complete(params));
expect(createMock).not.toHaveBeenCalled();
expect(sibling.complete).toHaveBeenCalledWith(params);
expect(result).toEqual({ delegated: true });
});
it.each([
['30m', { mode: 'explicit', ttl: '30m' }],
['12h', { mode: 'explicit', ttl: '12h' }],
])(
'delegates a %s breakpoint lifetime to the Responses provider, since Chat Completions rejects the field',
async (_ttl, breakpoint) => {
const { provider } = makeProvider();
const sibling = {
complete: vi.fn().mockResolvedValue({ delegated: true }),
};
provider.setResponsesProvider(
sibling as unknown as Parameters<
typeof provider.setResponsesProvider
>[0],
);
const params = {
model: 'gpt-6-luna',
messages: [
{
role: 'system',
content: [
{
type: 'text',
text: 'long prefix',
prompt_cache_breakpoint: breakpoint,
},
],
},
{ role: 'user', content: 'hi' },
],
};
const result = await withTestActor(() =>
provider.complete(params as never),
);
expect(createMock).not.toHaveBeenCalled();
expect(sibling.complete).toHaveBeenCalledWith(params);
expect(result).toEqual({ delegated: true });
},
);
it('keeps a breakpoint without a lifetime on the Chat Completions path, intact', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: { content: 'ok', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 1, completion_tokens: 1 },
});
await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [
{
role: 'system',
content: [
{
type: 'text',
text: 'long prefix',
prompt_cache_breakpoint: { mode: 'explicit' },
},
],
},
{ role: 'user', content: 'hi' },
],
} as never),
);
const [args] = createMock.mock.calls[0]!;
expect(args.messages[0]).toMatchObject({
role: 'system',
content: [
{
type: 'text',
text: 'long prefix',
prompt_cache_breakpoint: { mode: 'explicit' },
},
],
});
});
it('throws 400 for a 12h cache lifetime without a Responses sibling provider', async () => {
const { provider } = makeProvider();
await expect(
withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_options: { ttl: '12h' },
}),
),
).rejects.toMatchObject({ statusCode: 400 });
expect(createMock).not.toHaveBeenCalled();
});
it('requestPricing delegates to the Responses sibling (web_search\'s own hold), and no-ops without one', async () => {
const { provider } = makeProvider();
const args = {
@@ -473,31 +349,6 @@ describe('OpenAiChatProvider.complete request shape', () => {
expect(args.safety_identifier).toBe('puter-u42');
});
it('forwards 30m prompt_cache_options and the wire spelling of prompt_cache_retention', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hello' }],
prompt_cache_retention: 'in-memory',
prompt_cache_options: {
mode: 'explicit',
ttl: '30m',
prewarm: true,
} as never,
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.prompt_cache_retention).toBe('in_memory');
expect(args.prompt_cache_options).toEqual({
mode: 'explicit',
ttl: '30m',
});
});
it.each([
'gpt-6-astra',
'gpt-6.1-sol',
@@ -814,7 +665,7 @@ describe('OpenAiChatProvider.complete non-stream output', () => {
},
});
const result = await withTestActor(() =>
await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
@@ -834,13 +685,6 @@ describe('OpenAiChatProvider.complete non-stream output', () => {
cached_tokens: 1000 * Number(luna.costs.cached_tokens),
cache_write_tokens: 3000 * Number(luna.costs.cache_write_tokens),
});
// Chat Completions only writes OpenAI's default 30m entries.
expect((result as { usageDetails: unknown }).usageDetails).toEqual({
inputTokens: 1000,
outputTokens: 12,
cacheReadTokens: 1000,
cacheWrite30mTokens: 3000,
});
});
it('zeroes cached_tokens when prompt_tokens_details is missing', async () => {
@@ -35,10 +35,7 @@ import { make_openai_tools } from '../../utils/FunctionCalling.js';
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import {
clampReasoningEffort,
messagesHaveCacheBreakpointTtl,
openAICompatParams,
promptCacheOptionsToWire,
promptCacheRetentionToWire,
} from '../../utils/openaiParams.js';
import { OPEN_AI_MODELS } from './models.js';
import type { OpenAiResponsesChatProvider } from './OpenAiChatResponsesProvider.js';
@@ -129,11 +126,8 @@ export class OpenAiChatProvider implements IChatProvider {
temperature,
text,
prompt_cache_key,
prompt_cache_retention,
prompt_cache_options,
} = params;
let { messages, model } = params;
const cacheOptions = promptCacheOptionsToWire(prompt_cache_options);
if (
tools?.filter((e) => isWebSearchTool(e as Record<string, unknown>))
.length
@@ -163,21 +157,6 @@ export class OpenAiChatProvider implements IChatProvider {
}
return await this.#responsesProvider.complete(params);
}
// Only the Responses API writes 12h cache entries or takes a lifetime
// on a `prompt_cache_breakpoint`; Chat Completions rejects both.
if (
cacheOptions?.ttl === '12h' ||
messagesHaveCacheBreakpointTtl(messages)
) {
if (!this.#responsesProvider) {
throw new HttpError(
400,
'a prompt cache lifetime requires the OpenAI Responses provider, which is not configured',
{ legacyCode: 'bad_request' },
);
}
return await this.#responsesProvider.complete(params);
}
// Validate messages
if (!Array.isArray(messages)) {
throw new HttpError(400, '`messages` must be an array', {
@@ -201,9 +180,6 @@ export class OpenAiChatProvider implements IChatProvider {
const userIdentifier = upstreamUserIdentifier(actor);
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
const cacheKey = prompt_cache_key ?? userIdentifier;
const cacheRetention = promptCacheRetentionToWire(
prompt_cache_retention,
);
// Strip Anthropic-only shape a fallback-replayed message can carry
// (thinking/server-tool blocks, cache_control, citations) before the
@@ -240,12 +216,6 @@ export class OpenAiChatProvider implements IChatProvider {
user: userIdentifier,
safety_identifier: userIdentifier,
...(cacheKey !== undefined ? { prompt_cache_key: cacheKey } : {}),
...(cacheRetention !== undefined
? { prompt_cache_retention: cacheRetention }
: {}),
...(cacheOptions !== undefined
? { prompt_cache_options: cacheOptions }
: {}),
messages: messages,
model: modelUsed.id,
...(mappedTools?.length ? { tools: mappedTools } : {}),
@@ -284,11 +254,28 @@ export class OpenAiChatProvider implements IChatProvider {
);
return OpenAiUtil.handle_completion_output({
usage_calculator: ({ usage, setUsageCosts, setUsageDetails }) => {
// GPT-5.6 and later report cache writes inside
// `prompt_tokens`, like cached reads; Chat Completions only
// writes the default 30m entries, billed at 1.25x input.
const trackedUsage = OpenAiUtil.splitCachedPrompt(usage);
usage_calculator: ({ usage, setUsageCosts }) => {
const cachedTokens =
usage.prompt_tokens_details?.cached_tokens ?? 0;
// GPT-5.6 and later bill cache writes at 1.25x input. They're
// reported inside `prompt_tokens`, like cached reads.
// The SDK doesn't type `cache_write_tokens` yet.
const cacheWriteTokens =
(
usage.prompt_tokens_details as
{ cache_write_tokens?: number } | undefined
)?.cache_write_tokens ?? 0;
const trackedUsage = {
prompt_tokens:
(usage.prompt_tokens ?? 0) -
cachedTokens -
cacheWriteTokens,
completion_tokens: usage.completion_tokens ?? 0,
cached_tokens: cachedTokens,
...(cacheWriteTokens
? { cache_write_tokens: cacheWriteTokens }
: {}),
};
const metered = meterChatUsage(
this.#meteringService,
@@ -298,11 +285,6 @@ export class OpenAiChatProvider implements IChatProvider {
trackedUsage,
);
setUsageCosts(metered.costs);
setUsageDetails(
OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
writeTtl: '30m',
})!,
);
return metered.usage;
},
stream,
@@ -246,7 +246,6 @@ describe('OpenAiResponsesChatProvider model catalog', () => {
prompt_tokens: input,
cached_tokens: cached,
cache_write_tokens: input * 1.25,
cache_write_12h_tokens: input * 2.5,
completion_tokens: output,
},
context: 1_050_000,
@@ -845,7 +844,7 @@ describe('OpenAiResponsesChatProvider.complete non-stream output', () => {
},
});
const result = await withTestActor(() =>
await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
@@ -868,115 +867,6 @@ describe('OpenAiResponsesChatProvider.complete non-stream output', () => {
cached_tokens: 1000 * Number(luna.costs.cached_tokens),
cache_write_tokens: 3000 * Number(luna.costs.cache_write_tokens),
});
// Writes without a per-lifetime breakdown are OpenAI's default 30m
// entries, and the neutral breakdown says so.
expect((result as { usageDetails: unknown }).usageDetails).toEqual({
inputTokens: 1000,
outputTokens: 20,
cacheReadTokens: 1000,
cacheWrite30mTokens: 3000,
});
});
it('bills 12h cache writes at 2.5x input and the other writes at 1.25x', async () => {
const luna = OPEN_AI_MODELS.find((m) => m.id === 'gpt-6-luna')!;
const { provider } = makeProvider();
// One request can write both lifetimes: an explicit 12h breakpoint
// plus the implicit 30m one OpenAI adds at the end of the prompt.
responsesCreateMock.mockResolvedValueOnce({
output: [{ role: 'assistant' }],
output_text: 'ok',
usage: {
input_tokens: 6026,
output_tokens: 5,
input_tokens_details: {
cached_tokens: 0,
cache_write_tokens: 6023,
cache_write_tokens_details: {
ttl_12h_tokens: 6012,
ttl_30m_tokens: 11,
},
},
},
});
const result = await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_options: { ttl: '12h' },
}),
);
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
expect(usage).toEqual({
prompt_tokens: 3,
completion_tokens: 5,
cached_tokens: 0,
cache_write_tokens: 11,
cache_write_12h_tokens: 6012,
});
expect(luna.costs.cache_write_12h_tokens).toBe(
Number(luna.costs.prompt_tokens) * 2.5,
);
expect(overrides).toEqual({
prompt_tokens: 3 * Number(luna.costs.prompt_tokens),
completion_tokens: 5 * Number(luna.costs.completion_tokens),
cached_tokens: 0,
cache_write_tokens: 11 * Number(luna.costs.cache_write_tokens),
cache_write_12h_tokens:
6012 * Number(luna.costs.cache_write_12h_tokens),
});
expect((result as { usageDetails: unknown }).usageDetails).toEqual({
inputTokens: 3,
outputTokens: 5,
cacheWrite30mTokens: 11,
cacheWrite12hTokens: 6012,
});
});
it('forwards prompt_cache_options and the wire spelling of prompt_cache_retention', async () => {
const { provider } = makeProvider();
responsesCreateMock.mockResolvedValueOnce({
output: [{ role: 'assistant' }],
output_text: 'ok',
usage: { input_tokens: 1, output_tokens: 1 },
});
await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_retention: 'in-memory',
prompt_cache_options: {
mode: 'explicit',
ttl: '12h',
comparison_response_id: 'resp_1',
} as never,
}),
);
const [args] = responsesCreateMock.mock.calls[0]!;
expect(args.prompt_cache_retention).toBe('in_memory');
expect(args.prompt_cache_options).toEqual({
mode: 'explicit',
ttl: '12h',
});
});
it('rejects a cache lifetime OpenAI does not offer before calling upstream', async () => {
const { provider } = makeProvider();
await expect(
withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_options: { ttl: '2h' } as never,
}),
),
).rejects.toMatchObject({ statusCode: 400 });
expect(responsesCreateMock).not.toHaveBeenCalled();
});
it.each([
@@ -1178,69 +1068,6 @@ describe('OpenAiResponsesChatProvider.complete streaming', () => {
});
});
it('labels streamed cache writes by lifetime on the final usage chunk', async () => {
const luna = OPEN_AI_MODELS.find((m) => m.id === 'gpt-6-luna')!;
const { provider } = makeProvider();
responsesCreateMock.mockReturnValueOnce(
asAsyncIterable([
{ type: 'response.output_text.delta', delta: 'ok' },
{
type: 'response.completed',
response: {
usage: {
input_tokens: 6026,
output_tokens: 5,
input_tokens_details: {
cached_tokens: 0,
cache_write_tokens: 6023,
cache_write_tokens_details: {
ttl_12h_tokens: 6012,
ttl_30m_tokens: 11,
},
},
},
},
},
]),
);
const result = await withTestActor(() =>
provider.complete({
model: 'gpt-6-luna',
messages: [{ role: 'user', content: 'say ok' }],
stream: true,
prompt_cache_options: { ttl: '12h' },
}),
);
const harness = makeCapturingChatStream();
await (
result as {
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
}
).init_chat_stream({ chatStream: harness.chatStream });
const usageEvent = harness.events().find((e) => e.type === 'usage');
expect(usageEvent?.usage).toEqual({
prompt_tokens: 3,
completion_tokens: 5,
cached_tokens: 0,
cache_write_tokens: 11,
cache_write_12h_tokens: 6012,
});
expect(usageEvent?.usageDetails).toEqual({
inputTokens: 3,
outputTokens: 5,
cacheWrite30mTokens: 11,
cacheWrite12hTokens: 6012,
});
const [, , , overrides] = recordSpy.mock.calls[0]!;
expect(overrides).toMatchObject({
cache_write_tokens: 11 * Number(luna.costs.cache_write_tokens),
cache_write_12h_tokens:
6012 * Number(luna.costs.cache_write_12h_tokens),
});
});
it('emits a tool_use block when response.output_item.done arrives with a function_call', async () => {
const { provider } = makeProvider();
responsesCreateMock.mockReturnValueOnce(
@@ -32,8 +32,6 @@ import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
import {
clampReasoningEffort,
openAICompatParams,
promptCacheOptionsToWire,
promptCacheRetentionToWire,
rejectStatefulResponsesFields,
} from '../../utils/openaiParams.js';
import { responseSamplingParams } from '../../utils/responseSampling.js';
@@ -152,7 +150,6 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
prompt,
prompt_cache_key,
prompt_cache_retention,
prompt_cache_options,
store,
top_p,
truncation,
@@ -198,10 +195,6 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
const userIdentifier = upstreamUserIdentifier(actor);
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
const cacheKey = prompt_cache_key ?? userIdentifier;
const cacheRetention = promptCacheRetentionToWire(
prompt_cache_retention,
);
const cacheOptions = promptCacheOptionsToWire(prompt_cache_options);
if (tools) {
// Unravel tools to OpenAI Responses API format
@@ -295,11 +288,8 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
...(instructions !== undefined ? { instructions } : {}),
...(metadata !== undefined ? { metadata } : {}),
...(cacheKey !== undefined ? { prompt_cache_key: cacheKey } : {}),
...(cacheRetention !== undefined
? { prompt_cache_retention: cacheRetention }
: {}),
...(cacheOptions !== undefined
? { prompt_cache_options: cacheOptions }
...(prompt_cache_retention !== undefined
? { prompt_cache_retention }
: {}),
...(store !== undefined ? { store } : {}),
...(max_tokens !== undefined
@@ -320,17 +310,24 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
{ signal: Context.get('upstreamSignal') },
);
return OpenAiUtil.handle_completion_output_responses_api({
usage_calculator: ({
usage,
webSearchCalls,
setUsageCosts,
setUsageDetails,
}) => {
// GPT-5.6 and later report cache writes inside `input_tokens`,
// like cached reads, split by lifetime (30m and 12h entries
// cost 1.25x and 2.5x input to write).
usage_calculator: ({ usage, webSearchCalls, setUsageCosts }) => {
const cachedTokens =
(usage as any).input_tokens_details?.cached_tokens ?? 0;
// GPT-5.6 and later bill cache writes at 1.25x input. They're
// reported inside `input_tokens`, like cached reads.
const cacheWriteTokens =
(usage as any).input_tokens_details?.cache_write_tokens ??
0;
const trackedUsage: Record<string, number> = {
...OpenAiUtil.splitCachedPrompt(usage),
prompt_tokens:
((usage as any).input_tokens ?? 0) -
cachedTokens -
cacheWriteTokens,
completion_tokens: (usage as any).output_tokens ?? 0,
cached_tokens: cachedTokens,
...(cacheWriteTokens
? { cache_write_tokens: cacheWriteTokens }
: {}),
...(webSearchCalls
? { web_search_calls: webSearchCalls }
: {}),
@@ -359,11 +356,6 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
: {},
);
setUsageCosts?.(metered.costs);
setUsageDetails?.(
OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
writeTtl: '30m',
})!,
);
return metered.usage;
},
stream,
@@ -23,8 +23,6 @@ import type { IChatModel } from '../../types.js';
// Prompts over 272K input tokens are billed at 2x input (cached reads and
// cache writes included) and 1.5x output for the full request.
// GPT-5.6 and later bill cache writes by the lifetime requested: 1.25x input
// for the default 30m entries, 2.5x for 12h entries.
const GPT_LONG_CONTEXT_PRICING = {
threshold: 272_000,
input_multiplier: 2,
@@ -51,7 +49,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 100,
cached_tokens: 5,
cache_write_tokens: 100 * 1.25,
cache_write_12h_tokens: 100 * 2.5,
completion_tokens: 500,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -77,7 +74,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 200,
cached_tokens: 10,
cache_write_tokens: 200 * 1.25,
cache_write_12h_tokens: 200 * 2.5,
completion_tokens: 1000,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -103,7 +99,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 200,
cached_tokens: 20,
cache_write_tokens: 200 * 1.25,
cache_write_12h_tokens: 200 * 2.5,
completion_tokens: 1000,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -129,7 +124,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 10,
cached_tokens: 1,
cache_write_tokens: 10 * 1.25,
cache_write_12h_tokens: 10 * 2.5,
completion_tokens: 50,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -155,7 +149,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 1000,
cached_tokens: 100,
cache_write_tokens: 1000 * 1.25,
cache_write_12h_tokens: 1000 * 2.5,
completion_tokens: 5000,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -181,7 +174,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 400,
cached_tokens: 40,
cache_write_tokens: 400 * 1.25,
cache_write_12h_tokens: 400 * 2.5,
completion_tokens: 2000,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -206,7 +198,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 200,
cached_tokens: 20,
cache_write_tokens: 200 * 1.25,
cache_write_12h_tokens: 200 * 2.5,
completion_tokens: 1200,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
@@ -231,7 +222,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
prompt_tokens: 20,
cached_tokens: 2,
cache_write_tokens: 20 * 1.25,
cache_write_12h_tokens: 20 * 2.5,
completion_tokens: 120,
},
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
+2 -22
View File
@@ -145,19 +145,13 @@ export type StopReason =
| 'model_context_window_exceeded'
| (string & {});
/**
* Provider-neutral usage breakdown. `inputTokens` excludes cache reads/writes.
* Cache writes are keyed by the lifetime the entry was written with: Claude and
* Moonshot write 5m and 1h entries, OpenAI writes 30m and 12h entries.
*/
/** Provider-neutral usage breakdown. `inputTokens` excludes cache reads/writes. */
export interface UsageDetails {
inputTokens: number;
outputTokens: number; // includes reasoning
cacheReadTokens?: number;
cacheWrite5mTokens?: number;
cacheWrite30mTokens?: number;
cacheWrite1hTokens?: number;
cacheWrite12hTokens?: number;
reasoningTokens?: number;
webSearchRequests?: number;
webFetchRequests?: number;
@@ -174,22 +168,10 @@ export interface UsageDetails {
outputTokens: number;
cacheReadTokens?: number;
cacheWrite5mTokens?: number;
cacheWrite30mTokens?: number;
cacheWrite1hTokens?: number;
cacheWrite12hTokens?: number;
}>;
}
/**
* OpenAI `prompt_cache_options`, GPT-5.6 and later. `ttl` is the minimum
* lifetime of every cache entry the request writes; `12h` entries cost twice
* what `30m` entries do to write and only the Responses API writes them.
*/
export interface PromptCacheOptions {
mode?: 'implicit' | 'explicit';
ttl?: '30m' | '12h';
}
export interface ICompleteArguments {
messages: PuterMessage[];
provider?: string;
@@ -252,9 +234,7 @@ export interface ICompleteArguments {
metadata?: Record<string, string>;
prompt?: unknown;
prompt_cache_key?: string;
/** `in-memory` is accepted as a spelling of OpenAI's `in_memory`. */
prompt_cache_retention?: 'in-memory' | 'in_memory' | '24h' | undefined;
prompt_cache_options?: PromptCacheOptions;
prompt_cache_retention?: 'in-memory' | '24h' | undefined;
store?: boolean;
top_p?: number;
truncation?: 'auto' | 'disabled' | undefined;
+20 -78
View File
@@ -481,86 +481,43 @@ export const extractMeteredUsage = (usage) => {
};
/**
* @typedef {{
* cached_tokens?: number | null;
* cache_write_tokens?: number | null;
* cache_write_tokens_details?: {
* ttl_30m_tokens?: number | null;
* ttl_12h_tokens?: number | null;
* } | null;
* }} OpenAiCacheDetails
*/
/**
* OpenAI-shaped usage as it's metered. The prompt count reports cached reads
* and cache writes inside it, so they come out of it and are priced once each
* under their own keys: `cached_tokens` for reads, `cache_write_tokens` for
* writes at the model's default lifetime and `cache_write_12h_tokens` for
* writes at OpenAI's 12h lifetime. Without the per-lifetime breakdown every
* write is a default-lifetime write: Chat Completions writes no 12h entries,
* and Responses reports the breakdown whenever a request asked for them. Reads
* the Chat Completions (`prompt_tokens`) and Responses (`input_tokens`)
* spellings alike.
* Chat Completions usage as it's metered: `prompt_tokens` reports cached reads
* inside its count, so they come out of it and are priced only once, under
* `cached_tokens`.
*
* @param {{
* prompt_tokens?: number;
* completion_tokens?: number;
* prompt_tokens_details?: OpenAiCacheDetails | null;
* input_tokens?: number;
* output_tokens?: number;
* input_tokens_details?: OpenAiCacheDetails | null;
* }
* | undefined} usage
* @param {import('openai/resources/completions.mjs').CompletionUsage | undefined} usage
*/
export const splitCachedPrompt = (usage) => {
const details = usage?.prompt_tokens_details ?? usage?.input_tokens_details;
const details = usage?.prompt_tokens_details;
const cachedTokens = details?.cached_tokens ?? 0;
// Moonshot and GPT-5.6+ count the prompt tokens they wrote to the prefix
// cache here; they're billed at the model's write rate, not the prompt rate.
// Moonshot counts the prompt tokens it wrote to its prefix cache here;
// they're billed at the model's write rate, not the plain prompt rate.
const writeTokens =
typeof details?.cache_write_tokens === 'number'
? details.cache_write_tokens
: undefined;
const reported12h = details?.cache_write_tokens_details?.ttl_12h_tokens;
const write12hTokens =
typeof reported12h === 'number'
? Math.min(reported12h, writeTokens ?? 0)
: undefined;
return {
prompt_tokens:
(usage?.prompt_tokens ?? usage?.input_tokens ?? 0) -
cachedTokens -
(writeTokens ?? 0),
completion_tokens:
usage?.completion_tokens ?? usage?.output_tokens ?? 0,
(usage?.prompt_tokens ?? 0) - cachedTokens - (writeTokens ?? 0),
completion_tokens: usage?.completion_tokens ?? 0,
cached_tokens: cachedTokens,
...(writeTokens !== undefined
? { cache_write_tokens: writeTokens - (write12hTokens ?? 0) }
: {}),
...(write12hTokens !== undefined
? { cache_write_12h_tokens: write12hTokens }
? { cache_write_tokens: writeTokens }
: {}),
};
};
/**
* The `{prompt_tokens, completion_tokens, cached_tokens, cache_write_tokens,
* cache_write_12h_tokens}` shape every OpenAI-family `usage_calculator` already
* returns for billing, mapped to the provider-neutral `UsageDetails` the
* end-of-stream/non-stream result carries. `inputTokens`/`outputTokens` are the
* same numbers already billed — this never introduces a new figure, only
* exposes the breakdown.
* The `{prompt_tokens, completion_tokens, cached_tokens, cache_write_tokens}`
* shape every OpenAI-family `usage_calculator` already returns for billing,
* mapped to the provider-neutral `UsageDetails` the end-of-stream/non-stream
* result carries. `inputTokens`/`outputTokens` are the same numbers already
* billed — this never introduces a new figure, only exposes the breakdown.
*
* @param {Record<string, number> | undefined | null} trackedUsage
* @param {{ writeTtl?: '5m' | '30m' }} [opts] The lifetime `cache_write_tokens`
* were written with: 5m unless the provider says otherwise (OpenAI's default
* entries live 30m).
* @returns {import('../types.js').UsageDetails | undefined}
*/
export const usageDetailsFromTrackedUsage = (
trackedUsage,
{ writeTtl = '5m' } = {},
) => {
export const usageDetailsFromTrackedUsage = (trackedUsage) => {
if (!trackedUsage || typeof trackedUsage !== 'object') return undefined;
// OpenRouter, Infron and Ollama track `{prompt, completion,
// input_cache_read}` instead; any other shape is left to the driver.
@@ -569,19 +526,12 @@ export const usageDetailsFromTrackedUsage = (
if (input === undefined && output === undefined) return undefined;
const cacheRead =
trackedUsage.cached_tokens ?? trackedUsage.input_cache_read;
const cacheWrite = trackedUsage.cache_write_tokens;
return {
inputTokens: input ?? 0,
outputTokens: output ?? 0,
...(cacheRead ? { cacheReadTokens: cacheRead } : {}),
...(cacheWrite && writeTtl === '30m'
? { cacheWrite30mTokens: cacheWrite }
: {}),
...(cacheWrite && writeTtl !== '30m'
? { cacheWrite5mTokens: cacheWrite }
: {}),
...(trackedUsage.cache_write_12h_tokens
? { cacheWrite12hTokens: trackedUsage.cache_write_12h_tokens }
...(trackedUsage.cache_write_tokens
? { cacheWrite5mTokens: trackedUsage.cache_write_tokens }
: {}),
};
};
@@ -914,16 +864,12 @@ export const create_chat_stream_handler_responses_api =
// Missing usage is left undefined rather than fed to the calculator —
// see the sibling handler above, including why usage is reported
// before the block flushes.
let usageDetails;
const usage = last_usage
? usage_calculator({
usage: last_usage,
webSearchCalls,
tool_usage: toolUsage,
setUsageCosts: (c) => chatStream.setUsageCosts(c),
setUsageDetails: (d) => {
usageDetails = d;
},
})
: undefined;
chatStream.reportUsage(usage);
@@ -931,7 +877,7 @@ export const create_chat_stream_handler_responses_api =
reason:
incompleteReason ?? (sawFunctionCall ? 'tool_use' : 'end_turn'),
});
usageDetails ??= usageDetailsFromTrackedUsage(usage);
const usageDetails = usageDetailsFromTrackedUsage(usage);
if (usageDetails) chatStream.setUsageDetails(usageDetails);
textblock.end();
@@ -1069,7 +1015,6 @@ export const handle_completion_output = async (
* webSearchCalls?: number;
* tool_usage?: Record<string, unknown>;
* setUsageCosts?: (costs: Record<string, number>) => void;
* setUsageDetails?: (details: import('../types').UsageDetails) => void;
* }) => unknown} params.usage_calculator
* @param {() => Promise<void>} [params.finally_fn]
* @returns {ReturnType<import('../types').IChatProvider['complete']>}
@@ -1211,15 +1156,12 @@ export const handle_completion_output_responses_api = async ({
setUsageCosts: (c) => {
ret.usageCosts = c;
},
setUsageDetails: (d) => {
ret.usageDetails = d;
},
})
: {
input_tokens: completion.usage.input_tokens,
output_tokens: completion.usage.output_tokens,
};
if (usage_calculator && ret.usageDetails === undefined) {
if (usage_calculator) {
const usageDetails = usageDetailsFromTrackedUsage(ret.usage);
if (usageDetails) ret.usageDetails = usageDetails;
}
@@ -247,40 +247,6 @@ describe('usageDetailsFromTrackedUsage', () => {
});
});
it('labels cache writes by the lifetime the provider says it wrote them with', () => {
expect(
usageDetailsFromTrackedUsage(
{
prompt_tokens: 10,
completion_tokens: 5,
cached_tokens: 3,
cache_write_tokens: 11,
cache_write_12h_tokens: 6012,
},
{ writeTtl: '30m' },
),
).toEqual({
inputTokens: 10,
outputTokens: 5,
cacheReadTokens: 3,
cacheWrite30mTokens: 11,
cacheWrite12hTokens: 6012,
});
// Zero counts, as OpenAI reports on a cache hit, add no field.
expect(
usageDetailsFromTrackedUsage(
{
prompt_tokens: 10,
completion_tokens: 5,
cached_tokens: 3,
cache_write_tokens: 0,
cache_write_12h_tokens: 0,
},
{ writeTtl: '30m' },
),
).toEqual({ inputTokens: 10, outputTokens: 5, cacheReadTokens: 3 });
});
it('returns undefined for a missing/non-object usage', () => {
expect(usageDetailsFromTrackedUsage(undefined)).toBeUndefined();
expect(usageDetailsFromTrackedUsage(null)).toBeUndefined();
@@ -996,66 +962,6 @@ describe('splitCachedPrompt', () => {
cached_tokens: 3,
});
});
it('reads the Responses usage spelling', () => {
expect(
splitCachedPrompt({
input_tokens: 6026,
output_tokens: 5,
input_tokens_details: { cached_tokens: 0, cache_write_tokens: 6023 },
}),
).toEqual({
prompt_tokens: 3,
completion_tokens: 5,
cached_tokens: 0,
cache_write_tokens: 6023,
});
});
it('splits 12h writes out of the write count when the breakdown is reported', () => {
// One request can write both lifetimes: an explicit 12h breakpoint
// plus the implicit 30m one OpenAI adds at the end of the prompt.
expect(
splitCachedPrompt({
input_tokens: 6026,
output_tokens: 5,
input_tokens_details: {
cached_tokens: 0,
cache_write_tokens: 6023,
cache_write_tokens_details: {
ttl_12h_tokens: 6012,
ttl_30m_tokens: 11,
},
},
}),
).toEqual({
prompt_tokens: 3,
completion_tokens: 5,
cached_tokens: 0,
cache_write_tokens: 11,
cache_write_12h_tokens: 6012,
});
});
it('never reports more 12h writes than writes in total', () => {
expect(
splitCachedPrompt({
input_tokens: 100,
output_tokens: 1,
input_tokens_details: {
cached_tokens: 0,
cache_write_tokens: 40,
cache_write_tokens_details: { ttl_12h_tokens: 50 },
},
}),
).toEqual({
prompt_tokens: 60,
completion_tokens: 1,
cached_tokens: 0,
cache_write_tokens: 0,
cache_write_12h_tokens: 40,
});
});
});
// ── recorded usage costs ────────────────────────────────────────────
@@ -22,14 +22,11 @@ import { describe, expect, it } from 'vitest';
import type { ICompleteArguments } from '../types.js';
import {
clampReasoningEffort,
messagesHaveCacheBreakpointTtl,
openAICompatParams,
outputFormatFromResponseFormat,
outputFormatFromResponsesText,
outputFormatToResponseFormat,
outputFormatToResponsesText,
promptCacheOptionsToWire,
promptCacheRetentionToWire,
rejectStatefulResponsesFields,
toolChoiceFromWire,
toolChoiceToWire,
@@ -337,109 +334,3 @@ describe('rejectStatefulResponsesFields', () => {
).toThrow(/previous_response_id/);
});
});
describe('promptCacheRetentionToWire', () => {
it('spells the in-memory retention the way the upstreams expect', () => {
expect(promptCacheRetentionToWire('in-memory')).toBe('in_memory');
expect(promptCacheRetentionToWire('in_memory')).toBe('in_memory');
expect(promptCacheRetentionToWire('24h')).toBe('24h');
expect(promptCacheRetentionToWire(undefined)).toBeUndefined();
});
});
describe('promptCacheOptionsToWire', () => {
it('forwards mode and ttl and nothing else', () => {
expect(
promptCacheOptionsToWire({
mode: 'explicit',
ttl: '12h',
prewarm: true,
comparison_response_id: 'resp_1',
}),
).toEqual({ mode: 'explicit', ttl: '12h' });
expect(promptCacheOptionsToWire({ ttl: '30m' })).toEqual({
ttl: '30m',
});
});
it('sends nothing for an absent or empty object', () => {
expect(promptCacheOptionsToWire(undefined)).toBeUndefined();
expect(promptCacheOptionsToWire(null)).toBeUndefined();
expect(promptCacheOptionsToWire({})).toBeUndefined();
expect(promptCacheOptionsToWire({ prewarm: true })).toBeUndefined();
});
it('rejects a lifetime or mode OpenAI does not offer', () => {
expect(() => promptCacheOptionsToWire({ ttl: '2h' })).toThrow(
expect.objectContaining({ statusCode: 400 }),
);
expect(() => promptCacheOptionsToWire({ ttl: '24h' })).toThrow(
/30m.*12h/,
);
expect(() => promptCacheOptionsToWire({ mode: 'auto' })).toThrow(
/implicit.*explicit/,
);
expect(() => promptCacheOptionsToWire('12h')).toThrow(
/must be an object/,
);
});
});
describe('messagesHaveCacheBreakpointTtl', () => {
it('finds a breakpoint lifetime on any content part', () => {
expect(
messagesHaveCacheBreakpointTtl([
{ role: 'user', content: 'plain' },
{
role: 'system',
content: [
{ type: 'text', text: 'a' },
{
type: 'text',
text: 'b',
prompt_cache_breakpoint: {
mode: 'explicit',
ttl: '30m',
},
},
],
},
]),
).toBe(true);
expect(
messagesHaveCacheBreakpointTtl([
{
role: 'user',
content: {
type: 'text',
text: 'a',
prompt_cache_breakpoint: { ttl: '12h' },
},
},
]),
).toBe(true);
});
it('ignores breakpoints without a lifetime and odd message shapes', () => {
expect(
messagesHaveCacheBreakpointTtl([
{
role: 'system',
content: [
{
type: 'text',
text: 'a',
prompt_cache_breakpoint: { mode: 'explicit' },
},
'bare string part',
null,
],
},
{ role: 'user', content: null },
null,
]),
).toBe(false);
expect(messagesHaveCacheBreakpointTtl(undefined)).toBe(false);
expect(messagesHaveCacheBreakpointTtl('nope')).toBe(false);
});
});
@@ -29,7 +29,6 @@ import { HttpError } from '../../../core/http/HttpError.js';
import type {
ICompleteArguments,
OutputFormat,
PromptCacheOptions,
ReasoningEffort,
ToolChoice,
} from '../types.js';
@@ -306,80 +305,6 @@ export const openAICompatParams = (
return out;
};
// -- prompt caching ----------------------------------------------------------
/**
* The wire spelling of `prompt_cache_retention`. Puter also accepts the
* hyphenated `in-memory`, which every OpenAI-shaped upstream rejects.
*/
export const promptCacheRetentionToWire = (
retention: ICompleteArguments['prompt_cache_retention'],
): 'in_memory' | '24h' | undefined =>
retention === 'in-memory' ? 'in_memory' : retention;
const PROMPT_CACHE_MODES = new Set(['implicit', 'explicit']);
const PROMPT_CACHE_TTLS = new Set(['30m', '12h']);
/**
* `prompt_cache_options` as forwarded upstream: `mode` and `ttl` only, each
* checked against the values OpenAI accepts, so a bad value fails here with a
* clear message instead of as an upstream 400 the fallback loop may hide.
* `comparison_response_id` names a stored response, which Puter never keeps,
* and `prewarm` is dropped with it.
*/
export const promptCacheOptionsToWire = (
options: unknown,
): PromptCacheOptions | undefined => {
if (options === undefined || options === null) return undefined;
if (typeof options !== 'object' || Array.isArray(options)) {
throw new HttpError(400, '`prompt_cache_options` must be an object', {
legacyCode: 'bad_request',
});
}
const { mode, ttl } = options as Record<string, unknown>;
if (mode !== undefined && !PROMPT_CACHE_MODES.has(mode as string)) {
throw new HttpError(
400,
'`prompt_cache_options.mode` must be `implicit` or `explicit`',
{ legacyCode: 'bad_request' },
);
}
if (ttl !== undefined && !PROMPT_CACHE_TTLS.has(ttl as string)) {
throw new HttpError(
400,
'`prompt_cache_options.ttl` must be `30m` or `12h`',
{ legacyCode: 'bad_request' },
);
}
const wire: PromptCacheOptions = {
...(mode !== undefined
? { mode: mode as PromptCacheOptions['mode'] }
: {}),
...(ttl !== undefined ? { ttl: ttl as PromptCacheOptions['ttl'] } : {}),
};
return Object.keys(wire).length ? wire : undefined;
};
/**
* Whether any content part carries a `prompt_cache_breakpoint` with its own
* `ttl`. Chat Completions rejects the field whatever its value; only the
* Responses API honours a per-breakpoint lifetime.
*/
export const messagesHaveCacheBreakpointTtl = (messages: unknown): boolean =>
Array.isArray(messages) &&
messages.some((message) => {
const content = (message as { content?: unknown } | null)?.content;
const parts = Array.isArray(content) ? content : [content];
return parts.some(
(part) =>
(
part as {
prompt_cache_breakpoint?: { ttl?: unknown } | null;
} | null
)?.prompt_cache_breakpoint?.ttl !== undefined,
);
});
// -- Responses server-side state ---------------------------------------------
const STATEFUL_RESPONSES_FIELDS = [
@@ -463,26 +463,10 @@ describe('ImageGenerationDriver.generate provider routing', () => {
driver.generate({ prompt: 'a landscape' }),
);
expect(openaiImagesGenerateMock).toHaveBeenCalledWith(
expect.objectContaining({ model: 'gpt-image-2.6-flare' }),
expect.objectContaining({ model: 'gpt-image-2' }),
);
});
it.each(['gpt-image-2.6-flare', 'gpt-image-2.6-sunburst'])(
'routes %s and its OpenAI alias', async (model) => {
for (const id of [model, `openai/${model}`]) {
openaiImagesGenerateMock.mockResolvedValueOnce({
data: [{ url: 'https://oai/image.png' }],
});
await withDriverName('ai-image', () => driver.generate({
model: id, prompt: 'a landscape',
}));
expect(openaiImagesGenerateMock).toHaveBeenLastCalledWith(
expect.objectContaining({ model }),
);
}
},
);
it('rejects an excluded model regardless of caller casing', async () => {
await expect(withActor(() =>
driver.generate({
@@ -758,10 +742,7 @@ describe('ImageGenerationDriver.generate provider routing', () => {
}),
);
const reserveSpy = vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
);
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
const usageSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -855,7 +836,7 @@ describe('ImageGenerationDriver.generate provider routing', () => {
});
const reserveSpy = vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
'reserveCredits',
);
// Cloudflare answers the scheduled refresh with new rates at once;
@@ -1165,8 +1146,8 @@ describe('ImageGenerationDriver.generate Puter file inputs', () => {
);
beforeEach(() => {
vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable').mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
vi.spyOn(server.services.metering, 'hasEnoughCredits').mockResolvedValue(
true,
);
});
@@ -1668,10 +1649,7 @@ it.each([undefined, null, '', ' ', 42, {}])(
describe('ImageGenerationDriver cost factor', () => {
it('holds and meters at the AI cost factor', async () => {
const reserveSpy = vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
);
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
const usageSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -72,7 +72,7 @@ vi.mock('openai', () => {
// -- Test harness ----------------------------------------------------
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
>;
@@ -107,7 +107,7 @@ const makeProvider = (
beforeEach(() => {
generateMock.mockReset();
openAICtor.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -177,7 +177,7 @@ describe('BytePlusImageProvider.generate gates', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
});
@@ -196,7 +196,7 @@ describe('BytePlusImageProvider.generate gates', () => {
});
it('throws 402 BEFORE hitting ModelArk when the actor lacks credits', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => makeProvider().generate({ prompt: 'hi' })),
).rejects.toMatchObject({ statusCode: 402 });
@@ -713,7 +713,7 @@ it.each([
}),
),
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
});
@@ -73,7 +73,7 @@ let server: PuterServer;
let globalFetchSpy: MockInstance<typeof fetch>;
let fetchSpy: Mock<typeof fetch>;
let modelSearchFetch: Mock<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
>;
@@ -123,7 +123,7 @@ beforeEach(() => {
? modelSearchFetch(input, init)
: fetchSpy(input, init),
);
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -188,7 +188,7 @@ describe('CloudflareImageProvider.generate test_mode', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
@@ -210,7 +210,7 @@ describe('CloudflareImageProvider.generate argument validation', () => {
describe('CloudflareImageProvider.generate credit gate', () => {
it('throws 402 BEFORE hitting Cloudflare when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => provider.generate({ prompt: 'hi' })),
@@ -240,7 +240,7 @@ describe('CloudflareImageProvider.generate credit gate', () => {
actor,
);
const [, estimate] = admitSpy.mock.calls[0]!;
const [, estimate] = hasCreditsSpy.mock.calls[0]!;
expect(estimate).toBeGreaterThan(0);
expect(heldDuringCall).toBe(Math.ceil(estimate));
// The hold stays until the usage write lands.
@@ -708,7 +708,7 @@ describe('Cloudflare legacy image option', () => {
legacyCode: 'bad_request',
});
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
},
);
@@ -726,7 +726,7 @@ describe('Cloudflare legacy image option', () => {
message: 'Invalid input image (empty or not base64)',
});
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
});
});
@@ -823,12 +823,12 @@ describe('Cloudflare priced legacy models', () => {
imageSize: { kind: 'pixels', w: 512, h: 512 },
}));
expect(JSON.parse(fetchSpy.mock.calls[0]![1]!.body as string)).toMatchObject({ image: [1, 2, 3], mask: [4, 5, 6], num_steps: 20, width: 512, height: 512 });
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 0, undefined);
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 0);
});
it('rejects missing inpainting inputs before credit checks', async () => {
await expect(withTestActor(() => makeProvider().generate({ model: '@cf/runwayml/stable-diffusion-v1-5-inpainting', prompt: 'A cup' }))).rejects.toMatchObject({ statusCode: 400 });
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
@@ -844,7 +844,7 @@ describe('Cloudflare priced legacy models', () => {
it('charges Lucid at the price returned by the model API', async () => {
fetchSpy.mockResolvedValueOnce(new Response(new Uint8Array([1, 2, 3]), { headers: { 'content-type': 'image/png' } }));
await withTestActor(() => makeProvider().generate({ model: '@cf/leonardo/lucid-origin', prompt: 'A cup', imageSize: { kind: 'pixels', w: 512, h: 512 }, steps: 1 }));
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 713200, undefined);
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 713200);
});
});
@@ -897,10 +897,9 @@ describe('Cloudflare model-search pricing', () => {
await runPhoenix(provider);
expect(listed?.costs).toEqual({ tile_512: 1_000_000, step: 20_200 });
expect(admitSpy).toHaveBeenCalledWith(
expect(hasCreditsSpy).toHaveBeenCalledWith(
expect.anything(),
4 * 1_000_000 + 10 * 20_200,
undefined,
);
expect(batchIncrementUsagesSpy.mock.calls[0]![1]).toEqual([
expect.objectContaining({
@@ -931,10 +930,9 @@ describe('Cloudflare model-search pricing', () => {
await runPhoenix(makeProvider());
expect(modelSearchFetch).not.toHaveBeenCalled();
expect(admitSpy).toHaveBeenCalledWith(
expect(hasCreditsSpy).toHaveBeenCalledWith(
expect.anything(),
4 * 583_000 + 10 * 11_000,
undefined,
);
});
@@ -957,7 +955,7 @@ describe('Cloudflare model-search pricing', () => {
await refresh;
await runPhoenix(provider);
expect(admitSpy.mock.calls.map(([, cost]) => cost)).toEqual([
expect(hasCreditsSpy.mock.calls.map(([, cost]) => cost)).toEqual([
2_442_000, 4_202_000,
]);
});
@@ -979,10 +977,9 @@ describe('Cloudflare model-search pricing', () => {
await runPhoenix(provider);
expect(modelSearchFetch).toHaveBeenCalledTimes(2);
expect(admitSpy).toHaveBeenCalledWith(
expect(hasCreditsSpy).toHaveBeenCalledWith(
expect.anything(),
4_202_000,
undefined,
);
});
@@ -1018,7 +1015,7 @@ describe('Cloudflare model-search pricing', () => {
steps: 10,
} as never),
);
return admitSpy.mock.lastCall![1];
return hasCreditsSpy.mock.lastCall![1];
};
const settle = () => new Promise((resolve) => setImmediate(resolve));
const OLD = { tile_512: 583_000, step: 11_000 };
@@ -1072,10 +1069,9 @@ describe('Cloudflare model-search pricing', () => {
}),
);
expect(modelSearchFetch).toHaveBeenCalledOnce();
expect(admitSpy).toHaveBeenCalledWith(
expect(hasCreditsSpy).toHaveBeenCalledWith(
expect.anything(),
4 * 5280 + 4 * 10560,
undefined,
);
});
@@ -82,7 +82,7 @@ vi.mock('../../inputImage.js', async (orig) => ({
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
@@ -111,7 +111,7 @@ beforeEach(() => {
generateContentMock.mockReset();
toBase64DataUriMock.mockReset();
googleAICtor.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
@@ -167,7 +167,7 @@ describe('GeminiImageProvider.generate test_mode', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateContentMock).not.toHaveBeenCalled();
});
});
@@ -219,7 +219,7 @@ describe('GeminiImageProvider.generate argument validation', () => {
legacyCode: 'bad_request',
message: expect.stringContaining('Unsupported quality tier'),
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateContentMock).not.toHaveBeenCalled();
},
);
@@ -427,7 +427,7 @@ describe('GeminiImageProvider.generate Flash path (generateContent)', () => {
it('throws 402 BEFORE hitting Gemini when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() =>
@@ -466,7 +466,7 @@ describe('GeminiImageProvider.generate Flash path (generateContent)', () => {
actor,
);
const [, estimate] = admitSpy.mock.calls[0]!;
const [, estimate] = hasCreditsSpy.mock.calls[0]!;
expect(estimate).toBeGreaterThan(0);
expect(heldDuringCall).toBe(Math.ceil(estimate));
// The hold stays until the usage write lands.
@@ -643,7 +643,7 @@ describe('Gemini Nano Banana image generation', () => {
const request = generateContentMock.mock.calls[0][0];
expect(request.config.imageConfig.imageSize).toBe('1K');
// Input + 1120 image tokens at $30/1M + 1000 text tokens at $7.50/1M.
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 4_194_450, undefined);
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 4_194_450);
expect(batchIncrementUsagesSpy).toHaveBeenCalledWith(
expect.anything(),
expect.arrayContaining([
@@ -50,7 +50,7 @@ import { KlingImageProvider } from './KlingImageProvider.js';
let server: PuterServer;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -73,7 +73,7 @@ const makeProvider = () =>
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
incrementUsageSpy.mockResolvedValue({} as never);
});
@@ -318,7 +318,7 @@ describe('KlingImageProvider billing', () => {
});
it('rejects before calling Kling when credits run short', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(generate({})).rejects.toMatchObject({ statusCode: 402 });
expect(fetchSpy).not.toHaveBeenCalled();
});
@@ -97,7 +97,7 @@ vi.mock('../../inputImage.js', async (orig) => ({
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
>;
@@ -126,7 +126,7 @@ beforeEach(() => {
editMock.mockReset();
fetchImageAsBase64Mock.mockReset();
openAICtor.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -150,9 +150,9 @@ describe('OpenAiImageProvider construction', () => {
// ── Model catalog ───────────────────────────────────────────────────
describe('OpenAiImageProvider model catalog', () => {
it('returns gpt-image-2.6-flare as the default', () => {
it('returns gpt-image-2 as the default', () => {
const provider = makeProvider();
expect(provider.getDefaultModel()).toBe('gpt-image-2.6-flare');
expect(provider.getDefaultModel()).toBe('gpt-image-2');
});
it('drops the deprecated gpt-image-1 family and chatgpt-image-latest', () => {
@@ -195,7 +195,7 @@ describe('OpenAiImageProvider.generate test_mode', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
});
});
@@ -253,8 +253,7 @@ describe('OpenAiImageProvider.generate credit gate', () => {
});
let blocked = false;
vi.spyOn(redis, 'meterIncr').mockImplementation(async (...args) => {
const actorKey = `:actor:${actor.user!.uuid}:`;
if (!blocked && args.some((arg) => arg.includes(actorKey))) {
if (!blocked && args[5].includes(`:actor:${actor.user!.uuid}:`)) {
blocked = true;
await held;
}
@@ -288,7 +287,7 @@ describe('OpenAiImageProvider.generate credit gate', () => {
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() =>
provider.generate({
@@ -324,7 +323,7 @@ describe('OpenAiImageProvider.generate credit gate', () => {
actor,
);
const [, estimate] = admitSpy.mock.calls[0]!;
const [, estimate] = hasCreditsSpy.mock.calls[0]!;
expect(estimate).toBeGreaterThan(0);
expect(heldDuringCall).toBe(Math.ceil(estimate));
// The hold stays until the usage write lands.
@@ -728,8 +727,8 @@ describe('OpenAiImageProvider gpt-image-2.5 models', () => {
);
// Latent factor 48 vs 24 at the medium tier.
const gptImage2Estimate = admitSpy.mock.calls[0]![1];
const gptImage25Estimate = admitSpy.mock.calls[1]![1];
const gptImage2Estimate = hasCreditsSpy.mock.calls[0]![1];
const gptImage25Estimate = hasCreditsSpy.mock.calls[1]![1];
expect(gptImage25Estimate).toBeLessThan(gptImage2Estimate);
});
@@ -754,8 +753,8 @@ describe('OpenAiImageProvider gpt-image-2.5 models', () => {
}),
);
const highEstimate = admitSpy.mock.calls[0]![1];
const maxEstimate = admitSpy.mock.calls[1]![1];
const highEstimate = hasCreditsSpy.mock.calls[0]![1];
const maxEstimate = hasCreditsSpy.mock.calls[1]![1];
expect(maxEstimate).toBeGreaterThan(highEstimate);
});
});
@@ -908,64 +907,3 @@ it('keeps a landscape aspect hint landscape at gpt-image-2 sizes', async () => {
expect(w).toBeGreaterThan(h);
expect(Math.abs(w / h - 1.7)).toBeLessThan(0.1);
});
describe('OpenAiImageProvider GPT Image 2.6', () => {
it.each(['gpt-image-2.6-flare', 'gpt-image-2.6-sunburst'])(
'%s generates and edits with token-based billing', async (model) => {
const provider = makeProvider();
const response = {
data: [{ b64_json: 'AAAA' }],
usage: {
input_tokens: 100,
output_tokens: 800,
input_tokens_details: { text_tokens: 100, image_tokens: 0 },
},
};
generateMock.mockResolvedValue(response);
editMock.mockResolvedValue(response);
for (const input_image of [undefined, 'data:image/png;base64,AAAA']) {
const result = await withTestActor(() => provider.generate({
model, prompt: 'hi', quality: 'xhigh',
ratio: { w: 1024, h: 1024 }, input_image,
}));
expect(result).toBe('data:image/png;base64,AAAA');
}
for (const mock of [generateMock, editMock]) {
expect(mock.mock.calls[0]![0]).toMatchObject({
model, quality: 'xhigh', size: '1024x1024',
});
}
const [, entries] = batchIncrementUsagesSpy.mock.calls[0]!;
expect(entries).toEqual(expect.arrayContaining([
expect.objectContaining({
usageType: `openai:${model}:xhigh:1024x1024:output`,
costOverride: 2_400_000,
}),
]));
},
);
it.each([
['low', 1024, 1024, 823],
['medium', 1024, 1024, 1449],
['high', 1024, 1024, 1756],
['xhigh', 1024, 1024, 3122],
['max', 1024, 1024, 7024],
['auto', 1024, 1024, 1449],
['low', 2048, 1088, 571],
])('estimates %s at %sx%s from the 2.6 calculator', async (quality, w, h, tokens) => {
generateMock.mockResolvedValue({ data: [{ b64_json: 'AAAA' }] });
await withTestActor(() => makeProvider().generate({
model: 'gpt-image-2.6-flare', prompt: 'hi', quality: String(quality),
ratio: { w: Number(w), h: Number(h) },
}));
const [, entries] = batchIncrementUsagesSpy.mock.calls[0]!;
const output = (entries as Array<{ usageType: string; costOverride: number }>)
.find((entry) => entry.usageType.endsWith(':output'))!;
expect(output.usageType).toBe(`openai:gpt-image-2.6-flare:${quality}:${w}x${h}:output`);
// The metering conversion rounds fractional microcents upward.
expect(output.costOverride).toBeGreaterThanOrEqual(Number(tokens) * 3000);
expect(output.costOverride).toBeLessThanOrEqual(Number(tokens) * 3000 + 1);
});
});
@@ -56,8 +56,8 @@ interface OpenAIImageUsage {
/**
* OpenAI image generation provider for v2. Supports the GPT Image models
* (gpt-image-2 and the 2.5/2.6 Sunburst and Flare models), including
* image-to-image editing via `input_images` (the `images.edit` endpoint).
* (gpt-image-2, -2.5-sunburst, -2.5-flare), including image-to-image editing
* via `input_images` (the `images.edit` endpoint).
*/
export class OpenAiImageProvider implements IImageProvider {
#meteringService: AiMeteringService;
@@ -109,7 +109,7 @@ export class OpenAiImageProvider implements IImageProvider {
}
getDefaultModel(): string {
return 'gpt-image-2.6-flare';
return 'gpt-image-2';
}
async generate({
@@ -604,36 +604,17 @@ export class OpenAiImageProvider implements IImageProvider {
height: number,
quality?: string,
): number {
const isGptImage26 = model.startsWith('gpt-image-2.6');
const factors =
isGptImage26 || model.startsWith('gpt-image-2.5')
? OpenAiImageProvider.#GPT_IMAGE_2_5_LATENT_FACTORS
: OpenAiImageProvider.#GPT_IMAGE_2_LATENT_FACTORS;
const factors = model.startsWith('gpt-image-2.5')
? OpenAiImageProvider.#GPT_IMAGE_2_5_LATENT_FACTORS
: OpenAiImageProvider.#GPT_IMAGE_2_LATENT_FACTORS;
const factor = factors[quality ?? ''] ?? factors.medium;
const longEdge = Math.max(width, height);
const shortEdge = Math.min(width, height);
const shortGrid = (factor * shortEdge) / longEdge;
const floorGrid = Math.floor(shortGrid);
const shortLatent =
isGptImage26 && shortGrid - floorGrid === 0.5
? floorGrid + (floorGrid % 2)
: Math.round(shortGrid);
const shortLatent = Math.round((factor * shortEdge) / longEdge);
const latentW = width >= height ? factor : shortLatent;
const latentH = width >= height ? shortLatent : factor;
const baseArea = latentW * latentH;
const tokens = Math.ceil(
(baseArea * (2_000_000 + width * height)) / 4_000_000,
);
// GPT Image 2.6 bills more tokens at low and medium quality.
const estimatedQuality =
quality && quality in factors ? quality : 'medium';
const multiplier =
estimatedQuality === 'low'
? 4.2
: estimatedQuality === 'medium'
? 3.3
: 1;
return isGptImage26 ? Math.round(tokens * multiplier) : tokens;
return Math.ceil((baseArea * (2_000_000 + width * height)) / 4_000_000);
}
#estimateOutputCostFromTokens(
@@ -20,46 +20,6 @@
import type { IImageModel } from '../../types.js';
export const OPEN_AI_IMAGE_GENERATION_MODELS: IImageModel[] = [
{
puterId: 'openai:openai/gpt-image-2.6-sunburst',
id: 'gpt-image-2.6-sunburst',
aliases: ['openai/gpt-image-2.6-sunburst'],
name: 'GPT Image 2.6 Sunburst',
version: '2.6',
costs_currency: 'usd-cents',
index_cost_key: 'low:1024x1024',
costs: {
// Text tokens (per 1M tokens)
text_input: 500, // $5.00
text_cached_input: 125, // $1.25
// Image tokens (per 1M tokens)
image_input: 800, // $8.00
image_cached_input: 200, // $2.00
image_output: 3000, // $30.00
'low:1024x1024': 2.469,
},
allowedQualityLevels: ['low', 'medium', 'high', 'xhigh', 'max', 'auto'],
},
{
puterId: 'openai:openai/gpt-image-2.6-flare',
id: 'gpt-image-2.6-flare',
aliases: ['openai/gpt-image-2.6-flare'],
name: 'GPT Image 2.6 Flare',
version: '2.6',
costs_currency: 'usd-cents',
index_cost_key: 'low:1024x1024',
costs: {
// Text tokens (per 1M tokens)
text_input: 500, // $5.00
text_cached_input: 125, // $1.25
// Image tokens (per 1M tokens)
image_input: 800, // $8.00
image_cached_input: 200, // $2.00
image_output: 3000, // $30.00
'low:1024x1024': 2.469,
},
allowedQualityLevels: ['low', 'medium', 'high', 'xhigh', 'max', 'auto'],
},
{
puterId: 'openai:openai/gpt-image-2.5-sunburst',
id: 'gpt-image-2.5-sunburst',
@@ -84,7 +84,7 @@ vi.mock('../../../../util/secureHttp.js', () => ({
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
@@ -115,7 +115,7 @@ beforeEach(() => {
cancelMock.mockReset();
replicateCtor.mockReset();
secureFetchMock.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
@@ -177,7 +177,7 @@ describe('ReplicateImageGenerationProvider.generate test_mode', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(createPredictionMock).not.toHaveBeenCalled();
});
});
@@ -199,7 +199,7 @@ describe('ReplicateImageGenerationProvider.generate argument validation', () =>
describe('ReplicateImageGenerationProvider.generate credit gate', () => {
it('throws 402 BEFORE hitting Replicate when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => provider.generate({ prompt: 'hi' })),
).rejects.toMatchObject({ statusCode: 402 });
@@ -255,10 +255,7 @@ describe('ReplicateImageGenerationProvider.generate credit gate', () => {
it('holds credits for longer than a prediction can run', async () => {
const provider = makeProvider();
const actor = holdUser('ttl');
const reserveSpy = vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
);
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
createPredictionMock.mockResolvedValueOnce({
id: 'ttl',
status: 'succeeded',
@@ -277,9 +274,7 @@ describe('ReplicateImageGenerationProvider.generate credit gate', () => {
const provider = makeProvider();
const actor = holdUser('surcharge');
secureFetchMock.mockResolvedValueOnce(new Response('not-an-image'));
admitSpy.mockImplementationOnce((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
).mockResolvedValueOnce(false);
hasCreditsSpy.mockResolvedValueOnce(true).mockResolvedValueOnce(false);
await expect(
withTestActor(
@@ -955,7 +950,7 @@ it('does not start a prediction for an already-aborted request', async () => {
),
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'client_aborted' });
expect(createPredictionMock).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
});
it('stops polling and cancels at the prediction deadline', async () => {
@@ -1197,12 +1192,12 @@ it('rejects more than ten input images before fetching or pricing any', async ()
),
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
expect(secureFetchMock).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(createPredictionMock).not.toHaveBeenCalled();
});
it('checks credits on the output-only estimate before fetching input images', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() =>
makeProvider().generate({
@@ -71,7 +71,7 @@ vi.mock('together-ai', () => {
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -96,7 +96,7 @@ const makeProvider = () =>
beforeEach(() => {
generateMock.mockReset();
togetherCtor.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -158,7 +158,7 @@ describe('TogetherImageProvider.generate test_mode', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
});
});
@@ -189,7 +189,7 @@ describe('TogetherImageProvider.generate argument validation', () => {
describe('TogetherImageProvider.generate credit gate', () => {
it('throws 402 BEFORE hitting Together when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() =>
@@ -760,7 +760,7 @@ describe('Together supported sizes and references', () => {
width: 1024,
height: 1024,
});
expect(Number.isFinite(admitSpy.mock.calls[0][1])).toBe(true);
expect(Number.isFinite(hasCreditsSpy.mock.calls[0][1])).toBe(true);
});
it('resolves provider aliases without changing the selected model', async () => {
@@ -790,7 +790,7 @@ describe('Together supported sizes and references', () => {
}),
),
).rejects.toMatchObject({ statusCode: 400 });
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
});
@@ -859,7 +859,7 @@ it.each(['3K', 'unknown', ' 5k '])(
}),
),
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
},
);
@@ -895,7 +895,7 @@ describe('Together current API prices', () => {
input_image: 'https://example.com/reference.png',
}),
);
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 4685000, undefined);
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 4685000);
expect(incrementUsageSpy).toHaveBeenCalledWith(
expect.anything(),
'togetherai:google/flash-image-3.1:0.5K',
@@ -85,7 +85,7 @@ vi.mock('openai', () => {
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
>;
@@ -115,7 +115,7 @@ beforeEach(() => {
openAICtor.mockReset();
axiosGetMock.mockReset();
axiosGetMock.mockRejectedValue(new Error('Catalog unavailable'));
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -182,7 +182,7 @@ describe('XAIImageProvider.generate test_mode', () => {
expect(result).toBe(
'https://puter-sample-data.puter.site/image_example.png',
);
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
expect(batchIncrementUsagesSpy).not.toHaveBeenCalled();
});
@@ -231,7 +231,7 @@ describe('XAIImageProvider.generate argument validation', () => {
describe('XAIImageProvider.generate credit gate', () => {
it('throws 402 BEFORE hitting xAI when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() =>
@@ -718,7 +718,7 @@ describe('XAIImageProvider quality and reference limits', () => {
}),
),
).rejects.toMatchObject({ statusCode: 400 });
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(postMock).not.toHaveBeenCalled();
});
});
@@ -746,7 +746,7 @@ describe('xAI option validation', () => {
makeProvider().generate({ prompt: 'hi', ...options } as never),
),
).rejects.toMatchObject({ statusCode: 400 });
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(generateMock).not.toHaveBeenCalled();
});
});
+9 -9
View File
@@ -167,7 +167,7 @@ const sentRegions = () =>
let server: PuterServer;
let driver: OCRDriver;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -199,7 +199,7 @@ beforeEach(() => {
mistralCtor.mockReset();
// Spy on metering — keep the real impl so its recording side runs,
// but capture calls so per-test assertions can inspect them.
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -335,7 +335,7 @@ describe('OCRDriver.recognize (aws-textract)', () => {
};
it('throws 402 when the actor does not have enough credits', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
const { actor } = await makeUser();
await expect(
@@ -546,7 +546,7 @@ describe('OCRDriver.recognize (aws-textract)', () => {
describe('regional pricing', () => {
const onePage = { Blocks: [{ BlockType: 'PAGE' }] };
const checkedCosts = () =>
admitSpy.mock.calls.map(([, cost]) => cost);
hasCreditsSpy.mock.calls.map(([, cost]) => cost);
const textractCharges = () =>
incrementUsageSpy.mock.calls
.filter(([, type]) => type === TEXTRACT_USAGE)
@@ -719,7 +719,7 @@ describe('OCRDriver.recognize (mistral)', () => {
};
it('throws 402 when the actor does not have enough credits', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
const { actor } = await makeUser();
await expect(
@@ -1033,7 +1033,7 @@ describe('OCRDriver.recognize (mistral)', () => {
const pageType = 'mistral-ocr:mistral-ocr-4-1:page';
const annotationType = 'mistral-ocr:mistral-ocr-4-1:annotations:page';
// The pre-flight covers the page plus its annotation.
expect(admitSpy.mock.calls[0]![1]).toBe(
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(
OCR_COSTS[pageType] + OCR_COSTS[annotationType],
);
const ocrCalls = incrementUsageSpy.mock.calls.filter(
@@ -1080,7 +1080,7 @@ describe('OCRDriver credit pre-flight', () => {
const pageType = 'mistral-ocr:mistral-ocr-4-1:page';
const perPage = OCR_COSTS[pageType];
const pdfSource = (buffer: Buffer) => dataUrl(buffer, 'application/pdf');
const checkedCosts = () => admitSpy.mock.calls.map(([, cost]) => cost);
const checkedCosts = () => hasCreditsSpy.mock.calls.map(([, cost]) => cost);
const outstandingHolds = (actor: Actor) =>
server.stores.creditHold.outstanding(actor.user.uuid!);
@@ -1171,8 +1171,8 @@ describe('OCRDriver credit pre-flight', () => {
),
).rejects.toMatchObject({ statusCode: 402 });
// The estimate/hold path never ran.
expect(admitSpy).not.toHaveBeenCalled();
// The estimate/hold path (which calls hasEnoughCredits) never ran.
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(mistralOcrProcessMock).not.toHaveBeenCalled();
noUsage.mockRestore();
});
@@ -72,7 +72,7 @@ vi.mock('undici', async (importOriginal) => ({
let server: PuterServer;
let driver: VoiceChangerDriver;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -91,7 +91,7 @@ afterAll(async () => {
beforeEach(() => {
webFetchMock.mockReset();
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -197,7 +197,7 @@ describe('VoiceChangerDriver.convert argument validation', () => {
describe('VoiceChangerDriver.convert credit gate', () => {
it('throws 402 BEFORE hitting ElevenLabs when the actor lacks credits', async () => {
const { actor } = await makeUser();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withActor(actor, () =>
@@ -387,7 +387,7 @@ describe('VoiceChangerDriver.convert success path', () => {
const [base, factored] = metered();
expect(factored).toBe(base! * 2);
const held = admitSpy.mock.calls.map(([, amount]) => amount);
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
expect(held).toEqual([base, base! * 2]);
});
@@ -122,7 +122,7 @@ vi.mock('openai', () => {
let server: PuterServer;
let driver: SpeechToTextDriver;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -145,7 +145,7 @@ beforeEach(() => {
toFileMock.mockClear();
// Spy on metering — keep the real impl so its recording side runs,
// but capture calls so per-test assertions can inspect them.
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -360,7 +360,7 @@ describe('SpeechToTextDriver argument validation', () => {
describe('SpeechToTextDriver credit gate', () => {
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
const { actor } = await makeUser();
await expect(
@@ -686,7 +686,7 @@ describe('SpeechToTextDriver metering', () => {
expect(cost).toBe(SPEECH_TO_TEXT_COSTS[usageType]);
});
it('reserves the same total it later meters', async () => {
it('asks hasEnoughCredits for the same total it later meters', async () => {
const { actor } = await makeUser();
transcriptionsCreateMock.mockResolvedValueOnce({ text: 'x' });
@@ -700,7 +700,7 @@ describe('SpeechToTextDriver metering', () => {
const usageType = 'openai:gpt-transcribe:second';
const expected = SPEECH_TO_TEXT_COSTS[usageType] * 2;
const creditCall = admitSpy.mock.calls[0]!;
const creditCall = hasCreditsSpy.mock.calls[0]!;
expect(creditCall[1]).toBe(expected);
});
@@ -794,7 +794,7 @@ describe('SpeechToTextDriver metering', () => {
const [base, factored] = metered();
expect(factored).toBe(base! * 2);
const held = admitSpy.mock.calls.map(([, amount]) => amount);
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
expect(held).toEqual([base, base! * 2]);
});
});
@@ -72,7 +72,7 @@ const HOLD = COUNTED_TOKENS * AUDIO_RATE + 8192 * OUTPUT_RATE;
let server: PuterServer;
let sttDriver: SpeechToTextDriver;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchSpy: MockInstance<MeteringService['batchIncrementUsages']>;
const driver = {
@@ -102,7 +102,7 @@ beforeEach(() => {
.mockRejectedValue(new Error('unexpected fetch')) as MockInstance<
typeof fetch
>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
batchSpy = vi.spyOn(server.services.metering, 'batchIncrementUsages');
});
@@ -220,16 +220,13 @@ const metered = (): UsageInput[] =>
/** Records each credit hold taken, its options, and its release. */
const trackHolds = (order: string[] = []) => {
const metering = server.services.metering;
const realReserve = Object.getPrototypeOf(
metering,
).reserveCreditsIfAffordable.bind(metering) as typeof metering.reserveCreditsIfAffordable;
const realReserve = metering.reserveCredits.bind(metering);
const held: { amount: number; ttlMs?: number }[] = [];
const released = vi.fn();
vi.spyOn(metering, 'reserveCreditsIfAffordable').mockImplementation(
vi.spyOn(metering, 'reserveCredits').mockImplementation(
async (actor, amount, opts) => {
held.push({ amount, ttlMs: opts?.ttlMs });
const hold = await realReserve(actor, amount, opts);
if (!hold) return null;
return {
...hold,
release: async () => {
@@ -428,7 +425,7 @@ describe('GeminiSpeechToTextProvider test_mode', () => {
model: MODEL,
});
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(batchSpy).not.toHaveBeenCalled();
});
@@ -1266,12 +1263,12 @@ describe('GeminiSpeechToTextProvider countTokens preflight', () => {
describe('GeminiSpeechToTextProvider credit hold', () => {
it('throws 402 after counting tokens, before transcribing', async () => {
const { actor } = await makeUser();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
fetchSpy.mockResolvedValueOnce(Response.json({ totalTokens: 91 }));
await expect(
withActor(actor, () => driver.transcribe({ file: dataUrl() })),
).rejects.toMatchObject({ statusCode: 402 });
expect(admitSpy.mock.calls[0]![1]).toBe(HOLD);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(HOLD);
expect(fetchSpy).toHaveBeenCalledTimes(1);
expect(batchSpy).not.toHaveBeenCalled();
});
@@ -70,7 +70,7 @@ const driver = {
getReportedCosts: () => sttDriver.getReportedCosts(),
};
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -88,7 +88,7 @@ afterAll(async () => {
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -193,7 +193,7 @@ describe('XAISpeechToTextProvider test_mode', () => {
expect(Array.isArray(result.words)).toBe(true);
// No file required, no actor required, no fetch / metering hit.
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
@@ -258,7 +258,7 @@ describe('XAISpeechToTextProvider missing API key', () => {
describe('XAISpeechToTextProvider credit gate', () => {
it('throws 402 BEFORE hitting xAI when actor lacks credits', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
const { actor } = await makeUser();
await expect(
@@ -273,7 +273,7 @@ describe('XAISpeechToTextProvider credit gate', () => {
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('reserves estimated seconds × per-second ucents', async () => {
it('asks hasEnoughCredits for estimated seconds × per-second ucents', async () => {
const { actor } = await makeUser();
fetchSpy.mockResolvedValueOnce(
sttResponse({ text: 'ok', duration: 1 }),
@@ -285,7 +285,7 @@ describe('XAISpeechToTextProvider credit gate', () => {
driver.transcribe({ file: dataUrl(audio, 'audio/mp3') }),
);
expect(admitSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 2);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 2);
});
it('estimates 60 seconds for URL inputs (we cannot inspect size locally)', async () => {
@@ -298,7 +298,7 @@ describe('XAISpeechToTextProvider credit gate', () => {
driver.transcribe({ file: 'https://example.com/clip.mp3' }),
);
expect(admitSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 60);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 60);
});
});
+3 -3
View File
@@ -699,9 +699,9 @@ describe('TTSDriver cost factor', () => {
server.services.metering,
'incrementUsage',
);
const admitSpy = vi.spyOn(
const hasCreditsSpy = vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
'hasEnoughCredits',
);
const metered = () =>
incrementUsageSpy.mock.calls.map(([, , , cost]) => cost!);
@@ -725,7 +725,7 @@ describe('TTSDriver cost factor', () => {
const [base, factored] = metered();
expect(base).toBeGreaterThan(0);
expect(factored).toBe(Math.round(base! * 2));
const held = admitSpy.mock.calls.map(([, amount]) => amount);
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
expect(held).toEqual([base, Math.round(base! * 2)]);
});
});
@@ -84,7 +84,7 @@ vi.mock('@aws-sdk/client-polly', async () => {
let server: PuterServer;
let priceFetch: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -180,7 +180,7 @@ beforeEach(() => {
pollyCtor.mockReset();
priceFetch = vi.fn(servePriceLists());
vi.stubGlobal('fetch', priceFetch);
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -314,7 +314,7 @@ describe('AWSPollyTTSProvider.synthesize test_mode', () => {
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
content_type: 'audio',
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(pollySendMock).not.toHaveBeenCalled();
});
});
@@ -349,7 +349,7 @@ describe('AWSPollyTTSProvider.synthesize argument validation', () => {
describe('AWSPollyTTSProvider.synthesize credit gate', () => {
it('throws 402 BEFORE hitting Polly when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
sendDispatch();
await expect(
@@ -481,7 +481,7 @@ describe('AWSPollyTTSProvider.synthesize metering', () => {
expect(cost).toBe(expectedCost);
});
it('reserves the same total it later meters', async () => {
it('asks for hasEnoughCredits with the same total it later meters', async () => {
const provider = makeProvider();
sendDispatch();
@@ -495,7 +495,7 @@ describe('AWSPollyTTSProvider.synthesize metering', () => {
);
const expectedCost = AWS_POLLY_COSTS['generative'] * text.length;
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
});
});
@@ -599,7 +599,7 @@ describe('AWSPollyTTSProvider published pricing', () => {
engine: 'generative',
});
expect(admitSpy.mock.calls.map((c) => c[1])).toEqual([
expect(hasCreditsSpy.mock.calls.map((c) => c[1])).toEqual([
1700 * 5,
3000 * 5,
]);
@@ -648,7 +648,7 @@ describe('AWSPollyTTSProvider published pricing', () => {
await readStreamResult(result);
expect(refreshedNeural).toBe(19);
expect(admitSpy.mock.calls[0]![1]).toBe(1700 * 5);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(1700 * 5);
expect(incrementUsageSpy.mock.calls[0]![3]).toBe(1700 * 5);
});
@@ -86,10 +86,8 @@ describe('meterPerCharacter', () => {
it('holds and then meters exactly text length × price', async () => {
const check = vi
.spyOn(server.services.metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
.spyOn(server.services.metering, 'hasEnoughCredits')
.mockResolvedValue(true);
const record = vi.spyOn(server.services.metering, 'incrementUsage');
await drain(
@@ -103,7 +101,7 @@ describe('meterPerCharacter', () => {
),
);
expect(check).toHaveBeenCalledWith(actor, 7500, undefined);
expect(check).toHaveBeenCalledWith(actor, 7500);
expect(record).toHaveBeenCalledWith(
actor,
'openai:tts-1:character',
@@ -120,10 +118,8 @@ describe('meterPerCharacter', () => {
it('meters nothing when the upstream breaks before any audio', async () => {
vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
).mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
'hasEnoughCredits',
).mockResolvedValue(true);
const record = vi.spyOn(server.services.metering, 'incrementUsage');
await drain(
@@ -143,10 +139,8 @@ describe('meterPerCharacter', () => {
it('meters the full text once audio has started, even if the upstream then breaks', async () => {
vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
).mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
'hasEnoughCredits',
).mockResolvedValue(true);
const record = vi.spyOn(server.services.metering, 'incrementUsage');
await drain(
@@ -171,10 +165,8 @@ describe('meterPerCharacter', () => {
it('meters the full text when the caller hangs up before any audio', async () => {
vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
).mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
'hasEnoughCredits',
).mockResolvedValue(true);
const record = vi.spyOn(server.services.metering, 'incrementUsage');
const result = await meterPerCharacter(
@@ -200,10 +192,8 @@ describe('meterPerCharacter', () => {
it('meters nothing when synthesis fails', async () => {
vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
).mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
'hasEnoughCredits',
).mockResolvedValue(true);
const record = vi.spyOn(server.services.metering, 'incrementUsage');
const boom = new Error('vendor down');
@@ -223,8 +213,8 @@ describe('meterPerCharacter', () => {
it('refuses with a 402 before synthesizing when unaffordable', async () => {
vi.spyOn(
server.services.metering,
'reserveCreditsIfAffordable',
).mockResolvedValue(null);
'hasEnoughCredits',
).mockResolvedValue(false);
const synthesize = vi.fn(audio);
await expect(
@@ -57,7 +57,7 @@ import { ELEVENLABS_TTS_COSTS } from './costs.js';
let server: PuterServer;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -88,7 +88,7 @@ const audioResponse = (body = 'audio-bytes', contentType = 'audio/mpeg') =>
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -283,7 +283,7 @@ describe('ElevenLabsTTSProvider.synthesize test_mode', () => {
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
content_type: 'audio',
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
@@ -323,7 +323,7 @@ describe('ElevenLabsTTSProvider.synthesize argument validation', () => {
});
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
@@ -358,7 +358,7 @@ describe('ElevenLabsTTSProvider.synthesize argument validation', () => {
describe('ElevenLabsTTSProvider.synthesize credit gate', () => {
it('throws 402 BEFORE hitting ElevenLabs when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => provider.synthesize({ text: 'hi' })),
@@ -510,7 +510,7 @@ describe('ElevenLabsTTSProvider.synthesize request shape', () => {
fields: { key, got: value },
});
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(incrementUsageSpy).not.toHaveBeenCalled();
},
);
@@ -893,7 +893,7 @@ describe('ElevenLabsTTSProvider.synthesize metering', () => {
expect(cost).toBe(expectedCost);
});
it('reserves the same total it later meters', async () => {
it('asks for hasEnoughCredits with the same total it later meters', async () => {
const provider = makeProvider();
fetchSpy.mockResolvedValueOnce(audioResponse());
@@ -904,7 +904,7 @@ describe('ElevenLabsTTSProvider.synthesize metering', () => {
const expectedCost =
ELEVENLABS_TTS_COSTS['eleven_multilingual_v2'] * text.length;
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
});
});
@@ -61,7 +61,7 @@ const INTERACTIONS_URL =
let server: PuterServer;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchIncrementUsagesSpy: MockInstance<
MeteringService['batchIncrementUsages']
>;
@@ -183,16 +183,13 @@ const payingUser = () =>
/** Spies on the credit holds taken, recording the amount and each release. */
const trackHolds = () => {
const metering = server.services.metering;
const realReserve = Object.getPrototypeOf(
metering,
).reserveCreditsIfAffordable.bind(metering) as typeof metering.reserveCreditsIfAffordable;
const realReserve = metering.reserveCredits.bind(metering);
const held: number[] = [];
const released = vi.fn();
vi.spyOn(metering, 'reserveCreditsIfAffordable').mockImplementation(
vi.spyOn(metering, 'reserveCredits').mockImplementation(
async (actor, amount, opts) => {
held.push(amount);
const hold = await realReserve(actor, amount, opts);
if (!hold) return null;
return {
...hold,
release: async () => {
@@ -217,7 +214,7 @@ beforeEach(() => {
.mockRejectedValue(new Error('unexpected fetch')) as MockInstance<
typeof fetch
>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
batchIncrementUsagesSpy = vi.spyOn(
server.services.metering,
'batchIncrementUsages',
@@ -335,7 +332,7 @@ describe('GeminiTTSProvider.synthesize test_mode', () => {
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
content_type: 'audio',
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
@@ -409,7 +406,7 @@ describe('GeminiTTSProvider.synthesize argument validation', () => {
legacyCode: 'input_too_large',
fields: { key: 'text', max: 3000 },
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
@@ -437,7 +434,7 @@ describe('GeminiTTSProvider.synthesize argument validation', () => {
legacyCode: 'input_too_large',
fields: { key: 'instructions', max: 3000 },
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
@@ -446,7 +443,7 @@ describe('GeminiTTSProvider.synthesize argument validation', () => {
describe('GeminiTTSProvider.synthesize credit gate', () => {
it('throws 402 before calling Gemini when the actor lacks credits', async () => {
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => makeProvider().synthesize({ text: 'hi' })),
).rejects.toMatchObject({ statusCode: 402 });
@@ -476,10 +473,9 @@ describe('GeminiTTSProvider.synthesize credit gate', () => {
actor,
),
).rejects.toMatchObject({ statusCode: 402 });
expect(admitSpy).toHaveBeenCalledWith(
expect(hasCreditsSpy).toHaveBeenCalledWith(
actor,
FLASH_RESERVATION,
undefined,
);
expect(fetchSpy).not.toHaveBeenCalled();
expect(batchIncrementUsagesSpy).not.toHaveBeenCalled();
@@ -1369,10 +1365,9 @@ describe('GeminiTTSProvider.synthesize scheduled rates', () => {
server.clients.event.off('ai.cost.factor.*', factor);
}
expect(admitSpy).toHaveBeenCalledWith(
expect(hasCreditsSpy).toHaveBeenCalledWith(
expect.anything(),
reserved * 1.5,
undefined,
);
expect(held).toEqual([reserved * 1.5]);
expect(metered(':input').costOverride).toBe(7 * inputRate * 1.5);
@@ -84,7 +84,7 @@ vi.mock('openai', () => {
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -108,7 +108,7 @@ const mockAudioResponse = (bytes = 'opus-audio') => new Response(bytes);
beforeEach(() => {
speechCreateMock.mockReset();
openAICtor.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -205,7 +205,7 @@ describe('OpenAITTSProvider.synthesize test_mode', () => {
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
content_type: 'audio',
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(speechCreateMock).not.toHaveBeenCalled();
});
});
@@ -283,7 +283,7 @@ describe('OpenAITTSProvider.synthesize argument validation', () => {
describe('OpenAITTSProvider.synthesize credit gate', () => {
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => provider.synthesize({ text: 'hi' })),
@@ -298,9 +298,7 @@ describe('OpenAITTSProvider.synthesize credit gate', () => {
user: { id: 7, uuid: `tts-hold-${Date.now()}`, username: 'hold' },
});
const userId = actor.user!.uuid;
admitSpy.mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
hasCreditsSpy.mockResolvedValue(true);
let heldDuringCall = -1;
speechCreateMock.mockImplementationOnce(async () => {
heldDuringCall = await server.stores.creditHold.outstanding(userId);
@@ -326,9 +324,7 @@ describe('OpenAITTSProvider.synthesize credit gate', () => {
const actor = makeActor({
user: { id: 8, uuid: `tts-fail-${Date.now()}`, username: 'fail' },
});
admitSpy.mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
hasCreditsSpy.mockResolvedValue(true);
speechCreateMock.mockRejectedValueOnce(new Error('upstream down'));
await expect(
@@ -467,7 +463,7 @@ describe('OpenAITTSProvider.synthesize metering', () => {
expect(cost).toBe(expectedCost);
});
it('reserves the same total it later meters', async () => {
it('asks for hasEnoughCredits with the same total it later meters', async () => {
const provider = makeProvider();
speechCreateMock.mockResolvedValueOnce(mockAudioResponse());
@@ -477,8 +473,8 @@ describe('OpenAITTSProvider.synthesize metering', () => {
);
const expectedCost = OPENAI_TTS_COSTS['tts-1'] * text.length;
// The first reservation should match the metered cost.
const creditCall = admitSpy.mock.calls[0]!;
// First call to hasEnoughCredits should match the metered cost.
const creditCall = hasCreditsSpy.mock.calls[0]!;
expect(creditCall[1]).toBe(expectedCost);
});
});
@@ -56,7 +56,7 @@ import { SPEECHIFY_TTS_COSTS } from './costs.js';
let server: PuterServer;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -82,7 +82,7 @@ const audioResponse = (audioData = Buffer.from('audio-bytes').toString('base64')
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -159,7 +159,7 @@ describe('SpeechifyTTSProvider.synthesize test_mode', () => {
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
content_type: 'audio',
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
@@ -206,7 +206,7 @@ describe('SpeechifyTTSProvider.synthesize argument validation', () => {
describe('SpeechifyTTSProvider.synthesize credit gate', () => {
it('throws 402 BEFORE hitting Speechify when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => provider.synthesize({ text: 'hi' })),
@@ -324,7 +324,7 @@ describe('SpeechifyTTSProvider.synthesize metering', () => {
expect(cost).toBe(expectedCost);
});
it('reserves the same total it later meters', async () => {
it('asks for hasEnoughCredits with the same total it later meters', async () => {
const provider = makeProvider();
fetchSpy.mockResolvedValueOnce(audioResponse());
@@ -332,7 +332,7 @@ describe('SpeechifyTTSProvider.synthesize metering', () => {
await withTestActor(() => provider.synthesize({ text }));
const expectedCost = SPEECHIFY_TTS_COSTS['simba-3.2'] * text.length;
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
});
});
@@ -56,7 +56,7 @@ import { XAI_TTS_COSTS } from './costs.js';
let server: PuterServer;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
@@ -79,7 +79,7 @@ const audioResponse = (body = 'audio-bytes', contentType = 'audio/mpeg') =>
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
});
@@ -160,7 +160,7 @@ describe('XAITTSProvider.synthesize test_mode', () => {
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
content_type: 'audio',
});
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
@@ -195,7 +195,7 @@ describe('XAITTSProvider.synthesize argument validation', () => {
describe('XAITTSProvider.synthesize credit gate', () => {
it('throws 402 BEFORE hitting xAI when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() => provider.synthesize({ text: 'hi' })),
@@ -327,7 +327,7 @@ describe('XAITTSProvider.synthesize metering', () => {
expect(cost).toBe(expectedCost);
});
it('reserves the same total it later meters', async () => {
it('asks for hasEnoughCredits with the same total it later meters', async () => {
const provider = makeProvider();
fetchSpy.mockResolvedValueOnce(audioResponse());
@@ -335,7 +335,7 @@ describe('XAITTSProvider.synthesize metering', () => {
await withTestActor(() => provider.synthesize({ text }));
const expectedCost = XAI_TTS_COSTS['xai-tts'] * text.length;
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
});
});
@@ -94,7 +94,7 @@ vi.mock('../../util/secureHttp.js', async (importOriginal) => ({
let server: PuterServer;
let driver: VideoGenerationDriver;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
beforeAll(async () => {
server = await setupTestServer({
@@ -122,10 +122,8 @@ beforeEach(() => {
togetherVideosCreateMock.mockReset();
togetherVideosRetrieveMock.mockReset();
secureFetchMock.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
admitSpy.mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
hasCreditsSpy.mockResolvedValue(true);
vi.spyOn(server.services.metering, 'getRemainingUsage').mockResolvedValue(
100_000_000_000,
);
@@ -106,7 +106,7 @@ const MP4 = Buffer.concat([
let server: PuterServer;
let videoDriver: VideoGenerationDriver;
let fetchSpy: MockInstance<typeof fetch>;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let batchSpy: MockInstance<MeteringService['batchIncrementUsages']>;
const generate = (args: Record<string, unknown> = {}) =>
@@ -139,11 +139,9 @@ beforeEach(() => {
secureFetchMock.mockReset();
secureFetchMock.mockRejectedValue(new Error('unexpected secureFetch'));
// The full hold is more than a new account's allowance.
admitSpy = vi
.spyOn(server.services.metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
hasCreditsSpy = vi
.spyOn(server.services.metering, 'hasEnoughCredits')
.mockResolvedValue(true);
batchSpy = vi.spyOn(server.services.metering, 'batchIncrementUsages');
});
@@ -591,7 +589,7 @@ describe('GeminiVideoProvider test_mode', () => {
it('returns the sample clip without network, credits or metering', async () => {
expect(await testMode()).toBe(SAMPLE_URL);
expect(fetchSpy).not.toHaveBeenCalled();
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(batchSpy).not.toHaveBeenCalled();
});
@@ -1030,7 +1028,7 @@ describe('GeminiVideoProvider source video', () => {
const path = await writeVideo(user, 'dog.mp4', MP4);
const read = vi.spyOn(server.stores.s3Object, 'getObjectStream');
admitSpy.mockResolvedValue(null);
hasCreditsSpy.mockResolvedValue(false);
countOnly();
await expect(
withActor(user.actor, () => generate({ input_video: path })),
@@ -1575,12 +1573,12 @@ describe('GeminiVideoProvider countTokens preflight', () => {
describe('GeminiVideoProvider credit hold', () => {
it('throws 402 after counting tokens, before generating', async () => {
const { actor } = await makeUser();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
fetchSpy.mockResolvedValueOnce(Response.json({ totalTokens: 18 }));
await expect(withActor(actor, () => generate())).rejects.toMatchObject({
statusCode: 402,
});
expect(admitSpy.mock.calls[0]![1]).toBe(HOLD);
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(HOLD);
expect(fetchSpy).toHaveBeenCalledTimes(1);
expect(batchSpy).not.toHaveBeenCalled();
});
@@ -85,7 +85,7 @@ vi.mock('together-ai', () => {
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
let remainingUsageSpy: MockInstance<MeteringService['getRemainingUsage']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
@@ -117,13 +117,11 @@ beforeEach(() => {
videosCreateMock.mockReset();
videosRetrieveMock.mockReset();
togetherCtor.mockReset();
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
// Default to "has credits" so per-test setup only needs to override
// for the explicit credit-gate scenarios. SYSTEM_ACTOR has no uuid,
// so the live `getRemainingUsage` path can short-circuit to 0.
admitSpy.mockImplementation((who, amount, opts) =>
server.services.metering.reserveCredits(who, amount, opts),
);
hasCreditsSpy.mockResolvedValue(true);
remainingUsageSpy = vi.spyOn(server.services.metering, 'getRemainingUsage');
remainingUsageSpy.mockResolvedValue(AMPLE_CREDIT);
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
@@ -199,7 +197,7 @@ describe('TogetherVideoProvider.generate test_mode', () => {
provider.generate({ prompt: 'hi', test_mode: true }),
);
expect(result).toBe('https://assets.puter.site/txt2vid.mp4');
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
expect(videosCreateMock).not.toHaveBeenCalled();
});
});
@@ -240,7 +238,7 @@ const DIRECTOR = 'togetherai:minimax/video-01-director';
describe('TogetherVideoProvider.generate credit gate', () => {
it('throws 402 BEFORE hitting Together when actor lacks credits', async () => {
const provider = makeProvider();
admitSpy.mockResolvedValueOnce(null);
hasCreditsSpy.mockResolvedValueOnce(false);
await expect(
withTestActor(() =>
@@ -717,7 +715,7 @@ describe('TogetherVideoProvider.generate per-second models', () => {
expect('height' in sent).toBe(false);
expect('ratio' in sent).toBe(false);
// The flat-rate gate is not consulted for per-second models.
expect(admitSpy).not.toHaveBeenCalled();
expect(hasCreditsSpy).not.toHaveBeenCalled();
const seedance = TOGETHER_VIDEO_GENERATION_MODELS.find(
(m) => m.model === 'ByteDance/Seedance-2.5',
@@ -188,11 +188,7 @@ describe('app.message with live delivery over the socket wire', () => {
captureBus(state.server, 'outer.gui.notif.message', async () => {
created = (await withActor(asApp(holder, app), () =>
state.driver.create(
message({
text: 'Body',
priority: 'high',
channel: 'replies',
}),
message({ text: 'Body', channel: 'replies' }),
),
)) as Record<string, unknown>;
}).then((seen) => (desktop = seen)),
@@ -204,7 +200,7 @@ describe('app.message with live delivery over the socket wire', () => {
title: 'Hello',
text: 'Body',
channel: 'replies',
priority: 'high',
priority: 'normal',
ttl: 3600,
app: { uid: app.uid, name: 'App message test' },
});
@@ -216,7 +212,6 @@ describe('app.message with live delivery over the socket wire', () => {
app_uid: app.uid,
audience: 'app-user',
type: 'app.message',
value: expect.objectContaining({ priority: 'high' }),
});
// Not silent, but sent for the app alone: the desktop wire stays quiet.
expect(written).toEqual([
@@ -225,7 +220,7 @@ describe('app.message with live delivery over the socket wire', () => {
uid,
type: 'app.message',
appUid: app.uid,
value: expect.objectContaining({ priority: 'high' }), }),
}),
]);
expect(desktop).toEqual([]);
});
@@ -207,7 +207,6 @@ describe('app.message to the other party of a share', () => {
const written = await captureBus('notif.written', async () => {
sent = await send(inApp(owner, app), {
title: 'New message',
priority: 'high',
to: { handle },
});
});
@@ -223,7 +222,7 @@ describe('app.message to the other party of a share', () => {
expect(row!.value).toEqual({
type: 'app.message',
title: 'New message',
priority: 'high',
priority: 'normal',
ttl: 3600,
app: { uid: app.uid, name: 'Handle App' },
});
@@ -165,7 +165,6 @@ const fakeSchedules = (plan: Partial<DeferredSendPlan>) =>
createFor: vi.fn(),
list: vi.fn(),
get: vi.fn(),
update: vi.fn(),
remove: vi.fn(),
receivePeerChanges: vi.fn(),
prepareSendAt: vi.fn(async (): Promise<DeferredSendPlan> => ({
@@ -271,9 +270,7 @@ describe('notify({ sendAt }) with services.schedules', () => {
const schedules = fakeSchedules({ sendAt: at });
setSchedules(schedules);
await expect(
notify({ sendAt: at, text: 'later', priority: 'high' }),
).resolves.toEqual({
await expect(notify({ sendAt: at, text: 'later' })).resolves.toEqual({
scheduled: { scheduleId: 'sched-1', sendAt: at },
});
expect(schedules.prepareSendAt).toHaveBeenCalledWith(
@@ -286,7 +283,6 @@ describe('notify({ sendAt }) with services.schedules', () => {
message: expect.objectContaining({
title: 'Hi',
text: 'later',
priority: 'high',
}),
},
);
@@ -322,20 +318,11 @@ describe('deferredAppMessages.write', () => {
const key = uuidv4();
const id = await claimed(token);
await expect(
write(key, txFor(id, token), {
payload: JSON.stringify({
message: { title: 'Later', priority: 'high' },
}),
}),
).resolves.toBe('delivered');
await expect(write(key, txFor(id, token))).resolves.toBe('delivered');
const rows = await mailbox();
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({ uid: key, send_key: key });
expect(valueOf(rows[0])).toMatchObject({
title: 'Later',
priority: 'high',
});
expect(valueOf(rows[0])).toMatchObject({ title: 'Later' });
});
it('writes nothing when the guard no longer holds: another claim took it', async () => {
@@ -859,32 +859,3 @@ describe('NotificationDriver — rows written before the scope columns', () => {
expect(await selectUids(holder.actor)).toEqual([uid]);
});
});
describe('NotificationDriver displayed notification catch-up', () => {
it('selects named ids within the app and user scope', async () => {
const box = await mailbox();
await server.services.notification.markAcknowledged(box.rows.appUser, box.holder.userId);
const rows = await withActor(box.ownAppActor, () => driver.select({
subject: 'notif:app-user',
uids: Object.values(box.rows),
limit: 50,
}));
expect(rows).toEqual([expect.objectContaining({
uid: box.rows.appUser, acknowledged: expect.any(Number),
})]);
});
it.each([Array(51).fill('one'), [''], [7], 'one', ['x'.repeat(129)]])(
'rejects an invalid or oversized id batch: %s', async (uids) => {
const { actor } = await makeUser();
await expect(withActor(actor, () => driver.select({ uids })))
.rejects.toMatchObject({ statusCode: 400 });
},
);
it('returns no rows for an empty id batch', async () => {
const { actor } = await makeUser();
expect(await withActor(actor, () => driver.select({ uids: [] }))).toEqual([]);
});
});
@@ -46,7 +46,7 @@ import {
ownedAppUids,
} from '../../services/notification/notificationAudience.js';
import {
assertNotificationValueFits,
NOTIFICATION_VALUE_MAX_BYTES,
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
resolveNotificationWrite,
type NotificationTypeName,
@@ -120,6 +120,19 @@ interface MailboxScope {
appUid: string | null | undefined;
}
const assertValueFits = (value: Record<string, unknown>): void => {
if (
Buffer.byteLength(JSON.stringify(value), 'utf8') >
NOTIFICATION_VALUE_MAX_BYTES
) {
throw new HttpError(
400,
`\`value\` may not exceed ${NOTIFICATION_VALUE_MAX_BYTES} bytes`,
{ legacyCode: 'notification_value_too_large' },
);
}
};
const typeNotAllowed = (message: string): HttpError =>
new HttpError(403, message, {
legacyCode: 'forbidden',
@@ -264,7 +277,7 @@ export class NotificationDriver extends PuterDriver {
value,
);
}
assertNotificationValueFits(value);
assertValueFits(value);
if (type === APP_MESSAGE_TYPE) {
throw typeNotAllowed(`Only apps can create ${APP_MESSAGE_TYPE}`);
}
@@ -329,34 +342,13 @@ export class NotificationDriver extends PuterDriver {
: predicate;
const scope = this.#resolveScope(actor, args.subject);
const uids = args.uids;
if (
uids !== undefined &&
(!Array.isArray(uids) ||
uids.length > 50 ||
uids.some(
(uid) =>
typeof uid !== 'string' ||
uid.length === 0 ||
uid.length > 128,
))
) {
throw new HttpError(
400,
'uids must contain at most 50 notification ids',
{
legacyCode: 'bad_request',
},
);
}
const query: {
limit: number;
scope: MailboxScope | null;
filter?: string;
onlyUnacknowledged?: boolean;
excludeTypes?: readonly string[];
uids?: string[];
} = { limit, scope, ...(uids ? { uids: uids as string[] } : {}) };
} = { limit, scope };
// The whole mailbox, or a slice naming no app, is the desktop's view of
// it, which leaves out the types only an app has a use for. A slice
// naming the app still returns them.
@@ -438,7 +430,7 @@ export class NotificationDriver extends PuterDriver {
}
const handle = to === undefined ? undefined : parseNotifyTarget(to);
const message = parseAppMessage(fields);
assertNotificationValueFits(value);
assertValueFits(value);
const timing =
sendAt === undefined || sendAt === null
? null
@@ -547,7 +539,9 @@ export class NotificationDriver extends PuterDriver {
};
}
/** For extensions. A field, not a method, so `/drivers/call` can't reach it. */
/**
* For extensions. A field, not a method, so `/drivers/call` can't reach it.
*/
readonly deferredAppMessages = {
write: (input: DeferredWriteInput, tx: DeferredWriteTx) =>
this.#writeDeferredAppMessage(input, tx),
+29 -89
View File
@@ -324,10 +324,8 @@ describe('AI cost factor', () => {
it('gates and holds at the factored cost', async () => {
listen(1.3);
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
const reserve = vi.spyOn(metering, 'reserveCredits');
try {
const hold = await scoped.reserveAiCredits(
@@ -336,11 +334,7 @@ describe('AI cost factor', () => {
1000,
);
await hold?.release();
expect(check).toHaveBeenCalledExactlyOnceWith(
actor,
1300,
undefined,
);
expect(check).toHaveBeenCalledWith(actor, 1300);
expect(reserve).toHaveBeenCalledWith(actor, 1300, undefined);
} finally {
check.mockRestore();
@@ -348,40 +342,10 @@ describe('AI cost factor', () => {
}
});
it('admits one of two concurrent holds the balance only covers once', async () => {
listen(1.3);
vi.spyOn(metering, 'getAllowedUsage').mockResolvedValue({
remaining: 20_000_000,
monthUsageAllowance: 20_000_000,
addons: {},
});
const holds = await Promise.all(
[0, 1].map(() =>
scoped.reserveAiCredits(
actor,
'openai:gpt-realtime:realtime:audio_output',
12_500_000,
),
),
);
try {
expect(holds.filter((h) => h !== null)).toHaveLength(1);
// The factor applies once: 12.5M at 1.3 is 16.25M held.
expect(await metering.getOutstandingHolds(actor)).toBe(
16_250_000,
);
} finally {
await Promise.all(holds.map((h) => h?.release()));
vi.restoreAllMocks();
}
});
it('passes a hold lifetime through to the service', async () => {
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
const reserve = vi.spyOn(metering, 'reserveCredits');
try {
const hold = await scoped.reserveAiCredits(
@@ -403,12 +367,8 @@ describe('AI cost factor', () => {
it('takes no hold when the factored cost is unaffordable', async () => {
listen(1.3);
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation(async (who, amount, opts) =>
amount <= 1000
? metering.reserveCredits(who, amount, opts)
: null,
);
.spyOn(metering, 'hasEnoughCredits')
.mockImplementation(async (_actor, amount) => amount <= 1000);
const reserve = vi.spyOn(metering, 'reserveCredits');
try {
const hold = await scoped.reserveAiCredits(
@@ -449,10 +409,8 @@ describe('AI cost factor', () => {
it('holds the factored cost while run executes, then releases it', async () => {
listen(1.3);
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
try {
let heldDuringRun = 0;
const result = await withAiCreditHold(
@@ -490,10 +448,8 @@ describe('AI cost factor', () => {
};
server.clients.event.on('ai.cost.factor.*', slowFactor as never);
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
try {
await runWithContext({ actor }, () =>
withAiCreditHold(
@@ -529,10 +485,8 @@ describe('AI cost factor', () => {
it('releases the hold when the usage write fails', async () => {
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
const write = vi
.spyOn(metering, 'incrementUsage')
.mockRejectedValueOnce(new Error('ledger down'));
@@ -561,10 +515,8 @@ describe('AI cost factor', () => {
it('releases at once when nothing was recorded', async () => {
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
try {
await runWithContext({ actor }, () =>
withAiCreditHold(
@@ -583,10 +535,8 @@ describe('AI cost factor', () => {
it('releases the hold when run throws', async () => {
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
const boom = new Error('upstream down');
try {
await expect(
@@ -602,8 +552,8 @@ describe('AI cost factor', () => {
it('refuses with a 402 and never runs when the cost is unaffordable', async () => {
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockResolvedValue(null);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(false);
const run = vi.fn();
try {
await expect(
@@ -625,10 +575,8 @@ describe('AI cost factor', () => {
const relay = async (source: Readable) => {
const record = vi.fn();
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
try {
const result = await withStreamedAiCharge(
scoped,
@@ -676,10 +624,8 @@ describe('AI cost factor', () => {
landCharge = resolve;
});
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
try {
const result = await withStreamedAiCharge(
scoped,
@@ -709,10 +655,8 @@ describe('AI cost factor', () => {
vi.useFakeTimers({ toFake: ['setInterval', 'clearInterval'] });
const refresh = vi.spyOn(server.stores.creditHold, 'refresh');
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
let landCharge = () => {};
const charge = new Promise<void>((resolve) => {
landCharge = resolve;
@@ -753,10 +697,8 @@ describe('AI cost factor', () => {
vi.useFakeTimers({ toFake: ['setInterval', 'clearInterval'] });
const refresh = vi.spyOn(server.stores.creditHold, 'refresh');
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
let opened = () => {};
const opening = new Promise<void>((resolve) => {
opened = resolve;
@@ -799,10 +741,8 @@ describe('AI cost factor', () => {
it('releases the hold when recording the charge throws', async () => {
const check = vi
.spyOn(metering, 'reserveCreditsIfAffordable')
.mockImplementation((who, amount, opts) =>
metering.reserveCredits(who, amount, opts),
);
.spyOn(metering, 'hasEnoughCredits')
.mockResolvedValue(true);
try {
const result = await withStreamedAiCharge(
scoped,
+3 -5
View File
@@ -354,12 +354,10 @@ export function withAiCostFactor(
? await costFactor(actor, aiModelKey(usageType))
: 1;
const cost = factor === 1 ? amount : scaleCost(amount, factor);
const hold = await metering.reserveCreditsIfAffordable(
actor,
cost,
opts,
if (!(await metering.hasEnoughCredits(actor, cost))) return null;
return outlastingWrites(
await metering.reserveCredits(actor, cost, opts),
);
return hold && outlastingWrites(hold);
};
const reserveCredits = async (
-52
View File
@@ -336,58 +336,6 @@ describe('PuterServer host header validation', () => {
expect(api.headers['x-frame-options']).toBeUndefined();
});
it('allows deployment subdomains to frame the app browser embed', async () => {
const parent = `http://spreadsheet.puter.localhost:${port}`;
for (const path of ['/embed/apps', '/embed/apps/']) {
const res = await request(path, {
host: `puter.localhost:${port}`,
referer: `${parent}/sheet/1`,
});
expect(res.status).toBe(200);
expect(res.headers['x-frame-options']).toBeUndefined();
expect(res.headers['content-security-policy']).toBe(
`frame-ancestors 'self' ${parent}`,
);
expect(res.headers['cache-control']).toBe('no-store');
expect(res.body).toContain('/dist/apps-embed.min.js');
expect(res.body).not.toContain('/dist/bundle.min.js');
expect(res.body).toContain(`http://api.puter.localhost:${port}`);
}
const gui = await request('/embed/apps', {
host: `puter.localhost:${port}`,
referer: `http://puter.localhost:${port}/dashboard`,
});
expect(gui.headers['content-security-policy']).toBe("frame-ancestors 'self'");
const dashboard = await request('/dashboard', { host: `puter.localhost:${port}` });
expect(dashboard.headers['x-frame-options']).toBe('SAMEORIGIN');
const api = await request('/embed/apps', { host: `api.puter.localhost:${port}` });
expect(api.status).toBe(404);
});
it.each([
['a user-hosted site', 'http://someone.site.puter.localhost:PORT/'],
['a private app host', 'http://someone.app.puter.localhost:PORT/'],
['another site', 'http://evil.test:PORT/'],
['another port', 'http://spreadsheet.puter.localhost:1/'],
['a suppressed referrer', undefined],
])('keeps the embed same-origin only when framed by %s', async (_label, referer) => {
const res = await request('/embed/apps', {
host: `puter.localhost:${port}`,
...(referer ? { referer: referer.replace('PORT', String(port)) } : {}),
});
expect(res.status).toBe(200);
expect(res.headers['content-security-policy']).toBe("frame-ancestors 'self'");
});
it('strips SDK bootstrap query parameters from the embed route', async () => {
const res = await request('/embed/apps?puter.app_instance_id=other&puter.api_origin=https://other.test', {
host: `puter.localhost:${port}`,
});
expect(res.status).toBe(302);
expect(res.headers.location).toBe('/embed/apps');
expect(res.headers['x-frame-options']).toBe('SAMEORIGIN');
});
it('blocks a request the ip.validate listeners veto', async () => {
const handler = (_key: unknown, data: unknown) => {
(data as { allow: boolean }).allow = false;
+29 -107
View File
@@ -56,7 +56,6 @@ import {
} from '../../core/actor.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import type {
SqlGuard,
ScheduleEndReason,
ScheduleSubscriptionChange,
} from '../../core/schedules.js';
@@ -247,7 +246,6 @@ import {
isNotifToken,
kvHandleFromSubject,
kvHandleMatchFromSubject,
notifAnchorToken,
notifMatchSpansEveryApp,
parseSubject,
type FsOp,
@@ -682,7 +680,6 @@ export interface KvDispatchOptions {
/** One persisted notification, as the bus reports it. */
export interface NotifDispatchInput {
dismissed?: true;
/** The recipient, whose mailbox is the anchor. */
userId: number;
userUuid: string;
@@ -918,7 +915,7 @@ const scheduleSubscription = (): HttpError =>
const scheduleImmutable = (): HttpError =>
new HttpError(
400,
"Use updateSchedule() to change a schedule's context or timing",
"A schedule's context can't be changed; remove the schedule and create a new one",
{ legacyCode: 'events_schedule_immutable' },
);
@@ -1667,26 +1664,6 @@ export class EventsService extends PuterService {
});
});
this.clients.event.on('notif.acknowledged', (_key, data) => {
if (!this.notificationsFoldIn) return;
void (async () => {
const user = await this.stores.user.getById(data.userId);
if (!user?.uuid) return;
await this.dispatchNotif({
...data,
userUuid: user.uuid,
value: {},
createdAt: Date.now(),
dismissed: true,
});
})().catch((err) => {
console.warn(
'[events] notification dismissal dispatch failed',
err,
);
});
});
this.#armPendingSweep();
if (this.enabled) this.#registerJobs();
}
@@ -2893,7 +2870,6 @@ export class EventsService extends PuterService {
override?: readonly SuspendedReason[];
/** What a schedule ended by this records instead of `reason`. */
scheduleEndReason?: ScheduleEndReason;
guard?: SqlGuard;
} = {},
): Promise<DurableSubscription[]> {
if (rows.length === 0) return [];
@@ -2901,7 +2877,6 @@ export class EventsService extends PuterService {
const { suspended, bumps } =
await this.stores.durableSubscription.suspend(rows, reason, {
override: opts.override,
guard: opts.guard,
});
const policy = backlogPolicyFor(reason);
for (const row of suspended) {
@@ -3017,14 +2992,8 @@ export class EventsService extends PuterService {
* retry path's; this is the state it drives, and the developer is told
* because it is their code that stopped working. Called by extensions.
*/
async suspendForFailures(
subId: string,
guard?: SqlGuard,
): Promise<boolean> {
return this.#suspendOne(subId, 'failures', {
notifyDeveloper: true,
guard,
});
async suspendForFailures(subId: string): Promise<boolean> {
return this.#suspendOne(subId, 'failures', { notifyDeveloper: true });
}
/**
@@ -3699,16 +3668,13 @@ export class EventsService extends PuterService {
async #suspendOne(
subId: string,
reason: SuspendedReason,
options: { notifyDeveloper: boolean; guard?: SqlGuard },
options: { notifyDeveloper: boolean },
): Promise<boolean> {
if (!this.enabled) return false;
const row = await this.stores.durableSubscription.getBySubId(subId);
if (!row || row.suspendedAt !== null) return false;
const suspended = await this.#suspend([row], reason, {
guard: options.guard,
});
if (suspended.length === 0) return false;
await this.suspendSubscriptions([row], reason);
if (options.notifyDeveloper && row.appUid && row.handlerName)
await this.#notifySuspended(scopeOfRow(row), row.handlerName, 1);
return true;
@@ -4588,42 +4554,27 @@ export class EventsService extends PuterService {
return { matched, tokens };
}
if (item.family === 'notif' && item.notif) {
const input = {
...item.notif,
userId: item.ownerUserId,
createdAt: item.ts,
};
const matched = await this.dispatchNotif(input, {
forwarded: true,
id: item.id,
});
return { matched, tokens: [notifAnchorToken(item.notif.userUuid)] };
}
return { matched: false, tokens: [] };
}
/**
* Publish a posting or dismissal to the same mailbox audience. Postings
* also reach the desktop's wire; forwarded copies evaluate session rows
* only.
* Publish one persisted notification: the desktop's wire first, because
* that is what a person is waiting on, then whatever subscriptions asked
* for this slice of the mailbox.
*
* The fan-out is one token — a notification is addressed to a person — so
* there is no tree to walk and nothing to coalesce against.
*/
async dispatchNotif(
input: NotifDispatchInput,
options: { forwarded?: boolean; id?: string } = {},
): Promise<boolean> {
if (!this.notificationsFoldIn) return false;
async dispatchNotif(input: NotifDispatchInput): Promise<void> {
if (!this.notificationsFoldIn) return;
const subject = lookupNotifSubject('notif.created');
if (!subject) return false;
if (!subject) return;
const userId = input?.userId;
if (typeof userId !== 'number' || !input.userUuid || !input.uid)
return false;
if (typeof userId !== 'number' || !input.userUuid || !input.uid) return;
const context: NotifEventContext = {
key: input.dismissed ? 'notif.acknowledged' : 'notif.created',
...(input.dismissed ? { dismissed: true as const } : {}),
key: 'notif.created',
userId,
userUuid: input.userUuid,
uid: input.uid,
@@ -4631,9 +4582,9 @@ export class EventsService extends PuterService {
audience: (input.audience ??
'account') as NotifEventContext['audience'],
appUid: input.appUid ?? null,
notification: input.dismissed ? {} : (input.value ?? {}),
notification: input.value ?? {},
...(input.replaced === true ? { replaced: true as const } : {}),
id: options.id ?? (input.dismissed ? randomUUID() : input.uid),
id: input.uid,
ts: input.createdAt || Date.now(),
handlerDepth: input.handlerDepth,
};
@@ -4641,51 +4592,29 @@ export class EventsService extends PuterService {
// The adapter is not a subscription: the desktop is delivered to
// whether or not anything subscribed, which is what keeps the wire
// contract identical with the fold-in on.
if (!input.dismissed && !options.forwarded)
this.services.notification.deliverOverSocket(context);
this.services.notification.deliverOverSocket(context);
if (!(await this.#userHasAny(userId))) return false;
if (!(await this.#userHasAny(userId))) return;
const { local, remote } =
await this.stores.eventSubscription.watchedFor(
userId,
subject.tokens(context),
);
if (!options.forwarded && remote.size > 0)
this.services.eventForward.forwardEvent(regionsIn(remote), {
family: 'notif',
ownerUserId: userId,
id: context.id,
ts: context.ts,
notif: {
userUuid: input.userUuid,
uid: input.uid,
type: context.type,
audience: context.audience,
appUid: context.appUid,
value: context.notification,
...(input.dismissed ? { dismissed: true as const } : {}),
...(input.replaced ? { replaced: true as const } : {}),
handlerDepth: input.handlerDepth,
},
});
if (local.length === 0) return false;
const watched = await this.stores.eventSubscription.watchedTokens(
userId,
subject.tokens(context),
);
if (watched.length === 0) return;
const fetched = await this.stores.eventSubscription.getForTokens(
userId,
local,
{ keep: dispatchKeeps(options.forwarded) },
watched,
);
// A subscription naming no app is the desktop's view of the mailbox.
const rows = inAccountMailbox(context.type)
? fetched
: fetched.filter((row) => !notifMatchSpansEveryApp(row.match));
if (rows.length === 0) return true;
if (rows.length === 0) return;
await this.#route(subject, context, rows, userId, (matched) =>
this.#notifStillAuthorized(matched, context),
);
return true;
}
/**
@@ -6684,10 +6613,8 @@ export class EventsService extends PuterService {
*/
async invokeForSchedule(
invocation: WorkerInvocation,
options: Pick<WorkerRunOptions, 'stampEvent' | 'beforeSend'> = {},
options: Pick<WorkerRunOptions, 'stampEvent'> = {},
): Promise<ScheduleInvokeResult> {
if (options.beforeSend && !(await options.beforeSend()))
return { outcome: 'deferred' };
if (await this.#tooDeep(invocation)) return { outcome: 'too-deep' };
if (await this.#suspendIfHandlerGone(invocation))
return { outcome: 'deferred' };
@@ -6928,12 +6855,7 @@ export class EventsService extends PuterService {
delivery: AddressedDelivery,
): void {
if (this.#ended.has(row.subId) || this.#parkedUserRow(row)) return;
const event = delivery.envelope.event;
const key =
event.op === 'dismiss'
? `${subject}:dismiss:${event.uid}`
: subject;
this.#coalesce().push(coalesceKey(row.subId, key), delivery);
this.#coalesce().push(coalesceKey(row.subId, subject), delivery);
}
async #flush(delivery: AddressedDelivery): Promise<void> {
+1 -12
View File
@@ -88,7 +88,7 @@ export interface ForwardWatch {
/** One committed change, replayed against another region's session rows. */
export interface ForwardEvent {
kind: 'event';
family: 'fs' | 'kv' | 'notif';
family: 'fs' | 'kv';
ownerUserId: number;
/** Who caused it, so `self` is right on the far side. */
actingUserId?: number;
@@ -110,17 +110,6 @@ export interface ForwardEvent {
ancestors: Array<{ uid: string; path: string }>;
};
};
notif?: {
userUuid: string;
uid: string;
type: string;
audience: string;
appUid: string | null;
value: Record<string, unknown>;
dismissed?: true;
replaced?: true;
handlerDepth?: number;
};
kv?: {
userUuid: string;
appUid: string;
+1 -66
View File
@@ -402,7 +402,7 @@ const makeRegion = (
fields: { code: 'subject_does_not_exist' },
}),
},
notification: { notify: vi.fn(), deliverOverSocket: vi.fn() },
notification: { notify: vi.fn() },
metering: {
bufferIncrementUsages: (
actor: Actor,
@@ -2401,68 +2401,3 @@ describe('a durable row created in another region', () => {
);
});
});
describe('notification dismissals across connected pages', () => {
const config = { events: { enabled: true, notificationsFoldIn: true } } as Partial<IConfig>;
const appUid = 'notification-app';
const input = (uid = 'one') => ({
userId, userUuid: `user-${userId}`, uid,
type: 'app.message', audience: 'app-user', appUid,
value: {}, createdAt: Date.now(), dismissed: true as const,
});
it('reaches local and remote session pages without content or a forwarding loop', async () => {
const west = makeRegion('west', ['east'], config);
const east = makeRegion('east', ['west'], config);
for (const [region, socket] of [[west, 'local'], [east, 'remote']] as const) {
await region.events.subscribe(actorFor(appUid), socket, { subject: 'notif:app-user' });
}
await posted(east);
await quiet(100);
await west.events.dispatchNotif({ ...input(), value: { title: 'private' } });
await arrived(west);
await arrived(east);
for (const region of [west, east]) {
expect(region.sent).toHaveLength(1);
expect(region.sent[0].event).toMatchObject({ op: 'dismiss', uid: 'one', appUid });
expect(region.sent[0].event).not.toHaveProperty('notification');
}
expect(east.sent[0].event.id).toBe(west.sent[0].event.id);
const forwarded = west.posts.flatMap((p) => p.batch.items)
.find((item): item is ForwardEvent => item.kind === 'event');
expect(forwarded?.notif?.value).toEqual({});
expect(east.posts.flatMap((p) => p.batch.items).filter((i) => i.kind === 'event')).toEqual([]);
});
it('keeps separate dismissed ids in the same coalescing window', async () => {
const region = makeRegion('west', [], config);
await region.events.subscribe(actorFor(appUid), 'local', { subject: 'notif:app-user' });
await region.events.dispatchNotif(input('one'));
await region.events.dispatchNotif(input('two'));
await arrived(region, 2);
expect(region.sent.map((item) => 'uid' in item.event && item.event.uid)).toEqual(['one', 'two']);
});
it('does not deliver another app, account audience or another user to an app page', async () => {
const region = makeRegion('west', [], config);
await region.events.subscribe(actorFor(appUid), 'local', { subject: 'notif:app-user' });
await region.events.dispatchNotif({ ...input(), appUid: 'other-app' });
await region.events.dispatchNotif({ ...input(), audience: 'account' });
await region.events.dispatchNotif({ ...input(), userId: userId + 1, userUuid: 'other-user' });
await quiet(350);
expect(region.sent).toEqual([]);
});
it('also forwards postings through the same session path', async () => {
const west = makeRegion('west', ['east'], config);
const east = makeRegion('east', ['west'], config);
await east.events.subscribe(actorFor(appUid), 'remote', { subject: 'notif:app-user' });
await posted(east);
await quiet(100);
const { dismissed, ...postedInput } = input();
await west.events.dispatchNotif({ ...postedInput, value: { title: 'Hello' } });
await arrived(east);
expect(east.sent[0].event).toMatchObject({ op: 'post', uid: 'one', notification: { title: 'Hello' } });
});
});
+14 -24
View File
@@ -93,25 +93,24 @@ export interface ProjectedKvEvent extends ProjectedEventBase {
/**
* What a notification subscriber receives.
*
* For postings, `id` is the row's own uid: a client that fetched a missed
* notification and then received the live copy has to be able to tell they are
* the same thing, and the mailbox uid is the only identity both halves share.
* `uid` carries it under the name the mailbox verbs (`mark-ack`, `mark-shown`)
* take.
* `id` is the row's own uid rather than a per-dispatch uuid: a client that
* fetched a missed notification and then received the live copy has to be able
* to tell they are the same thing, and the mailbox uid is the only identity
* both halves share. `uid` carries it under the name the mailbox verbs
* (`mark-ack`, `mark-shown`) take.
*
* `notification` is the payload the caller sent, which is exactly what the
* socket adapter puts on `notif.message` — a subscriber reading this channel
* sees no less than the desktop does, and no internal row either. Dismissals
* use a fresh dispatch id and omit the payload.
* sees no less than the desktop does, and no internal row either.
*/
export interface ProjectedNotifEvent extends ProjectedEventBase {
op: 'post' | 'dismiss';
op: 'post';
uid: string;
type: string;
audience: NotificationAudience;
/** App the notification is about; `null` for platform notifications. */
appUid: string | null;
notification?: Record<string, unknown>;
notification: Record<string, unknown>;
/**
* Present when the notification replaced an earlier one under the same uid.
* `ts` is then the replacement time, the same on live and fetched copies,
@@ -121,9 +120,7 @@ export interface ProjectedNotifEvent extends ProjectedEventBase {
}
export type ProjectedEvent =
| ProjectedFsEvent
| ProjectedKvEvent
| ProjectedNotifEvent;
ProjectedFsEvent | ProjectedKvEvent | ProjectedNotifEvent;
export type GapReason =
| 'matched_subscription_limit'
@@ -203,7 +200,6 @@ export interface KvEventContext extends EventContextBase {
* recipient's uuid because the anchor token is the mailbox.
*/
export interface NotifEventContext extends EventContextBase {
dismissed?: true;
userId: number;
userUuid: string;
uid: string;
@@ -272,9 +268,7 @@ export interface NotifPublicSubject extends SubjectSpec<
}
export type PublicSubject =
| FsPublicSubject
| KvPublicSubject
| NotifPublicSubject;
FsPublicSubject | KvPublicSubject | NotifPublicSubject;
export interface UnpublishedInternalEvent {
event: EventKey;
@@ -352,14 +346,14 @@ const notifMatchTarget = (event: NotifEventContext): string =>
const notifMatchScope = (_anchorPath: string, slice: string): string => slice;
const notifProject = (delivery: NotifDeliveryContext): ProjectedNotifEvent => ({
id: delivery.dismissed ? delivery.id : delivery.uid,
id: delivery.uid,
subject: `notif:${delivery.appUid ?? delivery.userUuid}:${delivery.audience}`,
op: delivery.dismissed ? 'dismiss' : 'post',
op: 'post',
uid: delivery.uid,
type: delivery.type,
audience: delivery.audience,
appUid: delivery.appUid,
...(delivery.dismissed ? {} : { notification: delivery.notification }),
notification: delivery.notification,
...(delivery.replaced ? { replaced: true as const } : {}),
self: delivery.self,
ts: delivery.ts,
@@ -470,7 +464,7 @@ export const PUBLIC_SUBJECTS = [
{
family: 'notif',
subject: 'notif:*',
internal: ['notif.created', 'notif.acknowledged'],
internal: ['notif.created'],
tokens: notifTokens,
matchOn: notifMatchTarget,
matchSeparator: NOTIF_MATCH_SEPARATOR,
@@ -512,10 +506,6 @@ export const UNPUBLISHED_INTERNAL_EVENTS = [
event: 'notif.delivery.settled',
reason: 'A node-local report on how the live socket copy of a notification went; the notification itself publishes through `notif.created`.',
},
{
event: 'notif.handled',
reason: 'A node-local report that a page saw a notification live, for extensions deciding whether to push it; not a change to the notification.',
},
] as const satisfies readonly UnpublishedInternalEvent[];
// Every `fs.*`/`kv.*`/`notif.*` bus key needs a decision. A new key that is in
@@ -83,7 +83,6 @@ const recording = () => {
createFor: vi.fn(),
list: vi.fn(),
get: vi.fn(),
update: vi.fn(),
remove: vi.fn(),
receivePeerChanges: vi.fn(),
prepareSendAt: vi.fn(),
+2 -1
View File
@@ -48,7 +48,8 @@ export type SubjectFamily = 'fs' | 'kv' | 'notif';
export type FsOp = 'add' | 'write' | 'move' | 'remove' | 'meta';
export type NotifOp = 'post' | 'dismiss';
/** A notification is only ever posted; the mailbox verbs are their own surface. */
export type NotifOp = 'post';
export type SubjectOp = FsOp | KvOp | NotifOp;
@@ -218,7 +218,6 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
{
lane: options.lane,
stampEvent: options.stampEvent,
beforeSend: options.beforeSend,
},
);
if (result.deployBusy) return { outcome: 'deploy-busy' };
@@ -49,54 +49,6 @@ const makeReq = (over: Partial<Request> = {}): Request =>
...over,
}) as unknown as Request;
describe('app browser embed shell', () => {
it.each([
[{ env: 'dev' }, '/sdk/puter.dev.js'],
[{ env: 'prod' }, 'https://js.puter.com/v2/'],
[{ gui_puterjs_bundle: '/custom-sdk.js' }, '/custom-sdk.js'],
])('uses the deployment SDK configuration %j', (config, sdkUrl) => {
const service = makeService({
...config,
api_base_url: 'https://api.deployment.test',
domain: 'deployment.test',
});
const html = service.renderAppBrowserEmbed();
const configJson = html.match(/id="app-browser-config">(.*?)<\/script>/)![1];
expect(JSON.parse(configJson)).toEqual({
apiOrigin: 'https://api.deployment.test', sdkUrl,
});
expect(html).toContain('/dist/apps-embed.min.js');
expect(html).not.toContain('/dist/bundle.min.js');
expect(html).not.toContain('gui(');
});
it.each([
[{}, '/dist/apps-embed.min.js'],
[
{ gui_bundle: 'https://gui-cdn.deployment.test/bundle.min.js' },
'https://gui-cdn.deployment.test/apps-embed.min.js',
],
[
{ gui_bundle: '/static/bundle.min.js?v=42' },
'/static/apps-embed.min.js?v=42',
],
])('loads its script from beside the GUI bundle %j', (config, script) => {
const html = makeService(config).renderAppBrowserEmbed();
expect(html).toContain(`<script defer src="${script}"></script>`);
});
it('escapes configured values without breaking its JSON bootstrap', () => {
const sdkUrl = '/sdk.js?value=</script><script>unexpected()</script>';
const html = makeService({
gui_params: { title: '<script>unexpected()</script>' },
gui_puterjs_bundle: sdkUrl,
}).renderAppBrowserEmbed();
expect(html).not.toContain('<script>unexpected()');
const configJson = html.match(/id="app-browser-config">(.*?)<\/script>/)![1];
expect(JSON.parse(configJson).sdkUrl).toBe(sdkUrl);
});
});
/** Capture what the service sends, and return the rendered HTML. */
const render = async (
service: PuterHomepageService,
@@ -167,52 +167,6 @@ export class PuterHomepageService extends PuterService {
this.#guiParams[key] = val;
}
renderAppBrowserEmbed(): string {
const configJson = JSON.stringify({
apiOrigin: this.config.api_base_url,
sdkUrl:
this.config.env === 'dev'
? '/sdk/puter.dev.js'
: (this.config.gui_puterjs_bundle ??
'https://js.puter.com/v2/'),
}).replace(/</g, '\\u003c');
return `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>${encode(String(this.config.gui_params?.title ?? 'Puter'))}</title>
<link rel="stylesheet" href="/css/normalize.css">
<link rel="stylesheet" href="/css/dashboard.css">
<link rel="stylesheet" href="/css/apps-embed.css">
</head>
<body>
<main id="app-browser" class="dashboard"></main>
<script type="application/json" id="app-browser-config">${configJson}</script>
<script defer src="${encode(this.#appBrowserEmbedScript())}"></script>
</body>
</html>`;
}
/**
* The embed's script is built beside the GUI bundle, so it is served from
* wherever `gui_bundle` is (a CDN in production), with the same query.
*/
#appBrowserEmbedScript(): string {
const base = 'http://relative.invalid';
const bundle = new URL(
this.config.gui_bundle ?? '/dist/bundle.min.js',
base,
);
const script = new URL('apps-embed.min.js', bundle);
script.search = bundle.search;
return script.origin === base
? `${script.pathname}${script.search}`
: script.href;
}
/** `scriptPath` is a same-origin absolute path, like `/ext/push/worker.js`. */
registerServiceWorkerScript(scriptPath: string): void {
if (
@@ -4290,295 +4290,6 @@ describe('MeteringService', () => {
});
});
describe('reserveCreditsIfAffordable', () => {
const BUDGET = 20_000_000;
const HOLD = 16_200_000;
afterEach(() => {
vi.restoreAllMocks();
});
/** Spends the fresh actor's allowance down to exactly `balance`. */
const withBalance = async (balance: number) => {
const { remaining } = await target.getAllowedUsage(actor);
if (remaining > balance) {
await target.incrementUsage(
actor,
'kv:read',
1,
remaining - balance,
);
}
expect((await target.getAllowedUsage(actor)).remaining).toBe(
balance,
);
};
it('admits exactly one of two concurrent holds that only fit alone', async () => {
await withBalance(BUDGET);
const holds = await Promise.all([
target.reserveCreditsIfAffordable(actor, HOLD),
target.reserveCreditsIfAffordable(actor, HOLD),
]);
expect(holds.filter((h) => h !== null)).toHaveLength(1);
expect(await target.getOutstandingHolds(actor)).toBe(HOLD);
await Promise.all(holds.map((h) => h?.release()));
});
it('compares against the gross balance, counting live holds once', async () => {
await withBalance(BUDGET);
const other = await target.reserveCredits(actor, 5_000_000);
expect(
await target.reserveCreditsIfAffordable(actor, 15_000_001),
).toBeNull();
const fits = await target.reserveCreditsIfAffordable(
actor,
15_000_000,
);
expect(fits).not.toBeNull();
expect(await target.getOutstandingHolds(actor)).toBe(BUDGET);
// A release gives the headroom back for a retry.
await other.release();
expect(
await target.reserveCreditsIfAffordable(actor, 5_000_000),
).not.toBeNull();
});
it('counts allowance and purchased credit together', async () => {
await withBalance(1_000_000);
await target.updateAddonCredit(actor.user.uuid!, 2_000_000);
expect(
await target.reserveCreditsIfAffordable(actor, 3_000_001),
).toBeNull();
expect(
await target.reserveCreditsIfAffordable(actor, 3_000_000),
).not.toBeNull();
});
it('reads the balance again when usage lands and a hold is released after the read', async () => {
await withBalance(BUDGET);
// A request still running, about to record its usage and release.
const running = await target.reserveCredits(actor, HOLD);
const settle = async () => {
await target.incrementUsage(actor, 'kv:read', 1, HOLD);
await running.release();
};
// The admission's balance read finishes first, then the
// settlement lands before its reservation runs.
const read = target.getAllowedUsage.bind(target);
let settled = false;
vi.spyOn(target, 'getAllowedUsage').mockImplementation(
async (...args) => {
const balance = await read(...args);
if (!settled) {
settled = true;
await settle();
}
return balance;
},
);
expect(await target.reserveCreditsIfAffordable(actor, HOLD)).toBe(
null,
);
expect(await target.getOutstandingHolds(actor)).toBe(0);
});
it('would over-admit that interleaving without the version check', async () => {
await withBalance(BUDGET);
const running = await target.reserveCredits(actor, HOLD);
const read = target.getAllowedUsage.bind(target);
let settled = false;
vi.spyOn(target, 'getAllowedUsage').mockImplementation(
async (...args) => {
const balance = await read(...args);
if (!settled) {
settled = true;
await target.incrementUsage(actor, 'kv:read', 1, HOLD);
await running.release();
}
return balance;
},
);
vi.spyOn(server.stores.creditHold, 'version').mockResolvedValue('');
const hold = await target.reserveCreditsIfAffordable(actor, HOLD);
expect(hold).not.toBeNull();
await hold?.release();
});
it('marks the balance changed once a usage write lands', async () => {
const before = await server.stores.creditHold.version(
actor.user.uuid!,
);
await target.incrementUsage(actor, 'kv:read', 1, 1_000);
expect(
await server.stores.creditHold.version(actor.user.uuid!),
).not.toBe(before);
});
it('lets the balance decide, holding nothing, when the cache is unavailable', async () => {
await withBalance(BUDGET);
vi.spyOn(server.stores.creditHold, 'reserve').mockResolvedValue({
outcome: 'unavailable',
});
const hold = await target.reserveCreditsIfAffordable(actor, HOLD);
expect(hold).not.toBeNull();
expect(await target.getOutstandingHolds(actor)).toBe(0);
expect(
await target.reserveCreditsIfAffordable(actor, BUDGET + 1),
).toBeNull();
});
it('reads the balance again while it keeps changing, then refuses', async () => {
await withBalance(BUDGET);
const reserve = vi
.spyOn(server.stores.creditHold, 'reserve')
.mockResolvedValue({ outcome: 'stale' });
expect(await target.reserveCreditsIfAffordable(actor, HOLD)).toBe(
null,
);
expect(reserve).toHaveBeenCalledTimes(5);
// Every attempt checks the version; none falls back to skipping it.
expect(reserve.mock.calls.every(([, , , v]) => v !== '')).toBe(
true,
);
expect(await target.getOutstandingHolds(actor)).toBe(0);
});
it('admits once the balance holds still', async () => {
await withBalance(BUDGET);
const reserve = vi.spyOn(server.stores.creditHold, 'reserve');
reserve
.mockResolvedValueOnce({ outcome: 'stale' })
.mockResolvedValueOnce({ outcome: 'stale' });
const hold = await target.reserveCreditsIfAffordable(actor, HOLD);
expect(hold).not.toBeNull();
expect(reserve).toHaveBeenCalledTimes(3);
await hold?.release();
});
it('reads purchased credit consistently only while some is left', async () => {
await withBalance(BUDGET);
const get = vi.spyOn(server.stores.kv, 'get');
const consistentAddonReads = () =>
get.mock.calls.filter(
([args]) =>
args.consistentRead &&
String(args.key).endsWith(':addons'),
).length;
// With none, the cached record that says so answers.
await target.getAllowedUsage(actor);
get.mockClear();
await (
await target.reserveCreditsIfAffordable(actor, 1000)
)?.release();
expect(consistentAddonReads()).toBe(0);
await target.updateAddonCredit(actor.user.uuid!, 5_000);
get.mockClear();
await (
await target.reserveCreditsIfAffordable(actor, 1000)
)?.release();
expect(consistentAddonReads()).toBe(1);
});
it('marks the balance changed on credit, allowance and manual usage changes', async () => {
const uuid = actor.user.uuid!;
const versions = [await server.stores.creditHold.version(uuid)];
await target.updateAddonCredit(uuid, 1_000);
versions.push(await server.stores.creditHold.version(uuid));
await target.setActorCurrentMonthUsageTotal(actor, 5_000);
versions.push(await server.stores.creditHold.version(uuid));
expect(new Set(versions).size).toBe(3);
});
it('treats the system actor, a userless actor and amounts it can’t hold as the plain check does', async () => {
for (const [who, amount] of [
[SYSTEM_ACTOR, HOLD],
[actor, 0],
[actor, -5],
] as const) {
const hold = await target.reserveCreditsIfAffordable(
who,
amount,
);
expect(hold).not.toBeNull();
}
for (const amount of [Number.NaN, Number.POSITIVE_INFINITY]) {
expect(
await target.reserveCreditsIfAffordable(actor, amount),
).toBeNull();
}
await expect(
target.reserveCreditsIfAffordable({ user: {} } as Actor, HOLD),
).rejects.toThrow();
expect(await target.getOutstandingHolds(actor)).toBe(0);
});
it('passes the hold lifetime through and renews it', async () => {
await withBalance(BUDGET);
const hold = await target.reserveCreditsIfAffordable(actor, 750, {
ttlMs: 500,
});
await new Promise((r) => setTimeout(r, 620));
expect(await target.getOutstandingHolds(actor)).toBe(0);
await hold?.extend?.();
expect(await target.getOutstandingHolds(actor)).toBe(750);
await hold?.release();
expect(await target.getOutstandingHolds(actor)).toBe(0);
});
});
describe('settling a charge while a flush is out', () => {
it('splits a charge over the allowance using usage a flush has claimed', async () => {
const uuid = actor.user.uuid!;
const { monthUsageAllowance: allowance } =
await target.getActorSubscription(actor);
await target.updateAddonCredit(uuid, 1_000_000);
const Service = target.constructor as typeof MeteringService;
const threshold = Service.PRECISION_THRESHOLD;
// Keeps the first charge buffered instead of read exactly.
Service.PRECISION_THRESHOLD = 2;
await target.incrementUsage(actor, 'kv:read', 1, allowance - 5);
Service.PRECISION_THRESHOLD = threshold;
// A flush claims it and stalls before writing it onward.
const kv = server.stores.kv;
const realIncr = kv.incr.bind(kv);
let release: () => void = () => {};
const gate = new Promise<void>((r) => (release = r));
const write = vi
.spyOn(kv, 'incr')
.mockImplementation(async (...args) => {
await gate;
return realIncr(...args);
});
const flushing = server.stores.meteringBuffer.flushCycle();
await vi.waitFor(() => expect(write).toHaveBeenCalled());
write.mockImplementation(realIncr);
// 5 µ¢ of this is left in the allowance; the rest is credit.
await target.incrementUsage(actor, 'kv:read', 1, 10);
release();
await flushing;
write.mockRestore();
await waitFor(async () => {
const addons = await target.getActorAddons(actor, {
consistentRead: true,
});
expect(addons.consumedPurchaseCredits).toBe(5);
});
});
});
// ── Resolver registration ────────────────────────────────────────
describe('resolver registration', () => {
@@ -698,8 +698,6 @@ export class MeteringService extends PuterService {
actorAddons,
totalBatchCost,
);
// Before any hold this usage stands for is released.
await this.stores.creditHold.bump(userId);
this.maybeAlertOveruse({
actor,
@@ -731,10 +729,6 @@ export class MeteringService extends PuterService {
? ({ ...own, ...charged } as UsageByType)
: (charged ?? own);
} catch (e) {
// Part of the usage may have landed before the failure.
if (actor?.user?.uuid) {
await this.stores.creditHold.bump(actor.user.uuid);
}
console.error('[metering] batchIncrementUsages failed', {
actor,
usages,
@@ -1438,7 +1432,6 @@ export class MeteringService extends PuterService {
}
this.invalidateActorCredits(userId);
await this.stores.creditHold.bump(userId);
await Promise.all([
this.stores.meteringBuffer.forgetSummed(actorUsageKey),
this.stores.meteringBuffer.forgetSummed(
@@ -1552,86 +1545,7 @@ export class MeteringService extends PuterService {
opts.ttlMs,
);
if (!member) return NO_CREDIT_HOLD;
return this.#holdHandle(userId, member, opts.ttlMs);
}
/**
* Commit part of an actor's budget only if it fits beside what their other
* operations hold; null when it doesn't. The check and the hold are one
* step in the cache, and a usage write or release that lands after the
* balance read sends the reservation back to read it again. When the cache
* can't answer, the balance alone decides and nothing is held.
*/
async reserveCreditsIfAffordable(
actor: Actor,
amount: number,
opts: { ttlMs?: number } = {},
): Promise<CreditHold | null> {
const userId = actor?.user?.uuid;
if (
!userId ||
isSystemActor(actor) ||
!Number.isFinite(amount) ||
!(amount > 0)
) {
// Nothing to hold; the plain check decides, as it always has.
return (await this.hasEnoughCredits(actor, amount))
? NO_CREDIT_HOLD
: null;
}
for (let attempt = 1; ; attempt++) {
const version = await this.stores.creditHold.version(userId);
// Read after the version, so a write it misses bumps the version.
const { remaining } = await this.getAllowedUsage(actor, {
exact: true,
});
const budget = remaining || 0;
const result =
version === null
? ({ outcome: 'unavailable' } as const)
: await this.stores.creditHold.reserve(
userId,
amount,
budget,
version,
opts.ttlMs,
);
switch (result.outcome) {
case 'accepted':
return this.#holdHandle(userId, result.member, opts.ttlMs);
case 'insufficient':
return null;
case 'unavailable':
return budget >= amount ? NO_CREDIT_HOLD : null;
case 'stale':
break;
}
// A balance that never holds still long enough to read is refused
// rather than checked against a read that may be out of date.
if (attempt >= MeteringService.RESERVE_ATTEMPTS) {
console.warn(
`[metering] credit reservation refused for ${userId}: balance kept changing`,
);
return null;
}
await new Promise((resolve) =>
setTimeout(
resolve,
Math.random() *
MeteringService.RESERVE_BACKOFF_MS *
attempt,
),
);
}
}
/** Reservations tried while the balance keeps changing under them. */
private static readonly RESERVE_ATTEMPTS = 5;
/** Upper bound of the randomized wait before each retry, times the attempt. */
private static readonly RESERVE_BACKOFF_MS = 20;
#holdHandle(userId: string, member: string, ttlMs?: number): CreditHold {
let released = false;
return {
release: async () => {
@@ -1641,7 +1555,11 @@ export class MeteringService extends PuterService {
},
extend: async () => {
if (released) return;
await this.stores.creditHold.refresh(userId, member, ttlMs);
await this.stores.creditHold.refresh(
userId,
member,
opts.ttlMs,
);
},
};
}
@@ -1653,22 +1571,14 @@ export class MeteringService extends PuterService {
return this.stores.creditHold.outstanding(userId);
}
/**
* `exact` reads purchased credit consistently, for a spend decision that
* has to see the consumption a write just recorded. A cached record with no
* credit left still answers: it can only understate the balance.
*/
async getAllowedUsage(
actor: Actor,
{ exact = false }: { exact?: boolean } = {},
): Promise<{
async getAllowedUsage(actor: Actor): Promise<{
remaining: number;
monthUsageAllowance: number;
addons: UsageAddons;
}> {
const [userSubscription, addons, totals] = await Promise.all([
this.getActorSubscription(actor),
this.#addonsForSettle(actor, { consistentRead: exact }),
this.#addonsForSettle(actor),
this.currentMonthTotals(actor),
]);
@@ -2335,16 +2245,13 @@ export class MeteringService extends PuterService {
}
/** `getActorAddons`, through `#spentAddons`. Shared by gates and settles. */
async #addonsForSettle(
actor: Actor,
{ consistentRead }: { consistentRead?: boolean } = {},
): Promise<UsageAddons> {
async #addonsForSettle(actor: Actor): Promise<UsageAddons> {
const uuid = actor.user?.uuid;
const cached = uuid ? this.#spentAddons.get(uuid) : undefined;
if (cached && cached.expiresAt > Date.now()) return cached.addons;
const drops = this.#spentAddonsDrops;
const addons = await this.getActorAddons(actor, { consistentRead });
const addons = await this.getActorAddons(actor);
if (!uuid) return addons;
this.#spentAddons.delete(uuid);
const creditLeft =
@@ -2475,7 +2382,6 @@ export class MeteringService extends PuterService {
// Credit that lands while the account is being turned away has to take
// effect on the next request, not at the end of the cache window.
this.invalidateActorCredits(userId);
await this.stores.creditHold.bump(userId);
}
/**
@@ -2523,7 +2429,6 @@ export class MeteringService extends PuterService {
});
// Every node drops its cached policy, and the budget answer with it.
this.invalidateActorSubscription(userUuid);
await this.stores.creditHold.bump(userUuid);
}
/**
@@ -1,39 +0,0 @@
import { expect, it } from 'vitest';
import { EventClient } from '../../clients/event/EventClient.js';
import type { EventMap } from '../../clients/event/types.js';
import { NotificationService } from './NotificationService.js';
it.each([undefined, 0, 1_700_000_000_000])(
'emits replacement metadata for replacedAt %s',
async (replacedAt) => {
const event = new EventClient({} as never);
const written: EventMap['notif.written'][] = [];
event.on('notif.written', (_key, value) => {
written.push(value);
});
const service = new NotificationService(
{} as never,
{ event } as never,
{} as never,
);
await service.emitWritten(
{
userId: 1,
uid: 'notification-1',
type: 'app.message',
audience: 'app-user',
appUid: 'app-1',
value: { title: 'A message' },
updated: replacedAt !== undefined,
replacedAt,
},
{ sendId: 'send-1' },
);
expect(written).toHaveLength(1);
if (replacedAt === undefined)
expect(written[0]).not.toHaveProperty('replaced');
else expect(written[0].replaced).toBe(true);
},
);
@@ -354,7 +354,6 @@ describe('NotificationService.notifyUpdate', () => {
appUid,
value: { title: 'two messages', type: 'app.message' },
updated: true,
replaced: true,
});
expect(desktop.seen).toEqual([]);
expect(
@@ -23,14 +23,10 @@ import { handlerDepthOf, type Actor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import type { NotifEventContext } from '../events/registry.js';
import { PuterService } from '../types.js';
import { checkRateLimit } from '../../core/http/middleware/rateLimit.js';
import {
NOTIF_HANDLED_EVENT,
NOTIF_HANDLED_PER_MINUTE,
NotificationSocketAdapter,
notificationAckTimeoutMs,
notificationsFoldInEnabled,
type NotifSocket,
type ReceiptScope,
} from './notificationSocket.js';
import {
@@ -314,9 +310,6 @@ export class NotificationService extends PuterService {
...scope,
value: structuredClone(payload),
...(updated ? { updated: true as const } : {}),
...(written.replacedAt !== undefined
? { replaced: true as const }
: {}),
sendId: opts.sendId,
...(opts.replayable ? { replayable: true as const } : {}),
},
@@ -408,11 +401,6 @@ export class NotificationService extends PuterService {
return this.#socket().receipt(scope);
}
/** Installs the `notif.handled` verb on one connection. */
attachSocket(socket: NotifSocket, actor: Actor): void {
this.#socket().attachSocket(socket, actor);
}
/**
* Drop notifications past the retention window, in batches, and report how
* many went. Deleting is all there is to it: nothing is pushed, because a
@@ -448,12 +436,6 @@ export class NotificationService extends PuterService {
ackTimeoutMs: () => notificationAckTimeoutMs(this.config),
heldInPeerRegion: (userId, appUid) =>
this.#heldInPeerRegion(userId, appUid),
allowHandled: (userId) =>
checkRateLimit(
`socket:${NOTIF_HANDLED_EVENT}:${userId}`,
NOTIF_HANDLED_PER_MINUTE,
60_000,
),
});
return this.#adapter;
}
@@ -172,23 +172,13 @@ describe('parseAppMessage', () => {
rejects({ title: 't', sensitive: 'yes' }, 'sensitive');
});
it('accepts normal and high priority, defaulting to normal', () => {
expect(parseAppMessage({ title: 't' }).priority).toBe('normal');
expect(
parseAppMessage({ title: 't', priority: 'normal' }).priority,
).toBe('normal');
expect(
parseAppMessage({ title: 't', priority: 'high' }).priority,
).toBe('high');
it('refuses high priority to apps', () => {
expect(() => parseAppMessage({ title: 't', priority: 'high' })).toThrow(
"'high' is not available to apps",
);
rejects({ title: 't', priority: 'urgent' }, 'priority');
});
it.each(['urgent', 'low', '', null, 1, true])(
'rejects priority %s',
(priority) => {
rejects({ title: 't', priority }, 'priority');
},
);
it('bounds ttl to whole seconds within a day', () => {
expect(parseAppMessage({ title: 't', ttl: 1 }).ttl).toBe(1);
rejects({ title: 't', ttl: 0 }, 'ttl');
@@ -39,7 +39,7 @@ export const APP_MESSAGE_RATE_LIMITS = {
perDay: 1000,
} as const;
export type AppMessagePriority = 'normal' | 'high';
export type AppMessagePriority = 'normal';
/** An `app.message` value as stored, after normalization. */
export type AppMessage = {
@@ -322,11 +322,13 @@ export const parseAppMessage = (input: Record<string, unknown>): AppMessage => {
message.sensitive = input.sensitive;
}
if (input.priority !== undefined) {
if (input.priority !== 'normal' && input.priority !== 'high') {
throw invalid('priority', "must be 'normal' or 'high'");
}
message.priority = input.priority;
if (input.priority !== undefined && input.priority !== 'normal') {
throw invalid(
'priority',
input.priority === 'high'
? "'high' is not available to apps; use 'normal'"
: "must be 'normal'",
);
}
return message;
@@ -32,10 +32,7 @@ import {
import type { EventMap } from '../../clients/event/types.js';
import type { PuterServer } from '../../server.js';
import { accountSocketRoom, SocketService } from '../socket/SocketService.js';
import {
NOTIF_HANDLED_EVENT,
NOTIF_RECEIPT_EVENT,
} from './notificationSocket.js';
import { NOTIF_RECEIPT_EVENT } from './notificationSocket.js';
import {
allocateEphemeralPort,
createTestUser,
@@ -312,13 +309,12 @@ describe('live notification delivery receipts', () => {
complete: true,
peerRegions: false,
responders: [{ visibilityState: 'hidden' }],
handled: false,
};
expect(settled).toEqual(expected);
expect(state.settled).toEqual([expected]);
});
it('passes a tab`s focus through, typed strictly', async () => {
it('passes a tab`s focus and own display through, typed strictly', async () => {
const answers = [
{
received: true,
@@ -330,6 +326,7 @@ describe('live notification delivery receipts', () => {
received: true,
visibilityState: 'nope',
focused: 1,
showsOwn: 'yes',
},
];
for (const answer of answers) {
@@ -344,39 +341,11 @@ describe('live notification delivery receipts', () => {
expect(settled.outcome).toBe('received');
expect(settled.responders).toEqual(
expect.arrayContaining([
{ visibilityState: 'hidden', focused: false },
{ visibilityState: 'hidden', focused: false, showsOwn: true },
{ visibilityState: 'unknown' },
]),
);
expect(settled.responders).toHaveLength(2);
expect(settled.handled).toBe(false);
});
it('counts as handled only a visible tab that is focused, or an older one that is visible', async () => {
const cases: Array<[Record<string, unknown>, boolean]> = [
[{ visibilityState: 'visible', focused: true }, true],
[{ visibilityState: 'visible', focused: false }, false],
[{ visibilityState: 'hidden', focused: true }, false],
[{ visibilityState: 'visible' }, true],
];
for (const [answer, handled] of cases) {
const tab = await connect({ acks: [NOTIF_RECEIPT_EVENT] });
tab.on(NOTIF_RECEIPT_EVENT, (_data: unknown, ack: Ack) =>
ack({ received: true, ...answer }),
);
const { settled } = await ask();
expect(settled.handled, JSON.stringify(answer)).toBe(handled);
tab.disconnect();
await vi.waitFor(() =>
expect(
state.server.services.socket.has({
room: accountSocketRoom(state.userId),
}),
).toBe(false),
);
}
});
it('settles as unreceived when every tab answers that it never got it', async () => {
@@ -580,7 +549,6 @@ describe('live receipts from an app’s own sockets', () => {
complete: true,
peerRegions: false,
responders: [{ visibilityState: 'visible' }],
handled: true,
};
expect(settled).toEqual(expected);
expect(state.settled).toEqual([expected]);
@@ -751,59 +719,3 @@ describe('app receipts where the adapter cannot count servers', () => {
});
});
});
describe('a page reporting a notification it handled', () => {
const { state, connect, appSocket } = withAppReceipts({ cluster: true });
const reports: Array<EventMap['notif.handled']> = [];
const onHandled = (_key: string, data: unknown) => {
reports.push(data as EventMap['notif.handled']);
};
beforeAll(() => {
state.server.clients.event.on('notif.handled', onHandled);
});
afterEach(() => {
reports.length = 0;
});
afterAll(() => {
state.server?.clients.event.off('notif.handled', onHandled);
});
it('relays a desktop tab`s report for its user, without an app', async () => {
const tab = await connect({});
const uid = uuidv4();
tab.emit(NOTIF_HANDLED_EVENT, { uid });
await vi.waitFor(() =>
expect(reports).toEqual([{ userId: state.userId, uid }]),
);
});
it('relays an app page`s report with the app it acts as', async () => {
const page = await appSocket();
const uid = uuidv4();
page.emit(NOTIF_HANDLED_EVENT, { uid });
await vi.waitFor(() =>
expect(reports).toEqual([
{ userId: state.userId, uid, appUid: OWN_APP },
]),
);
});
it('drops a report without a notification uid', async () => {
const tab = await connect({});
tab.emit(NOTIF_HANDLED_EVENT, { uid: 'not-a-uid' });
tab.emit(NOTIF_HANDLED_EVENT, null);
tab.emit(NOTIF_HANDLED_EVENT, { uid: uuidv4() });
await vi.waitFor(() => expect(reports).toHaveLength(1));
await new Promise((resolve) => setTimeout(resolve, 100));
expect(reports).toHaveLength(1);
});
});
@@ -17,9 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { validate as validateUuid } from 'uuid';
import type { EventClient } from '../../clients/event/EventClient.js';
import type { Actor } from '../../core/actor.js';
import type {
EventMap,
NotifDeliveryOutcome,
@@ -55,10 +53,7 @@ import type { IConfig } from '../../types.js';
*/
export type NotifWire =
| 'notif.message'
| 'notif.unreads'
| 'notif.ack'
| 'notif.removed';
'notif.message' | 'notif.unreads' | 'notif.ack' | 'notif.removed';
export const NOTIF_DELIVERY_EVENT = 'outer.notif.delivery';
@@ -72,15 +67,6 @@ export const DEFAULT_NOTIFICATION_ACK_TIMEOUT_MS = 5_000;
*/
export const NOTIF_RECEIPT_EVENT = 'notif.receipt';
/**
* Sent by a page with `{ uid }` when it got that notification live while
* visible and focused, so a region that didn't ask the page can still hear it.
*/
export const NOTIF_HANDLED_EVENT = 'notif.handled';
/** `notif.handled` reports one user's sockets may send per minute. */
export const NOTIF_HANDLED_PER_MINUTE = 60;
/** One unread as the replay payload carries it. */
export interface UnreadNotification {
uid: string;
@@ -136,16 +122,12 @@ const responderOf = (response: unknown): NotifDeliveryResponder | null => {
...(typeof answer.focused === 'boolean'
? { focused: answer.focused }
: {}),
...(typeof answer.showsOwn === 'boolean'
? { showsOwn: answer.showsOwn }
: {}),
};
};
/**
* Visible and focused. An older client doesn't say whether it's focused, so it
* counts while visible.
*/
export const handledBy = (responder: NotifDeliveryResponder): boolean =>
responder.visibilityState === 'visible' && responder.focused !== false;
interface AdapterDeps {
event: EventClient;
socket: SocketService;
@@ -156,13 +138,6 @@ interface AdapterDeps {
userId: number,
appUid: string | null,
) => Promise<boolean>;
/** Spends one of the user's `notif.handled` reports; false past the limit. */
allowHandled: (userId: number) => Promise<boolean>;
}
/** The part of a socket `attachSocket` installs its verb on. */
export interface NotifSocket {
on(event: string, listener: (...args: never[]) => void): unknown;
}
type SettledDelivery = EventMap['notif.delivery.settled'];
@@ -278,7 +253,6 @@ export class NotificationSocketAdapter {
complete: asked.complete,
peerRegions,
responders,
handled: responders.some(handledBy),
};
this.#deps.event.emit(
'notif.delivery.settled',
@@ -292,28 +266,6 @@ export class NotificationSocketAdapter {
}
}
/**
* Relays a page's `notif.handled` as the node-local `notif.handled` event,
* scoped to the socket's user and app.
*/
attachSocket(socket: NotifSocket, actor: Actor): void {
const userId = actor.user?.id;
if (userId === undefined) return;
const appUid = actor.effectiveApp?.uid;
socket.on(NOTIF_HANDLED_EVENT, ((payload: unknown) => {
const uid = (payload as { uid?: unknown } | null)?.uid;
if (typeof uid !== 'string' || !validateUuid(uid)) return;
void this.#deps.allowHandled(userId).then((ok) => {
if (!ok) return;
this.#deps.event.emit(
'notif.handled',
{ userId, uid, ...(appUid ? { appUid } : {}) },
{},
);
});
}) as (...args: never[]) => void);
}
/** Whether this region holds a socket for the recipient right now. */
hasSocket(userId: number): boolean {
return this.#deps.socket.has({ room: userId });
@@ -17,8 +17,6 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { HttpError } from '../../core/http/HttpError.js';
/** Who the recipient is in relation to the notification. */
export type NotificationAudience = 'account' | 'developer' | 'app-user';
@@ -155,22 +153,6 @@ export type NotificationTypeName = (typeof NOTIFICATION_TYPES)[number]['type'];
/** Serialized `value` may not exceed this many bytes. */
export const NOTIFICATION_VALUE_MAX_BYTES = 16 * 1024;
export const assertNotificationValueFits = (
value: Record<string, unknown>,
): void => {
if (
Buffer.byteLength(JSON.stringify(value), 'utf8') >
NOTIFICATION_VALUE_MAX_BYTES
) {
throw new HttpError(
400,
`\`value\` may not exceed ${NOTIFICATION_VALUE_MAX_BYTES} bytes`,
{
legacyCode: 'notification_value_too_large',
},
);
}
};
const BY_NAME = new Map<string, NotificationType>(
NOTIFICATION_TYPES.map((entry) => [entry.type, entry]),
);
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+3 -53
View File
@@ -11,10 +11,7 @@ Direct clients are documented in the [Realtime WebSocket API reference](../../..
The relay forwards native OpenAI JSON events unchanged after validating the
event envelope, except that `rate_limits.updated` (Puter's provider account) is
dropped and provider organization IDs are removed from diagnostic messages in
`error`, `response.done.status_details.error`, and
`conversation.item.input_audio_transcription.failed.error`, and from provider
close reasons. Conversation content is preserved. An event must be an object with a nonempty string `type` of at
dropped and provider organization IDs are removed from error messages. An event must be an object with a nonempty string `type` of at
most 128 characters. Binary messages and malformed envelopes close the session.
Usage fields consumed by metering are validated before use. Event field order
does not affect billing.
@@ -49,8 +46,7 @@ traffic is not rate limited: OpenAI sends audio faster than real time, and
backpressure bounds what a slow client can queue. Usage is priced as each event
arrives, before any forwarding decision, and recorded serially off the
forwarding path through the AI cost-factor hook. Pricing follows the model the
session reports. A missing model uses the last known model, while an explicitly
unpriced model closes the connection; a client `session.update`
session reports, falling back to the connection's; a client `session.update`
that changes the model is refused. So are `conversation.item.retrieve` and MCP
tools in `session.update` or `response.create`, since a retrieved item or MCP
output can exceed the upstream message cap.
@@ -69,55 +65,9 @@ Each response holds a fixed amount of the account's budget
until its usage is recorded, so concurrent sessions can't overdraw it; a session
opens only if one hold fits. Input transcription is priced at the costlier of
the transcription models in force at commit and at completion. Session attempts
are limited per credential after signature verification. Local, bounded IP and
node admission limits protect signature verification and authentication lookups.
Upgrades use Express's configured proxy trust policy and the `ip.validate` gate.
Authentication uses `touch: false` on both initial connection and rechecks, so
opening or maintaining a realtime socket does not update session activity metadata.
Input transcription holds credit per item, at a fixed budget for the selected
transcription model and that model's cost factor. Audio and the configuration
that enables transcription are forwarded only while a reserved spare item
exists. Each provider commit takes one, and the pool refills to two spares in
the background, up to 64 pending plus spare items. A commit with no spare left
reserves its own item. A model change, requested or reported, releases the spares
and reserves new ones at the new model's price. Committed items keep their hold
and model. Disabling transcription keeps the spares until the connection ends.
If the account can't cover another item, client reads pause while the session
still holds items that may free credit, and the session is refused with
`insufficient_funds` once it holds none.
Audio the provider may commit without the relay seeing the acknowledgement is
tracked apart from the pool, as a duration at the session's input format. Without
turn detection that is all audio since the last acknowledged commit or clear.
With server turn detection it is only speech from `speech_started`, since the
provider drops silence, plus the buffer after an explicit commit. Each started
160 seconds of it claims one held item, and an acknowledged commit, a clear, or
an `input_audio_buffer_commit_empty` error returns those items to the spares.
The session is never closed for its length. If the upstream closes first, each
claimed item is billed. Each committed item retains its model
until completion, failure, or bounded settlement. Missing usage is billed the
item's fixed budget. Overflow closes the connection without evicting existing
pending identities.
The post-record continuation check asks whether settled credit remains. It
excludes outstanding holds and accepts fractional positive credit. Admission of
new billable work continues to use the atomic hold path, which includes every
outstanding reservation.
When the host listens for `ai.prompt.validate`, each response that starts runs
it on the text the client sent since the previous check: session and response
instructions, message text and transcripts, and function call output, as chat
messages. The response's cost follows as `ai.prompt.cost-calculated` under the
same completion id. A refusal closes the session with `permission_denied`; the
response may already be streaming by then. Pending text past 64 KiB is checked
on its own. Audio is not checked.
are limited per credential before authentication runs.
Credential checks run in batches of at most 16 with no overlapping sweeps.
Session and access-token revocation events close affected sockets and invalidate
their auth cache entries. Subscription eligibility is checked independently of
authentication and after subscription invalidations. A failed verification check after a 60-second
retry grace from the first failed lookup closes with a retryable internal error. Periodic checks remain the missed-event fallback.
When the drain starts, open sessions get a `puter.reconnect` event and new
sessions receive `server_restarting` and close with 1012. Open sessions can
finish an in-flight response for up to five seconds at shutdown, then close
+3 -8
View File
@@ -699,8 +699,7 @@ export class SocketService extends PuterService {
// `{ auth: { ... } }`, not the query string. puter-js uses
// `io(url, { auth: { auth_token } })`.
const handshakeAuth = socket.handshake.auth as
| Record<string, unknown>
| undefined;
Record<string, unknown> | undefined;
const tokenRaw =
typeof handshakeAuth?.auth_token === 'string'
? handshakeAuth.auth_token
@@ -1086,7 +1085,6 @@ export class SocketService extends PuterService {
// them. Off unless events are enabled, in which case the verbs
// answer with `events_disabled` rather than going unanswered.
this.services.events.attachSocket(socket, actor);
this.services.notification.attachSocket(socket, actor);
// Everything below is the desktop session's own traffic: two verbs
// that reach the user's other tabs, and a connect announcement
@@ -1223,8 +1221,7 @@ export class SocketService extends PuterService {
// their next poll of /cache/last-change-timestamp.
const originalSocketId = (
data.response as
| { original_client_socket_id?: string }
| undefined
{ original_client_socket_id?: string } | undefined
)?.original_client_socket_id;
await this.send({ room: userId }, 'cache.updated', {
timestamp,
@@ -1238,9 +1235,7 @@ export class SocketService extends PuterService {
#handleUploadProgress(data: UploadProgressPayload): void {
const meta = data.meta ?? {};
const userId = (meta.user_id ?? meta.userId) as
| number
| string
| undefined;
number | string | undefined;
if (!userId) {
console.warn('[socket] upload-progress missing user_id', { meta });
return;
@@ -466,10 +466,7 @@ export class DurableSubscriptionStore extends PuterStore {
async suspend(
rows: readonly DurableSubscription[],
reason: SuspendedReason,
opts: {
override?: readonly SuspendedReason[];
guard?: SqlGuard;
} = {},
opts: { override?: readonly SuspendedReason[] } = {},
): Promise<{ suspended: DurableSubscription[]; bumps: GenerationBump[] }> {
if (rows.length === 0) return { suspended: [], bumps: [] };
@@ -491,17 +488,8 @@ export class DurableSubscriptionStore extends PuterStore {
const written = await this.clients.db.write(
`UPDATE \`${TABLE}\` SET \`suspended_at\` = ?, ` +
'`suspended_reason` = ? ' +
`WHERE \`sub_id\` = ? AND ${condition}` +
(opts.guard
? ` AND EXISTS (SELECT 1 FROM \`${opts.guard.table}\` WHERE ${opts.guard.where})`
: ''),
[
at,
reason,
row.subId,
...override,
...(opts.guard?.values ?? []),
],
`WHERE \`sub_id\` = ? AND ${condition}`,
[at, reason, row.subId, ...override],
);
if (written.anyRowsAffected)
suspended.push({
@@ -971,20 +959,6 @@ export class DurableSubscriptionStore extends PuterStore {
return this.#reap(await this.#listForApp(appUid, batchSize));
}
scheduleSubscriptionContextEntry(
scheduleUid: string,
context: string | null,
guard: SqlGuard,
): BatchEntry {
assertDurableContext(context);
return {
statement:
`UPDATE \`${TABLE}\` SET \`context\` = ? WHERE \`token\` = ? ` +
`AND EXISTS (SELECT 1 FROM \`${guard.table}\` WHERE ${guard.where})`,
values: [context, scheduleToken(scheduleUid), ...guard.values],
};
}
/**
* Delete the row a schedule owns, whether or not the schedule ever recorded
* its id: a create that crashed between the two inserts leaves one it
@@ -21,7 +21,6 @@ import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { IConfig, IRedisConfig } from '../../types.js';
import type { CreditHoldStore } from './CreditHoldStore.js';
let server: PuterServer;
@@ -143,204 +142,3 @@ describe('CreditHoldStore', () => {
spy.mockRestore();
});
});
// Opt in to a real Redis Cluster with PUTER_TEST_REDIS_CLUSTER=host:port.
const clusterNode = process.env.PUTER_TEST_REDIS_CLUSTER;
const backends: { name: string; redis: IRedisConfig }[] = [
{ name: 'the redis mock', redis: { useMock: true } },
...(clusterNode
? [
{
name: `redis cluster ${clusterNode}`,
redis: {
useMock: false,
tls: false,
startupNodes: [
{
host: clusterNode.split(':')[0],
port: Number(clusterNode.split(':')[1]),
},
],
},
},
]
: []),
];
describe.each(backends)('CreditHoldStore.reserve on $name', ({ redis }) => {
// Two backends sharing one cache, as separate instances do in production.
let servers: PuterServer[];
let a: CreditHoldStore;
let b: CreditHoldStore;
const BUDGET = 20_000_000;
const HOLD = 16_200_000;
beforeAll(async () => {
servers = await Promise.all(
[0, 1].map(() => setupTestServer({ redis } as IConfig)),
);
[a, b] = servers.map((s) => s.stores.creditHold as CreditHoldStore);
});
afterAll(async () => {
await Promise.all((servers ?? []).map((s) => s.shutdown()));
});
const reserve = async (
on: CreditHoldStore,
id: string,
amount: number,
budget = BUDGET,
ttlMs?: number,
) => on.reserve(id, amount, budget, (await on.version(id))!, ttlMs);
it('admits exactly one of two concurrent holds that only fit alone', async () => {
const id = user();
const results = await Promise.all([
reserve(a, id, HOLD),
reserve(b, id, HOLD),
]);
expect(results.map((r) => r.outcome).sort()).toEqual([
'accepted',
'insufficient',
]);
expect(await a.outstanding(id)).toBe(HOLD);
expect(await b.outstanding(id)).toBe(HOLD);
});
it('admits exact headroom and refuses one unit more', async () => {
const id = user();
expect((await reserve(a, id, 15_000_000)).outcome).toBe('accepted');
expect((await reserve(b, id, 5_000_001)).outcome).toBe('insufficient');
expect((await reserve(b, id, 5_000_000)).outcome).toBe('accepted');
expect(await a.outstanding(id)).toBe(BUDGET);
});
it('rounds a fractional amount up once', async () => {
const id = user();
expect((await reserve(a, id, 9.2, 10)).outcome).toBe('accepted');
expect(await a.outstanding(id)).toBe(10);
expect((await reserve(a, id, 0.5, 10)).outcome).toBe('insufficient');
});
it('admits several holds that fit together, and keeps actors apart', async () => {
const [id, other] = [user(), user()];
const results = await Promise.all(
[a, b, a, b].map((s) => reserve(s, id, 5_000_000)),
);
expect(results.every((r) => r.outcome === 'accepted')).toBe(true);
expect((await reserve(a, other, HOLD)).outcome).toBe('accepted');
expect(await a.outstanding(id)).toBe(BUDGET);
expect(await a.outstanding(other)).toBe(HOLD);
});
it('leaves nothing behind when it refuses', async () => {
const id = user();
await reserve(a, id, HOLD);
await reserve(b, id, HOLD);
await reserve(b, id, HOLD, 0);
expect(await a.outstanding(id)).toBe(HOLD);
});
it('frees headroom on release and on expiry', async () => {
const id = user();
const first = await reserve(a, id, HOLD);
expect((await reserve(b, id, HOLD)).outcome).toBe('insufficient');
await a.release(id, first.outcome === 'accepted' ? first.member : null);
expect((await reserve(b, id, HOLD, BUDGET, 50)).outcome).toBe(
'accepted',
);
await vi.waitFor(
async () =>
expect((await reserve(a, id, HOLD)).outcome).toBe('accepted'),
{ timeout: 2000, interval: 25 },
);
});
it('keeps a long hold when a short one is reserved beside it', async () => {
const id = user();
await reserve(a, id, 1000);
await reserve(b, id, 500, BUDGET, 50);
const pttl = await (
servers[0].clients.redis as unknown as {
pttl: (key: string) => Promise<number>;
}
).pttl(`meter:holds:{${id}}`);
expect(pttl).toBeGreaterThan(500_000);
});
it('refuses a budget read before a usage write landed, but not one before a release', async () => {
const id = user();
// A release alone leaves the balance as it was.
const before = (await a.version(id))!;
const held = await reserve(a, id, 1000);
await b.release(id, held.outcome === 'accepted' ? held.member : null);
const after = await a.reserve(id, 1000, BUDGET, before);
expect(after.outcome).toBe('accepted');
await a.release(id, after.outcome === 'accepted' ? after.member : null);
const read = (await a.version(id))!;
await b.bump(id);
expect((await a.reserve(id, 1000, BUDGET, read)).outcome).toBe('stale');
expect(await a.outstanding(id)).toBe(0);
// Skipping the check still can't overlap another hold.
await reserve(a, id, HOLD);
expect((await a.reserve(id, HOLD, BUDGET, '')).outcome).toBe(
'insufficient',
);
});
it('keeps renewal and release consistent when they race', async () => {
const id = user();
const held = await reserve(a, id, HOLD, BUDGET, 200);
const member = held.outcome === 'accepted' ? held.member : null;
await Promise.all([a.refresh(id, member), b.release(id, member)]);
// Whichever ran last decides; a renewal that lost the race only
// brings the hold back until its owner releases it again.
await a.release(id, member);
expect(await a.outstanding(id)).toBe(0);
expect((await reserve(b, id, HOLD)).outcome).toBe('accepted');
});
it('refuses amounts it can’t hold without touching the hold set', async () => {
const id = user();
await reserve(a, id, 1000);
for (const amount of [0, -5_000, Number.NaN, Infinity]) {
expect((await reserve(a, id, amount)).outcome).toBe('insufficient');
}
expect(await a.outstanding(id)).toBe(1000);
});
it('starts a missing version from a random value', async () => {
const id = user();
expect(await a.version(id)).toBe('0');
await b.bump(id);
const first = (await a.version(id))!;
expect(first).not.toBe('1');
await b.bump(id);
expect(Number(await a.version(id))).toBe(Number(first) + 1);
});
it('reports an unreachable cache as unavailable, not as insufficient', async () => {
const id = user();
const client = servers[0].clients.redis as unknown as {
creditHoldReserve: () => Promise<unknown>;
get: () => Promise<unknown>;
};
const reserveSpy = vi
.spyOn(client, 'creditHoldReserve')
.mockRejectedValueOnce(new Error('cache down'));
expect((await a.reserve(id, 1000, BUDGET, '0')).outcome).toBe(
'unavailable',
);
const getSpy = vi
.spyOn(client, 'get')
.mockRejectedValueOnce(new Error('cache down'));
expect(await a.version(id)).toBeNull();
reserveSpy.mockRestore();
getSpy.mockRestore();
expect(await a.outstanding(id)).toBe(0);
});
});
+3 -167
View File
@@ -48,25 +48,6 @@ const KEY_TTL_SLACK_MS = 60 * 1000;
const holdsKey = (userId: string): string => `meter:holds:{${userId}}`;
/**
* Counts changes to an actor's balance and holds, so a reservation can tell
* whether the balance it read is still current. Shares the hold key's slot.
*/
const versionKey = (userId: string): string =>
`meter:holds:{${userId}}:version`;
/** Outlives any read-then-reserve window by orders of magnitude. */
const VERSION_TTL_MS = 60 * 60 * 1000;
/** What a conditional reservation came to. */
export type CreditReservation =
| { outcome: 'accepted'; member: string }
| { outcome: 'insufficient' }
// The balance changed since `version` was read; read it again.
| { outcome: 'stale' }
// The cache couldn't answer; nothing was held.
| { outcome: 'unavailable' };
// -- Scripts ----------------------------------------------------------
/**
@@ -98,8 +79,8 @@ return 1
* name (`<id>:<amount>`) so one sorted set carries both the deadline and the
* number, and pruning and summing happen in the same pass.
*/
const pruneAndSum = (now: string) => `
local dropped = redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', ${now})
const SUM_SCRIPT = `
local dropped = redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', ARGV[1])
local total = 0
local members = redis.call('ZRANGE', KEYS[1], 0, -1)
for i = 1, #members do
@@ -108,65 +89,13 @@ for i = 1, #members do
total = total + (tonumber(string.sub(members[i], sep + 1)) or 0)
end
end
`;
const SUM_SCRIPT = `${pruneAndSum('ARGV[1]')}
return { tostring(total), dropped }
`;
/**
* KEYS: the actor's hold set, its version. ARGV: member, deadline, key ttl,
* now, amount, budget, expected version ('' skips the check).
*
* Takes the hold only if it fits in `budget` beside the live holds, in one step
* so two reservations can't both fit in the same headroom. The version check
* refuses a budget read before a usage write or release that has since landed.
*/
const RESERVE_SCRIPT = `
if ARGV[7] ~= '' and (redis.call('GET', KEYS[2]) or '0') ~= ARGV[7] then
return { 'stale', 0 }
end
${pruneAndSum('ARGV[4]')}
if total + tonumber(ARGV[5]) > tonumber(ARGV[6]) then
return { 'insufficient', dropped }
end
local ttl = redis.call('PTTL', KEYS[1])
redis.call('ZADD', KEYS[1], ARGV[2], ARGV[1])
if tonumber(ARGV[3]) > ttl then
redis.call('PEXPIRE', KEYS[1], ARGV[3])
end
return { 'accepted', dropped }
`;
/**
* Bumps the version in KEYS[n], starting a missing one from a random value so a
* version read before the key expired can't match its successor.
*/
const bumpVersion = (key: string, seed: string, ttl: string) => `
if redis.call('EXISTS', ${key}) == 0 then
redis.call('SET', ${key}, ${seed})
end
redis.call('INCR', ${key})
redis.call('PEXPIRE', ${key}, ${ttl})
`;
/** KEYS: the actor's hold version. ARGV: version ttl, seed. */
const BUMP_SCRIPT = `
${bumpVersion('KEYS[1]', 'ARGV[2]', 'ARGV[1]')}
return 1
`;
/** A random starting version; see `bumpVersion`. */
const versionSeed = (): string =>
String(Math.floor((Math.random() * Number.MAX_SAFE_INTEGER) / 2));
type ScriptRunner = {
creditHoldTake(...args: string[]): Promise<number>;
creditHoldSum(...args: string[]): Promise<[string, number]>;
creditHoldReserve(...args: string[]): Promise<[string, number]>;
zrem(key: string, member: string): Promise<number>;
creditHoldBump(...args: string[]): Promise<number>;
get(key: string): Promise<string | null>;
};
// -- CreditHoldStore --------------------------------------------------
@@ -206,14 +135,6 @@ export class CreditHoldStore extends PuterStore {
numberOfKeys: 1,
lua: SUM_SCRIPT,
});
client.defineCommand('creditHoldReserve', {
numberOfKeys: 2,
lua: RESERVE_SCRIPT,
});
client.defineCommand('creditHoldBump', {
numberOfKeys: 1,
lua: BUMP_SCRIPT,
});
}
/**
@@ -274,92 +195,7 @@ export class CreditHoldStore extends PuterStore {
}
}
/**
* The actor's balance version, to pass to `reserve` once the balance has
* been read. Null when the cache can't answer.
*/
async version(userId: string): Promise<string | null> {
try {
return (await this.#redis.get(versionKey(userId))) ?? '0';
} catch (e) {
console.warn(
`[metering] credit hold version unreadable for ${userId}: ${(e as Error).message}`,
);
return null;
}
}
/**
* Commit `amount` only if it fits in `budget` beside the holds already
* live, as one step. `budget` is the gross balance read after `version`;
* pass `version: ''` to skip the staleness check.
*/
async reserve(
userId: string,
amount: number,
budget: number,
version: string,
ttlMs: number = DEFAULT_HOLD_TTL_MS,
): Promise<CreditReservation> {
// Nothing to hold, and a negative hold would lower everyone else's.
if (!userId || !Number.isFinite(amount) || amount <= 0) {
return { outcome: 'insufficient' };
}
const units = Math.ceil(amount);
const member = `${randomUUID()}:${units}`;
const now = Date.now();
try {
const [outcome, dropped] = await this.#redis.creditHoldReserve(
holdsKey(userId),
versionKey(userId),
member,
String(now + ttlMs),
String(ttlMs + KEY_TTL_SLACK_MS),
String(now),
String(units),
String(Math.floor(Math.min(budget, Number.MAX_SAFE_INTEGER))),
version,
);
if (dropped > 0) expiredHoldsCounter.add(dropped);
if (outcome === 'accepted') return { outcome, member };
if (outcome === 'insufficient' || outcome === 'stale') {
return { outcome };
}
throw new Error(`unexpected reservation outcome ${outcome}`);
} catch (e) {
console.warn(
`[metering] credit hold not reserved for ${userId}: ${(e as Error).message}`,
);
return { outcome: 'unavailable' };
}
}
/**
* Marks the actor's balance as changed, so a reservation that read it
* before the change reads it again. Call once a usage write has landed.
*/
async bump(userId: string): Promise<void> {
if (!userId) return;
try {
await this.#redis.creditHoldBump(
versionKey(userId),
String(VERSION_TTL_MS),
versionSeed(),
);
} catch (e) {
console.warn(
`[metering] credit hold version not bumped for ${userId}: ${(e as Error).message}`,
);
}
}
/**
* Give a hold back. Safe to call twice; the second call releases nothing.
*
* The version isn't bumped here: the usage a hold stands for bumps it when
* it's recorded, before the hold is released, and a release with no usage
* leaves the balance as it was.
*/
/** Give a hold back. Safe to call twice; the second call is a no-op. */
async release(userId: string, member: string | null): Promise<void> {
if (!userId || !member) return;
try {
@@ -420,625 +420,6 @@ describe('MeteringBufferStore', () => {
expect(await storedTotal(key)).toBe(0);
});
/** Holds `method` on `target` until the returned release is called. */
const holdCalls = <T extends object>(on: T, method: keyof T) => {
let release: () => void = () => {};
const gate = new Promise<void>((r) => (release = r));
const real = (on[method] as (...a: unknown[]) => unknown).bind(on);
const spy = vi
.spyOn(on, method as never)
.mockImplementation((async (...args: unknown[]) => {
await gate;
return real(...args);
}) as never);
return { spy, release };
};
const readTotal = async () =>
((await target.get({ key })).res as { total?: number } | null)
?.total ?? 0;
it('counts usage a flush has claimed but not yet written onward', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
const write = holdCalls(kv, 'incr');
try {
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
expect(await readTotal()).toBe(10);
write.release();
await flushing;
} finally {
write.spy.mockRestore();
}
expect(await readTotal()).toBe(10);
expect(await storedTotal(key)).toBe(10);
});
it('counts usage written onward until its settle lands', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
await target.flushCycle();
await target.incr({ key, pathAndAmountMap: { total: 5 } });
const redis = server.clients.redis as unknown as {
meterSettle: (...args: string[]) => Promise<number>;
};
const settle = holdCalls(redis, 'meterSettle');
try {
const flushing = target.flushCycle();
await vi.waitFor(() => expect(settle.spy).toHaveBeenCalled());
expect(await storedTotal(key)).toBe(15);
expect(await readTotal()).toBe(15);
settle.release();
await flushing;
} finally {
settle.spy.mockRestore();
}
expect(await readTotal()).toBe(15);
});
it('counts a claim in array reads too', async () => {
await target.incr({ key, pathAndAmountMap: { total: 4 } });
const write = holdCalls(kv, 'incr');
try {
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
const { res } = await target.get({ key: [key] });
expect((res as [{ total: number }])[0].total).toBe(4);
write.release();
await flushing;
} finally {
write.spy.mockRestore();
}
});
it('counts a claim in the running total incr returns', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
const write = holdCalls(kv, 'incr');
try {
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
const { res } = await target.incr({
key,
pathAndAmountMap: { total: 5 },
});
expect((res as { total: number }).total).toBe(15);
write.release();
await flushing;
} finally {
write.spy.mockRestore();
}
});
it('counts another flush’s claim in an exact read', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
const write = holdCalls(kv, 'incr');
try {
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
const { res } = await target.readExact({ key });
expect((res as { total: number }).total).toBe(10);
write.release();
await flushing;
} finally {
write.spy.mockRestore();
}
const { res } = await target.readExact({ key });
expect((res as { total: number }).total).toBe(10);
});
it('misses nothing in an exact read when a flush lands between its reads', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
const write = holdCalls(kv, 'incr');
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
// The store read sees the counter from before the flush's write;
// the flush then writes it and leaves its claim before any claim
// read that follows the store read.
let storeReadStarted = false;
let flushed!: Promise<void>;
const realGet = kv.get.bind(kv);
const get = vi.spyOn(kv, 'get').mockImplementation(async (args) => {
if (!args.consistentRead || args.key !== key) {
return realGet(args);
}
if (storeReadStarted) return realGet(args);
storeReadStarted = true;
const before = await realGet(args);
write.release();
flushed = flushing.then(() => {});
await flushed;
return before;
});
const redis = server.clients.redis as unknown as {
meterRead: (...args: string[]) => Promise<unknown>;
};
const realRead = redis.meterRead.bind(redis);
const read = vi
.spyOn(redis, 'meterRead')
.mockImplementation(async (...args: string[]) => {
if (storeReadStarted) {
await vi.waitFor(() => expect(flushed).toBeDefined());
await flushed;
}
return realRead(...args);
});
try {
const { res } = await target.readExact({ key });
expect((res as { total: number }).total).toBe(10);
} finally {
get.mockRestore();
read.mockRestore();
write.spy.mockRestore();
await flushing;
}
expect(await storedTotal(key)).toBe(10);
});
it('counts nothing twice in an exact read when a flush lands during its store read', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
const write = holdCalls(kv, 'incr');
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
// The claim was read with the amount still on it; the store read
// then sees it written.
let landed = false;
const realGet = kv.get.bind(kv);
const get = vi.spyOn(kv, 'get').mockImplementation(async (args) => {
if (args.consistentRead && args.key === key && !landed) {
landed = true;
write.release();
await flushing;
}
return realGet(args);
});
try {
const { res } = await target.readExact({ key });
expect((res as { total: number }).total).toBe(10);
} finally {
get.mockRestore();
write.spy.mockRestore();
await flushing;
}
});
it('counts usage behind another flush’s claim in an exact read', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
const write = holdCalls(kv, 'incr');
try {
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
// Arrives while the claim is out, so it can't be flushed yet.
await target.incr({ key, pathAndAmountMap: { total: 5 } });
const { res } = await target.readExact({ key });
expect((res as { total: number }).total).toBe(15);
write.release();
await flushing;
} finally {
write.spy.mockRestore();
}
});
/**
* Lets a held flush finish during the first consistent store read of
* `key`, which then answers with what the store holds afterwards.
*/
const landDuringStoreRead = (
release: () => void,
flushing: Promise<unknown>,
) => {
let landed = false;
const realGet = kv.get.bind(kv);
return vi.spyOn(kv, 'get').mockImplementation(async (args) => {
const ours = Array.isArray(args.key)
? args.key.includes(key)
: args.key === key;
if (args.consistentRead && ours && !landed) {
landed = true;
release();
await flushing;
}
return realGet(args);
});
};
for (const shape of ['single', 'array'] as const) {
it(`counts nothing twice in ${shape === 'single' ? 'a single' : 'an array'} read without a base when a flush lands during it`, async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
// No cached base: the counter is new to the store.
await target.forgetBase(key);
const write = holdCalls(kv, 'incr');
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
const get = landDuringStoreRead(write.release, flushing);
try {
const { res } = await target.get({
key: shape === 'single' ? key : [key],
});
const value = (
shape === 'single' ? res : (res as unknown[])[0]
) as { total: number };
expect(value.total).toBe(10);
} finally {
get.mockRestore();
write.spy.mockRestore();
await flushing;
}
});
}
it('reads a written-through increment once the cache is back', async () => {
await target.incr({ key, pathAndAmountMap: { total: 10 } });
await target.flushCycle();
const redis = server.clients.redis as unknown as {
meterIncr: (...args: string[]) => Promise<unknown>;
};
const failing = vi
.spyOn(redis, 'meterIncr')
.mockRejectedValueOnce(new Error('cache down'));
const { res, exact } = await target.incr({
key,
pathAndAmountMap: { total: 5 },
});
failing.mockRestore();
expect(exact).toBe(true);
expect((res as { total: number }).total).toBe(15);
expect(await readTotal()).toBe(15);
});
it('reads the store consistently when no base is cached', async () => {
await kv.incr({ key, pathAndAmountMap: { total: 3 } });
const get = vi.spyOn(kv, 'get');
try {
expect(await readTotal()).toBe(3);
expect(get).toHaveBeenCalledWith(
expect.objectContaining({ key, consistentRead: true }),
);
} finally {
get.mockRestore();
}
});
for (const shape of ['single', 'array', 'exact'] as const) {
it(`does not retry a ${shape} store read when only new increments arrive`, async () => {
await kv.incr({ key, pathAndAmountMap: { total: 100 } });
await target.incrAux({ key, pathAndAmountMap: { total: 10 } });
const dynamo = server.clients.dynamo;
const realGet = dynamo.get.bind(dynamo);
const realBatchGet = dynamo.batchGet.bind(dynamo);
let storeReads = 0;
const increment = async () => {
storeReads++;
await target.incrAux({
key,
pathAndAmountMap: { total: 5 },
});
};
const get = vi
.spyOn(dynamo, 'get')
.mockImplementation(async (...args) => {
const result = await realGet(...args);
if (args[1].key === key) await increment();
return result;
});
const batchGet = vi
.spyOn(dynamo, 'batchGet')
.mockImplementation(async (...args) => {
const result = await realBatchGet(...args);
if (args[0].some(({ items }) => items.key === key))
await increment();
return result;
});
const redis = server.clients.redis as unknown as {
meterRead: (...args: string[]) => Promise<unknown>;
};
const read = vi.spyOn(redis, 'meterRead');
try {
const { res } =
shape === 'exact'
? await target.readExact({ key })
: await target.get({
key: shape === 'single' ? key : [key],
});
const value = (
shape === 'array' ? (res as unknown[])[0] : res
) as { total: number };
expect(value.total).toBe(110);
expect(storeReads).toBe(1);
expect(read).toHaveBeenCalledTimes(2);
} finally {
get.mockRestore();
batchGet.mockRestore();
read.mockRestore();
}
expect(await readTotal()).toBe(115);
});
it(`retries a ${shape} read when new increments hide a completed flush`, async () => {
await kv.incr({ key, pathAndAmountMap: { total: 0 } });
await target.incrAux({ key, pathAndAmountMap: { total: 10 } });
const dynamo = server.clients.dynamo;
const write = holdCalls(dynamo, 'update');
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
const realGet = dynamo.get.bind(dynamo);
const realBatchGet = dynamo.batchGet.bind(dynamo);
let storeReads = 0;
const land = async () => {
if (++storeReads !== 1) return;
write.release();
await flushing;
await target.incrAux({
key,
pathAndAmountMap: { total: 20 },
});
};
const get = vi
.spyOn(dynamo, 'get')
.mockImplementation(async (...args) => {
if (args[1].key === key) await land();
return realGet(...args);
});
const batchGet = vi
.spyOn(dynamo, 'batchGet')
.mockImplementation(async (...args) => {
if (args[0].some(({ items }) => items.key === key))
await land();
return realBatchGet(...args);
});
try {
const { res } =
shape === 'exact'
? await target.readExact({ key })
: await target.get({
key: shape === 'single' ? key : [key],
});
const value = (
shape === 'array' ? (res as unknown[])[0] : res
) as { total: number };
expect(value.total).toBe(30);
expect(storeReads).toBe(2);
} finally {
get.mockRestore();
batchGet.mockRestore();
write.release();
await flushing;
write.spy.mockRestore();
}
});
}
it('keeps changed counters together in a batch retry', async () => {
const keys = [`${key}:a`, `${key}:b`];
await Promise.all(
keys.map((k) =>
kv.incr({ key: k, pathAndAmountMap: { total: 0 } }),
),
);
await Promise.all(
keys.map((k) =>
target.incrAux({ key: k, pathAndAmountMap: { total: 10 } }),
),
);
const dynamo = server.clients.dynamo;
const realBatchGet = dynamo.batchGet.bind(dynamo);
const write = holdCalls(dynamo, 'update');
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalledTimes(2));
const batches: string[][] = [];
const batchGet = vi
.spyOn(dynamo, 'batchGet')
.mockImplementation(async (...args) => {
batches.push(
args[0].map(({ items }) => items.key as string),
);
if (batches.length === 1) {
write.release();
await flushing;
}
return realBatchGet(...args);
});
const get = vi.spyOn(dynamo, 'get');
try {
expect((await target.get({ key: keys })).res).toEqual([
{ total: 10 },
{ total: 10 },
]);
expect(batches).toHaveLength(2);
expect(batches.map((batch) => batch.sort())).toEqual([
keys,
keys,
]);
expect(get).not.toHaveBeenCalled();
} finally {
batchGet.mockRestore();
get.mockRestore();
write.release();
await flushing;
write.spy.mockRestore();
}
});
it('detects a whole flush between snapshots with no outstanding claim', async () => {
await kv.incr({ key, pathAndAmountMap: { total: 0 } });
await target.incrAux({ key, pathAndAmountMap: { total: 10 } });
const dynamo = server.clients.dynamo;
const realGet = dynamo.get.bind(dynamo);
let reads = 0;
const get = vi
.spyOn(dynamo, 'get')
.mockImplementation(async (...args) => {
if (args[1].key === key && ++reads === 1) {
await target.flushCycle();
await target.incrAux({
key,
pathAndAmountMap: { total: 20 },
});
}
return realGet(...args);
});
try {
expect(await readTotal()).toBe(30);
expect(reads).toBe(2);
} finally {
get.mockRestore();
}
});
it('retries a decreasing buffer even when the flush metadata is unchanged', async () => {
await kv.incr({ key, pathAndAmountMap: { total: 100 } });
await target.incrAux({ key, pathAndAmountMap: { total: 10 } });
const dynamo = server.clients.dynamo;
const realGet = dynamo.get.bind(dynamo);
let reads = 0;
const get = vi
.spyOn(dynamo, 'get')
.mockImplementation(async (...args) => {
const result = await realGet(...args);
if (args[1].key === key && ++reads === 1) {
await target.incrAux({
key,
pathAndAmountMap: { total: -5 },
});
}
return result;
});
try {
expect(await readTotal()).toBe(105);
expect(reads).toBe(2);
} finally {
get.mockRestore();
}
});
it('detects applied paths while the claim id and fence token stay unchanged', async () => {
await kv.incr({ key, pathAndAmountMap: { total: 0 } });
await target.incrAux({ key, pathAndAmountMap: { total: 10 } });
const dynamo = server.clients.dynamo;
const redis = server.clients.redis as unknown as {
meterSettle: (...args: string[]) => Promise<number>;
};
const write = holdCalls(dynamo, 'update');
const settle = holdCalls(redis, 'meterSettle');
const flushing = target.flushCycle();
await vi.waitFor(() => expect(write.spy).toHaveBeenCalled());
const realGet = dynamo.get.bind(dynamo);
let reads = 0;
const get = vi
.spyOn(dynamo, 'get')
.mockImplementation(async (...args) => {
if (args[1].key === key && ++reads === 1) {
write.release();
await vi.waitFor(() =>
expect(settle.spy).toHaveBeenCalled(),
);
await target.incrAux({
key,
pathAndAmountMap: { total: 20 },
});
}
return realGet(...args);
});
try {
expect(await readTotal()).toBe(30);
expect(reads).toBe(2);
} finally {
get.mockRestore();
write.release();
settle.release();
await flushing;
write.spy.mockRestore();
settle.spy.mockRestore();
}
});
it('does not retry unchanged amounts when another counter advances the bucket token', async () => {
await kv.incr({ key, pathAndAmountMap: { total: 100 } });
let other = `${key}:other`;
for (let i = 0; bucketTag(other) !== bucketTag(key); i++)
other = `${key}:${i}`;
await kv.incr({ key: other, pathAndAmountMap: { total: 0 } });
await target.incrAux({
key: other,
pathAndAmountMap: { total: 10 },
});
const dynamo = server.clients.dynamo;
const realGet = dynamo.get.bind(dynamo);
let reads = 0;
const get = vi
.spyOn(dynamo, 'get')
.mockImplementation(async (...args) => {
if (args[1].key === key && ++reads === 1)
await target.flushCycle();
return realGet(...args);
});
try {
expect(await readTotal()).toBe(100);
expect(reads).toBe(1);
} finally {
get.mockRestore();
}
});
it('recovers a failed initial cache snapshot before retrying the batch', async () => {
await kv.incr({ key, pathAndAmountMap: { total: 100 } });
await target.incrAux({ key, pathAndAmountMap: { total: 10 } });
const redis = server.clients.redis as unknown as {
meterRead: (...args: string[]) => Promise<unknown>;
};
const read = vi
.spyOn(redis, 'meterRead')
.mockRejectedValueOnce(new Error('cache unavailable'));
const batchGet = vi.spyOn(server.clients.dynamo, 'batchGet');
try {
expect((await target.get({ key: [key] })).res).toEqual([
{ total: 110 },
]);
expect(batchGet).toHaveBeenCalledTimes(2);
} finally {
read.mockRestore();
batchGet.mockRestore();
}
});
it('bounds concurrent cache reads while preserving array order', async () => {
const keys = Array.from({ length: 25 }, (_, i) => `${key}:${i}`);
await Promise.all(
keys.map((k, i) =>
target.incrAux({
key: k,
pathAndAmountMap: { total: i + 1 },
}),
),
);
const redis = server.clients.redis as unknown as {
meterRead: (...args: string[]) => Promise<unknown>;
};
const realRead = redis.meterRead.bind(redis);
let active = 0;
let peak = 0;
const read = vi
.spyOn(redis, 'meterRead')
.mockImplementation(async (...args) => {
active++;
peak = Math.max(peak, active);
try {
await new Promise((resolve) => setTimeout(resolve, 1));
return await realRead(...args);
} finally {
active--;
}
});
try {
expect((await target.get({ key: keys })).res).toEqual(
keys.map((_, i) => ({ total: i + 1 })),
);
expect(peak).toBe(20);
expect(active).toBe(0);
} finally {
read.mockRestore();
}
});
it('reads an unknown counter as absent', async () => {
const { res } = await target.get({ key });
expect(res).toBeNull();
@@ -1525,16 +906,12 @@ describe('MeteringBufferStore', () => {
'1000000000',
);
// Another deployment's write, which only a replacing settle would
// bring into the base.
await kv.incr({ key, pathAndAmountMap: { total: 1000 } });
await target.incr({ key, pathAndAmountMap: { total: 5 } });
await target.flushCycle();
expect(await storedTotal(key)).toBe(1105);
// The newer base keeps its view, plus what this flush wrote.
expect(await storedTotal(key)).toBe(105);
expect(Number(await server.clients.redis.hget(base, 'total'))).toBe(
105,
100,
);
});
@@ -2061,30 +1438,6 @@ describe('MeteringBufferStore', () => {
).toEqual({});
});
it('reads a re-driven write that already landed, with a warm base', async () => {
await target.incr({ key, pathAndAmountMap: { total: 100 } });
await target.flushCycle();
await target.incr({ key, pathAndAmountMap: { total: 5 } });
const passThrough = kv.incr.bind(kv);
const lost = vi
.spyOn(kv, 'incr')
.mockImplementationOnce(
async (...args: Parameters<typeof kv.incr>) => {
await passThrough(...args);
throw new Error('request timed out');
},
);
await target.flushCycle();
lost.mockRestore();
await ageClaim();
await target.flushCycle();
expect(await storedTotal(key)).toBe(105);
const { res } = await target.get({ key });
expect((res as { total: number }).total).toBe(105);
});
it('applies a re-driven write whose first copy never landed', async () => {
await target.incr({ key, pathAndAmountMap: { total: 5 } });
const boom = vi
@@ -2389,17 +1742,6 @@ describe('MeteringBufferStore', () => {
order.push('dropFromClaim');
return realHdel(...args);
});
// A written batch leaves the claim through this script.
const redis = server.clients.redis as unknown as {
meterApply: (...args: string[]) => Promise<number>;
};
const realApply = redis.meterApply.bind(redis);
const applySpy = vi
.spyOn(redis, 'meterApply')
.mockImplementation((...args: string[]) => {
order.push('dropFromClaim');
return realApply(...args);
});
const alarmSpy = vi
.spyOn(server.clients.alarm, 'create')
.mockImplementation(() => {
@@ -2409,7 +1751,6 @@ describe('MeteringBufferStore', () => {
await target.flushCycle();
incrSpy.mockRestore();
hdelSpy.mockRestore();
applySpy.mockRestore();
alarmSpy.mockRestore();
expect(order).toEqual(['dropFromClaim', 'alarm']);
@@ -2442,13 +1783,7 @@ describe('MeteringBufferStore', () => {
// right up until the TTL takes the amounts with it.
const tag = bucketTag(key);
await server.clients.redis.del(`meter:dirty:{${tag}}`);
// Every few cycles a flush reconciles on its own; keep this one
// from doing so, whatever the earlier tests left the count at.
const scheduled = vi
.spyOn(target, 'reconcile')
.mockResolvedValue(undefined);
expect(await target.flushCycle()).toBe(0);
scheduled.mockRestore();
expect(await storedTotal(key)).toBe(0);
const logged = vi
@@ -83,19 +83,6 @@ interface IncrInput {
type FlatAmounts = Record<string, number>;
interface BufferedRead {
delta: FlatAmounts;
base: FlatAmounts;
claim: FlatAmounts;
claimId: string;
writeToken: string;
}
interface StoreAndBufferRead {
stored: unknown;
buffered: FlatAmounts;
}
/** A claim's write that was recorded before it was sent; see `INTENT_SCRIPT`. */
interface SettleIntent {
writer: string;
@@ -139,8 +126,8 @@ const ORPHAN_AGE_MS = 30_000;
/** Long enough that a counter for the current month can never expire. */
const BUFFER_TTL_MS = 40 * 24 * 60 * 60 * 1000;
/** Cache reads per call or counter writes per cycle in flight at once. */
const BUFFER_IO_CONCURRENCY = 20;
/** How many counters are written onward at once, to keep the load even. */
const SETTLE_CONCURRENCY = 20;
/** Roughly a minute between compression reports, so they don't flood the log. */
const CYCLES_PER_COMPRESSION_REPORT = 12;
@@ -165,12 +152,6 @@ const UNWRITABLE_PATH_MEMO_LIMIT = 1000;
*/
const EXACT_READ_MEMO_LIMIT = 10_000;
/**
* Store reads a consistent read makes while the buffer keeps shrinking under
* it; see `#readStoreAndBuffer`.
*/
const EXACT_READ_ATTEMPTS = 3;
/** Write units past which settling a counter raises the large-item alarm. */
const LARGE_SETTLE_WRITE_UNITS = 100;
@@ -358,15 +339,6 @@ const toScriptArgs = (amounts: Record<string, number>): string[] => {
return args;
};
/** Whether two sets of amounts hold the same paths and values. */
const sameAmounts = (a: FlatAmounts, b: FlatAmounts): boolean => {
const paths = Object.keys(a);
return (
paths.length === Object.keys(b).length &&
paths.every((path) => a[path] === b[path])
);
};
/** The subset of `amounts` at `paths`. */
const pickAmounts = (amounts: FlatAmounts, paths: string[]): FlatAmounts =>
Object.fromEntries(paths.map((path) => [path, amounts[path]!]));
@@ -408,12 +380,11 @@ export const isBilledCounter = (key: string): boolean =>
// -- Scripts ----------------------------------------------------------
/**
* KEYS: delta, base, dirty set, tracked set, claimed. ARGV: ttl, member,
* pending key prefix, then path/amount pairs.
* KEYS: delta, base, dirty set, tracked set. ARGV: ttl, member, then
* path/amount pairs.
*
* Returns the delta, the base and any outstanding claim together, so a running
* total can be served without a second round trip (see `READ_SCRIPT` for the
* claim). Amounts are floats — costs are fractional.
* Returns the delta and the base together so a running total can be served
* without a second round trip. Amounts are floats — costs are fractional.
*
* The counter joins both sets here. The dirty set is the working list a flush
* takes from and hands back; the tracked set is only ever added to here and
@@ -421,7 +392,7 @@ export const isBilledCounter = (key: string): boolean =>
* working list does not.
*/
const INCR_SCRIPT = `
for i = 4, #ARGV, 2 do
for i = 3, #ARGV, 2 do
redis.call('HINCRBYFLOAT', KEYS[1], ARGV[i], ARGV[i + 1])
end
redis.call('PEXPIRE', KEYS[1], ARGV[1])
@@ -429,48 +400,12 @@ redis.call('SADD', KEYS[3], ARGV[2])
redis.call('PEXPIRE', KEYS[3], ARGV[1])
redis.call('SADD', KEYS[4], ARGV[2])
redis.call('PEXPIRE', KEYS[4], ARGV[1])
local claim = {}
local nonce = redis.call('HGET', KEYS[5], ARGV[2])
if nonce then claim = redis.call('HGETALL', ARGV[3] .. nonce) end
return { redis.call('HGETALL', KEYS[1]), redis.call('HGETALL', KEYS[2]), claim }
return { redis.call('HGETALL', KEYS[1]), redis.call('HGETALL', KEYS[2]) }
`;
/**
* KEYS: delta, base, claimed, fence token. ARGV: counter, pending key prefix.
*
* Reads both without marking the counter for flushing, plus the counter's
* outstanding claim: amounts a flush has taken but not yet written onward are
* in neither the delta nor the base. The claim's key is built here, which is
* safe for the reason given at `RECONCILE_SCRIPT`.
*/
/** KEYS: delta, base. Reads both without marking the counter for flushing. */
const READ_SCRIPT = `
local claim = {}
local nonce = redis.call('HGET', KEYS[3], ARGV[1])
if nonce then claim = redis.call('HGETALL', ARGV[2] .. nonce) end
return {
redis.call('HGETALL', KEYS[1]), redis.call('HGETALL', KEYS[2]), claim,
nonce or '', redis.call('GET', KEYS[4]) or ''
}
`;
/**
* KEYS: pending, base. ARGV: path/amount pairs just written onward.
*
* Takes written paths off a claim and, when a base is cached, adds them to it
* in the same step, so a read counts them all the way to the settle that
* replaces the base with the store's own view. Without a base, reads go to the
* store, which already has them.
*/
const APPLY_SCRIPT = `
redis.call('HDEL', KEYS[1], '${INTENT_FIELD}')
local based = redis.call('EXISTS', KEYS[2]) == 1
for i = 1, #ARGV, 2 do
redis.call('HDEL', KEYS[1], ARGV[i])
if based then
redis.call('HINCRBYFLOAT', KEYS[2], ARGV[i], ARGV[i + 1])
end
end
return 1
return { redis.call('HGETALL', KEYS[1]), redis.call('HGETALL', KEYS[2]) }
`;
/**
@@ -716,29 +651,9 @@ end
return redis.call('HGETALL', KEYS[1])
`;
/**
* KEYS: pending, base, base sequence. ARGV: paths found already applied.
*
* Takes paths an earlier copy of the write already applied off a claim, and
* drops the cached base in the same step: the base may or may not have them,
* and the store does, so the next read seeds a fresh one from the store.
*/
const FENCED_SCRIPT = `
redis.call('HDEL', KEYS[1], '${INTENT_FIELD}')
for i = 1, #ARGV do
redis.call('HDEL', KEYS[1], ARGV[i])
end
redis.call('DEL', KEYS[2], KEYS[3])
return 1
`;
type ScriptRunner = {
meterIncr(...args: string[]): Promise<[string[], string[], string[]]>;
meterRead(
...args: string[]
): Promise<[string[], string[], string[], string, string]>;
meterApply(...args: string[]): Promise<number>;
meterFenced(...args: string[]): Promise<number>;
meterIncr(...args: string[]): Promise<[string[], string[]]>;
meterRead(...args: string[]): Promise<[string[], string[]]>;
meterDrain(...args: string[]): Promise<[string[], number]>;
meterClaim(...args: string[]): Promise<string[] | null>;
/** `[1, amounts]` when re-claimed, `[0, hadIndexEntry]` when not. */
@@ -855,8 +770,6 @@ export class MeteringBufferStore extends PuterStore {
`[metering] buffered incr failed, writing through: ${(e as Error).message}`,
);
const { res } = await this.#writeThrough(input);
// The cached base doesn't have this write; the next read reseeds it.
await this.forgetBase(input.key);
return { res, exact: true };
}
}
@@ -904,40 +817,43 @@ export class MeteringBufferStore extends PuterStore {
const keys = key;
const results: (unknown | null)[] = new Array(keys.length).fill(null);
const snapshots: (BufferedRead | null)[] = new Array(keys.length).fill(
null,
const deltas: FlatAmounts[] = new Array(keys.length).fill(
{} as FlatAmounts,
);
const misses: number[] = [];
const reads = await runWithConcurrencyLimitSettled(
keys,
BUFFER_IO_CONCURRENCY,
(k) => this.#readBuffered(k),
);
reads.forEach((read, i) => {
if (read.status === 'fulfilled') {
const { delta, base } = read.value;
if (Object.keys(base).length > 0) {
results[i] = mergedOrNull(base, delta);
return;
await Promise.all(
keys.map(async (k, i) => {
try {
const { delta, base } = await this.#readBuffered(k);
if (Object.keys(base).length > 0) {
results[i] = mergedOrNull(base, delta);
return;
}
// No cached base yet — the store is the only place left to
// ask, and it's cheaper to ask once for every key like it.
deltas[i] = delta;
misses.push(i);
} catch (e) {
console.warn(
`[metering] buffered read failed, reading through: ${(e as Error).message}`,
);
misses.push(i);
}
snapshots[i] = read.value;
} else {
console.warn(
`[metering] buffered read failed, reading through: ${(read.reason as Error).message}`,
);
}
misses.push(i);
});
}),
);
if (misses.length > 0) {
const fetched = await this.#readStoreAndBuffer(
misses.map((i) => keys[i]!),
misses.map((i) => snapshots[i]!),
);
// Unlike the per-key fallbacks above, a failure here is not
// absorbed: a caller deciding on a balance must know its answer
// is incomplete rather than silently read a buffered-only view.
const { res: fetched } = await this.stores.kv.get({
key: misses.map((i) => keys[i]!),
});
const fetchedValues = fetched as (unknown | null)[];
misses.forEach((i, idx) => {
const { stored, buffered } = fetched[idx]!;
results[i] = mergedOrNull(flattenAmounts(stored), buffered);
const persisted = flattenAmounts(fetchedValues[idx]);
results[i] = mergedOrNull(persisted, deltas[i]!);
});
}
@@ -981,11 +897,7 @@ export class MeteringBufferStore extends PuterStore {
`[metering] exact read could not flush ${key}: ${(e as Error).message}`,
);
}
// Whatever is still buffered, a claim another deployment holds and
// usage that arrived behind it included, isn't in the store yet.
const [{ stored, buffered }] = await this.#readStoreAndBuffer(key);
if (Object.keys(buffered).length === 0) return { res: stored };
return { res: mergedOrNull(flattenAmounts(stored), buffered) };
return this.stores.kv.get({ key, consistentRead: true });
}
#rememberExactRead(key: string, now: number): void {
@@ -1203,20 +1115,12 @@ export class MeteringBufferStore extends PuterStore {
this.#definedScripts = true;
const client = this.clients.redis;
client.defineCommand('meterIncr', {
numberOfKeys: 5,
numberOfKeys: 4,
lua: INCR_SCRIPT,
});
client.defineCommand('meterRead', {
numberOfKeys: 4,
lua: READ_SCRIPT,
});
client.defineCommand('meterApply', {
numberOfKeys: 2,
lua: APPLY_SCRIPT,
});
client.defineCommand('meterFenced', {
numberOfKeys: 3,
lua: FENCED_SCRIPT,
lua: READ_SCRIPT,
});
client.defineCommand('meterDrain', {
numberOfKeys: 2,
@@ -1268,23 +1172,18 @@ export class MeteringBufferStore extends PuterStore {
pathAndAmountMap,
}: IncrInput): Promise<{ delta: FlatAmounts; base: FlatAmounts }> {
const tag = bucketTag(key);
const [delta, base, claim] = await this.#redis.meterIncr(
const [delta, base] = await this.#redis.meterIncr(
deltaKey(tag, key),
baseKey(tag, key),
dirtyKey(tag),
trackedKey(tag),
claimedKey(tag),
String(BUFFER_TTL_MS),
key,
pendingKey(tag, ''),
...toScriptArgs(pathAndAmountMap),
);
this.#absorbedCount++;
absorbedCounter.add(1);
return {
delta: addAmounts(pairsToAmounts(delta), parseClaim(claim).amounts),
base: pairsToAmounts(base),
};
return { delta: pairsToAmounts(delta), base: pairsToAmounts(base) };
}
/**
@@ -1297,9 +1196,7 @@ export class MeteringBufferStore extends PuterStore {
* is over within seconds.
*/
async #seedBase(key: string): Promise<FlatAmounts> {
// Consistent, or a write that just landed could be missing from the
// base until the next settle replaces it.
const { res } = await this.stores.kv.get({ key, consistentRead: true });
const { res } = await this.stores.kv.get({ key });
const persisted = flattenAmounts(res);
const seeded = await this.#redis.meterSeed(
baseKey(bucketTag(key), key),
@@ -1315,107 +1212,25 @@ export class MeteringBufferStore extends PuterStore {
* This deployment's cached delta and base for a counter, straight from
* redis.
*/
async #readBuffered(key: string): Promise<BufferedRead> {
async #readBuffered(
key: string,
): Promise<{ delta: FlatAmounts; base: FlatAmounts }> {
const tag = bucketTag(key);
const [delta, base, claim, claimId, writeToken] =
await this.#redis.meterRead(
deltaKey(tag, key),
baseKey(tag, key),
claimedKey(tag),
tokenKey(tag),
key,
pendingKey(tag, ''),
);
const claimed = parseClaim(claim).amounts;
// Claimed amounts count like buffered ones until their settle.
return {
delta: addAmounts(pairsToAmounts(delta), claimed),
base: pairsToAmounts(base),
claim: claimed,
claimId,
writeToken,
};
const [delta, base] = await this.#redis.meterRead(
deltaKey(tag, key),
baseKey(tag, key),
);
return { delta: pairsToAmounts(delta), base: pairsToAmounts(base) };
}
async #readThrough(key: string): Promise<unknown | null> {
const snapshot = await this.#readBuffered(key);
const { delta, base } = snapshot;
if (Object.keys(base).length > 0) return mergedOrNull(base, delta);
const [{ stored, buffered }] = await this.#readStoreAndBuffer(key, [
snapshot,
]);
return mergedOrNull(flattenAmounts(stored), buffered);
}
const { delta, base } = await this.#readBuffered(key);
const resolved =
Object.keys(base).length > 0
? base
: flattenAmounts((await this.stores.kv.get({ key })).res);
/**
* Order buffer snapshots around consistent store reads to avoid losing
* flushed usage. Retry changed claims together; new increments alone do not
* move amounts to the store and do not invalidate the first snapshot.
*/
async #readStoreAndBuffer(
key: string | string[],
snapshots?: (BufferedRead | null)[],
): Promise<StoreAndBufferRead[]> {
const multi = Array.isArray(key);
const keys = multi ? key : [key];
const initial = snapshots ?? [await this.#readBuffered(keys[0]!)];
let pending = keys.map((k, i) => ({
key: k,
index: i,
before: initial[i] ?? null,
}));
const results: StoreAndBufferRead[] = new Array(keys.length);
for (let attempt = 1; pending.length > 0; attempt++) {
const { res } = await this.stores.kv.get({
key: multi ? pending.map((item) => item.key) : pending[0]!.key,
consistentRead: true,
});
const stored = multi ? (res as unknown[]) : [res];
const checks =
attempt < EXACT_READ_ATTEMPTS
? await runWithConcurrencyLimitSettled(
pending,
BUFFER_IO_CONCURRENCY,
(item) => this.#readBuffered(item.key),
)
: [];
const retry: typeof pending = [];
pending.forEach((item, i) => {
const check = checks[i];
if (check?.status === 'fulfilled') {
const after = check.value;
const before = item.before;
// The token catches a whole claim starting and finishing
// between snapshots, even when new increments hide it.
const unchanged =
before &&
(sameAmounts(before.delta, after.delta) ||
(before.writeToken === after.writeToken &&
before.claimId === after.claimId &&
sameAmounts(before.claim, after.claim) &&
Object.entries(before.delta).every(
([path, amount]) =>
(after.delta[path] ?? 0) >= amount,
) &&
Object.entries(after.delta).every(
([path, amount]) =>
amount >= (before.delta[path] ?? 0),
)));
if (!unchanged) {
retry.push({ ...item, before: after });
return;
}
} else if (check?.status === 'rejected' && !multi) {
throw check.reason;
}
results[item.index] = {
stored: stored[i],
buffered: item.before?.delta ?? {},
};
});
pending = retry;
}
return results;
return mergedOrNull(resolved, delta);
}
// -- Internals: flush ---------------------------------------------
@@ -1487,7 +1302,7 @@ export class MeteringBufferStore extends PuterStore {
const outcomes = await runWithConcurrencyLimitSettled(
work,
BUFFER_IO_CONCURRENCY,
SETTLE_CONCURRENCY,
(run) => run(),
);
for (const outcome of outcomes) {
@@ -1775,12 +1590,7 @@ export class MeteringBufferStore extends PuterStore {
if (err instanceof KvFencedWriteError) {
// Applied already, by an earlier copy of this write.
fencedCounter.add(1);
await this.#redis.meterFenced(
pendingKey(tag, nonce),
baseKey(tag, key),
baseSeqKey(tag, key),
...paths,
);
await this.#dropFromClaim(tag, nonce, paths);
continue;
}
if (!isPermanentSettleError(err)) throw e;
@@ -1808,11 +1618,7 @@ export class MeteringBufferStore extends PuterStore {
// before anything else can fail: a re-drive then picks up only what
// is still outstanding. The alarm below is the "anything else" — it
// comes after.
await this.#redis.meterApply(
pendingKey(tag, nonce),
baseKey(tag, key),
...toScriptArgs(batch),
);
await this.#dropFromClaim(tag, nonce, paths);
if (usage.write >= LARGE_SETTLE_WRITE_UNITS) {
this.#recordLargeSettle(key, usage.write);
@@ -1,90 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { EventClient } from '../../clients/event/EventClient.js';
import type { EventMap } from '../../clients/event/types.js';
import { NotificationStore } from './NotificationStore.js';
const setup = (row: Record<string, unknown> | null, changed = true) => {
const event = new EventClient({} as never);
const emitted: EventMap['notif.acknowledged'][] = [];
event.on('notif.acknowledged', (_key, data) => { emitted.push(data); });
const db = {
write: vi.fn(async () => ({ anyRowsAffected: changed })),
pread: vi.fn(async () => row ? [row] : []),
read: vi.fn(async () => []),
};
const store = new NotificationStore({}, { event, db } as never, {});
return { store, emitted, db };
};
const appRow = {
uid: 'one', user_id: 7, type: 'app.message', app_uid: 'app-one',
audience: 'app-user', value: { title: 'private', data: 'private' },
};
describe('acknowledgement emission', () => {
it('announces a committed app message dismissal without content', async () => {
const { store, emitted, db } = setup(appRow);
await expect(store.markAcknowledged('one', 7)).resolves.toBe(true);
expect(db.write.mock.calls[0][0]).toContain('`acknowledged` IS NULL');
expect(db.pread).toHaveBeenCalledWith(expect.stringContaining('`user_id` = ?'), ['one', 7]);
expect(emitted).toEqual([{ userId: 7, uid: 'one', type: 'app.message', audience: 'app-user', appUid: 'app-one' }]);
});
it.each([
{ ...appRow, type: 'share.received' },
{ ...appRow, app_uid: null },
null,
])('does not announce a row outside an app message scope: %s', async (row) => {
const { store, emitted } = setup(row);
await store.markAcknowledged('one', 7);
expect(emitted).toEqual([]);
});
it('does not announce or read back an unchanged row', async () => {
const { store, emitted, db } = setup(appRow, false);
await expect(store.markAcknowledged('one', 7)).resolves.toBe(false);
expect(db.pread).not.toHaveBeenCalled();
expect(emitted).toEqual([]);
});
it('preserves a committed acknowledgement if the announcement read fails', async () => {
const { store, db, emitted } = setup(appRow);
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
db.pread.mockRejectedValueOnce(new Error('unavailable'));
try {
await expect(store.markAcknowledged('one', 7)).resolves.toBe(true);
expect(emitted).toEqual([]);
} finally {
warn.mockRestore();
}
});
it('does not propagate markShown', async () => {
const { store, emitted } = setup(appRow);
await store.markShown('one', 7);
expect(emitted).toEqual([]);
});
});
describe('displayed notification state query', () => {
it('reads all named ids in one scoped query', async () => {
const { store, db } = setup(appRow);
await store.listByUserId(7, {
limit: 50, uids: ['one', 'two'],
scope: { appUid: 'app-one', audiences: ['app-user'] },
});
expect(db.pread).not.toHaveBeenCalled();
expect(db.read).toHaveBeenCalledExactlyOnceWith(
expect.stringContaining('`uid` IN (?, ?)'),
[7, 'one', 'two', 'app-user', 'app-one', 50],
);
expect(db.read.mock.calls[0][0]).toContain('`user_id` = ?');
expect(db.read.mock.calls[0][0]).toContain('`app_uid` = ?');
});
it('does not query an empty id set', async () => {
const { store, db } = setup(appRow);
expect(await store.listByUserId(7, { uids: [] })).toEqual([]);
expect(db.read).not.toHaveBeenCalled();
});
});
@@ -156,12 +156,11 @@ export class NotificationStore extends PuterStore {
* @param {number} userId
* @param {{
* limit?: number;
* uids?: readonly string[];
* onlyUnacknowledged?: boolean;
* filter?: string;
* scope?: {
* audiences: readonly string[];
* appUid?: string | null;
* appUid: string | null;
* } | null;
* excludeTypes?: readonly string[];
* }} [opts]
@@ -170,7 +169,6 @@ export class NotificationStore extends PuterStore {
userId,
{
limit = 200,
uids,
onlyUnacknowledged = false,
filter = undefined,
scope = null,
@@ -180,12 +178,6 @@ export class NotificationStore extends PuterStore {
const where = ['`user_id` = ?'];
const params = [userId];
if (uids) {
if (uids.length === 0) return [];
where.push(`\`uid\` IN (${uids.map(() => '?').join(', ')})`);
params.push(...uids);
}
if (excludeTypes.length > 0) {
where.push(
`\`type\` NOT IN (${excludeTypes.map(() => '?').join(', ')})`,
@@ -581,32 +573,6 @@ export class NotificationStore extends PuterStore {
'UPDATE `notification` SET `acknowledged` = ? WHERE `uid` = ? AND `user_id` = ? AND `acknowledged` IS NULL',
[now, uid, userId],
);
if (result.anyRowsAffected) {
try {
const [row] = await this.clients.db.pread(
'SELECT `type`, `app_uid`, `audience` FROM `notification` WHERE `uid` = ? AND `user_id` = ? LIMIT 1',
[uid, userId],
);
if (row?.type === 'app.message' && row.app_uid) {
this.clients.event.emit(
'notif.acknowledged',
{
userId,
uid,
type: row.type,
audience: row.audience,
appUid: row.app_uid,
},
{},
);
}
} catch (error) {
console.warn(
'[notification] dismissal announcement failed',
error,
);
}
}
return result.anyRowsAffected;
}
+8 -7
View File
@@ -256,7 +256,12 @@ export interface IPreludeConfig {
* an RCS agent provisioned in the Prelude account to actually use RCS.
*/
preferredChannel?:
'sms' | 'rcs' | 'whatsapp' | 'viber' | 'zalo' | 'telegram';
| 'sms'
| 'rcs'
| 'whatsapp'
| 'viber'
| 'zalo'
| 'telegram';
}
/**
@@ -1101,10 +1106,7 @@ interface IConfigOptional {
builtin_apps: Record<string, string>;
/** Force the bundled GUI even in dev. Default: false. */
use_bundled_gui: boolean;
/**
* Override the GUI bundle JS path. Default: `/dist/bundle.min.js`. The app
* browser embed loads `apps-embed.min.js` from the same directory.
*/
/** Override the GUI bundle JS path. Default: `/dist/bundle.min.js`. */
gui_bundle: string;
/** Override the GUI CSS path when bundled. Default: `/dist/bundle.min.css`. */
gui_css: string;
@@ -1391,8 +1393,7 @@ export interface ServerDrainInfo {
}
export interface WithCostsReporting extends WithLifecycle {
getReportedCosts?: () =>
// eslint-disable-next-line @typescript-eslint/no-explicit-any
getReportedCosts?: () => // eslint-disable-next-line @typescript-eslint/no-explicit-any
| Promise<Record<string, any>[]>
// eslint-disable-next-line @typescript-eslint/no-explicit-any
| Record<string, any>[];
+2 -28
View File
@@ -5,7 +5,7 @@ let sidebar = require('./src/sidebar');
const redirects = require('./src/redirects');
const menuItems = require('./src/menu.js');
const examples = require('./src/examples');
const { encode, decode } = require('html-entities');
const { encode } = require('html-entities');
const { JSDOM } = require('jsdom');
const esbuild = require('esbuild');
const MiniSearch = require('minisearch');
@@ -45,30 +45,11 @@ const site = 'https://docs.puter.com';
let usedPlaygroundExamples = new Set();
let anyErrors = false;
// Heading ids already used on the page being parsed. Reset before each parse.
let usedHeadingSlugs = new Set();
marked.use({
hooks: {
preprocess (markdown) {
usedHeadingSlugs = new Set();
return markdown;
},
},
renderer: {
// Add a link to each subheading
heading (text, level) {
// `text` is rendered HTML, so drop tags and entities before slugging.
const baseSlug = decode(removeTags(text))
.toLowerCase()
.replace(/[^\w]+/g, '-')
.replace(/^-+|-+$/g, '') || 'section';
// A repeated heading gets a numbered id, so each anchor is unique.
let slug = baseSlug;
for ( let i = 1; usedHeadingSlugs.has(slug); i++ ) {
slug = `${baseSlug}-${i}`;
}
usedHeadingSlugs.add(slug);
const slug = text.toLowerCase().replace(/[^\w]+/g, '-');
return `
<h${level} class="anchored-heading" id="${slug}">
@@ -77,13 +58,6 @@ marked.use({
</h${level}>`;
},
// Wrap each table so a wide one scrolls sideways instead of making
// the page wider than a phone screen. Same markup as marked's default.
table (header, body) {
if ( body ) body = `<tbody>${body}</tbody>`;
return `<div class="table-scroll"><table>\n<thead>\n${header}</thead>\n${body}</table>\n</div>\n`;
},
code (code, infostring, escaped) {
// Extract possible playground example ID from the info string
infostring = infostring.trim() || '';
+3 -3
View File
@@ -22,14 +22,14 @@ You can use AI models from various providers to perform tasks such as chat, text
<div class="example-content" data-section="ai-chat" style="display:block;">
#### Chat with GPT-6 Luna
#### Chat with GPT-5.6 Luna
```html;ai-chatgpt
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
puter.ai.chat(`What is life?`, { model: "gpt-6-luna" }).then(puter.print);
puter.ai.chat(`What is life?`, { model: "gpt-5.6-luna" }).then(puter.print);
</script>
</body>
</html>
@@ -212,7 +212,7 @@ These AI features are supported out of the box when using Puter.js:
You can see various Puter.js AI features in action from the following examples:
- AI Chat
- [Chat with GPT-6 Luna](/playground/ai-chatgpt/)
- [Chat with GPT-5.6 Luna](/playground/ai-chatgpt/)
- [Image Analysis](/playground/ai-gpt-vision/)
- [Stream the response](/playground/ai-chat-stream/)
- [Function Calling](/playground/ai-function-calling/)
+181 -258
View File
@@ -27,28 +27,24 @@ A string containing the prompt you want to complete.
An object containing the following properties:
| Option | Type | Description |
|--------|------|-------------|
| `model` | String | The model you want to use for the completion. If not specified, defaults to `gpt-5.4-nano`. More than 500 models are available from providers including OpenAI, Anthropic, Google, Alibaba Cloud, xAI, Mistral, OpenRouter, Infron, and others. For a full list, see the [AI models list](https://developer.puter.com/ai/models/) page. |
| `stream` | Boolean | A boolean indicating whether you want to stream the completion. Defaults to `false`. |
| `max_tokens` | Number | The maximum number of tokens to generate in the completion. By default, the specific model's maximum is used. |
| `temperature` | Number | A number between 0 and 2 indicating the randomness of the completion. Lower values make the output more focused and deterministic, while higher values make it more random. By default, the model's own temperature is used. See [Model-specific features](#model-specific-features). |
| `reasoning_effort` / `reasoning.effort` | String | Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Lower values give faster responses with less reasoning. OpenAI models, Moonshot's Kimi K3.1 Pro models, Meta's Muse Spark models, `gemini-4-argon`, `gemini-3.1-flash-lite` and `gemini-3-flash-preview` only. See [Model-specific features](#model-specific-features). |
| `thinking` | Object \| null | Extended-thinking control: `{ type: "adaptive" \| "enabled" \| "disabled" \| "between_tools", budgetTokens?, display? }`. Pass `null` to send no thinking configuration at all (the model's own default); leaving it unset falls back to the legacy derivation from `reasoning_effort`. `budgetTokens` only applies to `type: "enabled"`. |
| `verbosity` / `text.verbosity` | String | Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only. |
| `tools` | Array | Function definitions the AI can call. See [Function Calling](#function-calling) for details. |
| `toolChoice` | String \| Object | Which tool, if any, the model must call. Pass `"auto"` (the model decides), `"none"` (never call a tool), `"required"` (call at least one), or an object naming a specific tool: `{ type: "tool", name: "..." }`. Requires `tools`. |
| `parallelToolCalls` | Boolean | Whether the model may call more than one tool in a single turn. Left unset, tool calls run one at a time. |
| `streamToolInput` | Boolean | Streaming only. When `true`, a tool call's arguments arrive incrementally as `tool_use_start` / `tool_input_delta` chunks instead of a single `tool_use` chunk once the call completes. |
| `normalize` | Boolean | Controls the format of the non-streaming response. See [Response normalization](#response-normalization). |
| `compaction` | Boolean \| Object | Opt into inline context compaction for long conversations. See [Compaction](#compaction). |
| `signal` | AbortSignal | Abort the request, including while waiting for the next streamed chunk. Cancellation rejects with the signal reason (normally an `AbortError`). |
| `provider` | String | Pin the request to a specific provider, for example `openrouter` or `infron`. Without it, Puter selects a provider for the requested model. Call [`puter.ai.listModelProviders()`](/AI/listModelProviders) for the available values, and [`puter.ai.listModels(provider)`](/AI/listModels) for the models a given provider serves. See [Providers](#providers). |
| `fallback` | Boolean | Set to `false` to fail the request when the first provider tried fails, instead of retrying it on another provider that serves the same model. Defaults to `true`. See [Providers](#providers). |
| `outputFormat` | Object | Constrains the response to a JSON schema: `{ type: "json_schema", schema, name?, strict? }`. |
| `stopSequences` | Array of String | Up to the model's limit of strings; generation stops the moment one of them appears in the output. |
Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).
- `model` (String) - The model you want to use for the completion. If not specified, defaults to `gpt-5.4-nano`. More than 500 models are available from vendors including OpenAI, Anthropic, Google, Alibaba Cloud, xAI, Mistral, OpenRouter, Infron, and others. For a full list, see the [AI models list](https://developer.puter.com/ai/models/) page.
- `provider` (String) (Optional) - Pin the request to a specific vendor, for example `openrouter` or `infron`. Without it, Puter selects a vendor for the requested model. Call [`puter.ai.listModelProviders()`](/AI/listModelProviders) for the available values, and [`puter.ai.listModels(provider)`](/AI/listModels) for the models a given vendor serves.
- `fallback` (Boolean) (Optional) - Set to `false` to fail the request when the first vendor tried fails, instead of retrying it on another vendor that serves the same model. Defaults to `true`. With `provider` set as well, the request is answered by that vendor or fails. If that vendor does not serve the model, the request fails with a `400` instead of going to another vendor. The error is the same one you would get if every vendor had failed, for example `400 upstream_provider_unavailable` or `429 upstream_rate_limited`.
- `signal` (AbortSignal) (Optional) - Abort the request, including while waiting for the next streamed chunk. Cancellation rejects with the signal reason (normally an `AbortError`).
- `stream` (Boolean) - A boolean indicating whether you want to stream the completion. Defaults to `false`.
- `max_tokens` (Number) - The maximum number of tokens to generate in the completion. By default, the specific model's maximum is used.
- `temperature` (Number) - A number between 0 and 2 indicating the randomness of the completion. Lower values make the output more focused and deterministic, while higher values make it more random. By default, the model's own temperature is used. Some OpenAI reasoning models ignore `temperature` and `top_p`. GPT-5.6, GPT-6 Sol/Luna, and GPT-6.1 Terra accept them only with `reasoning_effort: 'none'`, and GPT-6 Astra, GPT-6.1 Sol, and GPT-5.3 Codex always ignore them. `gemini-4-argon` ignores `temperature` too. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` use it.
- `tools` (Array) (Optional) - Function definitions the AI can call. See [Function Calling](#function-calling) for details.
- `reasoning_effort` / `reasoning.effort` (String) (Optional) - Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. GPT-6.1 Terra accepts `none`, `low`, `medium` (default), `high`, `xhigh`, and `max`. Lower values give faster responses with less reasoning. OpenAI models, Moonshot's Kimi K3.1 Pro models, Meta's Muse Spark models, `gemini-4-argon`, `gemini-3.1-flash-lite` and `gemini-3-flash-preview` only; Muse Spark always reasons, so `none` is ignored for it. Kimi K3.1 Pro accepts `low`, `high` and `max`, uses `max` when you leave it unset, and fails with `400 bad_request` on any other value. `gemini-4-argon` accepts `low`, `medium` and `high`, and uses `medium` when you leave it unset. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` accept `minimal`, `low`, `medium` and `high`, and use Google's default level when you leave it unset. For these three Gemini models, any other value fails with `400 bad_request`, and messages can contain text and images only.
- `verbosity` / `text.verbosity` (String) (Optional) - Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only.
- `normalize` (Boolean) (Optional) - Controls the format of the non-streaming response. When `true`, the response is normalized to the OpenAI format regardless of the model's vendor: `message.content` is a string, tool calls appear as `message.tool_calls`, and `finish_reason` is one of `stop`, `length`, `tool_calls`, or `content_filter` — or the vendor's own stop reason, passed through unchanged when it has no OpenAI equivalent. When `false`, the response keeps the vendor's native format (for Anthropic models, an array of content blocks). When unset, the SDK-wide `puter.ai.normalize` applies — itself tri-state: set it to `true` to normalize every call regardless of release date, `false` to disable normalization for every call, and when it is left unset too (the default) the release-date policy applies: **models released on or after September 1, 2026 return normalized (OpenAI-format) responses by default**, and for older models the default is unchanged — `message.content` keeps its vendor-native shape. (A handful of reasoning fields were made consistent across all models independently of this option; see [Reasoning fields on existing models](#reasoning-fields-on-existing-models).) Streaming responses are unaffected — chunks already share one format across vendors. See [Response normalization](#response-normalization).
- `compaction` (Boolean | Object) (Optional) - Opt into inline context compaction for long conversations. Pass `true` to enable it with provider defaults, or `{ trigger_tokens: number }` to set the token threshold at which earlier context is summarized. When the model compacts, you receive a `compaction` chunk while streaming (or a `compaction` field on the result when not streaming) containing an opaque `encrypted_content` summary. Resend that item in `messages` on the next turn in place of the summarized history. The compaction chunk shape is identical across providers, so the same code works whether `model` is an OpenAI or Anthropic model. See [Compaction](#compaction).
- `toolChoice` (String | Object) (Optional) - Which tool, if any, the model must call. Pass `"auto"` (the model decides), `"none"` (never call a tool), `"required"` (call at least one), or an object naming a specific tool: `{ type: "tool", name: "..." }`. Requires `tools`.
- `parallelToolCalls` (Boolean) (Optional) - Whether the model may call more than one tool in a single turn. Left unset, tool calls run one at a time.
- `stopSequences` (Array of String) (Optional) - Up to the model's limit of strings; generation stops the moment one of them appears in the output.
- `outputFormat` (Object) (Optional) - Constrains the response to a JSON schema: `{ type: "json_schema", schema, name?, strict? }`.
- `thinking` (Object | null) (Optional) - Extended-thinking control: `{ type: "adaptive" | "enabled" | "disabled" | "between_tools", budgetTokens?, display? }`. Pass `null` to send no thinking configuration at all (the model's own default); leaving it unset falls back to the legacy derivation from `reasoning_effort`. `budgetTokens` only applies to `type: "enabled"`.
- `streamToolInput` (Boolean) (Optional) - Streaming only. When `true`, a tool call's arguments arrive incrementally as `tool_use_start` / `tool_input_delta` chunks instead of a single `tool_use` chunk once the call completes.
#### `testMode` (Boolean) (Optional)
@@ -121,21 +117,6 @@ Returns a `Promise` that resolves to either:
- A [`ChatResponse`](/Objects/chatresponse) object containing the chat response data, or
- An async iterable object of [`ChatResponseChunk`](/Objects/chatresponsechunk) (when `stream` is set to `true`) that you can use with a `for await...of` loop to receive the response in parts as they become available.
Break out of the `for await...of` loop to stop reading a chat stream. When using
`new ReadableStream(response)`, call `reader.cancel()` to stop it.
You can also pass an `AbortSignal` to stop waiting for output. See the [stop a stream example](/playground/ai-chat-stop-stream/).
Stopping a stream stops the output your app receives. Once the request has
reached the model, the model still finishes its response, and the whole response
is billed.
The `ReadableStream` adapter emits text chunks as UTF-8 text. Other chunk types
(including usage, tool calls, reasoning and errors) are serialized as JSON followed
by a newline. Use the async iterator to receive every chunk as an object.
## Errors
In case of an error, the `Promise` will reject with an error message.
For streaming responses, an error from the AI provider is returned as a chunk
@@ -152,6 +133,112 @@ the model may already have run (see
[Request timeouts](/rate-limits-and-quotas/#request-timeouts)). Use
`try...catch` around the call and loop to handle these failures.
### Stopping a stream
Break out of the `for await...of` loop to stop reading a chat stream. When using
`new ReadableStream(response)`, call `reader.cancel()` to stop it.
You can also pass an `AbortSignal` to stop waiting for output:
```js
const controller = new AbortController();
const stopButton = document.createElement('button');
stopButton.textContent = 'Stop';
stopButton.onclick = () => controller.abort();
document.body.append(stopButton);
try {
const response = await puter.ai.chat('Write a long story', {
stream: true,
signal: controller.signal,
});
for await (const part of response) {
if (part.type === 'text') console.log(part.text);
}
} catch (error) {
if (error.name !== 'AbortError') throw error;
} finally {
stopButton.remove();
}
```
Stopping a stream stops the output your app receives. Once the request has
reached the model, the model still finishes its response, and the whole response
is billed.
The `ReadableStream` adapter emits text chunks as UTF-8 text. Other chunk types
(including usage, tool calls, reasoning and errors) are serialized as JSON followed
by a newline. Use the async iterator to receive every chunk as an object.
## Vendors
We use different vendors for different models and try to use the best vendor available at the time of the request. Vendors currently include Alibaba Cloud, Anthropic, Azure OpenAI, DeepSeek, Google, Infron, Meta, MiniMax, Mistral, Moonshot AI, OpenAI, OpenRouter, Together AI, xAI, and Z.AI. Call [`puter.ai.listModelProviders()`](/AI/listModelProviders) for the current list, or pass `provider` in the options object to pin a request to one of them.
When the vendor serving a request fails, Puter retries the request on the next vendor that serves the same model (up to three attempts), and you are billed for the attempt that answers. Pass `fallback: false` to turn this off, so that a failure from the first vendor tried is returned to you instead:
```js
// Answered by Anthropic directly, or an error. Never sent to another vendor.
const response = await puter.ai.chat('Hello', {
model: 'claude-sonnet-4-6',
provider: 'claude',
fallback: false,
});
```
## Response Normalization
Most vendors respond in the OpenAI chat format, where `message.content` is a string and tool calls appear as `message.tool_calls`. Anthropic models historically respond in Anthropic's native format instead, where `message.content` is an array of content blocks such as `[{ type: "text", text: "..." }]`.
**Going forward, all models released on or after September 1, 2026 return responses in the OpenAI format**, no matter which vendor serves them — so the same response-handling code works across every new model. For models released before that date, the `normalize` default does not change: leave the option unset and `message.content` keeps its vendor-native shape.
### Reasoning fields on existing models
Separately from the `normalize` default, four reasoning-related fields were made consistent across vendors. These apply to **every** model, including ones released before the cutoff, and are not affected by `normalize`:
| Field | Before | Now |
| --- | --- | --- |
| `message.reasoning_content` | Present on providers following the DeepSeek convention (DeepSeek, OpenRouter and others) | **Renamed to `message.reasoning`.** Read `reasoning` instead — `reasoning_content` is no longer present on non-streaming responses. |
| `message.reasoning` on OpenAI Responses models | Always present as `null` | Absent when the model returned no reasoning summary; a string when it did. `if (msg.reasoning)` is unaffected; `'reasoning' in msg` changes. |
| `message.reasoning_details` on OpenAI Responses models | Not present | Present when the model returned reasoning items, carrying their `id` and `encrypted_content` for replay. |
| `finish_reason` on OpenAI Responses models | Always `"stop"` | `"tool_calls"` when the turn ended in tool calls, `"stop"` otherwise. |
If your code reads `message.reasoning_content` on a non-streaming response, that is the one change that removes a field — switch to `message.reasoning`.
You can control this per call with the `normalize` option:
```js
// Force the OpenAI format on any model, old or new:
const response = await puter.ai.chat("Hello", { model: "claude-sonnet-5", normalize: true });
console.log(response.message.content); // a string, or null on a tool-only turn
console.log(response.finish_reason); // "stop" | "length" | "tool_calls" | "content_filter" | vendor value
// Force the vendor-native format, even on a post-cutoff model:
const native = await puter.ai.chat("Hello", { model: "claude-sonnet-5", normalize: false });
```
Or SDK-wide with `puter.ai.normalize`:
```js
// Unset (the default): the release-date rule applies — models released on or
// after September 1, 2026 return the OpenAI format, older models stay native.
puter.ai.normalize = true; // force normalization: every chat() call
// returns the OpenAI format, old or new model
puter.ai.normalize = false; // disable normalization: every chat() call
// returns the vendor-native format
puter.ai.normalize = undefined; // back to the release-date rule
```
A `normalize` option on an individual call always overrides `puter.ai.normalize` in either direction. Normalized responses carry `normalized: true`.
On a normalized response, extended-thinking output (from reasoning models that expose it) is joined into `message.reasoning`, and Anthropic stop reasons are mapped to OpenAI values (`end_turn` → `stop`, `max_tokens` → `length`, `tool_use` → `tool_calls`, `refusal` → `content_filter`). A vendor stop reason with no OpenAI equivalent — Anthropic's `pause_turn`, for instance — passes through unchanged, so treat `finish_reason` as an open set. See [`finish_reason`](/Objects/chatresponse) for the full mapping.
Normalization does not cost you the ability to continue a reasoning turn. The opaque parts a provider needs back — Anthropic thinking-block signatures, OpenAI reasoning item ids and encrypted content — are preserved verbatim on `message.reasoning_details`. Resend that array as-is alongside the message when you continue an extended-thinking tool-use loop. The artifacts are vendor-specific and only meaningful to the model that produced them, so replay them to the same model — don't carry them across vendors. Kimi K3.1 Pro reads its `reasoning` string back the same way, so keep `reasoning` on the assistant message when you continue a conversation with it.
One caveat. The release-date rule applies to the model that actually serves the request — if a request is rerouted to a fallback provider, the served model's release date decides.
One thing to know about the release-date rule: a model's release date comes from the catalog of whichever provider serves it, and some providers report it from their own live listing. Models served through OpenRouter carry the date OpenRouter itself assigns, so a model newly listed there on or after September 1, 2026 is normalized by default without Puter shipping any change. Pin `normalize: false` if your code depends on a provider's native shape.
Streaming is unaffected by normalization: streamed [`ChatResponseChunk`](/Objects/chatresponsechunk) objects already share one format across all vendors, and the chunk types a model emits do not depend on the `normalize` option. Reasoning models stream their thinking as `reasoning` chunks on every provider, whether or not normalization applies.
## Function Calling
Function calling (also known as tool calling) allows AI models to request data or perform actions by calling functions you define. This enables the AI to access real-time information, interact with external systems, and perform tasks beyond its training data.
@@ -182,10 +269,51 @@ After executing the function, send the result back by including a message with:
- `tool_call_id` (String) - The `id` from the tool call
- `content` (String) - The function result as a string
See also [Gemini models](#gemini-models).
Gemini models can attach `extra_content.google.thought_signature` to a tool call. Send the assistant message back with its `tool_calls` unchanged, including `extra_content`, so the model can continue its reasoning. In a stream, the signature arrives on the `tool_use` chunk's `extra_content`.
See the [Function Calling example](/playground/ai-function-calling/) for a complete working implementation.
### Web Search
Specific to OpenAI models, you can use the built-in web search tool, allowing the AI to access up-to-date information from the internet.
Pass in the `tools` parameter with the type of `web_search`.
```js
{
model: 'openai/gpt-5.6-luna',
tools: [{type: "web_search"}]
}
```
The code implementation is available in our [web search example](/playground/ai-web-search/).
List of OpenAI models that support the web search can be found in their [API compatibility documentation](https://platform.openai.com/docs/guides/tools-web-search#api-compatibility).
## Prompt Caching
Specific to Anthropic models, you can use the cache control feature, allowing you to optimize costs for repeated prompts.
Pass in the `cache_control` parameter inside the object in the `messages` array.
```js
[
{
role: 'system',
content: 'a really long system prompt',
cache_control: { type: "ephemeral" }
},
{
role: 'user',
content: '<your message>'
},
]
```
You can find the implementation in our [prompt caching example](/playground/ai-claude-cache-control/). Find more details about cache control in [Anthropic documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
Moonshot's Kimi K3 and Kimi K3.1 Pro models cache prompt prefixes on their own and take no `cache_control`. Their `usage` reports the tokens read from the cache as `cached_tokens` and the tokens written to it as `cache_write_tokens`. Each is billed at the model's own rate from the [AI models list](https://developer.puter.com/ai/models/).
## Compaction
For long, multi-turn conversations that you keep on the client, enable
@@ -298,163 +426,7 @@ if ( result.compaction ) {
`trigger_tokens` below a provider's minimum) arrive as an `error` chunk, not a
thrown exception.
## Response Normalization
Most providers respond in the OpenAI chat format, where `message.content` is a string and tool calls appear as `message.tool_calls`. Anthropic models historically respond in Anthropic's native format instead, where `message.content` is an array of content blocks such as `[{ type: "text", text: "..." }]`.
**Going forward, all models released on or after September 1, 2026 return responses in the OpenAI format**, no matter which provider serves them — so the same response-handling code works across every new model. For models released before that date, the `normalize` default does not change: leave the option unset and `message.content` keeps its provider-native shape.
**Reasoning fields on existing models**
Separately from the `normalize` default, four reasoning-related fields were made consistent across providers. These apply to **every** model, including ones released before the cutoff, and are not affected by `normalize`:
| Field | Before | Now |
| --- | --- | --- |
| `message.reasoning_content` | Present on providers following the DeepSeek convention (DeepSeek, OpenRouter and others) | **Renamed to `message.reasoning`.** Read `reasoning` instead — `reasoning_content` is no longer present on non-streaming responses. |
| `message.reasoning` on OpenAI Responses models | Always present as `null` | Absent when the model returned no reasoning summary; a string when it did. `if (msg.reasoning)` is unaffected; `'reasoning' in msg` changes. |
| `message.reasoning_details` on OpenAI Responses models | Not present | Present when the model returned reasoning items, carrying their `id` and `encrypted_content` for replay. |
| `finish_reason` on OpenAI Responses models | Always `"stop"` | `"tool_calls"` when the turn ended in tool calls, `"stop"` otherwise. |
If your code reads `message.reasoning_content` on a non-streaming response, that is the one change that removes a field — switch to `message.reasoning`.
You can control this per call with the `normalize` option:
```js
// Force the OpenAI format on any model, old or new:
const response = await puter.ai.chat("Hello", { model: "claude-sonnet-5", normalize: true });
console.log(response.message.content); // a string, or null on a tool-only turn
console.log(response.finish_reason); // "stop" | "length" | "tool_calls" | "content_filter" | provider value
// Force the provider-native format, even on a post-cutoff model:
const native = await puter.ai.chat("Hello", { model: "claude-sonnet-5", normalize: false });
```
Or SDK-wide with `puter.ai.normalize`:
```js
// Unset (the default): the release-date rule applies — models released on or
// after September 1, 2026 return the OpenAI format, older models stay native.
puter.ai.normalize = true; // force normalization: every chat() call
// returns the OpenAI format, old or new model
puter.ai.normalize = false; // disable normalization: every chat() call
// returns the provider-native format
puter.ai.normalize = undefined; // back to the release-date rule
```
A `normalize` option on an individual call always overrides `puter.ai.normalize` in either direction. Normalized responses carry `normalized: true`.
On a normalized response, extended-thinking output (from reasoning models that expose it) is joined into `message.reasoning`, and Anthropic stop reasons are mapped to OpenAI values (`end_turn` → `stop`, `max_tokens` → `length`, `tool_use` → `tool_calls`, `refusal` → `content_filter`). A provider stop reason with no OpenAI equivalent — Anthropic's `pause_turn`, for instance — passes through unchanged, so treat `finish_reason` as an open set. See [`finish_reason`](/Objects/chatresponse) for the full mapping.
Normalization does not cost you the ability to continue a reasoning turn. The opaque parts a provider needs back — Anthropic thinking-block signatures, OpenAI reasoning item ids and encrypted content — are preserved verbatim on `message.reasoning_details`. Resend that array as-is alongside the message when you continue an extended-thinking tool-use loop. The artifacts are provider-specific and only meaningful to the model that produced them, so replay them to the same model — don't carry them across providers. Kimi K3.1 Pro reads its `reasoning` string back the same way, so keep `reasoning` on the assistant message when you continue a conversation with it.
One caveat. The release-date rule applies to the model that actually serves the request — if a request is rerouted to a fallback provider, the served model's release date decides.
One thing to know about the release-date rule: a model's release date comes from the catalog of whichever provider serves it, and some providers report it from their own live listing. Models served through OpenRouter carry the date OpenRouter itself assigns, so a model newly listed there on or after September 1, 2026 is normalized by default without Puter shipping any change. Pin `normalize: false` if your code depends on a provider's native shape.
Streaming is unaffected by normalization: streamed [`ChatResponseChunk`](/Objects/chatresponsechunk) objects already share one format across all providers, and the chunk types a model emits do not depend on the `normalize` option. Reasoning models stream their thinking as `reasoning` chunks on every provider, whether or not normalization applies.
## Providers
We use different providers for different models and try to use the best provider available at the time of the request. Providers currently include Alibaba Cloud, Anthropic, Azure OpenAI, DeepSeek, Google, Infron, Meta, MiniMax, Mistral, Moonshot AI, OpenAI, OpenRouter, Together AI, xAI, and Z.AI. Call [`puter.ai.listModelProviders()`](/AI/listModelProviders) for the current list, or pass `provider` in the options object to pin a request to one of them.
When the provider serving a request fails, Puter retries the request on the next provider that serves the same model (up to three attempts), and you are billed for the attempt that answers. Pass `fallback: false` to turn this off, so that a failure from the first provider tried is returned to you instead:
```js
// Answered by Anthropic directly, or an error. Never sent to another provider.
const response = await puter.ai.chat('Hello', {
model: 'claude-sonnet-4-6',
provider: 'claude',
fallback: false,
});
```
With `provider` set as well, the request is answered by that provider or fails. If that provider does not serve the model, the request fails with a `400` instead of going to another provider. The error is the same one you would get if every provider had failed, for example `400 upstream_provider_unavailable` or `429 upstream_rate_limited`.
## Model-specific features
### OpenAI models
#### `temperature`
Some OpenAI reasoning models ignore `temperature` and `top_p`. GPT-5.6, GPT-6 Sol/Luna, and GPT-6.1 Terra accept them only with `reasoning_effort: 'none'`, and GPT-6 Astra, GPT-6.1 Sol, and GPT-5.3 Codex always ignore them.
#### `reasoning_effort`
GPT-6.1 Terra accepts `none`, `low`, `medium` (default), `high`, `xhigh`, and `max`.
#### Web Search
Specific to OpenAI models, you can use the built-in web search tool, allowing the AI to access up-to-date information from the internet.
Pass in the `tools` parameter with the type of `web_search`.
```js
{
model: 'openai/gpt-6-luna',
tools: [{type: "web_search"}]
}
```
The code implementation is available in our [web search example](/playground/ai-web-search/).
List of OpenAI models that support the web search can be found in their [API compatibility documentation](https://platform.openai.com/docs/guides/tools-web-search#api-compatibility).
#### Prompt Caching
OpenAI models cache prompt prefixes on their own. Requests from the same app and user share one cache by default, and you can pick the cache with `prompt_cache_key`. Their `usage` reports the tokens read from the cache as `cached_tokens`.
GPT-5.6 and later models charge for cache writes and let you choose how long an entry lives. Pass `prompt_cache_options` with a `ttl` of `"30m"` (the default) or `"12h"`. The `usage` reports the tokens written with the 30 minute lifetime as `cache_write_tokens` and the tokens written with the 12 hour lifetime as `cache_write_12h_tokens`. Writing a 30 minute entry costs 1.25 times the model's input rate, and writing a 12 hour entry costs 2.5 times the input rate. Reads cost the same for both lifetimes. To mark where a reusable prefix ends, add `prompt_cache_breakpoint: { mode: "explicit" }` to a content part. A breakpoint can carry its own `ttl`, which overrides the request's. OpenAI only takes a 12 hour lifetime, or a lifetime on a breakpoint, through its Responses API, so a request that uses either is sent there.
```js
puter.ai.chat('<your message>', {
model: 'gpt-6-luna',
prompt_cache_options: { ttl: '12h' }
});
```
Older OpenAI models charge nothing for cache writes. Pass `prompt_cache_retention` with `"in_memory"` for an entry that lives a few minutes or `"24h"` for one that can live up to a day. The 24 hour option is the default, and GPT-5.5 and later models only support it.
The `usageDetails` object on every result carries the same counts under provider-neutral names, with one field per lifetime, such as `cacheWrite30mTokens` and `cacheWrite12hTokens`. See the [ChatResponse](/Objects/chatresponse/) object for the full list.
### Anthropic models
#### Prompt Caching
Specific to Anthropic models, you can use the cache control feature, allowing you to optimize costs for repeated prompts.
Pass in the `cache_control` parameter inside the object in the `messages` array.
```js
[
{
role: 'system',
content: 'a really long system prompt',
cache_control: { type: "ephemeral" }
},
{
role: 'user',
content: '<your message>'
},
]
```
You can find the implementation in our [prompt caching example](/playground/ai-claude-cache-control/). Find more details about cache control in [Anthropic documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
### Gemini models
#### `temperature`
`gemini-4-argon` ignores `temperature`. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` use it.
#### `reasoning_effort`
`gemini-4-argon` accepts `low`, `medium` and `high`, and uses `medium` when you leave it unset. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` accept `minimal`, `low`, `medium` and `high`, and use Google's default level when you leave it unset. For these three Gemini models, any other value fails with `400 bad_request`, and messages can contain text and images only.
#### Function calling
Gemini models can attach `extra_content.google.thought_signature` to a tool call. Send the assistant message back with its `tool_calls` unchanged, including `extra_content`, so the model can continue its reasoning. In a stream, the signature arrives on the `tool_use` chunk's `extra_content`.
### Gemini image models
## Image Generation (Gemini Image Models)
Certain Gemini models can generate and edit images as part of a chat conversation. These models accept text and image inputs, and return text and images in the response.
@@ -541,38 +513,22 @@ for await (const part of resp) {
}
```
### Moonshot Kimi models
#### `reasoning_effort`
Kimi K3.1 Pro accepts `low`, `high` and `max`, uses `max` when you leave it unset, and fails with `400 bad_request` on any other value.
#### Prompt Caching
Moonshot's Kimi K3 and Kimi K3.1 Pro models cache prompt prefixes on their own and take no `cache_control`. Their `usage` reports the tokens read from the cache as `cached_tokens` and the tokens written to it as `cache_write_tokens`. Each is billed at the model's own rate from the [AI models list](https://developer.puter.com/ai/models/).
### Meta Muse Spark models
#### `reasoning_effort`
Muse Spark always reasons, so `none` is ignored for it.
## Examples
### Ask GPT-6 Luna a question
<strong class="example-title">Ask GPT-5.6 Luna a question</strong>
```html;ai-chatgpt
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
puter.ai.chat(`What is life?`, { model: "gpt-6-luna" }).then(puter.print);
puter.ai.chat(`What is life?`, { model: "gpt-5.6-luna" }).then(puter.print);
</script>
</body>
</html>
```
### Image Analysis
<strong class="example-title">Image Analysis</strong>
```html;ai-gpt-vision
<html>
@@ -582,7 +538,7 @@ Muse Spark always reasons, so `none` is ignored for it.
<script>
puter.ai
.chat(`What do you see?`, `https://assets.puter.site/doge.jpeg`, {
model: "gpt-6-luna",
model: "gpt-5.6-luna",
})
.then(puter.print);
</script>
@@ -590,7 +546,7 @@ Muse Spark always reasons, so `none` is ignored for it.
</html>
```
### Video Analysis
<strong class="example-title">Video Analysis</strong>
```html;ai-video-analysis
<html>
@@ -607,7 +563,7 @@ Muse Spark always reasons, so `none` is ignored for it.
</html>
```
### Stream the response
<strong class="example-title">Stream the response</strong>
```html;ai-chat-stream
<html>
@@ -623,40 +579,7 @@ Muse Spark always reasons, so `none` is ignored for it.
</html>
```
### Stop a stream
```html;ai-chat-stop-stream
<html>
<body>
<button id="stop">Stop</button>
<div id="output" style="white-space: pre-wrap"></div>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
const controller = new AbortController();
const stopButton = document.getElementById('stop');
const output = document.getElementById('output');
stopButton.onclick = () => controller.abort();
try {
const response = await puter.ai.chat('Write a long story', {
stream: true,
signal: controller.signal,
});
for await (const part of response) {
if (part.type === 'text') output.append(part.text);
}
} catch (error) {
if (error.name !== 'AbortError') throw error;
} finally {
stopButton.remove();
}
})();
</script>
</body>
</html>
```
### Function Calling
<strong class="example-title">Function Calling</strong>
```html;ai-function-calling
<html>
@@ -716,7 +639,7 @@ Muse Spark always reasons, so `none` is ignored for it.
</html>
```
### Streaming Function Calling
<strong class="example-title">Streaming Function Calling</strong>
```html;ai-streaming-function-calling
<html>
@@ -798,7 +721,7 @@ Muse Spark always reasons, so `none` is ignored for it.
</html>
```
### Web Search
<strong class="example-title">Web Search</strong>
```html;ai-web-search
<html>
@@ -808,7 +731,7 @@ Muse Spark always reasons, so `none` is ignored for it.
puter.print(`Loading...`);
puter.ai
.chat("Summarize what the User-Pays Model is: https://docs.puter.com/user-pays-model/", {
model: "openai/gpt-6-luna",
model: "openai/gpt-5.6-luna",
tools: [{ type: "web_search" }],
})
.then(puter.print);
@@ -817,7 +740,7 @@ Muse Spark always reasons, so `none` is ignored for it.
</html>
```
### Prompt caching with Claude
<strong class="example-title">Prompt caching with Claude</strong>
```html;ai-claude-cache-control
<html>
@@ -872,7 +795,7 @@ Policy 8 - Account Management: Each Enterprise and Ultimate customer is assigned
</html>
```
### Image Generation
<strong class="example-title">Image Generation</strong>
```html;ai-image-chat
<html>
@@ -897,7 +820,7 @@ Policy 8 - Account Management: Each Enterprise and Ultimate customer is assigned
</html>
```
### Multi-Turn Image Editing
<strong class="example-title">Multi-Turn Image Editing</strong>
```html;ai-image-edit
<html>
@@ -949,7 +872,7 @@ Policy 8 - Account Management: Each Enterprise and Ultimate customer is assigned
</html>
```
### Working with Files
<strong class="example-title">Working with Files</strong>
```html;ai-resume-analyzer
<!DOCTYPE html>
+4 -5
View File
@@ -41,12 +41,12 @@ The conversation exposes these properties and methods.
| --- | --- |
| `state` | `prepared`, `connecting`, `connected`, or `closed`. |
| `model` | Resolved model name. |
| `connect()` | Opens the connection and applies settings. Resolves after the provider acknowledges session configuration. Concurrent calls share one connection Promise, including bounded retries for transient setup failures. Calling it while connected opens no additional connection. During a reconnection, it resolves when the reconnection succeeds and rejects if the conversation closes instead. |
| `connect()` | Opens the connection and applies settings. Resolves after the provider acknowledges session configuration. Concurrent calls share one attempt. Calling it while connected opens no additional connection. During a reconnection, it resolves when the reconnection succeeds and rejects if the conversation closes instead. |
| `configure(options)` | Updates the given session settings and resolves after the provider acknowledges this update. Settings it isn't given, including ones set through native events, are left alone. Accepts the factory options except `model`. One update can be pending at a time. Provider rules apply to changes such as switching a voice after audio has been generated. |
| `sendText(text)` | Adds a user message and requests a response. Text must be nonempty and at most 8192 characters. |
| `appendAudio(pcm)` | Appends an `Int16Array` containing PCM16, 24 kHz mono audio. A chunk contains at most 12000 samples. |
| `commitAudio()` | Commits buffered audio and requests a response. Requires turn detection to be off, through `turnDetection: false` or a native session update. The provider requires at least 100 ms of buffered audio. |
| `interrupt(position?)` | Stops built-in playback immediately and waits for cancellation and playback-history synchronization. A reply requested but not yet started is cancelled as soon as it starts. Custom players provide `{ itemId, contentIndex, playedMs }`. An idle call resolves immediately. Readiness failures reject the returned Promise with `session_not_ready` or `session_closed`. |
| `interrupt(position?)` | Stops built-in playback immediately and waits for cancellation and playback-history synchronization. A reply requested but not yet started is cancelled as soon as it starts. Custom players provide `{ itemId, contentIndex, playedMs }`. An idle call resolves immediately. |
| `send(event)` | Sends a native OpenAI JSON event. |
| `on(type, handler)` | Registers a listener and returns an unsubscribe function. Use `'*'` to receive every native and convenience event. |
| `off(type, handler)` | Removes a listener. |
@@ -54,9 +54,9 @@ The conversation exposes these properties and methods.
Attach listeners before connecting to receive initial session events. Closing during connection rejects the pending attempt.
If the connection drops because of network loss, a server restart or a provider error, the conversation reconnects on its own. Normal closes with code 1000 end the conversation. While it reconnects, `state` is `connecting`, a `reconnecting` event fires, and pending operations reject with `connection_lost`. The new session gets every setting the provider acknowledged, from the factory options, `configure()` and native `session.update` events. Then `ready` fires again with `reconnected: true`. Listeners, the microphone and the speaker carry over. Microphone audio captured while reconnecting is dropped.
If the connection drops because of network loss, a server restart or a provider error, the conversation reconnects on its own. While it reconnects, `state` is `connecting`, a `reconnecting` event fires, and pending operations reject with `connection_lost`. The new session gets every setting the provider acknowledged, from the factory options, `configure()` and native `session.update` events. Then `ready` fires again with `reconnected: true`. Listeners, the microphone and the speaker carry over. Microphone audio captured while reconnecting is dropped.
The conversation also restores recent context. Before `ready` fires, it adds back up to 40 of the most recent conversation items, 48 KiB in total. Restored items are user text, user speech that has a transcript, assistant replies as text, and function calls with their outputs. User speech has a transcript only when input transcription is configured through native session settings. An assistant reply that was interrupted, or still playing when the connection dropped, is cut to what the user saw or heard. The cut is estimated from how much of the audio played, so it can be off by a few words. Images, speech without a transcript, deleted items, and replies still generating when the connection dropped are left out. Restored items get new item IDs, don't fire item events, and count as input tokens on the next response. Item IDs from before the reconnect don't work in later events. Restoration uses a contiguous suffix of restorable items. An oversized newest item leaves that suffix empty. Older items left out to fit the window fire a `context_trimmed` event with the count in `omittedItems`; this is routine and not an error. A provider refusal or a failure to send the restored items fires one `error` event with code `context_restore_failed` per restoration. In both cases the conversation stays connected. Items are preserved without truncating text, function arguments or function results.
The conversation also restores recent context. Before `ready` fires, it adds back up to 40 of the most recent conversation items, 48 KiB in total. Restored items are user text, user speech that has a transcript, assistant replies as text, and function calls with their outputs. User speech has a transcript only when input transcription is configured through native session settings. An assistant reply that was interrupted, or still playing when the connection dropped, is cut to what the user saw or heard. The cut is estimated from how much of the audio played, so it can be off by a few words. Images, speech without a transcript, deleted items, and replies still generating when the connection dropped are left out. Restored items get new item IDs, don't fire item events, and count as input tokens on the next response. Item IDs from before the reconnect don't work in later events. If the provider refuses a restored item, an `error` event fires with code `context_restore_failed` and the conversation stays connected.
When Puter knows ahead of time that the server a session runs on will restart, it sends a `puter.reconnect` event, which also reaches your listeners. Without that notice, a restart drops the connection and the conversation reconnects as above. Within a few seconds, the conversation opens a replacement session in the background while the current one keeps working. It switches once no reply is generating or playing, no user turn is being spoken or transcribed, no function call is waiting for its output, and nothing you sent is waiting on the provider. The switch goes through the same steps as a reconnect, and the `reconnecting` event has code 1012. If the replacement can't open yet, the conversation keeps the current session and tries again later. A refusal that retrying can't fix, such as `insufficient_funds`, stops the attempts. If the server closes the session first, the conversation switches to the replacement, or reconnects if none is open.
@@ -72,7 +72,6 @@ The conversation emits these events alongside native events.
| --- | --- |
| `ready` | `model`, `provider`, and `reconnected: true` after a reconnection |
| `reconnecting` | `code`, `reason` of the dropped connection. After a `puter.reconnect` event, `code` is 1012. |
| `context_trimmed` | `omittedItems`, the number of restorable items left out of the restored context after a reconnection. Fires before `ready`. |
| `text` | `delta`, `responseId` |
| `transcript` | `role`, `text`, `final`. Assistant deltas use `final: false`. Completed assistant and user transcripts use `final: true`. User transcripts require transcription to be configured through native session settings. |
| `audio` | `pcm`, `sampleRate`, `responseId`, `itemId`, `contentIndex`. The PCM chunk is an `Int16Array`. |
+53 -47
View File
@@ -4,9 +4,7 @@ description: Extract text from images and documents using OCR to read printed te
platforms: [websites, apps, nodejs, workers]
---
Given an image or document, returns the text it contains.
Also known as OCR (Optical Character Recognition), this API can be used to extract text from images of printed text, handwriting, or any other text-based content. AWS Textract is the default; Mistral OCR reads multi-page documents and more file formats, returns Markdown, and can fill a JSON schema from the document.
Given an image or document, returns the text it contains. Also known as OCR (Optical Character Recognition), this API can be used to extract text from images of printed text, handwriting, or any other text-based content. AWS Textract is the default; Mistral OCR reads multi-page documents and more file formats, returns Markdown, and can fill a JSON schema from the document.
## Syntax
@@ -32,15 +30,60 @@ Every call has the same shape; only the model name changes which service reads t
| Option | Type | Description |
|--------|------|-------------|
| `model` | `String` | The OCR model to use. See [Model-specific features](#model-specific-features) for the models. The model picks its provider, so `provider` is not needed alongside it. Lookups are case-insensitive. |
| `provider` | `String` | Either `'aws-textract'` (default) or `'mistral'`. Aliases `'aws'`, `'textract'` and `'mistral-ocr'` are also accepted. Without a `model`, the provider's default model runs. When both are given, the model must belong to the provider. |
| `model` | `String` | The OCR model to use (see [Models](#models)). The model picks its provider, so `provider` is not needed alongside it. Lookups are case-insensitive. |
| `provider` | `String` | `'aws-textract'` (default) or `'mistral'`. Aliases `'aws'`, `'textract'` and `'mistral-ocr'` are also accepted. Without a `model`, the provider's default model runs. When both are given, the model must belong to the provider. |
| `testMode` | `Boolean` | When `true`, returns a sample response without using credits. Defaults to `false` |
Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).
#### Models
| Model | Provider | Notes |
|-------|----------|-------|
| `aws-textract` (alias `textract`) | AWS Textract | Default. Plain text, one line per detected line. |
| `mistral-ocr-latest` (aliases `mistral-ocr-4`, `mistral-ocr-4-1`) | Mistral | Mistral OCR 4.1, the default Mistral model. Markdown output. |
| `mistral-ocr-2512` (aliases `mistral-ocr-3`, `mistral-ocr-3-0`) | Mistral | Mistral OCR 3, at a lower per-page rate than OCR 4. |
`mistral-ocr-latest` is pinned to OCR 4.1 and moves to a newer model only when Puter adds it. Puter rejects the deprecated `mistral-ocr-2503` and `mistral-ocr-2505` and the retired `mistral-ocr-4-0` with `bad_request`.
#### AWS Textract options
AWS Textract takes no options beyond `model` and `provider`. It reads a single-page document as a whole, so there is nothing to select or tune; the Mistral options below are ignored.
#### Mistral options
These options apply to Mistral models. AWS Textract ignores them.
| Option | Type | Description |
|--------|------|-------------|
| `pages` | `Array<Number>` | Pages to process, as 0-based indexes |
| `includeImageBase64` | `Boolean` | Include extracted images in the provider response |
| `imageLimit` | `Number` | Maximum number of images to extract |
| `imageMinSize` | `Number` | Minimum height and width of an image to extract |
| `tableFormat` | `String` | `'markdown'` or `'html'`: extract tables in that format instead of leaving them inline |
| `extractHeader` | `Boolean` | Move page headers out of the returned text |
| `extractFooter` | `Boolean` | Move page footers out of the returned text |
| `documentAnnotationFormat` | `Object` | A JSON schema for a document-level annotation: `{ type: 'json_schema', json_schema: { name, schema } }`. When set, `img2txt()` resolves to the annotation instead of the text. Billed per page on top of OCR. |
| `documentAnnotationPrompt` | `String` | Instructions that guide the document annotation |
| `bboxAnnotationFormat` | `Object` | A JSON schema, in the same shape, for annotating each extracted image. Billed per page on top of OCR. |
In both annotation formats, `json_schema` may also be spelled `jsonSchema` and `schema` may be spelled `schemaDefinition`. Any other shape is rejected with `bad_request` before anything is billed.
For more details about each option, see the [Mistral OCR documentation](https://docs.mistral.ai/api/endpoint/ocr).
#### Input limits
| Model | Formats | Maximum size |
|-------|---------|--------------|
| AWS Textract | JPEG, PNG, TIFF, and **single-page** PDF | 10 MB |
| Mistral | PDF (up to 1,000 pages), images (JPEG, PNG, AVIF, TIFF, GIF, HEIC, BMP, WebP), and documents such as DOCX, PPTX, XLSX, EPUB and RTF | 50 MB |
The SDK checks the decoded size of data URI inputs against the selected model's limit before upload: 10 MB for Textract and 36 MB when a Mistral model or provider is specified (a data URI is a third larger than the file, and a request body is capped at 50 MB). URLs and Puter paths are limited by the backend. When neither model nor provider is specified, the SDK checks against 10 MB; explicitly select Mistral for larger inline inputs. Size failures use `input_too_large`. Textract rejects multi-page PDFs and other formats with `bad_request`; use a Mistral model for those.
## Return value
A `Promise` that resolves to a string, the recognized text, one line per line. The text's format depends on the model, and some model options change what the string holds. See [Model-specific features](#model-specific-features).
A `Promise` that resolves to a string.
- By default the string is the recognized text, one line per line: plain text from AWS Textract, Markdown from Mistral.
- When `documentAnnotationFormat` is set, the string is the document annotation: JSON that follows your schema.
The return shape is the same for every model.
@@ -58,46 +101,9 @@ A rejection carries the error body as the backend sent it: `{ message, code }`.
Other `upstream_*` codes mean the provider rejected the request or was unavailable; the `message` carries the provider's reason.
## Input limits
Each model accepts its own formats and sizes, listed in its section under [Model-specific features](#model-specific-features). The SDK checks the decoded size of a data URI against the selected model's limit before upload. When neither `model` nor `provider` is specified, it checks against the default model's limit. URLs and Puter paths are limited by the backend. An input over the limit fails with `input_too_large`.
## Model-specific features
### AWS Textract models
The model is `aws-textract` (alias `textract`), the default for every call. It returns plain text, one line per detected line.
AWS Textract reads JPEG, PNG, TIFF, and **single-page** PDF, up to 10 MB. It rejects multi-page PDFs and other formats with `bad_request`. Use a Mistral model for those.
AWS Textract takes no options beyond `model` and `provider`. It reads a single-page document as a whole, so there is nothing to select or tune; the Mistral options below are ignored.
### Mistral models
The default Mistral model is `mistral-ocr-latest` (aliases `mistral-ocr-4`, `mistral-ocr-4-1`), which is Mistral OCR 4.1. It stays on OCR 4.1 and moves to a newer model only when Puter adds it. Mistral OCR 3 is `mistral-ocr-2512` (aliases `mistral-ocr-3`, `mistral-ocr-3-0`), at a lower per-page rate than OCR 4. Both return Markdown.
Mistral reads PDF (up to 1,000 pages), images (JPEG, PNG, AVIF, TIFF, GIF, HEIC, BMP, WebP), and documents such as DOCX, PPTX, XLSX, EPUB and RTF, up to 50 MB. The SDK checks a data URI against 36 MB, because a data URI is a third larger than the file and a request body is capped at 50 MB. Select a Mistral model or provider to send a data URI over 10 MB, since without either the SDK checks against AWS Textract's limit.
| Option | Type | Description |
|--------|------|-------------|
| `pages` | `Array<Number>` | Pages to process, as 0-based indexes |
| `includeImageBase64` | `Boolean` | Include extracted images in the provider response |
| `imageLimit` | `Number` | Maximum number of images to extract |
| `imageMinSize` | `Number` | Minimum height and width of an image to extract |
| `tableFormat` | `String` | Either `'markdown'` or `'html'`: extract tables in that format instead of leaving them inline |
| `extractHeader` | `Boolean` | Move page headers out of the returned text |
| `extractFooter` | `Boolean` | Move page footers out of the returned text |
| `documentAnnotationFormat` | `Object` | A JSON schema for a document-level annotation: `{ type: 'json_schema', json_schema: { name, schema } }`. When set, `img2txt()` resolves to the annotation instead of the text, as JSON that follows your schema. Billed per page on top of OCR. |
| `documentAnnotationPrompt` | `String` | Instructions that guide the document annotation |
| `bboxAnnotationFormat` | `Object` | A JSON schema, in the same shape, for annotating each extracted image. Billed per page on top of OCR. |
In both annotation formats, `json_schema` may also be spelled `jsonSchema` and `schema` may be spelled `schemaDefinition`. Any other shape is rejected with `bad_request` before anything is billed.
For more details about each option, see the [Mistral OCR documentation](https://docs.mistral.ai/api/endpoint/ocr).
## Examples
### Extract the text contained in an image
<strong class="example-title">Extract the text contained in an image</strong>
```html;ai-img2txt
<html>
@@ -110,7 +116,7 @@ For more details about each option, see the [Mistral OCR documentation](https://
</html>
```
### Read the same image with Mistral OCR
<strong class="example-title">Read the same image with Mistral OCR</strong>
```html;ai-img2txt-mistral
<html>
@@ -125,7 +131,7 @@ For more details about each option, see the [Mistral OCR documentation](https://
</html>
```
### Extract structured data with a document annotation
<strong class="example-title">Extract structured data with a document annotation</strong>
```html;ai-img2txt-annotation
<html>
+3 -11
View File
@@ -18,7 +18,7 @@ Connect to this URL on the Puter API host:
wss://api.puter.com/puterai/openai/v1/realtime?model=gpt-realtime-2.1-mini
```
The `model` query parameter defaults to `gpt-realtime-2.1-mini`. The model is fixed for the connection. A session update that changes it is rejected. It also accepts `gpt-realtime-2.1`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini`.
Self-hosted clients use their deployment's API host. The `model` query parameter defaults to `gpt-realtime-2.1-mini`. The model is fixed for the connection. A session update that changes it is rejected. It also accepts `gpt-realtime-2.1`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini`.
Authenticate with a Puter session, app, or full-access API token. Puter supplies the provider credential. Server clients pass this header:
@@ -30,8 +30,6 @@ Browser WebSocket clients pass `realtime` and `openai-insecure-api-key.<PUTER_TO
The socket can open before authentication and provider setup finish. Listen for errors and closure immediately. Wait for `session.created`, send your session settings, then wait for `session.updated` before submitting a turn.
To open a replacement while another connection remains healthy, set the optional `purpose=handover` query parameter. These attempts share a smaller per-credential budget, checked before the general connection-attempt budget. Initial connections and dropped-connection recovery use the URL without this parameter. The limits are listed in [rate limits and quotas](/rate-limits-and-quotas/).
## Text example
Install the `ws` package and set `PUTER_TOKEN` to your Puter token. Run this example as a Node.js ES module:
@@ -111,14 +109,8 @@ A new connection starts an empty conversation. To keep context, add earlier item
## Errors, billing, and limits
API errors arrive as `{ "type": "error", "error": { "code": "...", "message": "..." } }`. Handle socket errors and close frames too. Invalid, missing, or revoked credentials use `invalid_api_key`. If Puter can't check a credential or the account's plan, the error is `internal_error` and the session closes with 1011, so try again after a short, randomized delay. Other relay codes include `permission_denied`, `model_not_found`, `insufficient_funds`, `subscription_required`, `rate_limit_exceeded`, `invalid_event`, `connection_timeout`, `provider_unavailable`, `upstream_error`, and `server_restarting`.
API errors arrive as `{ "type": "error", "error": { "code": "...", "message": "..." } }`. Handle socket errors and close frames too. Authentication failures use `invalid_api_key`. Other relay codes include `model_not_found`, `insufficient_funds`, `subscription_required`, `rate_limit_exceeded`, `invalid_event`, `connection_timeout`, `provider_unavailable`, `upstream_error`, and `server_restarting`.
Policy failures generally close with 1008, oversized client messages with 1009, upstream or setup failures with 1011, server restarts with 1012, and temporary server overload with 1013. Reconnect after 1011, 1012, or 1013. Authentication and model failures can arrive after the WebSocket handshake succeeds.
Policy failures generally close with 1008, oversized client messages with 1009, upstream or setup failures with 1011, and server restarts with 1012. Authentication and model failures can arrive after the WebSocket handshake succeeds.
The endpoint requires a paid Puter plan. Usage is charged after completed responses and input transcriptions. While a response generates, part of your balance is held for it, so a session opens only if your balance covers one response. If the provider connection closes before a response reports its usage, the response is charged the full amount held for it. Each account can open 20 sessions at once by default. Each JSON message you send is limited to 64 KiB, including reassembled fragments. Messages from the provider can be up to 4 MiB. Stream audio in small chunks. All message rates, queue caps, timeouts, and other bounds are published in [rate limits and quotas](/rate-limits-and-quotas/).
Input transcription holds a small amount of your balance per transcribed item, priced at the transcription model you select. Two items are reserved ahead of your audio and replenished as the provider commits them, so a session holds only its spare items plus the items still awaiting a transcript. Changing the transcription model reprices the spare items. If your balance can't cover another item, audio waits while earlier items settle; once none are left to settle, the session closes with `insufficient_funds`. Failed transcriptions, items without final usage, and audio whose commit the provider never acknowledges before the connection closes are each charged that model's fixed per-item amount. Unacknowledged audio is charged one item per started 160 seconds. With server turn detection, only detected speech counts, so silence is never charged. The amounts, timeout and audio limits are listed in [rate limits and quotas](/rate-limits-and-quotas/).
Provider-reported realtime models must have a known price. A missing model uses the last known model. An explicitly unpriced model closes the connection with `upstream_error`.
After recording usage, the connection can continue while settled credit is positive, including a fractional balance. Each new billable operation still needs its full reservation after accounting for other outstanding holds.
Loaded 100 of 251 files, more files were not shown because too many files have changed in this diff. Show more